linux/Documentation/video4linux/v4l2-framework.txt
<<
>>
Prefs
   1Overview of the V4L2 driver framework
   2=====================================
   3
   4This text documents the various structures provided by the V4L2 framework and
   5their relationships.
   6
   7
   8Introduction
   9------------
  10
  11The V4L2 drivers tend to be very complex due to the complexity of the
  12hardware: most devices have multiple ICs, export multiple device nodes in
  13/dev, and create also non-V4L2 devices such as DVB, ALSA, FB, I2C and input
  14(IR) devices.
  15
  16Especially the fact that V4L2 drivers have to setup supporting ICs to
  17do audio/video muxing/encoding/decoding makes it more complex than most.
  18Usually these ICs are connected to the main bridge driver through one or
  19more I2C busses, but other busses can also be used. Such devices are
  20called 'sub-devices'.
  21
  22For a long time the framework was limited to the video_device struct for
  23creating V4L device nodes and video_buf for handling the video buffers
  24(note that this document does not discuss the video_buf framework).
  25
  26This meant that all drivers had to do the setup of device instances and
  27connecting to sub-devices themselves. Some of this is quite complicated
  28to do right and many drivers never did do it correctly.
  29
  30There is also a lot of common code that could never be refactored due to
  31the lack of a framework.
  32
  33So this framework sets up the basic building blocks that all drivers
  34need and this same framework should make it much easier to refactor
  35common code into utility functions shared by all drivers.
  36
  37
  38Structure of a driver
  39---------------------
  40
  41All drivers have the following structure:
  42
  431) A struct for each device instance containing the device state.
  44
  452) A way of initializing and commanding sub-devices (if any).
  46
  473) Creating V4L2 device nodes (/dev/videoX, /dev/vbiX and /dev/radioX)
  48   and keeping track of device-node specific data.
  49
  504) Filehandle-specific structs containing per-filehandle data;
  51
  525) video buffer handling.
  53
  54This is a rough schematic of how it all relates:
  55
  56    device instances
  57      |
  58      +-sub-device instances
  59      |
  60      \-V4L2 device nodes
  61          |
  62          \-filehandle instances
  63
  64
  65Structure of the framework
  66--------------------------
  67
  68The framework closely resembles the driver structure: it has a v4l2_device
  69struct for the device instance data, a v4l2_subdev struct to refer to
  70sub-device instances, the video_device struct stores V4L2 device node data
  71and the v4l2_fh struct keeps track of filehandle instances.
  72
  73The V4L2 framework also optionally integrates with the media framework. If a
  74driver sets the struct v4l2_device mdev field, sub-devices and video nodes
  75will automatically appear in the media framework as entities.
  76
  77
  78struct v4l2_device
  79------------------
  80
  81Each device instance is represented by a struct v4l2_device (v4l2-device.h).
  82Very simple devices can just allocate this struct, but most of the time you
  83would embed this struct inside a larger struct.
  84
  85You must register the device instance:
  86
  87        v4l2_device_register(struct device *dev, struct v4l2_device *v4l2_dev);
  88
  89Registration will initialize the v4l2_device struct. If the dev->driver_data
  90field is NULL, it will be linked to v4l2_dev.
  91
  92Drivers that want integration with the media device framework need to set
  93dev->driver_data manually to point to the driver-specific device structure
  94that embed the struct v4l2_device instance. This is achieved by a
  95dev_set_drvdata() call before registering the V4L2 device instance. They must
  96also set the struct v4l2_device mdev field to point to a properly initialized
  97and registered media_device instance.
  98
  99If v4l2_dev->name is empty then it will be set to a value derived from dev
 100(driver name followed by the bus_id, to be precise). If you set it up before
 101calling v4l2_device_register then it will be untouched. If dev is NULL, then
 102you *must* setup v4l2_dev->name before calling v4l2_device_register.
 103
 104You can use v4l2_device_set_name() to set the name based on a driver name and
 105a driver-global atomic_t instance. This will generate names like ivtv0, ivtv1,
 106etc. If the name ends with a digit, then it will insert a dash: cx18-0,
 107cx18-1, etc. This function returns the instance number.
 108
 109The first 'dev' argument is normally the struct device pointer of a pci_dev,
 110usb_interface or platform_device. It is rare for dev to be NULL, but it happens
 111with ISA devices or when one device creates multiple PCI devices, thus making
 112it impossible to associate v4l2_dev with a particular parent.
 113
 114You can also supply a notify() callback that can be called by sub-devices to
 115notify you of events. Whether you need to set this depends on the sub-device.
 116Any notifications a sub-device supports must be defined in a header in
 117include/media/<subdevice>.h.
 118
 119You unregister with:
 120
 121        v4l2_device_unregister(struct v4l2_device *v4l2_dev);
 122
 123If the dev->driver_data field points to v4l2_dev, it will be reset to NULL.
 124Unregistering will also automatically unregister all subdevs from the device.
 125
 126If you have a hotpluggable device (e.g. a USB device), then when a disconnect
 127happens the parent device becomes invalid. Since v4l2_device has a pointer to
 128that parent device it has to be cleared as well to mark that the parent is
 129gone. To do this call:
 130
 131        v4l2_device_disconnect(struct v4l2_device *v4l2_dev);
 132
 133This does *not* unregister the subdevs, so you still need to call the
 134v4l2_device_unregister() function for that. If your driver is not hotpluggable,
 135then there is no need to call v4l2_device_disconnect().
 136
 137Sometimes you need to iterate over all devices registered by a specific
 138driver. This is usually the case if multiple device drivers use the same
 139hardware. E.g. the ivtvfb driver is a framebuffer driver that uses the ivtv
 140hardware. The same is true for alsa drivers for example.
 141
 142You can iterate over all registered devices as follows:
 143
 144static int callback(struct device *dev, void *p)
 145{
 146        struct v4l2_device *v4l2_dev = dev_get_drvdata(dev);
 147
 148        /* test if this device was inited */
 149        if (v4l2_dev == NULL)
 150                return 0;
 151        ...
 152        return 0;
 153}
 154
 155int iterate(void *p)
 156{
 157        struct device_driver *drv;
 158        int err;
 159
 160        /* Find driver 'ivtv' on the PCI bus.
 161           pci_bus_type is a global. For USB busses use usb_bus_type. */
 162        drv = driver_find("ivtv", &pci_bus_type);
 163        /* iterate over all ivtv device instances */
 164        err = driver_for_each_device(drv, NULL, p, callback);
 165        put_driver(drv);
 166        return err;
 167}
 168
 169Sometimes you need to keep a running counter of the device instance. This is
 170commonly used to map a device instance to an index of a module option array.
 171
 172The recommended approach is as follows:
 173
 174static atomic_t drv_instance = ATOMIC_INIT(0);
 175
 176static int drv_probe(struct pci_dev *pdev, const struct pci_device_id *pci_id)
 177{
 178        ...
 179        state->instance = atomic_inc_return(&drv_instance) - 1;
 180}
 181
 182If you have multiple device nodes then it can be difficult to know when it is
 183safe to unregister v4l2_device for hotpluggable devices. For this purpose
 184v4l2_device has refcounting support. The refcount is increased whenever
 185video_register_device is called and it is decreased whenever that device node
 186is released. When the refcount reaches zero, then the v4l2_device release()
 187callback is called. You can do your final cleanup there.
 188
 189If other device nodes (e.g. ALSA) are created, then you can increase and
 190decrease the refcount manually as well by calling:
 191
 192void v4l2_device_get(struct v4l2_device *v4l2_dev);
 193
 194or:
 195
 196int v4l2_device_put(struct v4l2_device *v4l2_dev);
 197
 198Since the initial refcount is 1 you also need to call v4l2_device_put in the
 199disconnect() callback (for USB devices) or in the remove() callback (for e.g.
 200PCI devices), otherwise the refcount will never reach 0.
 201
 202struct v4l2_subdev
 203------------------
 204
 205Many drivers need to communicate with sub-devices. These devices can do all
 206sort of tasks, but most commonly they handle audio and/or video muxing,
 207encoding or decoding. For webcams common sub-devices are sensors and camera
 208controllers.
 209
 210Usually these are I2C devices, but not necessarily. In order to provide the
 211driver with a consistent interface to these sub-devices the v4l2_subdev struct
 212(v4l2-subdev.h) was created.
 213
 214Each sub-device driver must have a v4l2_subdev struct. This struct can be
 215stand-alone for simple sub-devices or it might be embedded in a larger struct
 216if more state information needs to be stored. Usually there is a low-level
 217device struct (e.g. i2c_client) that contains the device data as setup
 218by the kernel. It is recommended to store that pointer in the private
 219data of v4l2_subdev using v4l2_set_subdevdata(). That makes it easy to go
 220from a v4l2_subdev to the actual low-level bus-specific device data.
 221
 222You also need a way to go from the low-level struct to v4l2_subdev. For the
 223common i2c_client struct the i2c_set_clientdata() call is used to store a
 224v4l2_subdev pointer, for other busses you may have to use other methods.
 225
 226Bridges might also need to store per-subdev private data, such as a pointer to
 227bridge-specific per-subdev private data. The v4l2_subdev structure provides
 228host private data for that purpose that can be accessed with
 229v4l2_get_subdev_hostdata() and v4l2_set_subdev_hostdata().
 230
 231From the bridge driver perspective you load the sub-device module and somehow
 232obtain the v4l2_subdev pointer. For i2c devices this is easy: you call
 233i2c_get_clientdata(). For other busses something similar needs to be done.
 234Helper functions exists for sub-devices on an I2C bus that do most of this
 235tricky work for you.
 236
 237Each v4l2_subdev contains function pointers that sub-device drivers can
 238implement (or leave NULL if it is not applicable). Since sub-devices can do
 239so many different things and you do not want to end up with a huge ops struct
 240of which only a handful of ops are commonly implemented, the function pointers
 241are sorted according to category and each category has its own ops struct.
 242
 243The top-level ops struct contains pointers to the category ops structs, which
 244may be NULL if the subdev driver does not support anything from that category.
 245
 246It looks like this:
 247
 248struct v4l2_subdev_core_ops {
 249        int (*log_status)(struct v4l2_subdev *sd);
 250        int (*init)(struct v4l2_subdev *sd, u32 val);
 251        ...
 252};
 253
 254struct v4l2_subdev_tuner_ops {
 255        ...
 256};
 257
 258struct v4l2_subdev_audio_ops {
 259        ...
 260};
 261
 262struct v4l2_subdev_video_ops {
 263        ...
 264};
 265
 266struct v4l2_subdev_pad_ops {
 267        ...
 268};
 269
 270struct v4l2_subdev_ops {
 271        const struct v4l2_subdev_core_ops  *core;
 272        const struct v4l2_subdev_tuner_ops *tuner;
 273        const struct v4l2_subdev_audio_ops *audio;
 274        const struct v4l2_subdev_video_ops *video;
 275        const struct v4l2_subdev_pad_ops *video;
 276};
 277
 278The core ops are common to all subdevs, the other categories are implemented
 279depending on the sub-device. E.g. a video device is unlikely to support the
 280audio ops and vice versa.
 281
 282This setup limits the number of function pointers while still making it easy
 283to add new ops and categories.
 284
 285A sub-device driver initializes the v4l2_subdev struct using:
 286
 287        v4l2_subdev_init(sd, &ops);
 288
 289Afterwards you need to initialize subdev->name with a unique name and set the
 290module owner. This is done for you if you use the i2c helper functions.
 291
 292If integration with the media framework is needed, you must initialize the
 293media_entity struct embedded in the v4l2_subdev struct (entity field) by
 294calling media_entity_init():
 295
 296        struct media_pad *pads = &my_sd->pads;
 297        int err;
 298
 299        err = media_entity_init(&sd->entity, npads, pads, 0);
 300
 301The pads array must have been previously initialized. There is no need to
 302manually set the struct media_entity type and name fields, but the revision
 303field must be initialized if needed.
 304
 305A reference to the entity will be automatically acquired/released when the
 306subdev device node (if any) is opened/closed.
 307
 308Don't forget to cleanup the media entity before the sub-device is destroyed:
 309
 310        media_entity_cleanup(&sd->entity);
 311
 312If the subdev driver intends to process video and integrate with the media
 313framework, it must implement format related functionality using
 314v4l2_subdev_pad_ops instead of v4l2_subdev_video_ops.
 315
 316In that case, the subdev driver may set the link_validate field to provide
 317its own link validation function. The link validation function is called for
 318every link in the pipeline where both of the ends of the links are V4L2
 319sub-devices. The driver is still responsible for validating the correctness
 320of the format configuration between sub-devices and video nodes.
 321
 322If link_validate op is not set, the default function
 323v4l2_subdev_link_validate_default() is used instead. This function ensures
 324that width, height and the media bus pixel code are equal on both source and
 325sink of the link. Subdev drivers are also free to use this function to
 326perform the checks mentioned above in addition to their own checks.
 327
 328There are currently two ways to register subdevices with the V4L2 core. The
 329first (traditional) possibility is to have subdevices registered by bridge
 330drivers. This can be done when the bridge driver has the complete information
 331about subdevices connected to it and knows exactly when to register them. This
 332is typically the case for internal subdevices, like video data processing units
 333within SoCs or complex PCI(e) boards, camera sensors in USB cameras or connected
 334to SoCs, which pass information about them to bridge drivers, usually in their
 335platform data.
 336
 337There are however also situations where subdevices have to be registered
 338asynchronously to bridge devices. An example of such a configuration is a Device
 339Tree based system where information about subdevices is made available to the
 340system independently from the bridge devices, e.g. when subdevices are defined
 341in DT as I2C device nodes. The API used in this second case is described further
 342below.
 343
 344Using one or the other registration method only affects the probing process, the
 345run-time bridge-subdevice interaction is in both cases the same.
 346
 347In the synchronous case a device (bridge) driver needs to register the
 348v4l2_subdev with the v4l2_device:
 349
 350        int err = v4l2_device_register_subdev(v4l2_dev, sd);
 351
 352This can fail if the subdev module disappeared before it could be registered.
 353After this function was called successfully the subdev->dev field points to
 354the v4l2_device.
 355
 356If the v4l2_device parent device has a non-NULL mdev field, the sub-device
 357entity will be automatically registered with the media device.
 358
 359You can unregister a sub-device using:
 360
 361        v4l2_device_unregister_subdev(sd);
 362
 363Afterwards the subdev module can be unloaded and sd->dev == NULL.
 364
 365You can call an ops function either directly:
 366
 367        err = sd->ops->core->g_std(sd, &norm);
 368
 369but it is better and easier to use this macro:
 370
 371        err = v4l2_subdev_call(sd, core, g_std, &norm);
 372
 373The macro will to the right NULL pointer checks and returns -ENODEV if subdev
 374is NULL, -ENOIOCTLCMD if either subdev->core or subdev->core->g_std is
 375NULL, or the actual result of the subdev->ops->core->g_std ops.
 376
 377It is also possible to call all or a subset of the sub-devices:
 378
 379        v4l2_device_call_all(v4l2_dev, 0, core, g_std, &norm);
 380
 381Any subdev that does not support this ops is skipped and error results are
 382ignored. If you want to check for errors use this:
 383
 384        err = v4l2_device_call_until_err(v4l2_dev, 0, core, g_std, &norm);
 385
 386Any error except -ENOIOCTLCMD will exit the loop with that error. If no
 387errors (except -ENOIOCTLCMD) occurred, then 0 is returned.
 388
 389The second argument to both calls is a group ID. If 0, then all subdevs are
 390called. If non-zero, then only those whose group ID match that value will
 391be called. Before a bridge driver registers a subdev it can set sd->grp_id
 392to whatever value it wants (it's 0 by default). This value is owned by the
 393bridge driver and the sub-device driver will never modify or use it.
 394
 395The group ID gives the bridge driver more control how callbacks are called.
 396For example, there may be multiple audio chips on a board, each capable of
 397changing the volume. But usually only one will actually be used when the
 398user want to change the volume. You can set the group ID for that subdev to
 399e.g. AUDIO_CONTROLLER and specify that as the group ID value when calling
 400v4l2_device_call_all(). That ensures that it will only go to the subdev
 401that needs it.
 402
 403If the sub-device needs to notify its v4l2_device parent of an event, then
 404it can call v4l2_subdev_notify(sd, notification, arg). This macro checks
 405whether there is a notify() callback defined and returns -ENODEV if not.
 406Otherwise the result of the notify() call is returned.
 407
 408The advantage of using v4l2_subdev is that it is a generic struct and does
 409not contain any knowledge about the underlying hardware. So a driver might
 410contain several subdevs that use an I2C bus, but also a subdev that is
 411controlled through GPIO pins. This distinction is only relevant when setting
 412up the device, but once the subdev is registered it is completely transparent.
 413
 414
 415In the asynchronous case subdevice probing can be invoked independently of the
 416bridge driver availability. The subdevice driver then has to verify whether all
 417the requirements for a successful probing are satisfied. This can include a
 418check for a master clock availability. If any of the conditions aren't satisfied
 419the driver might decide to return -EPROBE_DEFER to request further reprobing
 420attempts. Once all conditions are met the subdevice shall be registered using
 421the v4l2_async_register_subdev() function. Unregistration is performed using
 422the v4l2_async_unregister_subdev() call. Subdevices registered this way are
 423stored in a global list of subdevices, ready to be picked up by bridge drivers.
 424
 425Bridge drivers in turn have to register a notifier object with an array of
 426subdevice descriptors that the bridge device needs for its operation. This is
 427performed using the v4l2_async_notifier_register() call. To unregister the
 428notifier the driver has to call v4l2_async_notifier_unregister(). The former of
 429the two functions takes two arguments: a pointer to struct v4l2_device and a
 430pointer to struct v4l2_async_notifier. The latter contains a pointer to an array
 431of pointers to subdevice descriptors of type struct v4l2_async_subdev type. The
 432V4L2 core will then use these descriptors to match asynchronously registered
 433subdevices to them. If a match is detected the .bound() notifier callback is
 434called. After all subdevices have been located the .complete() callback is
 435called. When a subdevice is removed from the system the .unbind() method is
 436called. All three callbacks are optional.
 437
 438
 439V4L2 sub-device userspace API
 440-----------------------------
 441
 442Beside exposing a kernel API through the v4l2_subdev_ops structure, V4L2
 443sub-devices can also be controlled directly by userspace applications.
 444
 445Device nodes named v4l-subdevX can be created in /dev to access sub-devices
 446directly. If a sub-device supports direct userspace configuration it must set
 447the V4L2_SUBDEV_FL_HAS_DEVNODE flag before being registered.
 448
 449After registering sub-devices, the v4l2_device driver can create device nodes
 450for all registered sub-devices marked with V4L2_SUBDEV_FL_HAS_DEVNODE by calling
 451v4l2_device_register_subdev_nodes(). Those device nodes will be automatically
 452removed when sub-devices are unregistered.
 453
 454The device node handles a subset of the V4L2 API.
 455
 456VIDIOC_QUERYCTRL
 457VIDIOC_QUERYMENU
 458VIDIOC_G_CTRL
 459VIDIOC_S_CTRL
 460VIDIOC_G_EXT_CTRLS
 461VIDIOC_S_EXT_CTRLS
 462VIDIOC_TRY_EXT_CTRLS
 463
 464        The controls ioctls are identical to the ones defined in V4L2. They
 465        behave identically, with the only exception that they deal only with
 466        controls implemented in the sub-device. Depending on the driver, those
 467        controls can be also be accessed through one (or several) V4L2 device
 468        nodes.
 469
 470VIDIOC_DQEVENT
 471VIDIOC_SUBSCRIBE_EVENT
 472VIDIOC_UNSUBSCRIBE_EVENT
 473
 474        The events ioctls are identical to the ones defined in V4L2. They
 475        behave identically, with the only exception that they deal only with
 476        events generated by the sub-device. Depending on the driver, those
 477        events can also be reported by one (or several) V4L2 device nodes.
 478
 479        Sub-device drivers that want to use events need to set the
 480        V4L2_SUBDEV_USES_EVENTS v4l2_subdev::flags and initialize
 481        v4l2_subdev::nevents to events queue depth before registering the
 482        sub-device. After registration events can be queued as usual on the
 483        v4l2_subdev::devnode device node.
 484
 485        To properly support events, the poll() file operation is also
 486        implemented.
 487
 488Private ioctls
 489
 490        All ioctls not in the above list are passed directly to the sub-device
 491        driver through the core::ioctl operation.
 492
 493
 494I2C sub-device drivers
 495----------------------
 496
 497Since these drivers are so common, special helper functions are available to
 498ease the use of these drivers (v4l2-common.h).
 499
 500The recommended method of adding v4l2_subdev support to an I2C driver is to
 501embed the v4l2_subdev struct into the state struct that is created for each
 502I2C device instance. Very simple devices have no state struct and in that case
 503you can just create a v4l2_subdev directly.
 504
 505A typical state struct would look like this (where 'chipname' is replaced by
 506the name of the chip):
 507
 508struct chipname_state {
 509        struct v4l2_subdev sd;
 510        ...  /* additional state fields */
 511};
 512
 513Initialize the v4l2_subdev struct as follows:
 514
 515        v4l2_i2c_subdev_init(&state->sd, client, subdev_ops);
 516
 517This function will fill in all the fields of v4l2_subdev and ensure that the
 518v4l2_subdev and i2c_client both point to one another.
 519
 520You should also add a helper inline function to go from a v4l2_subdev pointer
 521to a chipname_state struct:
 522
 523static inline struct chipname_state *to_state(struct v4l2_subdev *sd)
 524{
 525        return container_of(sd, struct chipname_state, sd);
 526}
 527
 528Use this to go from the v4l2_subdev struct to the i2c_client struct:
 529
 530        struct i2c_client *client = v4l2_get_subdevdata(sd);
 531
 532And this to go from an i2c_client to a v4l2_subdev struct:
 533
 534        struct v4l2_subdev *sd = i2c_get_clientdata(client);
 535
 536Make sure to call v4l2_device_unregister_subdev(sd) when the remove() callback
 537is called. This will unregister the sub-device from the bridge driver. It is
 538safe to call this even if the sub-device was never registered.
 539
 540You need to do this because when the bridge driver destroys the i2c adapter
 541the remove() callbacks are called of the i2c devices on that adapter.
 542After that the corresponding v4l2_subdev structures are invalid, so they
 543have to be unregistered first. Calling v4l2_device_unregister_subdev(sd)
 544from the remove() callback ensures that this is always done correctly.
 545
 546
 547The bridge driver also has some helper functions it can use:
 548
 549struct v4l2_subdev *sd = v4l2_i2c_new_subdev(v4l2_dev, adapter,
 550               "module_foo", "chipid", 0x36, NULL);
 551
 552This loads the given module (can be NULL if no module needs to be loaded) and
 553calls i2c_new_device() with the given i2c_adapter and chip/address arguments.
 554If all goes well, then it registers the subdev with the v4l2_device.
 555
 556You can also use the last argument of v4l2_i2c_new_subdev() to pass an array
 557of possible I2C addresses that it should probe. These probe addresses are
 558only used if the previous argument is 0. A non-zero argument means that you
 559know the exact i2c address so in that case no probing will take place.
 560
 561Both functions return NULL if something went wrong.
 562
 563Note that the chipid you pass to v4l2_i2c_new_subdev() is usually
 564the same as the module name. It allows you to specify a chip variant, e.g.
 565"saa7114" or "saa7115". In general though the i2c driver autodetects this.
 566The use of chipid is something that needs to be looked at more closely at a
 567later date. It differs between i2c drivers and as such can be confusing.
 568To see which chip variants are supported you can look in the i2c driver code
 569for the i2c_device_id table. This lists all the possibilities.
 570
 571There are two more helper functions:
 572
 573v4l2_i2c_new_subdev_cfg: this function adds new irq and platform_data
 574arguments and has both 'addr' and 'probed_addrs' arguments: if addr is not
 5750 then that will be used (non-probing variant), otherwise the probed_addrs
 576are probed.
 577
 578For example: this will probe for address 0x10:
 579
 580struct v4l2_subdev *sd = v4l2_i2c_new_subdev_cfg(v4l2_dev, adapter,
 581               "module_foo", "chipid", 0, NULL, 0, I2C_ADDRS(0x10));
 582
 583v4l2_i2c_new_subdev_board uses an i2c_board_info struct which is passed
 584to the i2c driver and replaces the irq, platform_data and addr arguments.
 585
 586If the subdev supports the s_config core ops, then that op is called with
 587the irq and platform_data arguments after the subdev was setup. The older
 588v4l2_i2c_new_(probed_)subdev functions will call s_config as well, but with
 589irq set to 0 and platform_data set to NULL.
 590
 591struct video_device
 592-------------------
 593
 594The actual device nodes in the /dev directory are created using the
 595video_device struct (v4l2-dev.h). This struct can either be allocated
 596dynamically or embedded in a larger struct.
 597
 598To allocate it dynamically use:
 599
 600        struct video_device *vdev = video_device_alloc();
 601
 602        if (vdev == NULL)
 603                return -ENOMEM;
 604
 605        vdev->release = video_device_release;
 606
 607If you embed it in a larger struct, then you must set the release()
 608callback to your own function:
 609
 610        struct video_device *vdev = &my_vdev->vdev;
 611
 612        vdev->release = my_vdev_release;
 613
 614The release callback must be set and it is called when the last user
 615of the video device exits.
 616
 617The default video_device_release() callback just calls kfree to free the
 618allocated memory.
 619
 620There is also a video_device_release_empty() function that does nothing
 621(is empty) and can be used if the struct is embedded and there is nothing
 622to do when it is released.
 623
 624You should also set these fields:
 625
 626- v4l2_dev: must be set to the v4l2_device parent device.
 627
 628- name: set to something descriptive and unique.
 629
 630- vfl_dir: set this to VFL_DIR_RX for capture devices (VFL_DIR_RX has value 0,
 631  so this is normally already the default), set to VFL_DIR_TX for output
 632  devices and VFL_DIR_M2M for mem2mem (codec) devices.
 633
 634- fops: set to the v4l2_file_operations struct.
 635
 636- ioctl_ops: if you use the v4l2_ioctl_ops to simplify ioctl maintenance
 637  (highly recommended to use this and it might become compulsory in the
 638  future!), then set this to your v4l2_ioctl_ops struct. The vfl_type and
 639  vfl_dir fields are used to disable ops that do not match the type/dir
 640  combination. E.g. VBI ops are disabled for non-VBI nodes, and output ops
 641  are disabled for a capture device. This makes it possible to provide
 642  just one v4l2_ioctl_ops struct for both vbi and video nodes.
 643
 644- lock: leave to NULL if you want to do all the locking in the driver.
 645  Otherwise you give it a pointer to a struct mutex_lock and before the
 646  unlocked_ioctl file operation is called this lock will be taken by the
 647  core and released afterwards. See the next section for more details.
 648
 649- queue: a pointer to the struct vb2_queue associated with this device node.
 650  If queue is non-NULL, and queue->lock is non-NULL, then queue->lock is
 651  used for the queuing ioctls (VIDIOC_REQBUFS, CREATE_BUFS, QBUF, DQBUF,
 652  QUERYBUF, PREPARE_BUF, STREAMON and STREAMOFF) instead of the lock above.
 653  That way the vb2 queuing framework does not have to wait for other ioctls.
 654  This queue pointer is also used by the vb2 helper functions to check for
 655  queuing ownership (i.e. is the filehandle calling it allowed to do the
 656  operation).
 657
 658- prio: keeps track of the priorities. Used to implement VIDIOC_G/S_PRIORITY.
 659  If left to NULL, then it will use the struct v4l2_prio_state in v4l2_device.
 660  If you want to have a separate priority state per (group of) device node(s),
 661  then you can point it to your own struct v4l2_prio_state.
 662
 663- dev_parent: you only set this if v4l2_device was registered with NULL as
 664  the parent device struct. This only happens in cases where one hardware
 665  device has multiple PCI devices that all share the same v4l2_device core.
 666
 667  The cx88 driver is an example of this: one core v4l2_device struct, but
 668  it is used by both a raw video PCI device (cx8800) and a MPEG PCI device
 669  (cx8802). Since the v4l2_device cannot be associated with two PCI devices
 670  at the same time it is setup without a parent device. But when the struct
 671  video_device is initialized you *do* know which parent PCI device to use and
 672  so you set dev_device to the correct PCI device.
 673
 674- flags: optional. Set to V4L2_FL_USE_FH_PRIO if you want to let the framework
 675  handle the VIDIOC_G/S_PRIORITY ioctls. This requires that you use struct
 676  v4l2_fh. Eventually this flag will disappear once all drivers use the core
 677  priority handling. But for now it has to be set explicitly.
 678
 679If you use v4l2_ioctl_ops, then you should set .unlocked_ioctl to video_ioctl2
 680in your v4l2_file_operations struct.
 681
 682Do not use .ioctl! This is deprecated and will go away in the future.
 683
 684In some cases you want to tell the core that a function you had specified in
 685your v4l2_ioctl_ops should be ignored. You can mark such ioctls by calling this
 686function before video_device_register is called:
 687
 688void v4l2_disable_ioctl(struct video_device *vdev, unsigned int cmd);
 689
 690This tends to be needed if based on external factors (e.g. which card is
 691being used) you want to turns off certain features in v4l2_ioctl_ops without
 692having to make a new struct.
 693
 694The v4l2_file_operations struct is a subset of file_operations. The main
 695difference is that the inode argument is omitted since it is never used.
 696
 697If integration with the media framework is needed, you must initialize the
 698media_entity struct embedded in the video_device struct (entity field) by
 699calling media_entity_init():
 700
 701        struct media_pad *pad = &my_vdev->pad;
 702        int err;
 703
 704        err = media_entity_init(&vdev->entity, 1, pad, 0);
 705
 706The pads array must have been previously initialized. There is no need to
 707manually set the struct media_entity type and name fields.
 708
 709A reference to the entity will be automatically acquired/released when the
 710video device is opened/closed.
 711
 712ioctls and locking
 713------------------
 714
 715The V4L core provides optional locking services. The main service is the
 716lock field in struct video_device, which is a pointer to a mutex. If you set
 717this pointer, then that will be used by unlocked_ioctl to serialize all ioctls.
 718
 719If you are using the videobuf2 framework, then there is a second lock that you
 720can set: video_device->queue->lock. If set, then this lock will be used instead
 721of video_device->lock to serialize all queuing ioctls (see the previous section
 722for the full list of those ioctls).
 723
 724The advantage of using a different lock for the queuing ioctls is that for some
 725drivers (particularly USB drivers) certain commands such as setting controls
 726can take a long time, so you want to use a separate lock for the buffer queuing
 727ioctls. That way your VIDIOC_DQBUF doesn't stall because the driver is busy
 728changing the e.g. exposure of the webcam.
 729
 730Of course, you can always do all the locking yourself by leaving both lock
 731pointers at NULL.
 732
 733If you use the old videobuf then you must pass the video_device lock to the
 734videobuf queue initialize function: if videobuf has to wait for a frame to
 735arrive, then it will temporarily unlock the lock and relock it afterwards. If
 736your driver also waits in the code, then you should do the same to allow other
 737processes to access the device node while the first process is waiting for
 738something.
 739
 740In the case of videobuf2 you will need to implement the wait_prepare and
 741wait_finish callbacks to unlock/lock if applicable. If you use the queue->lock
 742pointer, then you can use the helper functions vb2_ops_wait_prepare/finish.
 743
 744The implementation of a hotplug disconnect should also take the lock from
 745video_device before calling v4l2_device_disconnect. If you are also using
 746video_device->queue->lock, then you have to first lock video_device->queue->lock
 747followed by video_device->lock. That way you can be sure no ioctl is running
 748when you call v4l2_device_disconnect.
 749
 750video_device registration
 751-------------------------
 752
 753Next you register the video device: this will create the character device
 754for you.
 755
 756        err = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
 757        if (err) {
 758                video_device_release(vdev); /* or kfree(my_vdev); */
 759                return err;
 760        }
 761
 762If the v4l2_device parent device has a non-NULL mdev field, the video device
 763entity will be automatically registered with the media device.
 764
 765Which device is registered depends on the type argument. The following
 766types exist:
 767
 768VFL_TYPE_GRABBER: videoX for video input/output devices
 769VFL_TYPE_VBI: vbiX for vertical blank data (i.e. closed captions, teletext)
 770VFL_TYPE_RADIO: radioX for radio tuners
 771
 772The last argument gives you a certain amount of control over the device
 773device node number used (i.e. the X in videoX). Normally you will pass -1
 774to let the v4l2 framework pick the first free number. But sometimes users
 775want to select a specific node number. It is common that drivers allow
 776the user to select a specific device node number through a driver module
 777option. That number is then passed to this function and video_register_device
 778will attempt to select that device node number. If that number was already
 779in use, then the next free device node number will be selected and it
 780will send a warning to the kernel log.
 781
 782Another use-case is if a driver creates many devices. In that case it can
 783be useful to place different video devices in separate ranges. For example,
 784video capture devices start at 0, video output devices start at 16.
 785So you can use the last argument to specify a minimum device node number
 786and the v4l2 framework will try to pick the first free number that is equal
 787or higher to what you passed. If that fails, then it will just pick the
 788first free number.
 789
 790Since in this case you do not care about a warning about not being able
 791to select the specified device node number, you can call the function
 792video_register_device_no_warn() instead.
 793
 794Whenever a device node is created some attributes are also created for you.
 795If you look in /sys/class/video4linux you see the devices. Go into e.g.
 796video0 and you will see 'name' and 'index' attributes. The 'name' attribute
 797is the 'name' field of the video_device struct.
 798
 799The 'index' attribute is the index of the device node: for each call to
 800video_register_device() the index is just increased by 1. The first video
 801device node you register always starts with index 0.
 802
 803Users can setup udev rules that utilize the index attribute to make fancy
 804device names (e.g. 'mpegX' for MPEG video capture device nodes).
 805
 806After the device was successfully registered, then you can use these fields:
 807
 808- vfl_type: the device type passed to video_register_device.
 809- minor: the assigned device minor number.
 810- num: the device node number (i.e. the X in videoX).
 811- index: the device index number.
 812
 813If the registration failed, then you need to call video_device_release()
 814to free the allocated video_device struct, or free your own struct if the
 815video_device was embedded in it. The vdev->release() callback will never
 816be called if the registration failed, nor should you ever attempt to
 817unregister the device if the registration failed.
 818
 819
 820video_device cleanup
 821--------------------
 822
 823When the video device nodes have to be removed, either during the unload
 824of the driver or because the USB device was disconnected, then you should
 825unregister them:
 826
 827        video_unregister_device(vdev);
 828
 829This will remove the device nodes from sysfs (causing udev to remove them
 830from /dev).
 831
 832After video_unregister_device() returns no new opens can be done. However,
 833in the case of USB devices some application might still have one of these
 834device nodes open. So after the unregister all file operations (except
 835release, of course) will return an error as well.
 836
 837When the last user of the video device node exits, then the vdev->release()
 838callback is called and you can do the final cleanup there.
 839
 840Don't forget to cleanup the media entity associated with the video device if
 841it has been initialized:
 842
 843        media_entity_cleanup(&vdev->entity);
 844
 845This can be done from the release callback.
 846
 847
 848video_device helper functions
 849-----------------------------
 850
 851There are a few useful helper functions:
 852
 853- file/video_device private data
 854
 855You can set/get driver private data in the video_device struct using:
 856
 857void *video_get_drvdata(struct video_device *vdev);
 858void video_set_drvdata(struct video_device *vdev, void *data);
 859
 860Note that you can safely call video_set_drvdata() before calling
 861video_register_device().
 862
 863And this function:
 864
 865struct video_device *video_devdata(struct file *file);
 866
 867returns the video_device belonging to the file struct.
 868
 869The video_drvdata function combines video_get_drvdata with video_devdata:
 870
 871void *video_drvdata(struct file *file);
 872
 873You can go from a video_device struct to the v4l2_device struct using:
 874
 875struct v4l2_device *v4l2_dev = vdev->v4l2_dev;
 876
 877- Device node name
 878
 879The video_device node kernel name can be retrieved using
 880
 881const char *video_device_node_name(struct video_device *vdev);
 882
 883The name is used as a hint by userspace tools such as udev. The function
 884should be used where possible instead of accessing the video_device::num and
 885video_device::minor fields.
 886
 887
 888video buffer helper functions
 889-----------------------------
 890
 891The v4l2 core API provides a set of standard methods (called "videobuf")
 892for dealing with video buffers. Those methods allow a driver to implement
 893read(), mmap() and overlay() in a consistent way.  There are currently
 894methods for using video buffers on devices that supports DMA with
 895scatter/gather method (videobuf-dma-sg), DMA with linear access
 896(videobuf-dma-contig), and vmalloced buffers, mostly used on USB drivers
 897(videobuf-vmalloc).
 898
 899Please see Documentation/video4linux/videobuf for more information on how
 900to use the videobuf layer.
 901
 902struct v4l2_fh
 903--------------
 904
 905struct v4l2_fh provides a way to easily keep file handle specific data
 906that is used by the V4L2 framework. New drivers must use struct v4l2_fh
 907since it is also used to implement priority handling (VIDIOC_G/S_PRIORITY)
 908if the video_device flag V4L2_FL_USE_FH_PRIO is also set.
 909
 910The users of v4l2_fh (in the V4L2 framework, not the driver) know
 911whether a driver uses v4l2_fh as its file->private_data pointer by
 912testing the V4L2_FL_USES_V4L2_FH bit in video_device->flags. This bit is
 913set whenever v4l2_fh_init() is called.
 914
 915struct v4l2_fh is allocated as a part of the driver's own file handle
 916structure and file->private_data is set to it in the driver's open
 917function by the driver.
 918
 919In many cases the struct v4l2_fh will be embedded in a larger structure.
 920In that case you should call v4l2_fh_init+v4l2_fh_add in open() and
 921v4l2_fh_del+v4l2_fh_exit in release().
 922
 923Drivers can extract their own file handle structure by using the container_of
 924macro. Example:
 925
 926struct my_fh {
 927        int blah;
 928        struct v4l2_fh fh;
 929};
 930
 931...
 932
 933int my_open(struct file *file)
 934{
 935        struct my_fh *my_fh;
 936        struct video_device *vfd;
 937        int ret;
 938
 939        ...
 940
 941        my_fh = kzalloc(sizeof(*my_fh), GFP_KERNEL);
 942
 943        ...
 944
 945        v4l2_fh_init(&my_fh->fh, vfd);
 946
 947        ...
 948
 949        file->private_data = &my_fh->fh;
 950        v4l2_fh_add(&my_fh->fh);
 951        return 0;
 952}
 953
 954int my_release(struct file *file)
 955{
 956        struct v4l2_fh *fh = file->private_data;
 957        struct my_fh *my_fh = container_of(fh, struct my_fh, fh);
 958
 959        ...
 960        v4l2_fh_del(&my_fh->fh);
 961        v4l2_fh_exit(&my_fh->fh);
 962        kfree(my_fh);
 963        return 0;
 964}
 965
 966Below is a short description of the v4l2_fh functions used:
 967
 968void v4l2_fh_init(struct v4l2_fh *fh, struct video_device *vdev)
 969
 970  Initialise the file handle. This *MUST* be performed in the driver's
 971  v4l2_file_operations->open() handler.
 972
 973void v4l2_fh_add(struct v4l2_fh *fh)
 974
 975  Add a v4l2_fh to video_device file handle list. Must be called once the
 976  file handle is completely initialized.
 977
 978void v4l2_fh_del(struct v4l2_fh *fh)
 979
 980  Unassociate the file handle from video_device(). The file handle
 981  exit function may now be called.
 982
 983void v4l2_fh_exit(struct v4l2_fh *fh)
 984
 985  Uninitialise the file handle. After uninitialisation the v4l2_fh
 986  memory can be freed.
 987
 988
 989If struct v4l2_fh is not embedded, then you can use these helper functions:
 990
 991int v4l2_fh_open(struct file *filp)
 992
 993  This allocates a struct v4l2_fh, initializes it and adds it to the struct
 994  video_device associated with the file struct.
 995
 996int v4l2_fh_release(struct file *filp)
 997
 998  This deletes it from the struct video_device associated with the file
 999  struct, uninitialised the v4l2_fh and frees it.
1000
1001These two functions can be plugged into the v4l2_file_operation's open() and
1002release() ops.
1003
1004
1005Several drivers need to do something when the first file handle is opened and
1006when the last file handle closes. Two helper functions were added to check
1007whether the v4l2_fh struct is the only open filehandle of the associated
1008device node:
1009
1010int v4l2_fh_is_singular(struct v4l2_fh *fh)
1011
1012  Returns 1 if the file handle is the only open file handle, else 0.
1013
1014int v4l2_fh_is_singular_file(struct file *filp)
1015
1016  Same, but it calls v4l2_fh_is_singular with filp->private_data.
1017
1018
1019V4L2 events
1020-----------
1021
1022The V4L2 events provide a generic way to pass events to user space.
1023The driver must use v4l2_fh to be able to support V4L2 events.
1024
1025Events are defined by a type and an optional ID. The ID may refer to a V4L2
1026object such as a control ID. If unused, then the ID is 0.
1027
1028When the user subscribes to an event the driver will allocate a number of
1029kevent structs for that event. So every (type, ID) event tuple will have
1030its own set of kevent structs. This guarantees that if a driver is generating
1031lots of events of one type in a short time, then that will not overwrite
1032events of another type.
1033
1034But if you get more events of one type than the number of kevents that were
1035reserved, then the oldest event will be dropped and the new one added.
1036
1037Furthermore, the internal struct v4l2_subscribed_event has merge() and
1038replace() callbacks which drivers can set. These callbacks are called when
1039a new event is raised and there is no more room. The replace() callback
1040allows you to replace the payload of the old event with that of the new event,
1041merging any relevant data from the old payload into the new payload that
1042replaces it. It is called when this event type has only one kevent struct
1043allocated. The merge() callback allows you to merge the oldest event payload
1044into that of the second-oldest event payload. It is called when there are two
1045or more kevent structs allocated.
1046
1047This way no status information is lost, just the intermediate steps leading
1048up to that state.
1049
1050A good example of these replace/merge callbacks is in v4l2-event.c:
1051ctrls_replace() and ctrls_merge() callbacks for the control event.
1052
1053Note: these callbacks can be called from interrupt context, so they must be
1054fast.
1055
1056Useful functions:
1057
1058void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev)
1059
1060  Queue events to video device. The driver's only responsibility is to fill
1061  in the type and the data fields. The other fields will be filled in by
1062  V4L2.
1063
1064int v4l2_event_subscribe(struct v4l2_fh *fh,
1065                         struct v4l2_event_subscription *sub, unsigned elems,
1066                         const struct v4l2_subscribed_event_ops *ops)
1067
1068  The video_device->ioctl_ops->vidioc_subscribe_event must check the driver
1069  is able to produce events with specified event id. Then it calls
1070  v4l2_event_subscribe() to subscribe the event.
1071
1072  The elems argument is the size of the event queue for this event. If it is 0,
1073  then the framework will fill in a default value (this depends on the event
1074  type).
1075
1076  The ops argument allows the driver to specify a number of callbacks:
1077  * add:     called when a new listener gets added (subscribing to the same
1078             event twice will only cause this callback to get called once)
1079  * del:     called when a listener stops listening
1080  * replace: replace event 'old' with event 'new'.
1081  * merge:   merge event 'old' into event 'new'.
1082  All 4 callbacks are optional, if you don't want to specify any callbacks
1083  the ops argument itself maybe NULL.
1084
1085int v4l2_event_unsubscribe(struct v4l2_fh *fh,
1086                           struct v4l2_event_subscription *sub)
1087
1088  vidioc_unsubscribe_event in struct v4l2_ioctl_ops. A driver may use
1089  v4l2_event_unsubscribe() directly unless it wants to be involved in
1090  unsubscription process.
1091
1092  The special type V4L2_EVENT_ALL may be used to unsubscribe all events. The
1093  drivers may want to handle this in a special way.
1094
1095int v4l2_event_pending(struct v4l2_fh *fh)
1096
1097  Returns the number of pending events. Useful when implementing poll.
1098
1099Events are delivered to user space through the poll system call. The driver
1100can use v4l2_fh->wait (a wait_queue_head_t) as the argument for poll_wait().
1101
1102There are standard and private events. New standard events must use the
1103smallest available event type. The drivers must allocate their events from
1104their own class starting from class base. Class base is
1105V4L2_EVENT_PRIVATE_START + n * 1000 where n is the lowest available number.
1106The first event type in the class is reserved for future use, so the first
1107available event type is 'class base + 1'.
1108
1109An example on how the V4L2 events may be used can be found in the OMAP
11103 ISP driver (drivers/media/platform/omap3isp).
1111
1112
1113V4L2 clocks
1114-----------
1115
1116Many subdevices, like camera sensors, TV decoders and encoders, need a clock
1117signal to be supplied by the system. Often this clock is supplied by the
1118respective bridge device. The Linux kernel provides a Common Clock Framework for
1119this purpose. However, it is not (yet) available on all architectures. Besides,
1120the nature of the multi-functional (clock, data + synchronisation, I2C control)
1121connection of subdevices to the system might impose special requirements on the
1122clock API usage. E.g. V4L2 has to support clock provider driver unregistration
1123while a subdevice driver is holding a reference to the clock. For these reasons
1124a V4L2 clock helper API has been developed and is provided to bridge and
1125subdevice drivers.
1126
1127The API consists of two parts: two functions to register and unregister a V4L2
1128clock source: v4l2_clk_register() and v4l2_clk_unregister() and calls to control
1129a clock object, similar to the respective generic clock API calls:
1130v4l2_clk_get(), v4l2_clk_put(), v4l2_clk_enable(), v4l2_clk_disable(),
1131v4l2_clk_get_rate(), and v4l2_clk_set_rate(). Clock suppliers have to provide
1132clock operations that will be called when clock users invoke respective API
1133methods.
1134
1135It is expected that once the CCF becomes available on all relevant
1136architectures this API will be removed.
1137
lxr.linux.no kindly hosted by Redpill Linpro AS, provider of Linux consulting and operations services since 1995.