Using the Jetson-IO Tool
If your camera module does not have an on-board EEPROM, you can create a DTB overlay file to statically configure the board for the attached camera. When you attach your camera module, apply the camera module DTB overlay using the
Jetson‑IO tool, then reboot. The new module works as soon as Jetson Linux starts.
To create and apply a DTB overlay file
1. Add your .dtsi file to the jetson board .dts file.
2. Set the status of your device tree nodes to “disabled”:
imx185_cam0: imx185_a@1a {
status = "disabled";
};
3. Add the override information to a new .dts file, for example:
<top>/hardware/nvidia/platform/t19x/common/kernel-dts/t19x-common-modules/tegra194-camera-overlay-file.dts
4. Update the .dtbo file with proper override information and a compatible string:
compatible = "nvidia,p3449-0000+p3668-0000", "nvidia,p3449-0000+p3668-0001", "nvidia,p3509-0000+p3668-0000", "nvidia,p3509-0000+p3668-0001";
fragment@0 {
target = <&imx185_cam0>;
__overlay__ {
status = "okay";
};
};
5. Compile the .dts file to generate a .dtbo file. Move the .dtbo file to /boot so that the Jetson‑IO tool can recognize it. Launch the Jetson‑IO tool and configure the DTB overlay.
Using the Main Platform Device Tree File
Register your new device by updating the main platform .dtsi file to include your new device .dtsi file. Because Jetson devices use Plugin Manager by default, you must first unregister Plugin Manager support, then add your device information to the main device tree .dtsi file.
To register a device using the main platform device tree file
1. Locate and edit the .dtsi file:
<top>/hardware/nvidia/platform/t19x/galen/kernel-dts/tegra194-p2888-0001-p2822-0000.dtsi
2. Remove the following line from the file:
#include "tegra194-camera-plugin-manager.dtsi"
3. Replace the following line:
#include "common/tegra194-p2822-camera-modules.dtsi"
With an #include statement specifying the DTSI file for your new device.
Save and close the file.
Note: | Use tegra194-p2822-camera-modules.dtsi as a model for generating your DTSI. In your file, change status = disable to status = okay. |
Verifying the V4L2 Sensor Driver
When the driver is complete, run v4l2-compliance and v4l2-ctl to ensure that it can capture raw data through the V4L2 interface.
For details on RAW memory format, see the “Video Input” topic in:
• For Jetson Nano series and Jetson TX1: Tegra X1 (SoC) Technical Referene Manual
• For Jetson TX2: Tegra X2 (Parker Series SoC) Technical Reference Manual
• For Jetson AGX Xavier series: Jetson AGX Xavier Technical Reference Manual
To run a v4l2-compliance test
• Enter the command:
v4l2-compliance -d /dev/video<n>
Where <n> is the number of the camera that the test is to use.
If your system has one camera, <n> is 0, i.e. the device path is /dev/video0.
If your system has multiple cameras, specify the index of the camera you want to use for the test.
To run a v4l2-ctl test
• Enter the command:
v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --stream-mmap --set-ctrl=sensor_mode=0 --stream-count=100 -d /dev/video<n>
Where <n> is the same as for v4l2-compliance, above.
You can use v4l2-ctl to capture RAW data by specifying the ‑‑stream-count switch for frames to capture and the ‑‑stream-to switch for the output path.
Both v4l2-compliance and v4l2-ctl are available as open source projects. Documentation for these commands is available from LinuxTV at:
Debugging Tips
When the driver is ready for testing, before running any camera applications, check to see if the camera device tree node(s) are populated correctly in the system.
If the driver is populated properly, the name of the camera’s device tree node is displayed:
/dev/video<n>
Where <n> is the index of the device tree node.
Any problems in the probing process require debugging. Typically, problems occur in the clock, GPIO, and regulator setup.
To verify that driver name matches the name in the Device Tree
The device' name must be the same in the device tree and your driver. This is how kernel binds the driver to your device.
In the device tree:
compatible = "nvidia,imx185";
In your driver:
static struct of_device_id imx185_of_match[] = {
{ .compatible = "nvidia,imx185", },
{ },
};
…
.driver = {
.name = "imx185",
.owner = THIS_MODULE,
.of_match_table = of_match_ptr(imx185_of_match),
},
To verify that all device names match the device tree
Verify that the device names read through parse_dt()match those in the device tree.
To verify that the Device tree values match the hardware
Ensure that the values assigned to the device tree fields match the hardware they describe. If regulator names, GPIO numbers, or clock names are out of date in the device tree, probing fails.
Use the device tree compiler (dtc) to convert your .dtb file back to .dtsi format, and examine the content. Ensure that your device tree node’s status field is set to the proper state.
To verify that functions run to completion
Make sure that all functions run to completion. Pay special attention to kernel logs, and look for any errors that could cause a function to exit unexpectedly.
It’s good practice to make your code print an error message at any point where it could fail. A silent failure is harder to catch.
To verify that default values are correctly linked
Verify that default values are all linked correctly in the control configuration. Also, verify that the default macros are updated appropriately with the values in the mode tables of the sensor.
If new controls have been added to the control configuration list, make sure that they contain the appropriate control handlers and default values.
After probing succeeds, problems may still occur with the control setting. A common problem is incorrect values in the register writes.
To verify that control register values are correct
Make sure that the control register setup contains the correct register address and formats according to the mode tables and data sheets. For more information, see
Setting Up Control Registers.
To verify that mode-specific settings are correct
Double check the mode-specific settings in the device tree, and update them if necessary. The most common issues cause sensor timeout due to incorrect values for these settings:
• num_lanes must be 1, 2, or 4, depending on your sensor configuration.
• tegra_sinterface must be set to the CSI port where the sensor is connected.
• discontinuous_clk must specify that the sensor is running in continuous clock mode (free running) or discontinuous clock mode (gated). If unsure, first set this value to no.
• mclk_multiplier must be equal or greater than pix_clk_hz / mclk to ensure that the ISP runs fast enough to process data from sensor.
To verify that I2C accesses are working properly
Examine the kernel logs closely for I2C errors associated with the camera sensor. If there are I2C errors, check the power_on() function to verify that the sensor power-on sequence is correct. The order of enabling power, clock, GPIO, and timing must comply with sensor specifications.
Configuring the Sensor Driver as a Loadable Kernel Module (LKM)
Configuring the sensor driver as a loadable kernel module can speed up development by eliminating the need to reflash the kernel image to the Jetson device. For more information, see
Loadable Kernel Module (LKM).
Mode Tables
The register’s mode tables reside in a separate header file named sensor_mode_tbls.h, and are included by the main driver. Those tables can be a list of reg_8 or reg_16 address-value pairs. Mode tables are separated by resolution.
The start and stop stream-register values must be in separate mode tables. When you separate them, delete the start stream-register values at the end of the resolution mode tables.
static imx185_reg imx185_start[] = {
{0x3000, 0x00 },
{IMX185_TABLE_WAIT_MS, IMX185_WAIT_MS_START},
{0x3002, 0x00},
{0x3049, 0x02},
{IMX185_TABLE_WAIT_MS, IMX185_WAIT_MS_STREAM},
{IMX185_TABLE_END, 0x00 }
};
static imx185_reg imx185_stop[] = {
{0x3000, 0x01 },
{IMX185_TABLE_WAIT_MS, IMX185_WAIT_MS_STOP},
{IMX185_TABLE_END, 0x00 }
};
Also, a table for the color bars of the test pattern is used if the test_mode flag is enabled, activating the test-pattern mode of the sensor. That table is required only if the sensor supports test patterns.
static imx185_reg tp_colorbars[] = {
...
};
At the end of the mode table header file is an enumeration of all the mode tables and a list that maps the enumeration to table pointers.
enum {
IMX185_MODE_1920X1080_CROP_30FPS,
...
};
static oimx185_reg *mode_table[] = {
[IMX185_MODE_1920X1080_CROP_30FPS] = imx185_1920x1080_crop_30fps,
...
};
Note: | The modes list in the table must match the modes listed in sensor device tree .dtsi file. |
The camera_common_frmfmt array sets up the format for the V4L2 framework. Each element in the array contains the mode’s resolutions, is_hdr flag, and enumeration.
static const struct camera_common_frmfmt imx185_frmfmt[] = {
{{1920, 1080}, 0, IMX185_MODE_1920X1080_CROP_30FPS},
...
};
To add a register mode table
1. Obtain the address-value pairs for the mode table you want to add.
Note: | If separate stream-on and stream-off mode tables exist, you may omit them from the mode table. |
2. Format the pairs according to the register structure and initialize a static array with the mode resolution as part of the name. For example:
static imx185_reg mode_####x####[] = {
{ 0x3000, 0x00 },
...
/* your addr and val pairs */
};
The mode table is now in place. End the table array with:
{ IMX185_TABLE_END, 0x00 }
3. Create a new element in the list of enumerations and add it to the array of mode tables:
enum {
...
IMX185_MODE_####X####,
}
static imx185_reg *mode_table[] = {
...
[IMX185_MODE_####X####] = mode_####x####,
};
4. Add the new mode to the camera_common_frmfmt array:
static const struct camera_common_frmfmt IMX185_frmfmt[] = {
...
{{####, ####}, 0, IMX185_MODE_####X####},
};
Register mode table considerations
A mode table can refer to thousands of registers. This increases the delay before the sensor starts streaming by the time required to program all of the registers. In extreme cases, the recipient engine may time out before the sensor outputs the first frame.
There are several possible ways to correct this problem:
• Reduce the number of registers referenced in the mode tables. You may be able to get smaller mode tables from the sensor vendor. Most registers are typically set to default values, and do not need to be updated.
• Arrange the registers in small groups with consecutive register addresses. The NVIDIA kernel driver supports regmap() bulk writes to as many as 16 consecutive registers, which can reduce the number of I2C/SPI transactions. Consult the sensor vendor to be sure you can do this without affecting sensor operations.
• Request a longer delay for the capture engine to prevent timeouts. You can add the device tree property set_mode_delay_ms to the sensor’s .dtsi file to extend the wait time for the first frame.
Camera Sensor Drivers Porting
Jetson Linux Driver Package (L4T) release 31 upgraded the Linux kernel from version 4.4 to 4.9. Sensor drivers developed on L4T Release 28 do not work with the current release. If you have such a driver, you must modify it to work with the release 31 driver interface. This section describes the necessary changes.
How to find differences between release 28 and the current release
• Compare the imx185.c sample driver source file from the release 28 source package to the one from the current release. The sample drivers are at these locations:
For release 28.2/28.2.1:
kernel/kernel-4.4/drivers/media/i2c/imx185.c
For release 31 and later:
kernel/nvidia/drivers/media/i2c/imx185.c
Changes for the V4L2 API
The media_entity_init() function is deprecated in kernel 4.9. NVIDIA provides a new wrapper function, tegra_media_entity_init(), to adapt to the changes. Modify the following code in the release 28 driver to make it compatible with kernel 4.9.
In the driver’s probe() function, replace:
priv->subdev->entity.type = MEDIA_ENT_T_V4L2_SUBDEV_SENSOR;
priv->subdev->entity.ops = &imx185_media_ops;
err = media_entity_init(&priv->subdev->entity, 1, &priv->pad, 0);
With:
priv->subdev->entity.ops = &imx185_media_ops;
err = tegra_media_entity_init(&priv->subdev->entity, 1,
&priv->pad, true, true);
Changes to Device Tree Properties
Release 31 and later support both the I2C and SPI buses, as well as the CPHY and DPHY interfaces. To avoid confusion, make the following changes in the current sensor driver device tree files:
• Rename csi-port to port-index for VI and CSI endpoints.
• Add a new phy_mode property to each sensor mode. The value can be DPHY or CPHY, depending on the sensor type.
Porting version 1.0 drivers to version 2.0
This section describes changes that you must make to port a version 1.0 driver to version 2.0.
Device tree
You must add factor, step, and default value properties to the sensor DTSI file to make gain, exposure, and framerate properties work properly.
NVIDIA recommends that you set default_gain to min_gain_val and default_framerate to max_framerate. For default_exp_time, don’t use min_exp_time or max_exp_time because they would make the image too dark or too bright when the sensor is started. Instead, use the exposure time setting from the mode tables.
Driver
Make the following changes to the driver code. You can also compare source code between version 2.0 and version 1.0 drivers (between imx185.c and imx185_v1.c) to see the differences.
1. Remove all hard coded sensor settings in the sensor driver. The driver should get them from the .dtsi file.
2. In probe(), allocate memory for the new struct tc_dev. Use it in the new tegracam_device_register() and tegracam_subdev_register() functions to set up the sensor context and register the device as a V4L2 subdevice.
3. If the driver must fulfill any special requirements during sensor probing, create a new board_setup() function for probe() to call.
4. Create a new ctrl_id_list[]and list all the controls needed by the sensor.
5. Create a tegracam_ctrl_ops structure that specifies the number of controls and provides a pointer to a handler for each control.
6. Convert all 32‑bit controls to 64‑bit controls.
7. Change the scaling factor used in version 1.0 to a more flexible scaling factor for version 2.0.
8. Be sure to include a group hold control in the version 2.0 driver, as it is now mandatory. If your hardware does not support the group hold feature, create a dummy group hold function.