Dual OV13850 cameras finally land on Firefly-RK3399 in Ubuntu 18.04!

Chip Platform Rockchip
1

Photo first, engineer style with a messy desktop :stuck_out_tongue_winking_eye:

dual-ov13850-cameras-firefly-rk3399-small.jpg
dual-ov13850-cameras-firefly-rk3399-small.jpg1024×768 237 KB

2

UPDATE:* 20190105

  • update to camera-engine-rkisp_1.5.0
  • update to the latest firefly stable kernel with dual cameras patches
  • fix setup script, remove /opt/gstreamer from environment.

For the bravery, I’ve written an installation manual which can be read on:

http://wiki.t-firefly.com/firefly-rk3399-repo/dualcamera/INSTALL.txt

Below is just a copy of it:

# Prerequisite

To install dual cameras experimental support, you need:

  • a Firefly-RK3399 board
  • flashed with firmware Firefly-RK3399-ubuntu18.04_20181017.img
  • two 30-pin OV13850 cameras from the Firefly Shop
  • Internet access from the board

# Installation

WARNING: Please BACKUP your data first. The following procedure will have the
risk of ruining your filesystem and system boot.

Download the script and run:

    wget -O dual-camera-setup.sh http://wiki.t-firefly.com/firefly-rk3399-repo/dualcamera/dual-camera-setup.txt
    sudo bash dual-camera-setup

This script will install the necessary packages and scripts, flash the new kernel.

After completion, please REBOOT your board to have the new kernel take effect.

# Run

After reboot, setup the pipeline by running:

sudo /etc/init.d/S50set_pipeline start

Then start the preview of dual cameras:

sudo dual-camera-rkisp.sh

The two windows vanish quickly, you need to run the command above again.

# Known bugs

  • First run may fail.

# Fixed bugs

  • Fixed: (20181228) Repeated run of sudo dual-camera-rkisp.sh will cause the cameras fail to start.
3

Wow awesome!
This is a tremendous development that can increase the scale of the firefly board users.

But..
How can it be?
Cameras are physically use the same address on the same bus.
What tricks were used?

4

@acidpero you are right, hitting the point.

At first, we are thinking of changing the OV13850’s i2c address physically by
its SID pin. The SCCB address will be 0x20 when SID pin is low, be 0x6c
when high.

But it is not the best option after all, right? Asking a user to iron a copper wire to the camera
is not good. Besides, Android can support dual cameras without it.

Then we read thoroughly the chip spec, and found the cure, which is quoted:

In the OV13850, the SCCB ID is controlled by the SID pin, and can be programmable. If SID is low, the sensor’s SCCB
address comes from register 0x300C which has a default value of 0x20. If SID is high, the sensor’s SCCB address comes
from register 0x3661 which has a default value of 0x6C.

That’s what we want, to change the i2c address on demand. After some trial and error, it proves that it is completely viable.

Later we have to deal with the sharing use of power regulators and reset pin,
and more tests to ensure that both single camera (connected with either mipi port)
and dual cameras work with the same kernel image and dtb.

5

penguin Posted at 12/28/2018 10:32
@acidpero you are right, hitting the point.

At first, we are thinking of changing the OV13850’s i …

Oh god. you nice hacker
How did you program that pin?
is that available on device driver code? or dts?
I can not catch it.
if u okay, get me some hint.

6

@acidpero , no, please read after that. we did not program that pin.

We end up programming the SC_SCCB_ID register:

0x300C SC_SCCB_ID 0x20 RW SCCB ID

7

Hi there

Are you able to get raw camera data with the driver? RGB888

8

Hi @loopdavid , I haven’t done that.

But according to http://opensource.rock-chips.com/wiki_Rockchip-isp1 ,
The RGB data can be retrieved from the isp1_selfpath.

9

penguin Posted at 1/14/2019 17:25
Hi @loopdavid , I haven’t done that.

But according to http://opensource.rock-chips.com/wiki_Rockc …

OK great thanks!

10

Hello - should this script work on the ROC-RK3399? We run the script but afterwards the unit would not boot.

11

This script is only tested and worked for Firefly-RK3399 board.

12

great job !
just a little question can we flash an image that is release later ? like this one (20190103)
https://pan.baidu.com/s/1PXbZXMAnU3k-KaNl4TgeAA#list/path=%2Fsharelink1414141670-305585565267069%2FUbuntu18.04(GPT)&parentPath=%2Fsharelink1414141670-305585565267069

13

It might work. Just go ahead and try.

14

Hi, really awesome work. I still can not get image from camera1. camera0 works ok.

root@firefly:~# dmesg | grep 1385
[ 0.138525] SMP: Total of 6 processors activated.
[ 1.218601] ov13850 1-0036: GPIO lookup for consumer reset
[ 1.218615] ov13850 1-0036: using device tree for GPIO lookup
[ 1.218655] of_get_named_gpiod_flags: parsed ‘reset-gpios’ property of node ‘/i2c@ff110000/ov13850@36[0]’ - status (0)
[ 1.218710] ov13850 1-0036: Looking up avdd-supply from device tree
[ 1.218898] ov13850 1-0036: Looking up dovdd-supply from device tree
[ 1.218995] ov13850 1-0036: Looking up dvdd-supply from device tree
[ 1.219331] ov13850 1-0036: GPIO lookup for consumer pwdn
[ 1.219341] ov13850 1-0036: using device tree for GPIO lookup
[ 1.219380] of_get_named_gpiod_flags: parsed ‘pwdn-gpios’ property of node ‘/i2c@ff110000/ov13850@36[0]’ - status (0)
[ 1.224184] ov13850 1-0036: Detected OV00d850 sensor, REVISION 0xb2
[ 1.228173] ov13850 1-0046: GPIO lookup for consumer pwdn
[ 1.228192] ov13850 1-0046: using device tree for GPIO lookup
[ 1.228253] of_get_named_gpiod_flags: parsed ‘pwdn-gpios’ property of node ‘/i2c@ff110000/ov13850@46[0]’ - status (0)
[ 1.232870] ov13850 1-0046: Detected OV00d850 sensor, REVISION 0xb2

root@firefly:/usr/local/bin# ./dual-camera-rkisp.sh
sensorbayer=SBGGR10_1X10
sensorname=“ov13850 1-0036”
sensorres=2112x1568
mpnode=/dev/video0
mpfmt=NV12
mpres=640x480
spnode=/dev/video1
spfmt=NV12
spres=640x480
parse user options success …
crop 2084,1568@(14,0)
crop 2084,1568@(14,0)
config sensor: ‘“ov13850 1-0036”:0[fmt:SBGGR10/2112x1568]’
config mipiphy input: ‘“rockchip-sy-mipi-dphy”:0[fmt:SBGGR10/2112x1568]’
config mipiphy output: ‘“rockchip-sy-mipi-dphy”:1[fmt:SBGGR10/2112x1568]’
config isp input fmt: ‘“rkisp1-isp-subdev”:0[fmt:SBGGR10/2112x1568]’
config isp input crop: ‘“rkisp1-isp-subdev”:0[crop:(0,0)/2112x1568]’
config isp output crop: ‘“rkisp1-isp-subdev”:2[crop:(0,0)/2112x1568]’
config isp output fmt: ‘“rkisp1-isp-subdev”:2[fmt:YUYV2X8/2112x1568]’
config mp crop: /dev/video0 --set-selection=target=crop,top=0,left=14,width=2084,height=1568
config mp fmt: /dev/video0 --set-fmt-video=width=640,height=480,pixelformat=NV12
config sp crop: /dev/video1 --set-selection=target=crop,top=0,left=14,width=2084,height=1568
config sp fmt: /dev/video1 --set-fmt-video=width=640,height=480,pixelformat=NV12
sensorbayer=SBGGR10_1X10
sensorname=“ov13850 1-0046”
sensorres=2112x1568
mpnode=/dev/video4
mpfmt=NV12
mpres=640x480
spnode=/dev/video5
spfmt=NV12
spres=640x480
parse user options success …
crop 2084,1568@(14,0)
crop 2084,1568@(14,0)
config sensor: ‘“ov13850 1-0046”:0[fmt:SBGGR10/2112x1568]’
config mipiphy input: ‘“rockchip-sy-mipi-dphy”:0[fmt:SBGGR10/2112x1568]’
config mipiphy output: ‘“rockchip-sy-mipi-dphy”:1[fmt:SBGGR10/2112x1568]’
config isp input fmt: ‘“rkisp1-isp-subdev”:0[fmt:SBGGR10/2112x1568]’
config isp input crop: ‘“rkisp1-isp-subdev”:0[crop:(0,0)/2112x1568]’
config isp output crop: ‘“rkisp1-isp-subdev”:2[crop:(0,0)/2112x1568]’
config isp output fmt: ‘“rkisp1-isp-subdev”:2[fmt:YUYV2X8/2112x1568]’
config mp crop: /dev/video4 --set-selection=target=crop,top=0,left=14,width=2084,height=1568
config mp fmt: /dev/video4 --set-fmt-video=width=640,height=480,pixelformat=NV12
config sp crop: /dev/video5 --set-selection=target=crop,top=0,left=14,width=2084,height=1568
config sp fmt: /dev/video5 --set-fmt-video=width=640,height=480,pixelformat=NV12
Setting pipeline to PAUSED …
media get entity by name: rockchip-mipi-dphy-rx is null
media get entity by name: lens is null
XCAM ERROR v4l2_device.cpp:253: VIDIOC_QUERYCAP returned: -1 (Inappropriate ioctl for device)
XCAM ERROR v4l2_device.cpp:253: VIDIOC_QUERYCAP returned: -1 (Inappropriate ioctl for device)
XCAM ERROR rkisp_control_loop_impl.cpp:69: get isp version failed !
Pipeline is live and does not need PREROLL …
XCAM ERROR rkisp_device.cpp:72: atomisp device(/dev/video0) try subdev format failed
Setting pipeline to PLAYING …
New clock: GstSystemClock
XCAM ERROR aec.c:1838: GAIN OUT OF RANGE: lasttime-gain: 0.001000-3.000000
Setting pipeline to PAUSED …
media get entity by name: rockchip-mipi-dphy-rx is null
media get entity by name: lens is null
XCAM ERROR v4l2_device.cpp:253: VIDIOC_QUERYCAP returned: -1 (Inappropriate ioctl for device)
XCAM ERROR v4l2_device.cpp:253: VIDIOC_QUERYCAP returned: -1 (Inappropriate ioctl for device)
XCAM ERROR rkisp_control_loop_impl.cpp:69: get isp version failed !
Pipeline is live and does not need PREROLL …
XCAM ERROR rkisp_device.cpp:72: atomisp device(/dev/video4) try subdev format failed
Setting pipeline to PLAYING …
New clock: GstSystemClock
XCAM ERROR aec.c:1838: GAIN OUT OF RANGE: lasttime-gain: 0.001000-3.000000
XCAM ERROR v4l2_device.cpp:505: device(/dev/video4) start failed on VIDIOC_STREAMON
XCAM ERROR v4l2_device.cpp:670: release buffer: munmap failed
XCAM ERROR v4l2_device.cpp:670: release buffer: munmap failed
XCAM ERROR v4l2_device.cpp:670: release buffer: munmap failed
XCAM ERROR v4l2_device.cpp:670: release buffer: munmap failed
XCAM ERROR device_manager.cpp:329: capture device start failed
ERROR: from element /GstPipeline:pipeline0/GstXCamSrc:xcamsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstXCamSrc:xcamsrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:01.083581630
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
^C
root@firefly:/usr/local/bin# handling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:12.682335287
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Thanks^0^

Will you open source for patch dirver and dts or pull the modification to firefly github?

15

PpHLNmt1R Posted at 1/31/2019 17:17
Hi, really awesome work. I still can not get image from camera1. camera0 works ok.

root@firefl …

Both camera works separately.

16

he same for me both cmera work on the exterior connection.
it look like the pin the closer to the heat sink doesn’t work for me :confused:

sudo dual-camera-rkisp.sh
sensorbayer=SBGGR10_1X10
sensorname=“ov13850 1-0036”
sensorres=2112x1568
mpnode=/dev/video0
mpfmt=NV12
mpres=640x480
spnode=/dev/video1
spfmt=NV12
spres=640x480
parse user options success …
crop 2084,1568@(14,0)
crop 2084,1568@(14,0)
config sensor: ‘“ov13850 1-0036”:0[fmt:SBGGR10/2112x1568]’
config mipiphy input: ‘“rockchip-sy-mipi-dphy”:0[fmt:SBGGR10/2112x1568]’
config mipiphy output: ‘“rockchip-sy-mipi-dphy”:1[fmt:SBGGR10/2112x1568]’
config isp input fmt: ‘“rkisp1-isp-subdev”:0[fmt:SBGGR10/2112x1568]’
config isp input crop: ‘“rkisp1-isp-subdev”:0[crop:(0,0)/2112x1568]’
config isp output crop: ‘“rkisp1-isp-subdev”:2[crop:(0,0)/2112x1568]’
config isp output fmt: ‘“rkisp1-isp-subdev”:2[fmt:YUYV2X8/2112x1568]’
config mp crop: /dev/video0 --set-selection=target=crop,top=0,left=14,width=2084,height=1568
config mp fmt: /dev/video0 --set-fmt-video=width=640,height=480,pixelformat=NV12
config sp crop: /dev/video1 --set-selection=target=crop,top=0,left=14,width=2084,height=1568
config sp fmt: /dev/video1 --set-fmt-video=width=640,height=480,pixelformat=NV12

sensorbayer=SBGGR10_1X10
sensorname=“ov13850 1-0046”
sensorres=2112x1568
mpnode=/dev/video4
mpfmt=NV12
mpres=640x480
spnode=/dev/video5
spfmt=NV12
spres=640x480
parse user options success …
crop 2084,1568@(14,0)
crop 2084,1568@(14,0)
config sensor: ‘“ov13850 1-0046”:0[fmt:SBGGR10/2112x1568]’
config mipiphy input: ‘“rockchip-sy-mipi-dphy”:0[fmt:SBGGR10/2112x1568]’
config mipiphy output: ‘“rockchip-sy-mipi-dphy”:1[fmt:SBGGR10/2112x1568]’
config isp input fmt: ‘“rkisp1-isp-subdev”:0[fmt:SBGGR10/2112x1568]’
config isp input crop: ‘“rkisp1-isp-subdev”:0[crop:(0,0)/2112x1568]’
config isp output crop: ‘“rkisp1-isp-subdev”:2[crop:(0,0)/2112x1568]’
config isp output fmt: ‘“rkisp1-isp-subdev”:2[fmt:YUYV2X8/2112x1568]’
config mp crop: /dev/video4 --set-selection=target=crop,top=0,left=14,width=2084,height=1568
config mp fmt: /dev/video4 --set-fmt-video=width=640,height=480,pixelformat=NV12
config sp crop: /dev/video5 --set-selection=target=crop,top=0,left=14,width=2084,height=1568
config sp fmt: /dev/video5 --set-fmt-video=width=640,height=480,pixelformat=NV12
Setting pipeline to PAUSED …
media get entity by name: rockchip-mipi-dphy-rx is null
media get entity by name: lens is null
XCAM ERROR v4l2_device.cpp:253: VIDIOC_QUERYCAP returned: -1 (Inappropriate ioctl for device)
XCAM ERROR v4l2_device.cpp:253: VIDIOC_QUERYCAP returned: -1 (Inappropriate ioctl for device)
XCAM ERROR rkisp_control_loop_impl.cpp:69: get isp version failed !
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
XCAM ERROR rkisp_device.cpp:72: atomisp device(/dev/video0) try subdev format failed
XCAM ERROR aec.c:1838: GAIN OUT OF RANGE: lasttime-gain: 0.001000-3.000000
Setting pipeline to PAUSED …
media get entity by name: rockchip-mipi-dphy-rx is null
media get entity by name: lens is null
XCAM ERROR v4l2_device.cpp:253: VIDIOC_QUERYCAP returned: -1 (Inappropriate ioctl for device)
XCAM ERROR v4l2_device.cpp:253: VIDIOC_QUERYCAP returned: -1 (Inappropriate ioctl for device)
XCAM ERROR rkisp_control_loop_impl.cpp:69: get isp version failed !
Pipeline is live and does not need PREROLL …
XCAM ERROR rkisp_device.cpp:72: atomisp device(/dev/video4) try subdev format failed
Setting pipeline to PLAYING …
New clock: GstSystemClock
XCAM ERROR aec.c:1838: GAIN OUT OF RANGE: lasttime-gain: 0.001000-3.000000

17

ERROR: from element /GstPipeline:pipeline0/GstXCamSrc:xcamsrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstXCamSrc:xcamsrc0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:01.068191325
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
ERROR: from element /GstPipeline:pipeline0/GstGtkSink:gtksink0: Output widget was destroyed
Additional debug info:
gstgtkbasesink.c(486): gst_gtk_base_sink_show_frame (): /GstPipeline:pipeline0/GstGtkSink:gtksink0
Execution ended after 0:00:38.340511582

here more info about the error

18

Happy New Year.
Long time no see. I’m using dual csi-mipi well

by the way,
ov13850 using 4 lane?

board and camera sensor schematics are showing 4 lanes.
but I think ‘rk3399-firefly-linux.dts’ not shown as using 4 lanes.
https://github.com/FireflyTeam/k … efly-linux.dts#L722

@penguin how do you think about this?

19

@PpHLNmt1R
source code is already at:

https://github.com/FireflyTeam/kernel/commits/stable-4.4-rk3399-linux

@acidpero
truly i’m not an expert on ov13850.
you can try to use 4 lanes by modifying dts node
and the ov13850 sensor driver.

20

@penguin Your instructions are useful and I can look at your repository, but without taking the entire repo apart could you outline what you have done to make dual cameras possible?
This is VERY important for portability and I do want to use something other than ubuntu. I was thinking Gentoo. It is ideal for making things small and customized down in size and increased in speed due to the differing options, both in terms of DEFINES and CFLAGS.
Thanks for your work! It means a lot to us!