Hardware decoder on linux for firefly-rk3288(source opened)

Chip Platform Rockchip
1

This is how to add hardware decoder to ffmpeg on linux for firefly-rk3288.

First, choose the vpu driver librarys.
We use the hybris library made by mac_l1,also you can choose rockchip’s vpu driver library ,which they has put it on a websites.
For convenience ,I put a mac_l1’s hybris lib’s copy to “baidu pan”. here is the link:
http://pan.baidu.com/s/1pKdqUQR

Install the driver library:

sudo dpkg  -i machybris-0.1.0_armhf.deb

if you get a mistake. please run this:

sudo apt-get -f install

and try again.

After install done ,reboot your board .

Install dependencies library :

sudo apt-get install libsdl-dev

this source code need compiled on the development board. if you want to compiled on PC ,you need modify the source code.

Install compile environment:

sudo apt-get install build-essential

Get the source code:

git clone https://github.com/isle0118/firefly-vpu-linux.git

Compile and Install:

./configure  --enable-shared --prefix=/usr --disable-ffmpeg --disable-ffprobe --disable-ffserver  --disable-static

make && sudo make install

Now, you can test the hardware decoder with ffplay .

the source file is here :

libavcodec/rkdec_decoder.c

if you want use other driver library or compiled on PC ,just need to modify this file.

Have fun !

2

Wow! I am impressed!

This opens up the way to get hw decoding in chromium, mplayer and vlc, as they all use ffmpeg/ avcodec as base.
Maybe we can even play netflix on chromium!

Also a new version of machybris (0.1.1) can be downloaded here: http://www.mac-l1.com/v0.1.1/machybris-rk3288_4.4.2-0.1.1_armhf.deb

Great work, isle!

3

your machybris is Amazing。we can do many things on it.

we really appreciate all your work!

4

Hi mac_l1:
can you share the method that how to use GPU for encoding?

5

346409941 Posted at 1/3/2016 11:05
Hi mac_l1:
can you share the method that how to use GPU for encoding?

Hi,
See: https://github.com/mac-l1/machybris_examples/blob/master/recorder/h264recorder.cpp#L158.
I used this code to capture the screen into a h264-movie using vpu.
This can probably quite easily be added into the shared ffmpeg / avcodec code above from isle.

6

Hi,

I get an error : librk_on2.so and libvpu.so are missing !

I think a package is missing…

7

OK it’s solved with just a reboot …:blush:
Do you know why it’s not working with 4K video ?

8

the vpu can decode 4K video smoothly. But the display was terrible.
so we didn’t add 4k support here.

9

Oh bad news…
It’s not possible to add 4K support (maybe with display improvement) ?

I think that the main advantage of this chip is 4K support.

10

Hi mac_l1:
can you give me screenrecoderh264 for testing HW encoder

11

mac_l1 Posted at 1/3/2016 14:11
Hi,
See: https://github.com/mac-l1/machybris_examples/blob/master/recorder/h264recorder.cpp#L158. …

Hi mac_l1:
can you give me screenrecoderh264 for testing HW encoder

12

Attached zip file with two files in it (one binary, one shell script).
Use the script to capture the screen, the recording is stopped by pressing a key:

./screenrecord_h264.sh test.mp4

You should have installed machybris before, and also avconv (sudo apt-get install libav-tools).

Every frame is captured from framebuffer device /dev/fb0. Using shared memory this RGB frame is converted to YUV420SP / NV21 format using the RGA hardware module. This YUV frame is then given to the encoder using the raw vpu api that generates an h264 bitstream. After key-press this h264 bitstream file is then encapsulated into a movie-container (mp4 in this case) by avconv utility.

You can see some android logging by:

sudo /system/bin/logcat

You can see some kernel logging by:

sudo tail -f /var/log/syslog

Hope this helps.

Cheers – mac_l1
h264recorder.zip (45.9 KB)

13

@mac_l1, @isle,

Do you think that it’s possible to add 4K support (maybe with display improvement) ?
I can’t see in sourcecode where 4K is forbidden ! Can you show me where i can find such thing ?

14

Hi fd101283.
the 4k video hasn’t be forbbidened on this sourcecode.
but when playing 4k video ,the ffplay sometimes get rong video width&height .like “avctx->width=0 & avctx->height=0”;in the initializtion function “rkdec_prepare”.
so you need give them the width&height value before “rkdec_ctx->ctx->init”.
after init is done ,the decoder works. Now your can get the real width&height “pframe->DisplayWidth” by “rkdec_ctx->vpu_mem_link(&pframe->vpumem);”.
this is what I get. when i playing 4k videos.

If someone can make the display woking well on 4k.that must be excellent.
But if you just need deocode with 4k , display with HD or FHD. you can use rga to do downscale.thay is a way to get 4k woks.
haha!

15

mac_l1 Posted at 1/6/2016 23:24
Attached zip file with two files in it (one binary, one shell script).
Use the script to capture th …

thank you for your help!
screenrecord_h264.sh called bin file h264recorder, Where can I get the source code of h264recorder?

thanks

16

346409941 Posted at 1/7/2016 15:54
thank you for your help!
screenrecord_h264.sh called bin file h264recorder, Where can I get the s …

Here is the source code:

git clone https://github.com/mac-l1/machybris_examples.git

Look in the recorder folder for the screenrecorder.
Cheers! mac_l1

17

Hi Isle,

Thanks alot for the explanation…It’s very great job that you’ve done. :slight_smile:

I try to modify source code in order to decode 4K. As a result i always get errors before decoding. This is the log :

Firefly@firefly:~/Downloads/firefly-vpu-linux$ ./ffplay Astra_Canal_UHD.ts
ffplay version 2.6.3 Copyright (c) 2003-2015 the FFmpeg developers
  built with gcc 4.8 (Ubuntu/Linaro 4.8.4-2ubuntu1~14.04)
  configuration: --enable-shared --prefix=/usr --disable-ffmpeg --disable-ffprobe --disable-ffserver --disable-static
  libavutil      54. 20.100 / 54. 20.100
  libavcodec     56. 26.100 / 56. 26.100
  libavformat    56. 25.101 / 56. 25.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 11.102 /  5. 11.102
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
[hevc @ 0xb3922dc0] nb threads_frame = 1, nb threads_slice 1, thread_type = null
[hevc @ 0xb3908220] nb threads_frame = 1, nb threads_slice 1, thread_type = null
[hevc @ 0xb3922880] nb threads_frame = 1, nb threads_slice 1, thread_type = null
[hevc_rkvpu @ 0xb39044a0] PPS id out of range: 0
[hevc @ 0xb3908220] PPS id out of range: 0
[hevc @ 0xb3908220] Error parsing NAL unit #2.
Error while decoding frame
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
    Last message repeated 2 times
get frame failed ret -10040 aq=    0KB vq=    0KB sq=    0B f=0/0
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
get frame failed ret -1004
Something wrong during decode!!!
18

Oh,
“[hevc_rkvpu @ 0xb39044a0] PPS id out of range: 0”
This is very strange.
have you try another 4k video ?and get the some mistake?

19

Yes, the same stream works well with android player…
And i also try with some other stream and it give the same problem…

20

This is the stream which gives problems :

http://www.filedropper.com/astracanaluhd