This lesson builds the digital human demo natively on the board, installs it as a system
service, then drives the avatar to speak by sending text over a Unix socket.
The whole flow was verified on a real board (license OK + HDMI output). The demo depends on a
ROCKDH algorithm license: without an account you simply need thekey_digital_human.lic
file placed in the right location — no account required.
Objectives
- Master the board-native build flow for SDK source demos (
build-linux.sh, no cross-compiler) - Master the
system/+install.shboard deployment pattern - Learn to place the rkauth license file and verify the license is active
1. Prepare the Environment
1.1 Hardware
| Item | Requirement |
|---|---|
| Board | RK3588-class board (the RK1828 card is NOT required — this demo’s algorithms run on the host board); tested on Firefly AIO-3588L-QUA-RK182X |
| Display | HDMI connected to the board: the digital human is a rendering demo — no screen, no picture (HDMI-A-1 must be connected) |
| Audio | HDMI audio or a Bluetooth speaker (without it you get picture but no sound, see FAQ Q4) |
| PC | Only needs ssh/scp access to the board; no cross-compiler |
1.2 Board Dependencies
The board needs gcc / g++ / cmake (the SDK system image ships them by default):
gcc --version | head -1
cmake --version | head -1
1.3 SDK Directory
examples/digital_man/
├── build-linux.sh # one-shot build entry (run on the board)
├── CMakeLists.txt
├── src/ # source code
├── third_party/ # digital_human SDK + rockit (~1.3GB)
├── system/ # build output lands here (bin/ + lib/ + userdata/DH + install.sh)
└── README_CN.md
2. Task Overview
Runtime layout:
Text source ──(Unix dgram socket /tmp/dh_ipc.sock)──> digital_man resident service
├─ text → TTS voice
├─ lip-sync / expression driving
└─ Rockchip rendering → HDMI output
- Managed by the systemd service
digital_man(ExecStart=/bin/rknn3_make_digital_man_demo) - The external interface is minimal: send
id=end=text(e.g.1=1=hello) to the socket to make it speak
3. Steps
Step 1: Pack and Copy to the Board
# On the PC (archive ~650MB)
cd <SDK>/examples
tar czf /tmp/digital_man-src.tgz digital_man
# Copy to the board
scp /tmp/digital_man-src.tgz firefly@<board-IP>:/tmp/
# Extract on the board (~1.3GB extracted)
ssh firefly@<board-IP>
sudo tar xzf /tmp/digital_man-src.tgz -C /userdata
ls /userdata/digital_man # CMakeLists.txt build-linux.sh src system third_party ...
Step 2: [MUST DO BEFORE BUILDING] Point the audio card at YOUR board
The audio card is hardcoded in the source (src/VideoProducer.cpp):
memcpy(aoAttr.u8CardName, "hw:1,0", 7); // 1 = the board's second sound card
Card numbering differs between boards. List your playback devices first, then decide what to fill in:
aplay -l # list all playback devices; note card N and its name
Measured on this board (Firefly AIO-3588L-QUA-RK182X):
card 0: rockchiphdmi0 [rockchip-hdmi0] ← HDMI audio
card 1: rockchipbt [rockchip,bt] ← Bluetooth SCO
The original hw:1,0 points at the Bluetooth card — hence “picture but no sound”.
To output over HDMI change it to hw:0,0:
cd /userdata/digital_man
sed -i 's/hw:1,0/hw:0,0/' src/VideoProducer.cpp
grep -n "u8CardName" src/VideoProducer.cpp # confirm it now reads hw:0,0
Order matters: this must happen before compiling; only then does the change take effect.
If HDMI is card2/card3 on your board, fill in the matchinghw:N,0.
Step 3: Build on the Board
cd /userdata/digital_man
chmod +x build-linux.sh
./build-linux.sh
Do NOT set
GCC_COMPILER: the board-side build uses the system gcc/g++;
setting it may actually break toolchain detection.
Success markers (end of the log):
-- Installing: /userdata/digital_man/system/bin/rknn3_make_digital_man_demo
-- Installing: /userdata/digital_man/system/lib/libdigital_human_library.so
Step 4: Install (libraries / models / service)
cd /userdata/digital_man/system
chmod +x install.sh
sudo ./install.sh
What gets installed:
| Target | Content |
|---|---|
/lib/*.so |
rockit, rockx2, digital_human, rknnrt, etc. |
/userdata/DH/ |
models and frontend dictionaries (~780MB) |
/etc/systemd/system/digital_man.service |
service unit |
Step 5: Place the License File
With a lic file you don’t need an account — just place it:
sudo cp key_digital_human.lic /userdata/key_digital_human.lic
sudo chmod 644 /userdata/key_digital_human.lic
grep license_path /userdata/DH/rockx_auth_config.json # must be /userdata/key_digital_human.lic
To generate the lic yourself (with an account):
cd /userdata/DH/rkauth_tool/Linux/aarch64/
./rkauth_tool_bin -u <user> -p <password> -o /userdata/key_digital_human.lic -m digital_human
Step 6: Start the Service
sudo systemctl daemon-reload
sudo systemctl restart digital_man
sudo systemctl enable digital_man # boot autostart (install.sh does NOT enable it)
systemctl is-active digital_man # active
Note:
digital_man.serviceis not enabled after installation; after a reboot it will not
run, and/tmp/dh_ipc.sockwill be missing. Fix withsystemctl enable digital_man.
Step 7: Verify (four checks)
# ① Is the license active?
journalctl -u digital_man -n 20 --no-pager | grep -E "rkauth|socket initialized"
# TextControl: socket initialized at /tmp/dh_ipc.sock
# I rkauth: auth by license_path /userdata/key_digital_human.lic
# I rkauth: verify license key ok ← license active
# ② Is HDMI actually outputting?
cat /sys/kernel/debug/dri/0/summary | head -8
# Video Port0: ACTIVE
# Connector:HDMI-A-1
# Display mode: 1920x1080p60.00
# ③ Is the IPC ready?
ls -l /tmp/dh_ipc.sock
# ④ Does audio point at the card you chose? (key check)
PID=$(pgrep -f rknn3_make_digital_man_demo | head -1)
ls -l /proc/$PID/fd | grep pcm
# expected: pcmC0D0p → card0 (HDMI)
# if still pcmC1D0p, the code wasn't changed or wasn't rebuilt
Step 8: Send Text to the Digital Human (one command)
First install a tiny one-liner helper (only once):
sudo tee /usr/local/bin/dh >/dev/null <<'EOF'
#!/bin/bash
set -e
SOCK=/tmp/dh_ipc.sock
if [ "$1" = "-f" ]; then TEXT=$(cat "$2")
elif [ -n "$1" ]; then TEXT="$*"
elif [ ! -t 0 ]; then TEXT=$(cat)
else echo "usage: dh <text> | echo text | dh | dh -f file" >&2; exit 1; fi
[ -S "$SOCK" ] || { echo "error: $SOCK not found (service down?)" >&2; exit 1; }
python3 - "$TEXT" <<'PY'
import socket, sys
s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
s.sendto(("1=1=" + sys.argv[1]).encode("utf-8"), "/tmp/dh_ipc.sock")
print("said to digital human: " + sys.argv[1])
PY
EOF
sudo chmod +x /usr/local/bin/dh
Then interaction is trivial:
dh Hello, I am a digital human # say a sentence directly
echo "another sentence" | dh # pipe
dh -f speech.txt # read from file
The message format is id=end=text (separated by =, NOT id=1;end=1;text=) — the most common pitfall:
// parsing logic in src/TextControl.cpp
p1 = msg.find('='); // first '='
p2 = msg.find('=', p1 + 1); // second '='
nTextId = stoi(msg.substr(0, p1)); // first field = id
bIsEnd = stoi(msg.substr(p1+1, p2-p1-1));// second field = is-final-segment flag
strText = msg.substr(p2 + 1); // everything after = text body
Send:
python3 - <<'EOF'
import socket
s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
# correct: 1=1=text
s.sendto("1=1=Hello everyone, welcome to the Firefly digital human class".encode("utf-8"), "/tmp/dh_ipc.sock")
print("sent")
EOF
After sending, confirm TTS is really working via the log:
journalctl -u digital_man -n 10 --no-pager
# [TTS Thread] Got audio status: 0
# zh,Hello everyone, welcome to the Firefly digital human class.!.!.!
# RTF: 0.12 ← real-time factor, lower is faster
4. Verification
| Item | Criterion | Measured |
|---|---|---|
| Build | system/bin/rknn3_make_digital_man_demo exists (file reports aarch64 ELF) |
|
| Install | /userdata/DH/data exists, /lib/libdigital_human_library.so exists |
|
| Service | systemctl is-active digital_man = active, /tmp/dh_ipc.sock exists |
|
| License | log shows I rkauth: verify license key ok |
|
| Picture | dri/0/summary shows Video Port0: ACTIVE + HDMI-A-1 + 1920x1080 |
|
| Render load | process CPU ~110% (ps -o %cpu -p $(pgrep -f rknn3_make_digital_man_demo)) |
|
| Speaking | lip movement on screen / audible voice after sending text | needs eyes on screen |
Do NOT judge by “did the log print the received text”: in practice the service
does not log every text message.
5. FAQ
Q1: Log shows W rkauth: license_path /userdata/key_digital_human.lic can not read?
The lic is missing or in the wrong place. Put key_digital_human.lic under /userdata/ with
chmod 644, then systemctl restart digital_man; the path must match license_path in
rockx_auth_config.json.
Q2: Service active but screen black / log shows Layer ... VoDev 0 is not enabled?
No monitor attached or not detected. Check cat /sys/class/drm/card0-HDMI-A-1/status; it must be
connected. Best practice: connect the display before powering on.
Q3: install.sh put desktop shortcuts under /home/linaro/Desktop but my user is firefly?
The script assumes ROCKDH’s default user linaro; just copy the .desktop files to
/home/firefly/Desktop/.
Q4: Picture but no sound?
Root cause: the audio card is hardcoded (u8CardName in src/VideoProducer.cpp); the default
hw:1,0 points at the Bluetooth card.
① List playback devices
aplay -l # card numbers + names
aplay -L # list all PCMs (including default/plug etc.)
② Choose the playback device (edit code → rebuild → reinstall → restart)
cd /userdata/digital_man
sed -i 's/hw:1,0/hw:0,0/' src/VideoProducer.cpp # use YOUR card number
./build-linux.sh
cd system && sudo ./install.sh
sudo systemctl restart digital_man
③ Verify it really uses that card
PID=$(pgrep -f rknn3_make_digital_man_demo | head -1)
ls -l /proc/$PID/fd | grep pcm
# pcmC0D0p = card0 (HDMI after this change) pcmC1D0p = card1 (old Bluetooth)
Once correct, speaker-test -D hw:0,0 reports Device or resource busy — that is NOT an error;
it means the digital human has exclusive ownership of the card and is playing.
The audio card is baked into the binary at compile time, so “changing the card number” must
happen before./build-linux.sh.
Q5: Service active, HDMI connected, but the screen stays black?
First check the log for this line:
vo_gpu_composer {splice:183} Layer 0 is not enabled!
If present, the desktop is stealing the screen: the board boots to graphical.target by
default, GNOME (gdm) holds /dev/dri/card0, and Rockit VO cannot enable its layer. Confirm:
ps -eo pid,comm | grep -E "gnome-shell|gdm" # gnome-shell present = desktop occupying screen
fuser -v /dev/dri/card0 # who holds card0
systemctl get-default # graphical.target = desktop present
The digital human uses direct VO output and must own the display exclusively. Switch to
multi-user (no desktop) then start it:
sudo systemctl isolate multi-user.target
sudo systemctl restart digital_man
Afterwards the log shows VoDev 0 Layer 0 update ... (no more Layer 0 is not enabled), and
/sys/kernel/debug/dri/0/summary reports Video Port0: ACTIVE and Esmart0-win0: ACTIVE.
Note:
systemctl isolate multi-user.targettears down the whole graphics stack; the
screen blanks briefly then the demo picture appears. To restore the desktop run
sudo systemctl isolate graphical.target(but it will grab the screen again and the digital
human loses the picture). To make “boot straight into digital human” permanent:
sudo systemctl set-default multi-user.target.
Q6: No reaction after sending text / it reads gibberish?
Check the message format first. The code expects id=end=text (two = separators), e.g.
1=1=hello. If you send id=1;end=1;text=hello, the second field parses as 1;end
(stoi fails → 0) and the body becomes 1;text=hello, so it either stays silent or reads out
something odd.
# correct
python3 -c 'import socket;s=socket.socket(socket.AF_UNIX,socket.SOCK_DGRAM);s.sendto("1=1=hello".encode(),"/tmp/dh_ipc.sock")'
Other checks: ① service active; ② /tmp/dh_ipc.sock exists;
③ journalctl -u digital_man -n 10 should show [TTS Thread] and RTF: lines.
Lesson Summary
- Source demos in three moves: pack & push → board-side
build-linux.sh→install.sh+systemctl - Board-native builds need no cross toolchain — simplest path; the board only needs gcc/cmake
- Licensing is file-based: place
key_digital_human.licat the right path and it activates;
verify license key okin the log is the criterion - For rendering demos, “really showing a picture” is confirmed via
Video Port0: ACTIVEindri/0/summary