1. Set resolution ratios for all four cameras.

2. Store the output tensors of the four cameras to one UserBuffer tensor with a batch size of four.

3. Execute the network with four batch sizes and get the result from the output UserBuffer.

4. UserBuffer is recommended, as it provides better performance

The following cameras are compatible for RB5 (Computer Vision)

1. ISP Camera in the Qualcomm® Robotics RB5 Vision Kit https://www.thundercomm.com/product/qualcomm-robotics-rb5-development-kit/#versions

2. USB camera - Any USB camera works. The Intel® RealSense™ depth camera D435i can be also used as USB camera."

The command syntax to start the TCP server is:

gst-launch-1.0 v4l2src device="/dev/video1" ! image/jpeg,framerate=30/1,width=1280,height=720 ! tcpserversink port=8900 host=192.168.55.12

In the above command: 

device="your/camera/videoId"

host="rb5 ip address"

To capture a frame in openCV, use the following command:

frame = cv2.VideoCapture("tcp://192.168.55.12:8900 ") //tcp://rb5ip:port

1. Power off the device (unplug the power cable and USB cable).

2. Plug in the power cable (12V).

3. Press and hold the F_DL key, and connect the board to the PC via a type-C USB (switches the device to EDL mode).

4. Release the F_DL key after the board connects to the PC

Since RB5 is an Ubuntu 18.04-based device, you can assign a static IP address using the procedures described here.

The GStreamer command allows you to store the video in the specified location of the device, for example, the /data/ directory. The device can  connect to the PC and data can be pulled with the adb command:

adb pull data/'filename' c:\aaa

SSH can also be used.

Use this command ot SSH onto the device:

echo  "PermitRootLogin yes"  >> /etc/ssh/sshd_config

/etc/init.d/ssh restart

Here are some sample audio projects that address commond use cases:

Comprehend natural language commands (voice or text) using AWS Lex services

Text to speech with AWS tts-ros2 application

Wake Word example

gst-launch-1.0 -e qtiqmmfsrc name=qmmf ! capsfilter caps="video/x-raw,format=NV12,width=640,height=480,framerate=30/1" ! jpegenc ! multifilesink location=test_image.jpg

gst-launch-1.0 -e qtiqmmfsrc name=qmmf ! video/x-h264,format=NV12, width=1280, height=720,framerate=30/1 ! h264parse config-interval=1 ! mpegtsmux name=muxer ! queue ! tcpserversink port=8900 host=192.168.1.120

The following packages (and corresponding versions) are required to flash the RB5 image:

coreutils 8.28, fakechroot 2.19, fakeroot 1.22, kmod 24-1ubuntu3.2, libc6-arm64-cross 2.27, python 2.7.15, qemu-user-static 1:2.11+dfsg1ubuntu7.28, udev 237-3ubuntu10.42, unzip 6.0, wget 1.19.4.

To enable a Weston client on HDMI, use the following commands:

sh-4.4# mkdir -p /usr/bin/weston_socket

sh-4.4# export XDG_RUNTIME_DIR=/usr/bin/weston_socket

sh-4.4# weston --tty=1 --connector=29

To put the device in QDL mode:

1. Unplug the power connector.

2. Connect the USB cable to the host PC.

3. Press and hold the F_DL button and plug in the power connector. Keep holding the F_DL button for 3 to 5 seconds until the device enters QDL mode.