This project is designed to empower developers with the knowledge and tools necessary to effectively utilize Qualcomm® Robotics ROS (ROS) software packages. These packages serve as essential building blocks for robotics development, providing a superior foundation for creating advanced robotic applications.
Objective
What is Qualcomm Robotics ROS?
Qualcomm Robotics ROS is a comprehensive suite of open-source ROS 2 packages designed for robotics applications. It utilizes upstream ROS community projects and enables developers to easily access our platform hardware capabilities, accelerating the development of innovative robotics projects.
What hardware capabilities and acceleration features can you utilize with Qualcomm Robotics platforms in ROS?
Qualcomm Robotics platforms offer several hardware acceleration features designed to enhance the performance and efficiency of ROS applications:
- Qualcomm Robotics ROS Transport:
- Hardware-Accelerated Message Transport: Facilitates zero-copy message data transfer between ROS nodes and hardware components using type adaptation, ensuring efficient communication.
- Type Adaptation Feature (REP 2007): Available in ROS 2 Humble, this feature allows direct serialization to the user-requested type and intra-process communication without conversion, minimizing data handling overhead.
- lib_mem_dmabuf: The qrb_ros_transport is based on this open-source library, ensuring broad compatibility across all Linux-based platforms.
- Image Pipeline Acceleration:
- Qualcomm Robotics ROS Image Pipeline: Enhanced for hardware acceleration using Qualcomm Technologies’ EVA/GPU, this pipeline supports high-performance image processing.
- Current Features: Includes hardware-accelerated image resizing and color space conversion.
- Future Enhancements: Plans to fully accelerate processing of rectified mono/color images, stereo disparity images, and stereo point clouds.
- qrb_ros_image_resize:
- NV12 Image Downscaling: Supports EVA acceleration for resizing images in NV12 color space, commonly used in Qualcomm’s smart devices.
- qrb_ros_color_space_convert:
- Color Space Conversion: Facilitates conversion between NV12 and RGB888 color spaces, leveraging Qualcomm’s hardware capabilities.
- Qualcomm Robotics ROS Sensor:
- Sensor Data Acquisition: Utilizes a framework to obtain sensor data from the sensors DSP via FastRPC direct channel.
- qrb_ros_imu: Provides high-performance IMU data acquisition.
- qrb_sensor_client: A dynamic library that reduces latency between the ROS node and the driver, achieving time consumption around 0.4ms, significantly improving performance.
These features collectively enhance the efficiency and performance of robotics applications on Qualcomm Robotics platforms, making them a superior choice for developers.
Running camera
Download Code
- Create a path to store the code on the Host PC.
- Run the following commands to clone the code repository in this path.
git clone https://github.com/quic-qrb-ros/lib_mem_dmabuf.git
git clone https://github.com/quic-qrb-ros/qrb_ros_transport.git
git clone https://github.com/quic-qrb-ros/qrb_ros_camera.git
git clone https://github.com/quic-qrb-ros/qrb_ros_colorspace_convert.gitBuild
Ensure that you have set up the Qualcomm® Intelligent Robotics Product (IRP) SDK environment before building the code.
Then, you can use the following commands to build the code in Qualcomm IRP SDK.
export AMENT_PREFIX_PATH="${OECORE_TARGET_SYSROOT}/usr;${OECORE_NATIVE_SYSROOT}/usr"
export PYTHONPATH=${PYTHONPATH}:${OECORE_TARGET_SYSROOT}/usr/lib/python3.10/site-packages
colcon build --merge-install --cmake-args \
-DPython3_ROOT_DIR=${OECORE_TARGET_SYSROOT}/usr \
-DPython3_NumPy_INCLUDE_DIR=${OECORE_TARGET_SYSROOT}/usr/lib/python3.10/site-packages/numpy/core/include \
-DSYSROOT_LIBDIR=${OECORE_TARGET_SYSROOT}/usr/lib \
-DSYSROOT_INCDIR=${OECORE_TARGET_SYSROOT}/usr/include \
-DPYTHON_SOABI=cpython-310-aarch64-linux-gnu -DCMAKE_STAGING_PREFIX=$(pwd)/install \
-DCMAKE_PREFIX_PATH=$(pwd)/install/share \
-DBUILD_TESTING=OFFPush and install
Run these commands to push and install the output SDK on the device.
cd install
tar czvf qrb_ros_example.tar.gz lib share
scp qrb_ros_example.tar.gz root@[ip-addr]:/opt/
ssh root@[ip-addr]
(ssh) tar -zxf /opt/qrb_ros_example.tar.gz -C /opt/qcom/qirp-sdk/usr/Firstly, we need to create launch.py file that looks like:
import launch
import os
from ament_index_python.packages import get_package_share_directory
from launch_ros.actions import ComposableNodeContainer
from launch_ros.actions import Node
from launch_ros.descriptions import ComposableNode
def generate_launch_description():
camera_info_config_file_path = os.path.join(
get_package_share_directory('qrb_ros_camera'),
'config', 'camera_info_imx577.yaml'
)
camera_info_path = camera_info_config_file_path
print(camera_info_path)
"""Generate launch description with multiple components."""
container = ComposableNodeContainer(
name='my_container',
namespace='',
package='rclcpp_components',
executable='component_container',
composable_node_descriptions=[
ComposableNode(
package='qrb_ros_camera',
plugin='qrb_ros::camera::CameraNode',
name='camera_node',
remappings=[('/image', '/image_raw')],
parameters=[{
'camera_info_path': camera_info_path,
'fps': 30,
'width': 1920,
'height': 1080,
'cameraId': 0,
'publish_latency_type': 1,
}]
),
ComposableNode(
package='qrb_ros_colorspace_convert',
plugin='qrb_ros::colorspace_convert::ColorspaceConvertNode',
parameters=[{
'conversion_type': "nv12_to_rgb8",
'latency_fps_test': False,
}],
extra_arguments=[{'use_intra_process_comms': True, 'log_level': 'INFO'}],
)
],
output='screen',
)
return launch.LaunchDescription([container,
Node(
package='image_transport',
executable='republish',
output='screen',
name='republish_node',
arguments=[
'raw', # Input
'compressed', # Output
],
remappings=[
('in', '/image'),
('out/compressed', '/image_compressed'),
]
),
])
Secondly, we need to push this file to device.
scp launch.py root@[ip-addr]:/opt/Thirdly, we will use these cmd to run it on the device.
ssh root@[ip-addr]
(ssh) export XDG_RUNTIME_DIR=/dev/socket/weston/
(ssh) export WAYLAND_DISPLAY=wayland-1
(ssh) export HOME=/opt
(ssh) source /opt/qcom/qirp-sdk/qirp-setup.sh
(ssh) export ROS_DOMAIN_ID=1
(ssh) source /usr/bin/ros_setup.bash
(ssh) ros2 launch /opt/launch.py
