feat(gemini2): add OrbbecSDK_ROS2/

This commit is contained in:
X-lanni
2025-07-04 15:30:11 +08:00
parent 5cf886315d
commit 1e82e0115a
185 changed files with 864328 additions and 0 deletions
+40
View File
@@ -0,0 +1,40 @@
## Aligning Depth to Color in ROS 2
This section explains how to align depth images with color images to create an overlay image using ROS 2. This is particularly useful for applications requiring synchronized visual information from different sensor modalities.
### Commands to Align and View Depth and Color Images
1. **Basic Depth to Color Alignment:**
To simply align the depth image to the color image, use the following command:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true
```
This command activates the depth registration feature without opening a viewer.
2. **Viewing Depth to Color Overlay:**
If you wish to view the depth to color overlay, you need to enable the viewer by using the command below:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true enable_d2c_viewer:=true
```
This launches the camera node with depth to color registration and opens a viewer to display the overlay image.
### Selecting Topics in RViz2
To visualize the aligned images in RViz2:
1. Launch RViz2 after running one of the above commands.
2. Select the topic for the depth to color overlay image. An example topic selection is shown here:
![Topic Selection for Depth to Color Overlay](./images/image3.png)
### Example of Depth to Color Overlay
After selecting the appropriate topic in RViz2, you will be able to see the depth to color overlay image. Here's what it might look like:
![Depth to Color Overlay Image](./images/image4.jpg)
@@ -0,0 +1,40 @@
## 在ROS 2中将深度图像与彩色图像对齐
本节介绍如何使用ROS 2将深度图像与彩色图像对齐,以创建叠加图像。这对于需要来自不同传感器模态的同步视觉信息的应用程序特别有用。
### 对齐并查看深度和彩色图像的命令
1. **基本的深度到彩色对齐:**
要简单地将深度图像对齐到彩色图像,使用以下命令:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true
```
此命令激活深度注册功能,但不打开查看器。
2. **查看深度到彩色叠加:**
如果您希望查看深度到彩色叠加,您需要通过使用以下命令来启用查看器:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py depth_registration:=true enable_d2c_viewer:=true
```
这将启动具有深度到彩色注册的相机节点,并打开一个查看器以显示叠加图像。
### 在RViz2中选择主题
要在RViz2中可视化对齐的图像:
1. 在运行上述命令之一后启动RViz2。
2. 选择深度到彩色叠加图像的主题。这里显示了一个示例主题选择:
![深度到彩色叠加主题选择](./images/image3.png)
### 深度到彩色叠加示例
在RViz2中选择合适的主题后,您将能够看到深度到彩色叠加图像。这是它可能看起来的样子:
![深度到彩色叠加图像](./images/image4.jpg)
+154
View File
@@ -0,0 +1,154 @@
# Fast DDS Optimization for Orbbec Camera with ROS2
When operating with the default configuration, Fast DDS exhibits suboptimal transmission efficiency, resulting in
significant image transmission delays when used with the Orbbec camera in ROS2. This document provides guidance on
optimizing Fast DDS to enhance image transfer efficiency.
## 1. Adjusting System Parameters
### IP Fragmentation Time
- **Path**: `/proc/sys/net/ipv4/ipfrag_time` (default: 30 seconds)
- **Purpose**: Defines the duration that IP fragments are kept in memory.
- **Adjustment**: Decrease this value to reduce the time window where no fragments are received, which can help reduce
delays. Consider the specific needs of your environment as this setting affects all incoming fragments.
**Example**: Set to 3 seconds.
```bash
sudo sysctl net.ipv4.ipfrag_time=3
```
### IP Fragmentation Memory Threshold
- **Path**: `/proc/sys/net/ipv4/ipfrag_high_thresh` (default: 262144 bytes)
- **Purpose**: Sets the maximum memory used to reassemble IP fragments.
- **Adjustment**: Increase this value to allow more memory for fragment reassembly, which can improve handling of larger
data packets.
**Example**: Increase to 128 MB.
```bash
sudo sysctl net.ipv4.ipfrag_high_thresh=134217728
```
### Maximum Buffer Sizes
- **Purpose**: Configures the maximum buffer sizes for receiving and sending data, which is critical for high-throughput
data transmission.
- **Adjustment**: Set the maximum buffer sizes for both receiving and sending operations.
**Commands**:
```bash
sudo sysctl -w net.core.rmem_max=2147483647
sudo sysctl -w net.core.rmem_default=2147483647
sudo sysctl -w net.core.wmem_max=2147483647
sudo sysctl -w net.core.wmem_default=2147483647
```
Alternatively, make these settings permanent by adding them to the `/etc/sysctl.d/10-fastrtps-max.conf` file.
```bash
sudo gedit /etc/sysctl.d/10-fastrtps-max.conf
```
add blow lines to the file:
```bash
net.core.rmem_max=2147483647
net.core.rmem_default=2147483647
net.core.wmem_max=2147483647
net.core.wmem_default=2147483647
```
then save and exit the file. run `sudo sysctl -p` to apply the changes.
For detailed guidance, refer
to [ROS 2 DDS Tuning Documentation](https://docs.ros.org/en/foxy/How-To-Guides/DDS-tuning.html).
## 2. Fast DDS Configuration
Below is an example of a Fast DDS configuration file optimized for ROS2 usage with the Orbbec camera. This configuration
enhances the overall data transmission by adjusting buffer sizes and transport settings.
### Configuration File: `shm_fastdds.xml`
Place this file in the `$HOME` directory.
```xml
<?xml version="1.0" encoding="UTF-8"?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
<transport_descriptors>
<transport_descriptor>
<transport_id>UDP_transport</transport_id>
<type>UDPv4</type>
<maxInitialPeersRange>10</maxInitialPeersRange>
<maxMessageSize>65000</maxMessageSize>
<sendBufferSize>1048576</sendBufferSize>
<receiveBufferSize>1048576</receiveBufferSize>
</transport_descriptor>
</transport_descriptors>
<participant profile_name="participant_profile_ros2" is_default_profile="true">
<rtps>
<name>profile_for_ros2_context</name>
<userTransports>
<transport_id>UDP_transport</transport_id>
</userTransports>
<useBuiltinTransports>false</useBuiltinTransports>
<sendSocketBufferSize>1048576</sendSocketBufferSize>
<listenSocketBufferSize>1048576</listenSocketBufferSize>
<builtin>
<initialPeersList>
<locator>
<udpv4>
<address>127.0.0.1</address>
</udpv4>
</locator>
</initialPeersList>
</builtin>
</rtps>
</participant>
<data_writer profile_name="default publisher profile" is_default_profile="true">
<qos>
<publishMode>
<kind>ASYNCHRONOUS</kind>
</publishMode>
<latencyBudget>
<duration>
<sec>0</sec>
<nanosec>1000000</nanosec>
</duration>
</latencyBudget>
</qos>
<historyMemoryPolicy>PREALLOCATED_WITH_REALLOC</historyMemoryPolicy>
</data_writer>
<data_reader profile_name="default subscription profile" is_default_profile="true">
<qos>
<data_sharing>
<kind>AUTOMATIC</kind>
</data_sharing>
<latencyBudget>
<duration>
<sec>0</sec>
<nanosec>1000000</nanosec>
</duration>
</latencyBudget>
</qos>
<historyMemoryPolicy>PREALLOCATED_WITH_REALLOC</historyMemoryPolicy>
</data_reader>
</profiles>
```
### Environment Variables
Set the following environment variables to use the custom Fast DDS profile:
```bash
export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
export FASTRTPS_DEFAULT_PROFILES_FILE=$HOME/shm_fastdds.xml
export RMW_FASTRTPS_USE_QOS_FROM_XML=1
```
This configuration aims to optimize the data flow and reduce transmission delays, improving the responsiveness and
reliability of the Orbbec camera system in a ROS2 environment.
Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 143 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 736 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 70 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 116 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 103 KiB

+108
View File
@@ -0,0 +1,108 @@
## Using Multiple Cameras with the Orbbec ROS 2 Package
This section describes how to configure and use multiple Orbbec cameras simultaneously in a ROS 2 environment.
### Identifying Camera USB Ports
#### Script to List Connected Cameras
To determine which USB ports the cameras are connected to, you can use the following bash script. This script lists all Orbbec devices attached to the system along with their USB port and serial number.
```bash
#!/bin/bash
VID="2bc5"
for dev in /sys/bus/usb/devices/*; do
if [ -e "$dev/idVendor" ]; then
vid=$(cat "$dev/idVendor")
if [ "$vid" == "${VID}" ]; then
port=$(basename $dev)
product=$(cat "$dev/product" 2>/dev/null) # product name
serial=$(cat "$dev/serial" 2>/dev/null) # serial number
echo "Found Orbbec device $product, usb port $port, serial number $serial"
fi
fi
done
```
Save this script to a file and execute it in your terminal to output a list of connected cameras.
### Launching Multiple Cameras
#### Setup for Multiple Camera Launch
You can launch multiple cameras by specifying different USB ports for each camera. Below is an example Python script that uses the ROS 2 launch system to start two cameras with individual configurations.
```python
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription, GroupAction, ExecuteProcess
from launch.launch_description_sources from ament_index_python.packages import get_package_share_directory
import os
def generate_launch_description():
package_dir = get_package_share_directory('orbbec_camera')
launch_file_dir = os.path.join(package_dir, 'launch')
launch1_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(os.path.join(launch_file_dir, 'gemini_330_series.launch.py')),
launch_arguments={'camera_name': 'camera_01', 'usb_port': '2-3.4.4.4.1', 'device_num': '2', 'sync_mode': 'free_run'}.items()
)
launch2_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(os.path.join(launch_file_dir, 'gemini_330_series.launch.py')),
launch_arguments={'camera_name': 'camera_02', 'usb_port': '2-3.4.4.4.3', 'device_num': '2', 'sync_mode': 'free_run'}.items()
)
ld = LaunchDescription([
GroupAction([launch1_include]),
GroupAction([launch2_include])
])
return ld
```
#### Running the Launch File
To execute the launch configuration for multiple cameras, use the command:
```bash
ros2 launch orbbec_camera multi_camera.launch.py
```
### Configuring the TF Tree for Multiple Cameras
#### Example TF Configuration for Two Cameras
When using multiple cameras, it's essential to calibrate them and publish a static TF tree for each camera. The following Python script configures the TF tree based on your calibration results:
```python
from launch import LaunchDescription
from launch_ros.actions import Node
def generate_launch_description():
ld = LaunchDescription([
Node(
package='tf2_ros',
executable='static_transform_publisher',
name='camera_01_tf',
arguments=['0', '0', '0', '0', '0', '0', 'base_link', 'camera_01_link'],
output='screen'
),
Node(
package='tf2_ros',
executable='static_transform_publisher',
name='camera_02_tf',
arguments=['0', '0', '0', '0', '0', '0', 'base_link', 'camera_02_link'],
output='screen'
)
])
return ld
```
Save this configuration as `multi_camera_tf.launch.py` in the launch directory of the Orbbec camera package. To run it, use:
```bash
ros2 launch orbbec_camera multi_camera_tf.launch.py
```
+109
View File
@@ -0,0 +1,109 @@
## 使用Orbbec ROS 2包配置多个摄像头
本节介绍如何在ROS 2环境中同时配置和使用多个Orbbec摄像头。
### 识别摄像头USB端口
#### 列出连接的摄像头的脚本
要确定摄像头连接到哪些USB端口,您可以使用以下bash脚本。该脚本列出了连接到系统的所有Orbbec设备及其USB端口和序列号。
```bash
#!/bin/bash
VID="2bc5"
for dev in /sys/bus/usb/devices/*; do
if [ -e "$dev/idVendor" ]; then
vid=$(cat "$dev/idVendor")
if [ "$vid" == "${VID}" ]; then
port=$(basename $dev)
product=$(cat "$dev/product" 2>/dev/null) # 产品名称
serial=$(cat "$dev/serial" 2>/dev/null) # 序列号
echo "发现Orbbec设备 $productusb端口 $port,序列号 $serial"
fi
fi
done
```
将此脚本保存为一个文件,并在您的终端中执行,以输出连接的摄像头列表。
### 启动多个摄像头
#### 多摄像头启动配置
您可以通过为每个摄像头指定不同的USB端口来启动多个摄像头。下面是一个使用ROS 2启动系统启动两个摄像头的Python脚本示例。
```python
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription, GroupAction, ExecuteProcess
from launch.launch_description_sources import PythonLaunchDescriptionSource
from ament_index_python.packages import get_package_share_directory
import os
def generate_launch_description():
package_dir = get_package_share_directory('orbbec_camera')
launch_file_dir = os.path.join(package_dir, 'launch')
launch1_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(os.path.join(launch_file_dir, 'gemini_330_series.launch.py')),
launch_arguments={'camera_name': 'camera_01', 'usb_port': '2-3.4.4.4.1', 'device_num': '2', 'sync_mode': 'free_run'}.items()
)
launch2_include = IncludeLaunchDescription(
PythonLaunchDescriptionSource(os.path.join(launch_file_dir, 'gemini_330_series.launch.py')),
launch_arguments={'camera_name': 'camera_02', 'usb_port': '2-3.4.4.4.3', 'device_num': '2', 'sync_mode': 'free_run'}.items()
)
ld = LaunchDescription([
GroupAction([launch1_include]),
GroupAction([launch2_include])
])
return ld
```
#### 运行启动文件
要执行多摄像头的启动配置,请使用命令:
```bash
ros2 launch orbbec_camera multi_camera.launch.py
```
### 配置多摄像头的TF树
#### 两个摄像头的TF配置示例
使用多个摄像头时,校准它们并为每个摄像头发布静态TF树是必不可少的。以下Python脚本基于您的校准结果配置TF树:
```python
from launch import LaunchDescription
from launch_ros.actions import Node
def generate_launch_description():
ld = LaunchDescription([
Node(
package='tf2_ros',
executable='static_transform_publisher',
name='camera_01_tf',
arguments=['0', '0', '0', '0', '0', '0', 'base_link', 'camera_01_link'],
output='screen'
),
Node(
package='tf2_ros',
executable='static_transform_publisher',
name='camera_02_tf',
arguments=['0', '0', '0', '0', '0', '0', 'base_link', 'camera_02_link'],
output='screen'
)
])
return ld
```
将此配置保存为 `multi_camera_tf.launch.py` 在Orbbec摄像头包的启动目录中。运行它,请使用:
```bash
ros2 launch orbbec_camera multi_camera_tf.launch.py
```
+53
View File
@@ -0,0 +1,53 @@
## Enabling and Visualizing Point Cloud in ROS 2
This section demonstrates how to enable point cloud data output from the camera node and visualize it using RViz2, similarly to the initial camera node setup discussed in the [Starting Camera Node](./start_camera_node.MD) document.
### Enabling Depth Point Cloud
#### Command to Enable Depth Point Cloud
To activate the point cloud data stream for depth information, use the following command:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py enable_point_cloud:=true
```
#### Visualizing Depth Point Cloud in RViz2
After running the above command, perform the following steps to visualize the depth point cloud:
1. Open RViz2.
2. Add a `PointCloud2` display.
3. Select the `/camera/depth/points` topic for visualization.
4. Set the fixed frame to `camera_link` to properly align the data.
##### Example Visualization
Here is what the depth point cloud might look like in RViz2:
![Depth Point Cloud Visualization](./images/image5.jpg)
### Enabling Colored Point Cloud
#### Command to Enable Colored Point Cloud
To enable the colored point cloud feature, enter the following command:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py enable_colored_point_cloud:=true
```
#### Visualizing Colored Point Cloud in RViz2
To visualize the colored point cloud data:
1. Launch RViz2 following the command execution.
2. Add a `PointCloud2` display panel.
3. Choose the `/camera/depth_registered/points` topic from the list.
4. Ensure the fixed frame is set to `camera_link`.
##### Example Visualization
The result of the colored point cloud in RViz2 should look similar to this:
![Colored Point Cloud Visualization](./images/image6.jpg)
+53
View File
@@ -0,0 +1,53 @@
## 在ROS 2中启用和可视化点云
本节演示如何从相机节点启用点云数据输出,并使用RViz2进行可视化,类似于之前讨论的[启动相机节点](./start_camera_node.MD)文档中的初始相机节点设置。
### 启用深度点云
#### 启用深度点云的命令
要激活深度信息的点云数据流,请使用以下命令:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py enable_point_cloud:=true
```
#### 在RViz2中可视化深度点云
运行上述命令后,执行以下步骤来可视化深度点云:
1. 打开RViz2。
2. 添加一个`PointCloud2`显示。
3. 选择`/camera/depth/points`主题进行可视化。
4. 将固定帧设置为`camera_link`以正确对齐数据。
##### 示例可视化
以下是在RViz2中可能看到的深度点云的样子:
![深度点云可视化](./images/image5.jpg)
### 启用彩色点云
#### 启用彩色点云的命令
要启用彩色点云功能,请输入以下命令:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py enable_colored_point_cloud:=true
```
#### 在RViz2中可视化彩色点云
要可视化彩色点云数据:
1. 在执行命令后启动RViz2。
2. 添加一个`PointCloud2`显示面板。
3. 从列表中选择`/camera/depth_registered/points`主题。
4. 确保固定帧设置为`camera_link`
##### 示例可视化
在RViz2中彩色点云的结果应类似于这样:
![彩色点云可视化](./images/image6.jpg)
+83
View File
@@ -0,0 +1,83 @@
## Starting the Camera Node in ROS 2
This guide provides instructions on how to launch the camera node with a colored point cloud feature enabled using ROS 2.
### Command to Start the Node
To start the camera node, execute the following command in your terminal:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py enable_colored_point_cloud:=true
```
This command initiates the camera node and enables the colored point cloud.
### Published Topics
Once the camera node is running, it will publish data on several ROS topics. Below is a list of the available topics:
- **IMU Data and IMU Information:**
- `camera/accel/imu_info`
- `camera/gyro/imu_info`
- `camera/gyro_accel/sample`
- **Color Camera Topics:**
- `/camera/color/camera_info`
- `/camera/color/image_raw`
- `/camera/color/image_raw/compressed`
- `/camera/color/image_raw/compressedDepth`
- `/camera/color/image_raw/theora`
- `/camera/color/metadata`
- **Depth Camera Topics:**
- `/camera/depth/camera_info`
- `/camera/depth/image_raw`
- `/camera/depth/image_raw/compressed`
- `/camera/depth/image_raw/compressedDepth`
- `/camera/depth/image_raw/theora`
- `/camera/depth/metadata`
- `/camera/depth/points`
- `/camera/depth_filter_status`
- `/camera/depth_registered/points`
- `/camera/depth_to_color`
- `/camera/depth_to_left_ir`
- `/camera/depth_to_right_ir`
- **Infrared Camera Topics:**
- `/camera/left_ir/camera_info`
- `/camera/left_ir/image_raw`
- `/camera/left_ir/image_raw/compressed`
- `/camera/left_ir/image_raw/compressedDepth`
- `/camera/left_ir/image_raw/theora`
- `/camera/left_ir/metadata`
- `/camera/right_ir/camera_info`
- `/camera/right_ir/image_raw`
- `/camera/right_ir/image_raw/compressed`
- `/camera/right_ir/image_raw/compressedDepth`
- `/camera/right_ir/image_raw/theora`
- `/camera/right_ir/metadata`
- **Miscellaneous Topics:**
- `/diagnostics`
- `/parameter_events`
- `/rosout`
- `/rosout_agg`
### Visualizing Data in RViz2
To view the PointCloud or Image data, use RViz2:
1. Launch RViz2.
2. Select the topic you wish to visualize from the list of published topics.
3. Add the selected topic to RViz2 to start viewing the data.
### Example Visualizations
Here are examples of how the visualization might appear in RViz2:
- **PointCloud Visualization**
![PointCloud View](./images/image1.jpg)
- **Image Data Visualization**
![Image Data View](./images/image2.jpg)
@@ -0,0 +1,82 @@
## 在ROS 2中启动相机节点
本指南提供了使用ROS 2启动启用彩色点云功能的相机节点的指令。
### 启动节点的命令
要启动相机节点,请在终端执行以下命令:
```bash
ros2 launch orbbec_camera gemini_330_series.launch.py enable_colored_point_cloud:=true
```
此命令将启动相机节点并启用彩色点云。
### 发布的主题
一旦相机节点运行,它将在多个ROS主题上发布数据。以下是可用主题的列表:
- **IMU数据和IMU信息:**
- `camera/accel/imu_info`
- `camera/gyro/imu_info`
- `camera/gyro_accel/sample`
- **彩色相机主题:**
- `/camera/color/camera_info`
- `/camera/color/image_raw`
- `/camera/color/image_raw/compressed`
- `/camera/color/image_raw/compressedDepth`
- `/camera/color/image_raw/theora`
- `/camera/color/metadata`
- **深度相机主题:**
- `/camera/depth/camera_info`
- `/camera/depth/image_raw`
- `/camera/depth/image_raw/compressed`
- `/camera/depth/image_raw/compressedDepth`
- `/camera/depth/image_raw/theora`
- `/camera/depth/metadata`
- `/camera/depth/points`
- `/camera/depth_filter_status`
- `/camera/depth_registered/points`
- `/camera/depth_to_color`
- `/camera/depth_to_left_ir`
- `/camera/depth_to_right_ir`
- **红外相机主题:**
- `/camera/left_ir/camera_info`
- `/camera/left_ir/image_raw`
- `/camera/left_ir/image_raw/compressed`
- `/camera/left_ir/image_raw/compressedDepth`
- `/camera/left_ir/image_raw/theora`
- `/camera/left_ir/metadata`
- `/camera/right_ir/camera_info`
- `/camera/right_ir/image_raw`
- `/camera/right_ir/image_raw/compressed`
- `/camera/right_ir/image_raw/compressedDepth`
- `/camera/right_ir/image_raw/theora`
- `/camera/right_ir/metadata`
- **杂项主题:**
- `/diagnostics`
- `/parameter_events`
- `/rosout`
- `/rosout_agg`
### 在RViz2中可视化数据
要查看点云或图像数据,请使用RViz2:
1. 启动RViz2。
2. 从发布的主题列表中选择您希望可视化的主题。
3. 将选定的主题添加到RViz2中开始查看数据。
### 示例可视化
以下是在RViz2中可视化可能出现的示例:
- **点云可视化**
![点云视图](./images/image1.jpg)
- **图像数据可视化**
![图像数据视图](./images/image2.jpg)