Lab6: Camera and Line Following
Introduction
In this lab, you will learn how to enable the Mini Pupper camera, view images remotely via WSL, and implement a line-following algorithm using ROS2.
Prerequisites
- Completed WSL Setup (see Preparation > WSL Setup)
- Completed Lab1-Lab5
- Mini Pupper with camera connected
- ROS2 installed on both WSL and Mini Pupper
Part 1: Enable Camera in Mini Pupper
Step 1: Modify Configuration File
Edit the Mini Pupper configuration file:
nano ~/ros2_ws/src/mini_pupper_ros/mini_pupper_bringup/config/mini_pupper_2.yaml
Set camera to true:
sensors:
lidar: false
imu: true
camera: true
ports:
lidar: '/dev/ttyAMA1'
Step 2: Copy Camera Calibration File
mkdir -p ~/.ros/camera_info
cp ~/mini_pupper_ros/mmal_service_16.1.yaml ~/.ros/camera_info
Part 2: Configure Camera Settings
Option 1: Using v4l2-ctl
Set camera resolution directly:
# Set to 160x120 resolution
v4l2-ctl --device=/dev/video0 --set-fmt-video=width=160,height=120,pixelformat=YUYV
# Verify the format was set
v4l2-ctl --device=/dev/video0 --get-fmt-video
Option 2: Modify Camera Launch File
Edit mini_pupper_driver/launch/camera.launch.py (around line 39):
parameters=[
{
'output_encoding': output_encoding,
'image_size': [160, 120], # Lower resolution for better performance
'frame_rate': 30,
}
],
Step 3: Rebuild the Package
cd ~/ros2_ws
colcon build --packages-select mini_pupper_driver
source install/setup.bash
Part 3: Launch Mini Pupper with Camera
On Mini Pupper
Launch all hardware with drift correction:
export ROS_DOMAIN_ID=30
source ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_bringup bringup.launch.py \
hardware_connected:=True drift_correction:=0.1
Part 4: View Camera Image
Option 1: Display on Mini Pupper LCD
export ROS_DOMAIN_ID=30
source ~/ros2_ws/install/setup.bash
ros2 run mini_pupper_driver display_interface --ros-args \
-r mini_pupper_lcd/image_raw:=/image_raw
Note: This runs image processing on the Mini Pupper, which may affect performance.
Option 2: View on WSL (Recommended)
Step 1: Download and Install Webcam Package
Download mini_pupper_webcam.zip on your PC/WSL.
cd ~/ros2_ws/src
unzip mini_pupper_webcam.zip
cd ~/ros2_ws
colcon build --packages-select mini_pupper_webcam
source install/setup.bash
Step 2: Open Firewall for Web Viewer
In PowerShell (Admin):
New-NetFirewallRule -DisplayName "ROS2 Webcam TCP 5000" -Direction Inbound -Protocol TCP -LocalPort 5000 -Action Allow
New-NetFirewallRule -DisplayName "ROS2 Webcam TCP 5001" -Direction Inbound -Protocol TCP -LocalPort 5001 -Action Allow
Step 3: Launch Web Viewer
export ROS_DOMAIN_ID=30
source ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_webcam single_viewer.launch.py
Open browser: http://localhost:5000
Part 5: Line Following
The line following code uses camera input to detect and follow a line on the ground.
Launch Line Following
In a new terminal (on Mini Pupper or WSL):
export ROS_DOMAIN_ID=30
source ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_recognition recognition.launch.py
To disable PID control:
ros2 launch mini_pupper_recognition recognition.launch.py pid:=false
How It Works
- Camera captures image
- OpenCV processes image to detect line
- PID controller calculates steering correction
- Velocity commands sent to robot
Architecture Diagram
┌─────────────────────────────────────────────────────────────────────┐
│ WSL (PC) │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Web Viewer │ │ RViz │ │ Teleop │ │
│ │ (Port 5000) │ │ Visualization │ │ Keyboard │ │
│ └────────▲────────┘ └────────▲────────┘ └────────┬────────┘ │
│ │ │ │ │
└───────────┼──────────────────────┼──────────────────────┼──────────┘
│ │ │
│ ROS2 Topics (ROS_DOMAIN_ID=30) │
│ WiFi Network │
│ │ │
┌───────────┼──────────────────────┼──────────────────────┼──────────┐
│ │ │ ▼ │
│ ┌────────┴────────┐ ┌────────┴────────┐ ┌─────────────────┐ │
│ │ /image_raw │ │ /joint_states │ │ /cmd_vel │ │
│ │ Camera Topic │ │ Robot State │ │ Velocity Cmd │ │
│ └────────▲────────┘ └────────▲────────┘ └────────┬────────┘ │
│ │ │ │ │
│ ┌────────┴──────────────────────┴──────────────────────▼────────┐ │
│ │ Mini Pupper Bringup │ │
│ │ (Camera, IMU, Motor Controllers, Line Following) │ │
│ └───────────────────────────────────────────────────────────────┘ │
│ Mini Pupper │
└────────────────────────────────────────────────────────────────────┘
Exercises
Exercise 1: Custom Line Color
Modify the line following code to detect a different color line (e.g., blue instead of black).
Exercise 2: Adjust PID Parameters
Experiment with different PID values to improve line following performance.
Exercise 3: Add Visual Feedback
Display the detected line on the web viewer with overlay graphics.
Troubleshooting
| Issue | Solution |
|---|---|
| No camera image | Check camera: true in yaml, verify /dev/video0 exists |
| Topics not visible on WSL | Verify ROS_DOMAIN_ID matches, check firewall |
| Line following not working | Ensure good lighting, adjust color thresholds |
| Web viewer not accessible | Open TCP port 5000 in Windows Firewall |
Debugging Commands
# Check camera topic
ros2 topic list | grep image
ros2 topic hz /image_raw
# Check ROS2 communication
ros2 node list
ros2 topic list
# View ROS2 graph
rqt_graph
Summary
In this lab, you learned:
- How to enable and configure the Mini Pupper camera
- How to view camera images on WSL via web interface
- How to implement line following with ROS2
- How to set up WSL and Mini Pupper communication