Robot Control using Raspberry Pi CM4

Workshop - A minimal introduction to Minipupper V2

Li Bai

lbai@temple.edu

Temple University

October 6, 2024

System Requirements

One PC Windows 11 (Home edition or Professional edition)
32GB MicroSD Card
Minipupper V2

Process of the Day

flowchart LR
  A(("Software setup")) --> B(("Network setup"))
  B --> C(("Hardware check: camera and audio"))
  C --> D(("Google Gemini"))
  C --> E((" Linux ROS2 "))
  E --> F(("camera view"))
  E --> G(("line following"))
  
  %% Define custom styles for each node
  classDef nodeStyle width:150px, text-align:center, white-space:pre-wrap;
  classDef coloredNode fill:#1f96,stroke:#333,stroke-width:2px,width:150px,white-space:pre-wrap,text-align:center;


  %% Apply the custom style to the nodes
  class A,C,D,E,F,G nodeStyle;
  class B coloredNode;
  
  %% Add slide references to each block (adjust according to your presentation setup)
  click A "#software" "Go to Software setup slide"
  click B "#network" "Go to Network setup slide"
  click C "#hardware" "Go to Hardware slide"
  click D "#gemini" "Go to Google Gemini slide"
  click E "#ros2" "Go to ROS slide"
  click F "#ros2" "Go to ROS slide"
  click G "#linefollowing" "Go to ROS line following slide"

Install VScode

edit file in text mode

Install Window Terminal

seems to be better than power shell

Install Balena Etcher

Download Balena Etcher https://etcher.balena.io/ to copy the new image to your MicroSD card

Install WSL

run cmd in administrative mode
setup username and password

WSL folders on Windows

Setup Raspberry Pi - Balena Etcher (preflashed)

Download Balena Etcher https://etcher.balena.io/ to copy the new image to your MicroSD card

Check to ensure your SD card is selected to flash to SDHC, NOT your computer!

NOTE: the following step will not apply to you if you own a computer using Windows

Connection Methods

method 1: Mobile hotspot on Windows 11/10
- easy, but cannot work between wsl and minipupper
method 2: Wireless router
- work with ROS operation (WSL & robot), do not require mutlicast enable on the router
method 3: eduroam (need to generate a hash in ubuntu 18.04)
- not work with ROS2 operation (WSL & robot), but require mutlicast enable on the router

open power shell and

cd ~/Downloads
wget https://raw.githubusercontent.com/lbaitemple/mini-pupper-aws/refs/heads/fall2024/wifipasswd/wireless.txt -O wireless.txt
code wireless.txt

make sure you see a new driver system-boot

Wireless router - method 2

Setup
wireless.txt
file
insert card
test

open window terminal (wireless.txt downloaded already)

cd ~/Downloads
code wireless.txt

get TU wireless credential in class

make sure you see a new driver system-boot

eduroam - method 3

Generating a Password Hash (case sensitive, space sensitive)
Passowrd Hash: cryptographic representation of a password

Generate your Password Hash

wsl
hash
credential
mv file
insert card
IP
test

Now we open a new power shell terminal to setup another wsl distro

wsl --install ubuntu-18.04

echo -n password | iconv -t UTF-16LE | openssl md4

replace “password” in command line with your password used to log in to school organization email
enter to regenerate unique hash

save value for your WiFi access (in google doc or notebook)!

Line 1: use your school email address
Line 2: use the hash passcode that you previously saved in a document or notebook

make sure you see a new driver system-boot

IP address does not show up on first time boot up, wait for power on sound, then disconnect power and reconnect

reboot second time, you can see the IP address

If you see the IP address on the screen in the second time power on, type http://[ipaddress]:8080 for a web remote control

Web Control - Activate the robot

Web Control - Make it for dance move

Web Control - Make it for walk

Revise the WiFi in Jupyter Lab

Jupyter Lab
WiFi Configuration
Caution

Open a browser, type http://[ipaddress]:8888
password mangdang

sudo nano /etc/netplan/50-cloud-init.yaml

at least keep one connection working
50-cloud-init.yaml is indentation sensentive, any mistake will have no IP address shown
If file changed, try the following commands to ensure if you have any errors before reboot

sudo netplan apply
sudo netplan generate --debug

the worst sitation is to use a HDMI monitor and a USB cable to troubleshoot the yaml file

Minipupper Powered On

which method is better?
- best method 2 (need have multicast enabled),
- easy method 1 (not always reliable)
- good method 3 (no multicast and school dependent)
Common problem with WSL not seeing the robot in the LAN
- WSL & robot operation - bridge vs nat mode
- nat mode create a seperated LAN (no good)
- bridge mode so robot can talk with WSL

WSL Mirror Mode

config
content
reset
verify

[wsl2]
networkingMode=mirrored

wsl --shutdown

wslinfo --networking-mode

Firewall enable port (multicast)

ROS2 UDP port Access

Hardware Check

download the mangdang using the following command

git clone http://github.com/lbaitemple/mangdang

setup
camera check
audio check

run ieee2024.ipynb

run camera_test.ipynb
you check camera by running each cell in jupyter notebook

record 5 second audio and play back

cd ~/mini_pupper_bsp/demos/
python audio_test.py

Gemini

download the code to

cd ~
git clone https://github.com/lbaitemple/apps-md-robots
cd gemini-md-bot
pip install -r requirements.txt

add credential in env.sample

cp env.sample .env
python gemini-md-bot.py

ROS2

ROS2 can be installed on minipupper and WSL
ensure multicast port open in your Wi-Fi network

ROS2 Installation

install on WSL
install on Minipupper

git clone https://github.com/mangdangroboticsclub/mini_pupper_ros.git -b ros2-dev mini_pupper_ros
cd mini_pupper_ros
bash ./pc_install.sh

git clone https://github.com/mangdangroboticsclub/mini_pupper_ros.git -b ros2-dev mini_pupper_ros
cd mini_pupper_ros
bash ./pupper_install.sh

Copy camera ymal file to .ros

mkdir -p ~/.ros/camera_info
cp ~/mini_pupper_ros/mmal_service_16.1.yaml ~/.ros/camera_info

enable camera in config.yaml as follows

sensors:
  lidar: false
  imu: true
  camera: true
ports: 
  lidar: '/dev/ttyAMA1'

Code revised and Recomplie

cd ~\ros2_ws
colcon build

colcon build --packages-select  mini_pupper_driver

Demo (cmd_vel)

ensure your computer and laptop on the router network
prefer ROS_DOMAIN_ID is setup differnetly for different peole

minipupper
PC (WSL)

export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
. ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_bringup bringup.launch.py

export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
ros2 topic list

if you see /cmd_vel, run

ros2 topic pub --once /cmd_vel geometry_msgs/msg/Twist "linear:
  x: 1.0
  y: 0.0
  z: 0.0
angular:
  x: 0.0
  y: 0.0
  z: 0.0"

Music logic

sequenceDiagram
  participant Minipupper
  participant Router
  Minipupper->>WSL: Hello WSL, I run the music server!
  loop Healthcheck
    WSL->>WSL: Let me check<br> ros2 service list (sub) <br> /play_music  /stop_music exist
  end
  Note right of WSL: Find ROS <br/>services!
  WSL-->>Minipupper: Great! call /play_music 
  WSL->>Minipupper: base on what I can hear, call /stop_music
  Minipupper-->>WSL: help me check red dot so I can stop!

Demo (music_service)

ensure your computer and laptop on the router network

minipupper
PC (WSL)

install dependcies

sudo apt-get install ffmpeg portaudio19-dev -y
pip install pydub pyaudio pydub

start music server

export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
. ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_music music.launch.py

export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
. ~/ros2_ws/install/setup.bash
ros2 service list

if you see /music_servrice, run

ros2 service call /play_music mini_pupper_interfaces/srv/PlayMusic "{file_name: 'robot1.mp3', start_second: 3}"

stop music

ros2 service call /stop_music mini_pupper_interfaces/srv/StopMusic

Line following logic

sequenceDiagram
  participant Minipupper
  participant Router
  Minipupper->>WSL: Hello WSL, I run the bringup!
  loop Healthcheck
    WSL->>WSL: Let me check<br> cmd_vel (sub)<br> and image_raw (pub)
  end
  Note right of WSL: Find ROS <br/>topics!
  WSL-->>Minipupper: Great! subscribe image_raw, publish cmd_vel 
  WSL->>Minipupper: base on line shown, here is your cmd_vel
  Minipupper-->>WSL: help me check red dot so I can stop!

Demo (line following)

ensure your computer and laptop on the router network
make sure your camera works

minipupper
check camera (WSL)
PC (WSL)

start bring up

export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
. ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_bringup bringup.launch.py

check image_raw toic

export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
ros2 topic list | grep image

if image_raw exists, run the following

ros2 run rqt_image_view rqt_image_view

recognition node to recognize line, pid:=true

. ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_recognition recognition.launch.py #pid:=false if want to disable pid

Dance logic

sequenceDiagram
  participant Minipupper
  participant Router
  Minipupper->>WSL: Hello WSL, I run the bringup!
  Minipupper->>WSL: Hello WSL, I run the music_service!
  Minipupper->>WSL: Hello WSL, I run the dance_service!
  loop Healthcheck
    WSL->>WSL: Let me check<br> cmd_vel (sub)<br> music_service 
  end
  Note right of WSL: Find ROS <br/>topics and services!
  WSL->>Minipupper: ros2 service list

Demo (Dance)

ensure your computer and laptop on the router network
make sure you launch bring and music service on minipupper

minipupper
minipupper (dance)
PC (WSL)

start bring up

# terminal 1
export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
. ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_bringup bringup.launch.py

music service

# terminal 2
export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
. ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_music music.launch.py

dance service

# terminal 3
export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
. ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_dance dance.launch.py

check what service avaiable

export ROS_LOCALHOST_ONLY=0
export ROS_DOMAIN_ID=0
. ~/ros2_ws/install/setup.bash
ros2 service list