Head-Tracking Robot

Introduction

While the Lego Mindstorms kit typically aims to introduce children to the basics of engineering and coding, my project took it a few steps further. By pairing a Lego Mindstorms NXT robot with the accelerometer in an Android smartphone, I created a robot whose head mimics the tilt and turn of the phone. This experiment was an exploration in real-time sensor data processing, robotic control, and communication protocols.

Android Smartphone Accelerometer + LeJOS Lego Mindstorm V2

Decoding a Wii Nunchuck accelerometer with Arduino

Technologies Used

  • Hardware: Lego Mindstorms NXT, Android smartphone, Nintendo Wii Nunchuck, Arduino, Protoboard
  • Software: LeJOS (Java for Lego Mindstorms)
  • Communication: Bluetooth protocol

Project Goals

  1. Develop a real-time head-tracking robot using Lego NXT and smartphone accelerometer.
  2. Decode the Nintendo Wii Nunchuck protocol for use in similar head-tracking applications.
  3. Implement a solution for relative head rotation tracking against full-body rotation.

Challenges Faced

  • Limited NXT Capabilities: The Lego NXT needed to be hacked to unlock standard coding features.
  • Protocol Decoding: Both the Bluetooth connection and the Wii Nunchuck protocol needed to be decoded.
  • Sensor Limitations: The accelerometers could not distinguish between full-body rotation and head-only rotation.

Solutions and Approaches

  • LeJOS Installation: I hacked the Lego NXT to run LeJOS, unlocking more advanced programming capabilities.
  • Protocol Research: Invested time in decoding the Bluetooth protocol for NXT-smartphone communication and the protocol for the Wii Nunchuck.
  • Sensor Fusion: Utilized sensor data from both the smartphone and Lego NXT to implement real-time head tracking.

Key Features

  • Bluetooth Communication: Real-time data transfer between the Android smartphone and the Lego robot.
  • Sensor Integration: Use of smartphone accelerometer for head-tracking.
  • Protocol Decoding: Successfully decoded and utilized the Wii Nunchuck protocol.

Outcomes and Impact

  • Successful Implementation: Both the smartphone-controlled and the Nunchuck-controlled versions were successfully developed.
  • Sensor Fusion: Demonstrated a proof-of-concept for integrating different types of sensor data for robotic control.

Lessons Learned

  • Hardware Limitations: The constraints of using consumer-grade or ‘toy’ hardware for complex tasks became evident.
  • Protocol Mastery: Gained valuable experience in decoding and understanding communication protocols.

Future Directions

  • Relative Rotation: Plan to implement a third pair of accelerometers/gyroscopes to distinguish between head-only and full-body rotation.

Release Date

2013