Control an iPod with gestures

Apple’s iPod is a great product for active people. It’s small, has a long play time, and is easily loaded with customized selections of music. But it does have drawbacks for those who are physically challenged, including cold weather users.

In colder climates, avid iPod users have difficulty comfortably controlling their MP3 player. In order to manipulate the touch-based iPod controls or Apple’s standard remote control, pockets must be opened and gloves or mittens removed, losing precious heat to the cold. One possible solution is to use general body motion to control the iPod as opposed to the standard touch control. This approach requires no dexterous manipulation, only general movements.

Using gross physical movements makes controlling an iPod easier for active sports such as riding a bike - you can keep your eyes on the road and don't have to use fingers. For another group of people who lack dexterity and can't use their fingers to manipulate the iPod controls like arthritis sufferers, a movement based control system can bring control over everyday technology.

A remote control for the iPod that responds to a user’s motion requires a sensor to determine motion, a control system to interpret motions, and a remote connection to the iPod. A Memsic MXD2125 accelerometer responds to the user’s motion. It is a two dimensional accelerometer and senses acceleration in the range of -2g to 2g. It employs pulse width modulation (PWM) to output acceleration measurements. The acceleromoter measures both static and dynamic acceleration.

The sensor heats a bubble of air to provide the stimulus. An array of thermocouples detects the bubble position and thus acceleration. An Atmel ATmega8 8-bit AVR microcontroller interprets motions to determine when the user is trying to control the iPod. The microcontroller polls the accelerometer for data, interprets the readings, and sends commands to the iPod.

A Maxim MAX3233E RS232 line-level converter translates the serial output of the microcontroller to RS232 voltage levels needed by a PC's serial port during system debug. Internal charge pumps are used in the MAX3233 which reduces necessary external circuitry.

The 3rd generation iPod has a remote connector on its top next to the headphone jack.

3rd Generation iPod viewed from above.

The remote connector has 4 pins- VCC, GND, TX data, and RX data. The remote connector supplies power to the remote control as well as receives its commands. To reduce the final size of the remote control itself, the MAX3233E RS232 converter and programming circuitry is located on an external debug board. The debug board connects via a flat flex cable to the remote control board which has the accelerometer and microcontroller. The remote control has 1 red power LED and 3 general purpose LEDs connected to spare IO pins on the microcontroller.


Debug board (left) <-> Remote board (right).

Debug board and Remote board schematics may be viewed by clicking on the links.

Connecting the remote control to the iPod uses a technique originally developed by the iPod Linux project. First, a connector is fashioned using an old ISA card and some 2-row header pins. Pin connectors are crimped onto the ends of the wire then insulated with heat shrink tubing.


ISA connector-based remote connector.

The finished remote control is roughly the size of a quarter, slightly smaller than the standard Apple-supplied remote.


Finished remote control.

Reading Accelerations
The MXD2125 accelerometer has two outputs - one for X accelerations, one for Y. These accelerations are represented by the duty cycle of a square wave. A 50% duty cycle is equivalent to 0g. Duty cycle's above 50% are positive accelerations, duty cycles below 50% are negative accelerations. A formula provided by Memsic converts the duty cycle reading into acceleration. To read the accelerations, the INT0 and INT1 pins of the ATmega8 are connected to the MXD2125 outputs. These pins allow triggering of external interrupts. Additionally, they can be configured to trigger on positive or negative edges, or logic level changes. This allows the ATmega8 to read the duty cycle using the following process:

Interrupt on a positive edge, record the time

Reconfigure the pin to trigger on a negative edge

Interrupt on negative edge, record time

Disable interrupts on the pin

Repeat the process on the other input, thus reading both accelerations.

These times, in addition to the constant period of the MXD2125 output, are used to calculate the duty cycles, and thus the accelerations.


Interpreting acceleration.

Interrupt-based duty cycle calculation
Interpretation of accelerations caused by movement is the tricky part of the design. The accelerometer detects both static and dynamic acceleration in two dimensions. Therefore, it can detect movement in a plane, or, using simple geometry, the angle of tilt of the sensor. The angle of tilt is measuring what portion of Earth's gravity is acting horizontally on the sensor rather than acceleration caused by movement.

Since everyone moves at different rates while gravity is constant, it is much easier to accurately detect tilts. Thus the angle of tilt of the accelerometer is most often used in the algorithm to interpret commands. A state machine implemented in software interprets the user's motions into commands. To prevent the remote from generating random commands while running, waving, etc, the remote must be put into a "command mode" by holding the remote flat for approximately 1 second before it begins issuing commands.

Once the remote begins accepting commands, multiple commands can be issued until the remote is held still for approximately 1 more second. In general use, no commands are sent while the remote control is being moved since it is in a "Wait" state. After it is held flat for 1 second, an LED lights up signaling it is ready to accept a command. Each time the remote is tilted to the right, the iPod skips the next track and a LED on the remote flashes showing the user that a command has been sent. Each time the remote is tilted to the left, the iPod skips to the previous track and a LED on the remote flashes showing the user that a command has been sent.

The control movements can be more complex, but the less complex control algorithm demonstrates the remote's functionality. A more complex and robust control algorithm can also be implemented such as a detection algorithm that controls volume by responding to circular motions.

In order for the ATmega8 to send commands to the iPod, it must mimic the standard iPod remote control's output. The iPod remote uses a 7-byte standard 8N1 19200 baud serial protocol. Apple has bit-mapped the buttons which means that the remote can indicate that more than one button is pressed by setting more than one bit. It appears that a total of 16 buttons may be accommodated by the protocol.

Dance to control your iPod
The Motion-Based iPod remote control provides an alternative mechanism for controlling the popular Apple iPod suitable for active iPod listeners and listeners with a physical limitation of their manual dexterity. While the remote control was programmed to be worn on the user's wrist, it could easily be reprogrammed to be worn on a headphone, in a pocket, or any other place where the user can make repeatable movements. Since the iPod has very few commands, the control algorithm could be fairly simple and still effective.

Other remote controllable appliances may have more commands which would be better served by a more complex motion detection algorithm. One such addition to the linear control is circular motions. Other modifications to the Motion-Based Control include employing a wireless link to the controlled device, adding 3D accelerometers for more complex sensing of movement, adding audio feedback to assist the user in knowing what commands have been deciphered by the remote control, and adapting the control to a wide variety of appliances.

About the Author

Peter Tyser is employed by Extreme Engineering Solutions as an engineer specializing in firmware and Linux development. The iPod remote control won first prize for individuals at the University of Wisconsin Engineering Expo. Peter can be contacted at petertyser@gmail.com