LoboVault Home

The development of a robotic test bed with applications in Q-learning


Please use this identifier to cite or link to this item: http://hdl.handle.net/1928/17331

The development of a robotic test bed with applications in Q-learning

Show full item record

Title: The development of a robotic test bed with applications in Q-learning
Author: Appel, Titus
Advisor(s): Fierro, Rafael
Committee Member(s): Oishi, Meeko
Lumia, Ron
Department: University of New Mexico. Dept. of Electrical and Computer Engineering
Subject: Q-Learning
reinforcement learning
LC Subject(s): Autonomous robots--Design and construction.
Reinforcement learning.
Feedback control systems.
Intelligent transportation systems--Design and construction.
Degree Level: Masters
Abstract: In this work, we show the design, development, and testing of an autonomous ground vehicle for experiments in learning and intelligent transportation research. We then implement the Q-Learning algorithm to teach the robot to navigate towards a light source. The vehicle platform is based on the Tamiya TXT-1 chassis which is outfitted with an onboard computer for processing high-level functions, a microcontroller for controlling the low-level tasks, and an array of sensors for collecting information about its surroundings. The TXT-1 robot is a unique research testbed that encourages the use of a modular design, low-cost COTS hardware, and open-source software. The TXT-1 is designed using different modules or blocks that are separated based on functionality. The different functional blocks of the TXT-1 are the motors, power, low-level controller, high-level controller, and sensors. This modular design is important when considering upgrading or maintaining the robot. The research platform uses an Apple Mac Mini as its on-board computer for handling high-level navigation tasks like processing sensor data and computing navigation trajectories. ROS, the robot operating system, is used on the computer as a development environment to easily implement algorithms to validate on the robot. A ROS driver was created so that the TXT-1 low-level functions can be sensed and commanded. The TXT-1 low-level controller is designed using an ARM7 processor development board with FreeRTOS, OpenOCD, and the CodeSourcery development tools. The RTOS is used to provide a stable, real-time platform that can be used for many future generations of TXT-1 robots. A communication protocol is created so that the high and low-level processors can communicate. A power distribution system is designed and built to deliver power to all of the systems efficiently and reliably while using a single battery type. Velocity controllers are developed and implemented on the low-level controller. These control the linear and angular velocities using the wheel encoders in a PID feedback loop. The angular velocity controller uses gain scheduling to overcome the system’s nonlinearity. The controllers are then tested for adequate velocity response and tracking. The robot is then tested by using the Q-Learning algorithm to teach the robot to navigate towards a light source. The Q-Learning algorithm is first described in detail, and then the problem is formulated and the algorithm is tested in the Stage simulation environment with ROS. The same ROS code is then used on the TXT-1 to implement the algorithm in hardware. Because of delays encountered in the system, the Q-Learning algorithm is modified to use the sensed action to update the Q-Table, which gives promising results. As a result of this research, a novel autonomous ground vehicle was built and the Q-Learning source finding problem was implemented.
Graduation Date: December 2011
URI: http://hdl.handle.net/1928/17331

Files in this item

Files Size Format View Description
Appel-Robot_Q-Learning.pdf 6.993Mb PDF View/Open Thesis

This item appears in the following Collection(s)

Show full item record

UNM Libraries

Search LoboVault


My Account