In this project, I created a real time human upper body tracking system based on skin detection using Altera's DE2 board, a VGA monitor and a camera. Video streams were obtained from a camera, filtered, averaged and stored in a down sampled memory block. The down sampled frames were used to compute the location of the head and arms of the user. Movements of the head, chest and arms are displayed on VGA screen as a 3D robot constructed from several 3D boxes. The 3D projections are changed based on the user's view of the camera to create a more realistic 3D robot. The user can change the current VGA view through a set of switches. The resulting system is able to mimic the user's real time body movements.