High Level Design

This project consisted of two main parts: face detection and drawing projections of a 3D cube onto 2D space. The following diagram shows the overall schematic of our design:

Face Detection

There are many different algorithms to track human faces. One popular method tries to express images as a linear combination of reference faces or eigenfaces. Other methods use the color content of images to determine which pixels can represent human skin. We chose to use a color algorithm because we did not believe that we had enough memory to implement our project and store a large set of reference faces.

For each video frame, we determined which pixels represented human skin based on the red and green color components. We performed various averaging operations to remove noise effects and to remove details like glasses and moustaches that are not skin tone but may affect face detection. The face’s location was determined and passed to the Nios II to perform the projection calculations.

Projection and Display

The projection calculations were performed on the Nios II so they could be done in floating point. The Nios took the offset from the user’s face to the center of the VGA screen as input and returned the coordinates of the 8 corners of the projected cube. The cube was then displayed using a Bresenham line drawing algorithm in hardware to connect these 8 points.

Hardware/Software Tradeoffs

We had to optimize our C code to have enough memory to run our project. The Terasic camera modules used the SDRAM as a frame buffer, and they used a complicated SDRAM controller. We were using SRAM to store the 12 line segments of the projected cube to display. We originally wanted to use SDRAM for the Nios II as well, since it had the most capacity. Unfortunately, we found that the Nios II could not share the SDRAM with the camera. We decided to use the on-chip memory for the Nios II, since that seemed easier than deciphering the SDRAM controller and storing the frames elsewhere. The biggest problem with on-chip memory was its size. We could not make allocate too much memory to the Nios II, otherwise our entire project would not fit on the FPGA, but we still needed to allocate enough memory for our C program. We were forced to optimize our C program to reduce how much memory was required. Therefore, in order to avoid importing math.h, we replaced the trigonometric functions we needed with simpler algorithms that only require arithmetic operations (see Appendix). We also compressed the other libraries we used. This limits the functionality of the Nios II, but we only needed basic pointer and arithmetic operations.