Results and Conclusion

Results

When the system was put together, we were able to load two consecutive frames of images in the SDRAM, compute the minimum sum of absolute differences within a search area. We also outlined the reference area and search box, shaded the match area with the smallest sum, and displayed the complete image on a VGA monitor. The system could track moving object or change in intensities between the two frames successfully. Since the noise floor of the sensor was quite high the system might not be sensitive enough to pick up the subtle movement, so as long as the intensity difference was large enough to be detected by the CMOS sensor, the system can accurately track the motion. Figures 1 illustrates what happened when a sharp black and white boundary was encountered. We can clearly see that the shaded area, which represents the area with the least change in the image, changed its position within the search area. Figure 2 and Figure 3 show the responses at two other interfaces. All of the pictures show that the algorithm and hardware setup were proven to work. Click Here to see a demo video.

Figure 1: The response of the system when a sharp boundary was encountered

Figure 2: Response to sharp boundaries when the camera was being turned slowly

Figure 3: Response to gray boundaries when the camera was being turned slowly

Conclusion

This project worked at video frame rate to compute exhaustive affine optical flow for small 8x8 window and an 24x24 search space, taking advantage of the inherently parallel nature of the FPGA. We did however notice that increased pixel depth provided much less reliable tracking results. We concluded this was due to the fact that their was far too much noise on the raw intensity values of the CMOS sensor (which we were aware of from our research). To make the project work much more reliably, a simple Sobel edge detector should have been applied BEFORE computer the optical flow. This would have a two fold advantage. First off, the sobel operator would significantly reduce the sensitivity to the noise of the CMOS sensor. Secondly, the optical flow calculation would now only be computed on binary images, so the algorithm could be highly optimized to use fewer LE's on the FPGA. However, using only the most significant bits proved to be relatively workable in our system.