by Keith Jamison (kwj5) and Morgan Winer (mhw23)
ECE 476 Final Project - Spring 2006
Monday 7:30PM Lab
|
Our project is a serial port-controlled, high resolution color television picture display.
We divided the project into the image transfer and storage portion and the television display portion. We opted for an RS-232 port for picture transmission because it involves a relatively simple, well-known protocol that will allow us to change the picture on a whim, rather than hard-coding an image into the microcontroller program. In order to reach our goals in due time, we used an external chip to generate the synchronization timing, and another external chip to generate the color TV signal from a timed set of red, green, and blue voltages. Also, we greatly simplified our microcontroller code by having all of our active code inside a single external interrupt, triggered by the synchronization generator chip. With this method, we avoid duplicating the signal bookkeeping that the external chips also keep track of. |
The high-level design can be broken down into 3 major segments: serial port communication between the PC and microcontroller, image storage and retrieval from static RAM, and TV signal generation. The following diagram illustrates the major components and interactions of the system:The microcontroller interfaces with a PC application via a RS-232 serial port to downloading an image into a static RAM chip on a whiteboard. To display pixels, the microcontroller simply deposits an address to the static RAM, which then provides the pixel to the DAC circuit. Each pixel is a single byte, allowing for 256 possible colors with 3 bits for red, 3 bits for green, and 2 bits for blue. The DAC takes these 8 bits of pixel data as input, separates out the individual colors' bits, then uses three separate, smaller resistor networks to convert each color's bits to an analog signal that is appropriate for the AD 724 chip which converts them into a single NTSC signal that is displayed on the television. The ELM304 provides a solid-white NTSC signal that is routed to both the microcontroller and the AD 724. The sync pulses from this signal trigger the line display interrupts on the microcontroller, and the blank signal is overlaid with color information by the AD 724. Design Tradeoffs Our design involved several tradeoffs between hardware and software complexity. Most notably, we opted to use software to discern horizontal and vertical syncs, rather than relying on an external sync separator. Though this required a delay in signal initiation for each line, this delay did not affect the visible portion of the television display. Furthermore, it was necessary to ensure that the color signals were turned off at the end of each line in order to preserve the integrity of the sync information. We opted for the simplest solution, which was to black out the rightmost two pixels of each line on the PC before sending it to the microcontroller, and as before this did not affect the visible portion of the display. This and other tradeoffs are discussed in greater detail in the low-level design sections that follow as well as in the conclusions. Standards Since this project exports a signal to a standard television, it must conform to the NTSC television standard. The NTSC standard has been used before, but only with black & white signals. For this project, we needed to include a 3.58 MHz sideband for the color TV signal, and also provide the red, green, and blue (RGB) values for this sideband. Our compliance with this standard is discussed in more detail in the hardware and software design sections. The RS-232 serial port protocol was used for data loading from a PC. This protocol is based on the EIA232 standard, and compliance with this standard was handled by the System.IO.SerialPort class on the PC and the MAX233 chip on our board. |
|
The circuit is largely the linking of the AtMega32 microcontroller with the ELM 304 NTSC Video Generator chip and the Analog Devices 724 RGB to NTSC encoder chip. These are also integrated with our static RAM image storage which is a Dallas Semiconductor DS1270 16 Mbit non-volatile RAM chip. (See appendix for a detailed schematic)
The most involved portion of circuitry was in building the resistor-network digital-to-analog converters. There are three of them: a 3-bit DAC for a "red" signal, a 3-bit DAC for a "green" signal, and a 2-bit DAC for a "blue" signal. As shown in the circuit schematic, each resistor network is a simple set of voltage adders, with input resistance adjusted to provide different "weights" for each bit of binary input, scaled by powers of two. For example, each input resistor has twice the resistance of the one preceding it. The output is then voltage-divided with another resistor from 5v down to 0.714v. This is because the AD724 chip expects the red, green, and blue inputs to be within the 0v -> 0.714v range. The main problem we encountered with our original circuit was that the voltage regulator recommended for use with the protoboard (the LM340LAZ-5) was not capable of providing the necessary current to power the components of the system. The LM340LAZ-5 is only capable of supplying 100mA, and with the NVSRAM alone requiring 85mA, this was far from sufficient. As a result, the regulator would get extremely hot very quickly, and none of the components on the board would function properly. We therefore removed the LM340LAZ-5 and replaced it with its larger cousin, the LM340, which supplies up to 1A and completely fixed our problem. ELM 304 The ELM 304 is central to our design, and it is important that we describe its behavior in some detail at this time. The ELM 304 has two main modes of operation, determined by the status of pin 4: pure white raster (pin 4 high) and grey color bar raster (pin 4 low). In white raster mode (the mode we used), the signal is as follows: This diagram from the NTSC's online tutorial website demonstrates the timing constraints involved in each portion of a display line of the NTSC signal. At the end of each low sync pulse (highlighted in green), the signal enters the “back porch” which lasts for 4.7µs before either displaying a new (white) line if it was a horizontal sync or continuing through the rest of the vertical sync. The chip’s output pins V1-V2 have different values during each of these periods, and V3 is not relevant in this mode: These values will be referenced later when discussing the software that interprets the signal. |
|
The software portion of this system can be broken down into four sections: initialization, the display interrupt, PC-side image preparation, and the serial port image transfer system. The first three were implemented solely on the microcontroller, while the fourth required code on both the microcontroller and PC.
Initial Design Considerations Because we were interfacing directly with 16 bits of static RAM addressing, 8 bits of static RAM data, and at least 5 bits for extra communications with chips and the serial port, we had to carefully designate Atmel pins for specific tasks. We allocated the eight port A pins for the high byte of static RAM addressing, the eight port B pins for the low byte of static RAM addressing, and the eight port C pins for bi-directional static RAM data communication. As for port D, pins 0 and 1 are reserved by the Atmel chip for serial port communication, and we needed pins 2 and 3 for external interrupt input from the ELM 304 chip. We used port D's pin 4 for controlling the static RAM's active-low write enable line. We chose to develop the PC application in Microsoft Visual C# 2005 (acquired for free via Cornell ECE’s MSDNAA license agreement), which runs on Microsoft’s .NET 2.0 framework and provides inherent functionality for loading and manipulating images as well as serial port communication. This allowed us to quickly and easily implement the functionality we desired. Mega32 Initialization Upon booting the microcontroller, the code initializes communication I/O ports A through D, and fills the static RAM with vertical color bars as a test image until an actual picture is downloaded (see results). Specific registers that are used for keeping television display state are also initialized, then external interrupt 0 is activated, and the main() function goes into an empty infinite loop. Mega32 Display Interrupt The entire screen display and serial code is contained inside of external interrupt 0. Because the external interrupt is the only active code that will be running, we end every interrupt by re-enabling interrupts and another empty infinite loop or a "sleep" command. With this method each interrupt is essentially going to interrupt itself, and we can skip the saving and restoring of microcontroller registers during the interrupt, trimming at least 8 cycles of delay per interrupt. The external interrupt is triggered at the end of the ELM 304’s sync pulse (when the V1 pin goes from low to high). The external interrupt code immediately starts counting out just over 75 cycles (at 16MHz, this is just over 4.7µs), which is the timed delay of the "back porch" part of the signal, the time between the start of a display line and the actual displaying of pixels. After this time has passed, the code checks the ELM's "V2" signal, which will be high if the ELM is displaying "white" pixels. The atmega32 then starts outputting addresses to the static RAM, thereby feeding pixels to the DAC/tv-output circuit. Since this "wait-then-check" method takes approximately 5 cycles that should be spent outputting pixels, this in essence trades almost 2 pixels off the left side of the screen that aren't normally visible anyways for a simpler display mechanism. If, on the other hand, the interrupt handler code checks the ELM's "V2" signal, and the result is a logic 0, then the Mega32 interprets this as a sign it should be in the vertical synchronization part of the TV display. The code that executes during vertical sync is the code that checks if there is an image transfer waiting on the serial port, and if so, services the request by stopping TV display and transferring the image to static RAM. PC-Side Image Preparation The PC application allows images of almost any file type (JPEG, GIF, TIF, BMP, PNG, among others) to be loaded via a Load File menu option, dragged from another application or folder, or pasted from another application via the Windows clipboard. The .NET platform provides the functionality to complete all of these tasks with one or two simple function calls. Once loaded, the image is copied into an off-screen buffer with a fixed 24 bits/pixel (1 byte each for red, green, and blue) format, which is repainted onto the screen during each repainting event. This image can be either stretched or cropped to fit the desired resolution. Before transmitting the image, the color values of each pixel must be flattened from 24bpp into the 8bpp format our system uses for storage. This is accomplished by simply truncating each channel and concatenating the most significant 3, 3, and 2 bits from the red, green, and blue values, respectively. Finally, because a byte value of 0x01 represents a “stop” signal for the microcontroller, any pixel with this color value after flattening is changed to 0x00 (black). This particular choice of stop byte and replacement value were chosen to minimize the visual impact of the change, since “almost black” merely becomes “pure black”. A “preview” option is available that allows the user to see what their image will look like after flattening. The following illustrates how such an image might look before and after this process (original image from http://tenbyten.org): One of the primary annoyances during the flattening process, is that the .NET architecture goes out of its way to keep developers from accessing the raw image data. In order to mitigate the risks associated with memory pointers, they provide a “GetPixel” function which is extremely slow for pixel-by-pixel operations. Thankfully, we found an article by Christian Graus entitled Image Processing for Dummies with C# and GDI+ Part 1 - Per Pixel Filters which details a workaround in which we enter a designated “unsafe” mode, allowing us to modify the image data directly via memory pointer. This technique sped the flattening preview process by at least an order of magnitude, though it does have the detriment of not be able to run under certain very strict security settings. As mentioned previously, the rightmost two pixels of each line are blacked out before transmitting the data. This solves a problem we encountered during testing, whereby only pictures with a black border would display properly on the screen. We eventually determined that this was because the voltages corresponding to the final pixel values of each line were remaining into horizontal sync portion of the signal. Since the AD724 merely overlays whatever color information it receives at any point in time over the white NTSC signal, this meant that the horizontal sync in the output signal to the television would be comingled with an erroneous color signal and the image could not be displayed. It would be possible to solve this by editing the Atmel code to stop feeding addresses to SRAM two pixels earlier and then manually force the DAC to zero, but our solution of blacking out the two rightmost pixels before even sending it to the microcontroller was far simpler and no less effective. In fact, since the television cuts off the edges of the signal it is fed, this technique had no visible effect on the output. Serial Port Communication Protocol The microcontroller only checks for new data during a vertical sync, and if we simply began sending our data, some of it may be lost at the beginning. Therefore, the PC application first sends a single byte to alert the microcontroller and then waits for confirmation that the microcontroller has entered “Transfer Mode” before continuing with the transfer. This was accomplished by registering a DataReceived callback function with the System.IO.SerialPort object: serialPort.DataReceived += new SerialDataReceivedEventHandler(serialPort_DataReceived);where serialPort is an instance of the System.IO.SerialPoint class, and serialPort_DataReceived is the callback function we’ve written. This function is executed in a separate thread from the main application, and while it is still capable of accessing any application-wide variables (such as the internal image buffer), in order to interact with any display elements such as the transfer progress bar, we had to declare a specialized “function delegate” (ie: .NET’s take on the function pointer), and tell the main form’s thread to invoke invoke a specific function of that delegate’s type, with whatever arguments it would like: //declarationsIn any case, when the DataReceived callback executes, it flattens the image buffer into a byte array, blacks out the right two lines, and transmits this array via a call to the System.IO.SerialPort class’s write() function. On the Atmel, after sending the reply to the PC’s start byte, the code simply loops and reads data in at desired baud until the value 0x01 is found, or 64k of static RAM has been filled. The “stop” value check for 0x01 was implemented so that a transfer could be cancelled in the middle if so desired. Once a transfer is cancelled, both the Atmel and the PC application resume their normal non-transfer behavior, and can initiate a new transfer at any time. When a transfer is complete, the Atmel code finishes by clearing the serial port read buffer, then resumes television display with the new picture. |
|
Image Quality In our final design, we were able to obtain completely stable images with a broad range of vivid colors, as can be seen in the color bar image we load into SRAM at initialization: And in the following images captured from the screen: Though the color flattening was noticeable on many of the images we displayed, we are clearly capable of displaying enough colors for a video game system, such as the Nintendo scheme we had originally envisioned. Transmission Speeds The highest transmission rate for which image integrity was maintained was 57.6 Kbit/s. At this speed, a complete 256x224 8bpp image took approximately 8 seconds to download. Attempts were made to transmit at higher speeds, but anything above 57.6 Kbit/s distorted the colors in the same way and to the same extent. This distortion the same at 115.2 Kbit/s (still below 120 Kbit/s, maximum rate for the MAX233 chip, according to its datasheet) as it was at 1Mbit/s (the maximum rate the RS-232 protocol allows). Testing Methods Throughout the project, we employed mostly oscilloscope-oriented testing methods to debug issues in our circuit. Our circuit will only display anything if everything works at once, so we could not do partial testing like we would have preferred. When an issue arose, we would begin by testing the output of the ELM 304's V1 pin. If it was still generating 63.5 us, 5v sync pulses, then we would know that the ELM was not at fault. We would next move to the DAC circuit. If the ELM was functioning, then the ATMega32 would be providing proper output, and we would only know that if voltages were appearing on the DAC pins. If the DAC was indeed functioning correctly, then we would progress to the AD724's outputs, and try to see precisely the signal that the television was being fed, and continue debugging from there. Since the non-volatile RAM had troublesome connections at times, we would then have to resort to disabling the video RAM altogether and generate a "vertical color bars" test pattern. This would be generated directly from the ATMega32, and skip the RAM altogether. With such a setup, we were able to better diagnose circuit problems by eliminating the RAM and ATMega32 as sources of trouble. |
|
Our design exactly matched the goal of projecting high-resolution images on a color television. The pictures display in excess of 250 distinct colors, and can be updated any number of times without having to restart the microcontroller.
If we did have to repeat the project though, we would plan far further ahead with the video RAM chips. Sampling and working with the RAM chips by far caused us the most headaches, and we were relieved when we finally found the DS1270 chip solution. Additionally, if this project were redone, we would find a way to transfer images via the serial port without interrupting the television display. We would have to split transfers across multiple display frames, even if we were to upgrade the serial driver to a higher baud rate. We would also make use of excess RAM storage for multiple images, possibly animation. Our design uses the Maxim Max233CPP serial driver for RS-232 communication, which is fully compliant with the EIA232 standard that it conforms to. Since our project generated an NTSC color video signal, we of course have to conform to the NTSC standard too. The NTSC standard was developed by the National Television System Committee, and is a separate, established television standard that differs from the alternate PAL and SECAM television standards. We chose to implement NTSC over PAL or SECAM because NTSC is the standard that is most prevalent in the US. Our project's microcontroller code is fully original, and completely of our own design. The circuit schematic we built is loosely based off the circuit of Alan Levy's creation at Cornell University while working with Prof. Bruce Land on color television generation. However, our circuit extends upon his posted design by including a non-volatile RAM chip, and a more complicated digital-to-analog converter. With respect to the translation of NES code, the information that the translator was written with was the sum of what a large league of hobbyists had learned about the NES in the years since its release. This information is open to the public and published on the Internet. We in no way listed our project as a competitor or "knock-off" of the NES, nor do we have any trademark-infringing name for it. The IEEE Code of Ethics deal with responsibility, honesty, integrity, and justice throughout the process of electrical engineering. With the code in mind, our project was completely uninvolved with any health issues, social bias or libel, or dishonor. We were honest with manufacturers when ordering sample parts for academic use. When fellow students asked for assistance we always helped when possible, and we credit those we accepted help from. And finally, we feel our project appropriately encourages the understanding of technology, and how it was, is, and will be integrated into society. As an example, we were presented with many questions as to how we generated the television signal, and how we would deal with the issue of NES game copyright. Rather than attempting to keep the television signal generation method to ourselves, we openly and freely shared information with whomever asked. Others who had worked on color television before have begun incorporating our method with their own now because of this collaboration. Also, we stated at the time of the project proposal that we do not condone or encourage the infringing of copyrighted material, in this case NES games. There are a lot of NES games that people still own, and they have the right to play those games, whether on an NES, or an extended version of our project. Without ownership though, a user would be at fault both ethically and morally. So close, yet so far... One of the original extended goals of this project was to actively translate and play old Nintendo Entertainment System (NES) games on the ATMega32 microcontroller. The NES used a modified 6502 microprocessor for running the game code, combined with a customized sprite-based graphics processing chip to display a few dozen colors at a time on a full-screen color television. We have made significant progress towards this lofty goal, though are far from completing it. Several of the sources we found most useful in devising our solution are listed under the references portion of this document. We finished a binary machine code translation program from NES game code into ATMega32 machine code. The 6502-like processor inside the NES shared many of the same features and instruction types as the ATMega32, but unlike the ATMega, it also supports 13 memory addressing modes. The translator is essentially a pack of lookup tables, with the actual 8-bit NES instruction as the table index, and the result being the ATMega32 instructions written to file. For a test file of 16KB of NES code, the translator resulted in approximately 48KB of ATMega32 code. Considering that an ATMega32 opcode is 2 bytes to the NES' 1-byte, this means that the translator generated on average 1.5 instructions for every instruction input. Since the NES was run at about 1.7 MHz and the ATMega32 runs at 16 MHz, this is an acceptable translation result as far as speed is concerned. The game sprite graphics are much more easily decoded, and would function in software much the same way that the original NES handled them in hardware. This involves displaying a background of 8x8 sprites, and overlaying the characters, all while scrolling the display left and right. It is anticipated that equivalent operations could be completed without noticeable slowdown. If fully implemented, this would require a pair of ATMega32 microcontrollers running in tandem: one for the NTSC video signal creation, and one to run the game code. This would also require that the two processors share access to the video RAM, such that the game can update the video display during NTSC vertical sync timing segments. Furthermore, the translator would need to properly adjust memory locations and program offsets, which it currently does not. |
Commented Source Code
|
