The inference of neural networks is very complicated. It is very unwise to simply connect different PEs and assume data flow will be flawless. For this reason, in our design, we added a storage unit (M10K in our case) after each PE to store the results. Every PE will be reading from the M10k memory in the layer before it and writing to the M10k memory after it. This way we can both instantiate the M10k memory with initialized value and test different PEs properly and ensure all the parts work correctly when integrated. As for small constants such as a bias for each PE, we simply used a register and directly connected it to the PEs.
One thing to notice about the M10k design in Altera FPGAs is that the M10k block used in all of their FPGAs incorporates an internal latch at the input ports. Therefore, there will be at least a cycle of delay in the reading process. Altera M10k blocks have 2 different modes of reading: synchronous read and asynchronous read. Synchronous read means the data will be ready at the q port on the next posedge of the clock if the input is latched in the internal latch at the current cycle's posedge of the clock. This means there will be 2 cycles of delay until the data is available after the address shows up on M10k's input address port. Asynchronous read means the data will be ready at the q port as soon as it is latched, with a little combinatory delay. Therefore, unless the clock cycle is too fast and the combinatory delay causes a setup timing error, the data is valid 1 cycle after the address shows up on M10k's input address port.
By default, M10k modules instantiated in Qsys are in synchronous mode. This is the case because Altera needs a way to analyze the timing of their M10ks to ensure they would function with the proposed design. This is also the case when instantiating M10k memory using Altera IP core in Quartus' IP catalog. However, in MegaWizard Plug-in Manager, the reading mode can be changed by changing the "Read output port(s)" selection under the section "Regs/Clkens/Adrs" if needed. The images below show the selection for the two modes:
The schematic on the left top corner of the wizard gives a nice idea of how the inputs and the outputs are latched or not latched.
When writing your own Verilog code of M10k modules and relying on the compiler to infer an M10k module, it becomes tricky. The following is an example Verilog code provided to infer the M10k module:
module M10K#(
parameter ITE_NUM = 100,
parameter DATA_WIDTH = 10,
parameter ADDR_WIDTH = 10
)
(
output reg [DATA_WIDTH - 1:0] q,
input [DATA_WIDTH - 1:0] d,
input [ADDR_WIDTH - 1:0] write_address, read_address,
input we, clk
);
// force M10K ram style
// 307200 words of 8 bits
reg [DATA_WIDTH-1:0] mem [ITE_NUM-1:0] /* synthesis ramstyle = "no_rw_check, M10K" */;
always @ (posedge clk) begin
if (we) begin
mem[write_address] <= d;
end
q <= mem[read_address]; // q doesn't get d in this clock cycle
end
endmodule
Since the hardware must reflect the functionality described by Verilog, when inferring M10k, Quartus will use the corresponding mode that matches the Verilog. Since it is not standard to write a pipeline at inputs when using HDL to describe memory and memories are usually designed to be synchronous reads, Quartus will infer the M10k in asynchronous read mode to ensure the behavior of the M10k follows the Verilog code. To make M10k be inferred in synchronous read mode, an extra pipeline at the output need to be added like the following:
module M10K#(
parameter ITE_NUM = 100,
parameter DATA_WIDTH = 10,
parameter ADDR_WIDTH = 10
)
(
output reg [DATA_WIDTH - 1:0] q,
input [DATA_WIDTH - 1:0] d,
input [ADDR_WIDTH - 1:0] write_address, read_address,
input we, clk
);
// force M10K ram style
// 307200 words of 8 bits
reg [DATA_WIDTH-1:0] mem [ITE_NUM-1:0] /* synthesis ramstyle = "no_rw_check, M10K" */;
reg [DATA_WIDTH - 1:0] out_q;
always @ (posedge clk) begin
if (we) begin
mem[write_address] <= d;
end
out_q <= mem[read_address]; // q doesn't get d in this clock cycle
q <= out_q;
end
endmodule
One drawback of our design is that Quartus can only generate memory units of the size of 2's exponent. This means if we want to separate all the results and parameters and put them into individual memory units, we are wasting lots of memory. For example, there are 750 weights for the fully connected layer and since 750 is larger than 512, we had to instantiate an M10k memory of size 1024 to hold all the weights.
Additionally, we chose our data type to be 27 bits wide to fully utilize the DSP blocks. However, Quartus can only generate memory units with a width of 20 or 32. This means we are wasting about 16% of our memory no matter what.
Nonetheless, even with all these wastes of memory, we still only occupied less than 3% of the memory on the DE1-SOC board with our design. This suggests that a much more complicated neural network can be implemented on this board without ever needing to worry about running out of memory.
