Computation Bit Width Growth - 2.3 English

Video Processing Subsystem Product Guide (PG231)

Document ID
PG231
Release Date
2022-04-27
Version
2.3 English

Full precision (DATA_WIDTH+16+log2(NTaps) bits) is maintained during the horizontal and/or vertical FIR convolution operation.

FIR filter outputs are rounded to DATA_WIDTH bits by adding half an output LSB in the full precision domain prior to truncation. Clipping and clamping of the output data prevents overflows and underflows. Data is clipped and clamped at 2DATA_WIDTH-1 and 0.