Filter Coefficients and Offsets - 2.3 English

Video Processing Subsystem Product Guide (PG231)

Document ID
Release Date
2.3 English

The coefficients are presented in 16.12 fixed point format. The 16-bit signed integer values (2’s compliment) are equivalent to real numbers in the [-8 .. 8] range.

The offset value has a width of the data width plus 1. It is a signed integer with a range as shown in This Equation.

Equation 3-9      [-2Data_Width, 2Data_Width-1]

Matrix computation outputs are rounded to DATA_WIDTH bits by adding half an output LSB prior to truncation.

Output values greater than the Clip value are replaced with the Clip value. Output values smaller than the Clamp value are replaced with the Clamp value. The Clip and Clamp values have the same width as the data width. They are unsigned integers with a range of [0 .. 2Data_Width-1].