FP32 to Int8 Tensor Quantization: Scale, Zero Point, Clipping and Outliers

You are writing the numeric core of an int8 inference path: FP32 tensors (weights or activations) are converted to signed 8-bit integers so that they take a quarter of the memory and can feed integer kernels. The interview starts with a small NumPy function and then works through five follow-ups: adding a zero point, when symmetric quantization is not enough, how scale and zero point are derived and work together, clipping and saturation, and how quantization error behaves when the input is skewed or has outliers. Signed int8 covers the integers from `qmin = -128` to `qmax = 127`. ### Constraints and Clarifications - `x` is a NumPy array of FP32 values with any shape, including an empty array or a 0-dimensional array. Every function returns an array with the same shape as its input. - `scale` is a positive real number supplied by the caller in Part 1; Part 4 asks where it comes from. - Use NumPy only, no quantization library, and keep the code vectorized. - Be ready to check your code by hand on `x = [-1.0, 0.0, 1.0]` with `scale = 0.1`. ### Clarifying Questions - Which tie rule should rounding follow for values such as `2.5`: round half to even (NumPy's default) or round half away from zero? - What should happen for a scale that is zero, negative, infinite or NaN, and for NaN or infinite entries in `x`? - Should inputs of another float dtype (for example float64) be accepted and converted, or rejected? - Is the quantized tensor only stored, or will it feed integer matrix-multiplication kernels? That affects whether a nonzero zero point is acceptable. ### Part 1 — Symmetric per-tensor quantization Implement: ```python def quantize_fp32_to_int8(x: np.ndarray, scale: float) -> np.ndarray: ``` Divide every element of `x` by `scale`, round to the nearest integer, bring the result into the int8 range `[-128, 127]`, and return an int8 array with the same shape as `x`. Then work through `x = [-1.0, 0.0, 1.0]`, `scale = 0.1` by hand and state the output. ```hint Look at the last step Before relying on `astype(np.int8)`, ask what it does to a value such as `200.0` that is already outside the int8 range. ``` #### What This Part Should Cover - A vectorized implementation that preserves shape and returns int8 - An explicit, stated rounding rule - How out-of-range values are handled before the integer conversion - Validation of `scale` and of non-finite inputs ### Part 2 — Asymmetric quantization with a zero point Add an integer `zero_point` parameter so that each element is quantized as ```text q = clamp(round(x / scale) + zero_point, qmin, qmax) ``` and write the matching dequantization, which reconstructs an approximation of the original value: ```text x_hat = scale * (q - zero_point) ``` Show a small quantize-then-dequantize round trip and state how large the reconstruction error can be for values inside the representable range. ```hint Mind the integer width When you compute `q - zero_point` for dequantization, check whether the result still fits in the dtype you are computing in. ``` #### What This Part Should Cover - A correct asymmetric quantizer and dequantizer - Why the zero point must be an integer inside `[qmin, qmax]` - A worked round trip and a bound on the in-range reconstruction error ### Part 3 — When symmetric quantization is not enough Symmetric quantization fixes `zero_point = 0`, so the representable window is centered on zero. Explain when that choice wastes precision, using a concrete tensor distribution, and when symmetric quantization is still the better choice. ```hint Count the usable codes Take a tensor whose values are all non-negative and count how many of the 256 int8 codes a symmetric quantizer can ever produce for it. ``` #### What This Part Should Cover - The distributions for which a centered window wastes codes, with a quantitative example - The resolution lost in that case, compared with an asymmetric quantizer - Why symmetric quantization is still attractive, including its effect on integer matrix multiplication ### Part 4 — Deriving scale and zero point Given an observed real range `[x_min, x_max]` for a tensor (for example, from running calibration data through the model), derive the `scale` and `zero_point` for asymmetric int8 quantization, and the `scale` for symmetric quantization. Explain how the two parameters work together, and what should be true of the real value `0.0` after quantization. ```hint Pin the endpoints Require `x_min` to map to `qmin` and `x_max` to map to `qmax` under the affine map, then look at where `0.0` lands. ``` #### What This Part Should Cover - The derivation of `scale` and `zero_point` from a range, with rounding and clamping of the zero point - Exact representation of `0.0` and why it matters - Degenerate ranges and a numeric check of the derived parameters ### Part 5 — Clipping, saturation, and skewed or outlier-heavy inputs Discuss what should happen to values whose `round(x / scale) + zero_point` falls outside `[-128, 127]`, and what goes wrong if nothing handles them. Then explain how the quantization error behaves when the input distribution is skewed or contains outliers, and what you would change about how the range is chosen. ```hint Two sources of error Separate the error on values that fall inside the representable window from the error on values that fall outside it, and see how each one changes as the window gets wider. ``` #### What This Part Should Cover - Saturation versus wraparound, with a concrete failing value - How rounding error and clipping error trade off as the range changes - The effect of a single outlier or a long tail on per-tensor min/max scaling, with numbers - Practical range-setting or granularity changes that reduce the error ### What a Strong Answer Covers - Correct, vectorized NumPy code with explicit rounding, saturation and dtype handling throughout - A derivation of the quantization parameters that keeps `0.0` exact - Quantitative reasoning about error (step size, maximum rounding error, clipping error) rather than qualitative claims - Awareness of what zero points and scale granularity cost in real integer kernels - Edge cases: non-finite values, degenerate ranges, empty tensors and ties in rounding ### Follow-up Questions - How do per-tensor and per-channel scales differ for the weight matrix of a linear or convolution layer, and why is per-channel cheap for weights but awkward for activations? - Why are activations usually harder to quantize than weights, and how does calibrating activation ranges differ from choosing weight ranges? - An integer kernel multiplies two int8 tensors and accumulates in int32. How is the result converted back to int8 for the next layer, and where can overflow or saturation occur? - How would you test these functions so that a regression in rounding or saturation is caught automatically?