Build a fake quantizer with jax.custom_vjp whose forward pass rounds values onto an integer grid and whose backward pass is a clipped straight-through estimator, then return the quantized values and the gradient that flows back through it.
With the step size scale and the integer range from qmin to qmax, the forward pass maps every entry of x to
q(x)=scale⋅clip(round(x/scale), qmin, qmax)where round sends halves to the nearest even integer, as jnp.round does, so 2.5 becomes 2 and 3.5 becomes 4. The true derivative of q is zero almost everywhere, so the backward rule replaces it. Given an upstream gradient c with the same shape as x, called the cotangent, the backward rule returns c⊙m, where ⊙ is elementwise multiplication and the mask m is 1 for entries with qmin≤x/scale≤qmax (both ends included) and 0 for every other entry.
Return a dict with two keys, values holding q(x) and grads holding the gradient for x that the cotangent produces through your custom rule, each a float32 JAX array with the same shape as x.
Input: x = [0.3,1.25,-0.75,2.6,-3.0], scale = 0.5, qmin = -4, qmax = 3, cotangent = [1.0,1.0,1.0,1.0,1.0]
Output: {"values":[0.5,1.0,-1.0,1.5,-2.0],"grads":[1.0,1.0,1.0,0.0,0.0]}
Explanation: Dividing by 0.5 gives 0.6, 2.5, -1.5, 5.2 and -6. Rounding halves to even gives 1, 2, -2, 5 and -6, and clipping to [-4, 3] turns the last two into 3 and -4. The first three entries are inside the range, so their cotangent passes through, and the two clipped entries get zero.
Input: x = [[0.1,0.6],[3.9,-0.2]], scale = 0.25, qmin = 0, qmax = 15, cotangent = [[2.0,-1.0],[0.5,3.0]]
Output: {"values":[[0.0,0.5],[3.75,0.0]],"grads":[[2.0,-1.0],[0.0,0.0]]}
Input: x = [2.0,2.5,-2.0,-2.25,0.5], scale = 1.0, qmin = -2, qmax = 2, cotangent = [1.0,2.0,3.0,4.0,5.0]
Output: {"values":[2.0,2.0,-2.0,-2.0,0.0],"grads":[1.0,0.0,3.0,0.0,5.0]}
f = jax.custom_vjp(f); f.defvjp(f_fwd, f_bwd) attaches your own backward rule.
f_fwd returns (output, residuals) and f_bwd(residuals, g) returns a tuple like (grad_x,).
y, pullback = jax.vjp(f, x); (grad_x,) = pullback(cotangent).
Accepts: array
Accepts: number
Accepts: integer
Accepts: integer
Accepts: array
Build a fake quantizer with jax.custom_vjp whose forward pass rounds values onto an integer grid and whose backward pass is a clipped straight-through estimator, then return the quantized values and the gradient that flows back through it.
With the step size scale and the integer range from qmin to qmax, the forward pass maps every entry of x to
q(x)=scale⋅clip(round(x/scale), qmin, qmax)where round sends halves to the nearest even integer, as jnp.round does, so 2.5 becomes 2 and 3.5 becomes 4. The true derivative of q is zero almost everywhere, so the backward rule replaces it. Given an upstream gradient c with the same shape as x, called the cotangent, the backward rule returns c⊙m, where ⊙ is elementwise multiplication and the mask m is 1 for entries with qmin≤x/scale≤qmax (both ends included) and 0 for every other entry.
Return a dict with two keys, values holding q(x) and grads holding the gradient for x that the cotangent produces through your custom rule, each a float32 JAX array with the same shape as x.
Input: x = [0.3,1.25,-0.75,2.6,-3.0], scale = 0.5, qmin = -4, qmax = 3, cotangent = [1.0,1.0,1.0,1.0,1.0]
Output: {"values":[0.5,1.0,-1.0,1.5,-2.0],"grads":[1.0,1.0,1.0,0.0,0.0]}
Explanation: Dividing by 0.5 gives 0.6, 2.5, -1.5, 5.2 and -6. Rounding halves to even gives 1, 2, -2, 5 and -6, and clipping to [-4, 3] turns the last two into 3 and -4. The first three entries are inside the range, so their cotangent passes through, and the two clipped entries get zero.
Input: x = [[0.1,0.6],[3.9,-0.2]], scale = 0.25, qmin = 0, qmax = 15, cotangent = [[2.0,-1.0],[0.5,3.0]]
Output: {"values":[[0.0,0.5],[3.75,0.0]],"grads":[[2.0,-1.0],[0.0,0.0]]}
Input: x = [2.0,2.5,-2.0,-2.25,0.5], scale = 1.0, qmin = -2, qmax = 2, cotangent = [1.0,2.0,3.0,4.0,5.0]
Output: {"values":[2.0,2.0,-2.0,-2.0,0.0],"grads":[1.0,0.0,3.0,0.0,5.0]}
f = jax.custom_vjp(f); f.defvjp(f_fwd, f_bwd) attaches your own backward rule.
f_fwd returns (output, residuals) and f_bwd(residuals, g) returns a tuple like (grad_x,).
y, pullback = jax.vjp(f, x); (grad_x,) = pullback(cotangent).
Accepts: array
Accepts: number
Accepts: integer
Accepts: integer
Accepts: array