Compare analytic gradients with numerical estimates for a scalar computation graph:
e=ab+c. L=ef.Here a, b, c, and f are scalar inputs, e is an intermediate value, and L is the loss. Derive the analytic gradients using the chain rule. Numerical gradients use the supplied positive step h and forward differences, changing one input while keeping the others fixed.
Return a four-item tuple: the loss as a Python float, a list of four analytic Python-float gradients, a list of four numerical Python-float gradients, and their largest absolute pairwise difference as a Python float. Both lists use a, b, c, f order.
Input: a=2.0, b=-3.0, c=10.0, f=-2.0, h=0.001
Output: (-8.0, [6.0,-4.0,-2.0,4.0], [6.0,-4.0,-2.0,4.0], 0.0)
Explanation: The downstream multiplier affects the first three gradients; the two gradient lists agree to the displayed precision.
Input: a=0.0, b=0.0, c=0.0, f=0.0, h=0.01
Output: (0.0,[0.0,0.0,0.0,0.0],[0.0,0.0,0.0,0.0],0.0)
Input: a=0.5, b=-1.25, c=2.0, f=1.5, h=0.0001
Output: (2.0625, [-1.875,0.75,1.5,1.375], [-1.875,0.75,1.5,1.375], 0.0)
A fresh list.copy prevents perturbations from accumulating across checks.
zip pairs matching analytic and numerical gradients.
abs and max summarize the largest component error.
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Compare analytic gradients with numerical estimates for a scalar computation graph:
e=ab+c. L=ef.Here a, b, c, and f are scalar inputs, e is an intermediate value, and L is the loss. Derive the analytic gradients using the chain rule. Numerical gradients use the supplied positive step h and forward differences, changing one input while keeping the others fixed.
Return a four-item tuple: the loss as a Python float, a list of four analytic Python-float gradients, a list of four numerical Python-float gradients, and their largest absolute pairwise difference as a Python float. Both lists use a, b, c, f order.
Input: a=2.0, b=-3.0, c=10.0, f=-2.0, h=0.001
Output: (-8.0, [6.0,-4.0,-2.0,4.0], [6.0,-4.0,-2.0,4.0], 0.0)
Explanation: The downstream multiplier affects the first three gradients; the two gradient lists agree to the displayed precision.
Input: a=0.0, b=0.0, c=0.0, f=0.0, h=0.01
Output: (0.0,[0.0,0.0,0.0,0.0],[0.0,0.0,0.0,0.0],0.0)
Input: a=0.5, b=-1.25, c=2.0, f=1.5, h=0.0001
Output: (2.0625, [-1.875,0.75,1.5,1.375], [-1.875,0.75,1.5,1.375], 0.0)
A fresh list.copy prevents perturbations from accumulating across checks.
zip pairs matching analytic and numerical gradients.
abs and max summarize the largest component error.
Accepts: number
Accepts: number
Accepts: number
Accepts: number
Accepts: number