Check a scalar expression's analytic leaf gradients against forward finite differences. Each leaf has id and value. Each operation has id, op, and parents, an ordered list of operand IDs. The supported operations are add and mul with two operands, and tanh with one. Operation records need not be in dependency order.
Only the selected output and its ancestors contribute. This problem uses a tree: each reachable node occurs in at most one operation's parent position.
ginum=hf(x+hei)−f(x)Here f is the selected scalar expression, x contains the reachable leaf values, i identifies one leaf, h is the positive step, and e is the unit vector that changes only that leaf. The analytic gradient is the derivative of f with respect to that leaf.
Return a tuple of the analytic-gradient dictionary, numerical-gradient dictionary, and maximum absolute difference between matching gradients. Both dictionaries map reachable leaf IDs to 64-bit floating-point values in original leaf order; the maximum error is also a 64-bit floating-point number. If the output is a leaf, its analytic gradient is one.
Input: leaves = [{"id":"x","value":0.7},{"id":"y","value":-0.4}], operations = [{"id":"product","op":"mul","parents":["x","y"]},{"id":"out","op":"tanh","parents":["product"]}], output_id = "out", h = 0.0001
Output: ({"x": -0.370209, "y": 0.647866}, {"x": -0.370205, "y": 0.647878}, 0.000012)
Explanation: Perturbing each leaf separately gives gradients close to the analytic derivatives.
Input: leaves = [{"id":"a","value":1.5},{"id":"b","value":-2}], operations = [{"id":"out","op":"mul","parents":["a","b"]}], output_id = "out", h = 0.000001
Output: ({"a": -2.0, "b": 1.5}, {"a": -2.0, "b": 1.5}, 0.0)
Input: leaves = [{"id":"x","value":-2},{"id":"y","value":-3.5}], operations = [{"id":"out","op":"add","parents":["x","y"]}], output_id = "out", h = 0.00001
Output: ({"x": 1.0, "y": 1.0}, {"x": 1.0, "y": 1.0}, 0.0)
Dictionaries keyed by node ID can cache forward values and derivatives.
reversed can traverse a valid forward evaluation order backward.
Copying the leaf values keeps each finite-difference perturbation independent.
Accepts: array
Accepts: array
Accepts: string
Accepts: number
Check a scalar expression's analytic leaf gradients against forward finite differences. Each leaf has id and value. Each operation has id, op, and parents, an ordered list of operand IDs. The supported operations are add and mul with two operands, and tanh with one. Operation records need not be in dependency order.
Only the selected output and its ancestors contribute. This problem uses a tree: each reachable node occurs in at most one operation's parent position.
ginum=hf(x+hei)−f(x)Here f is the selected scalar expression, x contains the reachable leaf values, i identifies one leaf, h is the positive step, and e is the unit vector that changes only that leaf. The analytic gradient is the derivative of f with respect to that leaf.
Return a tuple of the analytic-gradient dictionary, numerical-gradient dictionary, and maximum absolute difference between matching gradients. Both dictionaries map reachable leaf IDs to 64-bit floating-point values in original leaf order; the maximum error is also a 64-bit floating-point number. If the output is a leaf, its analytic gradient is one.
Input: leaves = [{"id":"x","value":0.7},{"id":"y","value":-0.4}], operations = [{"id":"product","op":"mul","parents":["x","y"]},{"id":"out","op":"tanh","parents":["product"]}], output_id = "out", h = 0.0001
Output: ({"x": -0.370209, "y": 0.647866}, {"x": -0.370205, "y": 0.647878}, 0.000012)
Explanation: Perturbing each leaf separately gives gradients close to the analytic derivatives.
Input: leaves = [{"id":"a","value":1.5},{"id":"b","value":-2}], operations = [{"id":"out","op":"mul","parents":["a","b"]}], output_id = "out", h = 0.000001
Output: ({"a": -2.0, "b": 1.5}, {"a": -2.0, "b": 1.5}, 0.0)
Input: leaves = [{"id":"x","value":-2},{"id":"y","value":-3.5}], operations = [{"id":"out","op":"add","parents":["x","y"]}], output_id = "out", h = 0.00001
Output: ({"x": 1.0, "y": 1.0}, {"x": 1.0, "y": 1.0}, 0.0)
Dictionaries keyed by node ID can cache forward values and derivatives.
reversed can traverse a valid forward evaluation order backward.
Copying the leaf values keeps each finite-difference perturbation independent.
Accepts: array
Accepts: array
Accepts: string
Accepts: number