Compute a scalar expression's output and its derivative with respect to each contributing leaf. Each leaf is a dictionary with id and value. Each operation is a dictionary with id, op, and parents; op is "add", "mul", or "tanh", and parents lists its operand IDs in order. Records need not be in dependency order.
Only output_id and its ancestors contribute. The reachable graph is a tree: a node occurs in at most one operation's parent position. The selected output may itself be a leaf.
∂y∂y=1Here y is the selected output. Return a tuple containing y and a dictionary mapping each reachable leaf ID to its gradient. The output and gradient values must be 64-bit floating-point numbers. Preserve leaf input order in the dictionary and exclude disconnected leaves.
Input: leaves=[{"id":"x","value":2.0},{"id":"y","value":-3.0},{"id":"bias","value":1.0}], operations=[{"id":"product","op":"mul","parents":["x","y"]},{"id":"out","op":"add","parents":["product","bias"]}], output_id="out"
Output: (-5.0,{"x":-3.0,"y":2.0,"bias":1.0})
Explanation: The product supplies the first output operand, while the distinct bias leaf supplies the second.
Input: leaves=[{"id":"a","value":3.0},{"id":"b","value":-1.0}], operations=[{"id":"out","op":"add","parents":["a","b"]}], output_id="out"
Output: (2.0,{"a":1.0,"b":1.0})
Input: leaves=[{"id":"x","value":0.75},{"id":"bias","value":-0.2}], operations=[{"id":"sum","op":"add","parents":["x","bias"]},{"id":"out","op":"tanh","parents":["sum"]}], output_id="out"
Output: (0.50052,{"x":0.74948,"bias":0.74948})
Dictionaries keyed by node ID can hold forward values and gradients separately.
reversed can traverse a valid forward evaluation order backward.
np.tanh accepts a scalar; its saved output is useful during backpropagation.
Accepts: array
Accepts: array
Accepts: string
Compute a scalar expression's output and its derivative with respect to each contributing leaf. Each leaf is a dictionary with id and value. Each operation is a dictionary with id, op, and parents; op is "add", "mul", or "tanh", and parents lists its operand IDs in order. Records need not be in dependency order.
Only output_id and its ancestors contribute. The reachable graph is a tree: a node occurs in at most one operation's parent position. The selected output may itself be a leaf.
∂y∂y=1Here y is the selected output. Return a tuple containing y and a dictionary mapping each reachable leaf ID to its gradient. The output and gradient values must be 64-bit floating-point numbers. Preserve leaf input order in the dictionary and exclude disconnected leaves.
Input: leaves=[{"id":"x","value":2.0},{"id":"y","value":-3.0},{"id":"bias","value":1.0}], operations=[{"id":"product","op":"mul","parents":["x","y"]},{"id":"out","op":"add","parents":["product","bias"]}], output_id="out"
Output: (-5.0,{"x":-3.0,"y":2.0,"bias":1.0})
Explanation: The product supplies the first output operand, while the distinct bias leaf supplies the second.
Input: leaves=[{"id":"a","value":3.0},{"id":"b","value":-1.0}], operations=[{"id":"out","op":"add","parents":["a","b"]}], output_id="out"
Output: (2.0,{"a":1.0,"b":1.0})
Input: leaves=[{"id":"x","value":0.75},{"id":"bias","value":-0.2}], operations=[{"id":"sum","op":"add","parents":["x","bias"]},{"id":"out","op":"tanh","parents":["sum"]}], output_id="out"
Output: (0.50052,{"x":0.74948,"bias":0.74948})
Dictionaries keyed by node ID can hold forward values and gradients separately.
reversed can traverse a valid forward evaluation order backward.
np.tanh accepts a scalar; its saved output is useful during backpropagation.
Accepts: array
Accepts: array
Accepts: string