TensorTonicTensorTonic
Problems
Study PlansProjectsInterviewPricingFeedback
Study Plan
Train a Tiny Micrograd MLP
Neural NetworksOptimizationLoss Functions
Hard

Train the supplied scalar-output multilayer perceptron for exactly steps full-batch gradient-descent updates. Each weight matrix stores one neuron per row, and every layer applies tanh, including the final layer.

Hℓ=tanh⁡(Hℓ−1WℓT+bℓ)H_{\ell}=\tanh(H_{\ell-1}W_{\ell}^{\mathsf T}+\mathbf{b}_{\ell})Hℓ​=tanh(Hℓ−1​WℓT​+bℓ​) L=∑i=1B(pi−yi)2L=\sum_{i=1}^{B}(p_i-y_i)^2L=i=1∑B​(pi​−yi​)2 θt+1=θt−η∇θL(θt)\boldsymbol{\theta}_{t+1}=\boldsymbol{\theta}_t-\eta\nabla_{\boldsymbol{\theta}}L(\boldsymbol{\theta}_t)θt+1​=θt​−η∇θ​L(θt​)

Here the initial H is inputs, the layer index follows the weight and bias lists, W is a layer's weight matrix, and b is its bias vector broadcast across rows. The final layer has width one; p is its prediction for example i, y is its target, B is the batch size, and L is the summed loss. Theta contains all network parameters, t counts updates, and eta is learning_rate. Each update uses gradients at the same current parameter values.

Record the loss before each update. With zero steps, return the initial network results and an empty loss history. An empty batch has zero loss and leaves parameters unchanged.

Return a tuple containing the final prediction tensor of shape (B,), final zero-dimensional loss tensor, Python list of trained weight tensors, Python list of trained bias tensors, and Python list of zero-dimensional pre-update loss tensors. Parameter lists preserve the supplied order and shapes; loss history has length steps. All tensors preserve the common input dtype and device.

Loading visualization...

Examples

Input: inputs = [[1],[-1]], targets = [1,-1], weights = [[[0.5]]], biases = [[0]], learning_rate = 0.1, steps = 0

Output: ([0.462117, -0.462117], 0.578636, [[[0.5]]], [[0.0]], [])

Explanation: With zero updates, the returned predictions and loss come from the original parameters.

Input: inputs = [[1]], targets = [1], weights = [[[0]]], biases = [[0]], learning_rate = 0.1, steps = 1

Output: ([0.379949], 0.384463, [[[0.2]]], [[0.2]], [1.0])

Input: inputs = [[1]], targets = [0], weights = [[[1]]], biases = [[0]], learning_rate = 0.05, steps = 2

Output: ([0.700481], 0.490674, [[[0.934122]]], [[-0.065878]], [0.580026, 0.537865])

Hint 1

A list can retain layer activations for the backward calculation.

Hint 2

transpose swaps the two axes of a weight matrix.

Hint 3

clone creates working parameters without changing the supplied tensors.

Requirements

  • Use PyTorch tensor arithmetic to compute gradients manually; automatic differentiation and stored tensor gradients are not allowed.
  • Leave inputs, targets, weights, and biases unchanged.
  • Preserve the shared float16, float32, or float64 dtype and device; examples and custom Run inputs use float64.

Constraints

  • inputs has shape (B, n), and targets has shape (B,); B may be zero.
  • weights and biases are nonempty Python lists of equal length, in forward layer order.
  • Each matrix has shape (output width, input width), and each bias vector matches its output width; adjacent widths match.
  • All layer widths are positive, and the final layer has width one.
  • steps is a nonnegative Python integer; learning_rate is a finite nonnegative Python number.
  • All tensors contain finite values and share one floating-point dtype and device; noncontiguous tensors are valid.
  • All intermediate values and results are finite.
Try Similar Problems
Apply One Full SGD Training StepHardApply One Policy-Value Head Training StepHardCalculate Squared-Error Loss GradientsMediumLogistic Regression Training LoopMediumGradient Clipping (Global Norm)MediumRun Reverse-Mode AutodiffHardEvaluate a Multi-Layer PerceptronHardGradient-Check an Autodiff GraphHardDropout from ScratchHardSVM with Hinge Loss (SGD)HardPPO with Reference-Policy KLHardBradley-Terry Reward Model LossMedium

Accepts: array

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

You must run your code first.

Train a Tiny Micrograd MLP

Neural NetworksOptimizationLoss Functions
Hard

Train the supplied scalar-output multilayer perceptron for exactly steps full-batch gradient-descent updates. Each weight matrix stores one neuron per row, and every layer applies tanh, including the final layer.

Hℓ=tanh⁡(Hℓ−1WℓT+bℓ)H_{\ell}=\tanh(H_{\ell-1}W_{\ell}^{\mathsf T}+\mathbf{b}_{\ell})Hℓ​=tanh(Hℓ−1​WℓT​+bℓ​) L=∑i=1B(pi−yi)2L=\sum_{i=1}^{B}(p_i-y_i)^2L=i=1∑B​(pi​−yi​)2 θt+1=θt−η∇θL(θt)\boldsymbol{\theta}_{t+1}=\boldsymbol{\theta}_t-\eta\nabla_{\boldsymbol{\theta}}L(\boldsymbol{\theta}_t)θt+1​=θt​−η∇θ​L(θt​)

Here the initial H is inputs, the layer index follows the weight and bias lists, W is a layer's weight matrix, and b is its bias vector broadcast across rows. The final layer has width one; p is its prediction for example i, y is its target, B is the batch size, and L is the summed loss. Theta contains all network parameters, t counts updates, and eta is learning_rate. Each update uses gradients at the same current parameter values.

Record the loss before each update. With zero steps, return the initial network results and an empty loss history. An empty batch has zero loss and leaves parameters unchanged.

Return a tuple containing the final prediction tensor of shape (B,), final zero-dimensional loss tensor, Python list of trained weight tensors, Python list of trained bias tensors, and Python list of zero-dimensional pre-update loss tensors. Parameter lists preserve the supplied order and shapes; loss history has length steps. All tensors preserve the common input dtype and device.

Loading visualization...

Examples

Input: inputs = [[1],[-1]], targets = [1,-1], weights = [[[0.5]]], biases = [[0]], learning_rate = 0.1, steps = 0

Output: ([0.462117, -0.462117], 0.578636, [[[0.5]]], [[0.0]], [])

Explanation: With zero updates, the returned predictions and loss come from the original parameters.

Input: inputs = [[1]], targets = [1], weights = [[[0]]], biases = [[0]], learning_rate = 0.1, steps = 1

Output: ([0.379949], 0.384463, [[[0.2]]], [[0.2]], [1.0])

Input: inputs = [[1]], targets = [0], weights = [[[1]]], biases = [[0]], learning_rate = 0.05, steps = 2

Output: ([0.700481], 0.490674, [[[0.934122]]], [[-0.065878]], [0.580026, 0.537865])

Hint 1

A list can retain layer activations for the backward calculation.

Hint 2

transpose swaps the two axes of a weight matrix.

Hint 3

clone creates working parameters without changing the supplied tensors.

Requirements

  • Use PyTorch tensor arithmetic to compute gradients manually; automatic differentiation and stored tensor gradients are not allowed.
  • Leave inputs, targets, weights, and biases unchanged.
  • Preserve the shared float16, float32, or float64 dtype and device; examples and custom Run inputs use float64.

Constraints

  • inputs has shape (B, n), and targets has shape (B,); B may be zero.
  • weights and biases are nonempty Python lists of equal length, in forward layer order.
  • Each matrix has shape (output width, input width), and each bias vector matches its output width; adjacent widths match.
  • All layer widths are positive, and the final layer has width one.
  • steps is a nonnegative Python integer; learning_rate is a finite nonnegative Python number.
  • All tensors contain finite values and share one floating-point dtype and device; noncontiguous tensors are valid.
  • All intermediate values and results are finite.
Try Similar Problems
Apply One Full SGD Training StepHardApply One Policy-Value Head Training StepHardCalculate Squared-Error Loss GradientsMediumLogistic Regression Training LoopMediumGradient Clipping (Global Norm)MediumRun Reverse-Mode AutodiffHardEvaluate a Multi-Layer PerceptronHardGradient-Check an Autodiff GraphHardDropout from ScratchHardSVM with Hinge Loss (SGD)HardPPO with Reference-Policy KLHardBradley-Terry Reward Model LossMedium

Accepts: array

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

You must run your code first.