TensorTonicTensorTonic
Problems
Study PlansProjectsInterviewPricingFeedback
Study Plan
Apply One Full SGD Training Step
Optimization
Hard

Perform one full-batch gradient-descent step for a tanh neuron, using the supplied inputs, targets, weights, bias, and learning rate.

pi=tanh⁡(xiTw+b)p_i=\tanh(\mathbf{x}_i^{\mathsf T}\mathbf{w}+b)pi​=tanh(xiT​w+b) L=∑i=1B(pi−yi)2L=\sum_{i=1}^{B}(p_i-y_i)^2L=i=1∑B​(pi​−yi​)2 θnew=θ−η∇θL\boldsymbol{\theta}_{\mathrm{new}}=\boldsymbol{\theta}-\eta\nabla_{\boldsymbol{\theta}}Lθnew​=θ−η∇θ​L

Here x is one input row, w is weights, b is bias, p is its prediction, y is its target, B is the batch size, and L is the summed loss. The parameter vector theta contains the weights and bias, and eta is learning_rate. Both parameter updates use gradients at the original parameter values. An empty batch has zero loss and gradients and leaves the parameters unchanged.

Return a tuple of six PyTorch tensors in this order: loss before the update, loss after the update, updated weights, updated bias, weight gradients, and bias gradient. Losses and bias values are zero-dimensional; weight tensors retain the input weight shape. Preserve the common input dtype and device.

Loading visualization...

Examples

Input: inputs = [[1]], targets = [1], weights = [0], bias = 0, learning_rate = 0.1

Output: (1.0, 0.384463, [0.2], 0.2, [-2.0], -2.0)

Explanation: Both parameter gradients are -2; the update changes the weight and bias to 0.2 and reduces the loss.

Input: inputs = [[1,-1],[0,2]], targets = [0.5,-0.25], weights = [0.2,-0.3], bias = 0.1, learning_rate = 0

Output: (0.046366, 0.046366, [0.2, -0.3], 0.1, [0.052727, -0.720004], -0.280911)

Input: inputs = [[0]], targets = [0], weights = [2], bias = 0, learning_rate = 0.5

Output: (0.0, 0.0, [2.0], 0.0, [0.0], 0.0)

Hint 1

torch.mv evaluates a matrix-vector product for a batch.

Hint 2

sum along dimension zero combines per-example contributions.

Hint 3

clone can keep updated parameters separate from the supplied tensors.

Requirements

  • Use PyTorch tensor arithmetic to derive gradients manually; automatic differentiation and stored tensor gradients are not allowed.
  • Do not modify the supplied tensors.
  • Preserve the shared float16, float32, or float64 dtype and device; examples and custom Run inputs use float64.

Constraints

  • inputs has shape (B, n), targets has shape (B,), weights has shape (n,), and bias is zero-dimensional.
  • B may be zero; learning_rate is a finite nonnegative Python number.
  • All tensors contain finite values and share one floating-point dtype and device; noncontiguous tensors are valid.
  • All predictions, losses, gradients, and updated parameters are finite.
Try Similar Problems
Train a Tiny Micrograd MLPHardApply a Gradient-Descent StepMediumRecover an IsoFLOP Compute-Optimal FrontierHardApply One Policy-Value Head Training StepHardL-BFGS Two-Loop RecursionHardFit Learning-Rate and Batch-Size ScalingHardCalculate Squared-Error Loss GradientsMediumLogistic Regression Training LoopMediumGRPO Group-Relative AdvantageHardHungarian MatchingHardGradient ClippingHardL-BFGSHard

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

You must run your code first.

Apply One Full SGD Training Step

Optimization
Hard

Perform one full-batch gradient-descent step for a tanh neuron, using the supplied inputs, targets, weights, bias, and learning rate.

pi=tanh⁡(xiTw+b)p_i=\tanh(\mathbf{x}_i^{\mathsf T}\mathbf{w}+b)pi​=tanh(xiT​w+b) L=∑i=1B(pi−yi)2L=\sum_{i=1}^{B}(p_i-y_i)^2L=i=1∑B​(pi​−yi​)2 θnew=θ−η∇θL\boldsymbol{\theta}_{\mathrm{new}}=\boldsymbol{\theta}-\eta\nabla_{\boldsymbol{\theta}}Lθnew​=θ−η∇θ​L

Here x is one input row, w is weights, b is bias, p is its prediction, y is its target, B is the batch size, and L is the summed loss. The parameter vector theta contains the weights and bias, and eta is learning_rate. Both parameter updates use gradients at the original parameter values. An empty batch has zero loss and gradients and leaves the parameters unchanged.

Return a tuple of six PyTorch tensors in this order: loss before the update, loss after the update, updated weights, updated bias, weight gradients, and bias gradient. Losses and bias values are zero-dimensional; weight tensors retain the input weight shape. Preserve the common input dtype and device.

Loading visualization...

Examples

Input: inputs = [[1]], targets = [1], weights = [0], bias = 0, learning_rate = 0.1

Output: (1.0, 0.384463, [0.2], 0.2, [-2.0], -2.0)

Explanation: Both parameter gradients are -2; the update changes the weight and bias to 0.2 and reduces the loss.

Input: inputs = [[1,-1],[0,2]], targets = [0.5,-0.25], weights = [0.2,-0.3], bias = 0.1, learning_rate = 0

Output: (0.046366, 0.046366, [0.2, -0.3], 0.1, [0.052727, -0.720004], -0.280911)

Input: inputs = [[0]], targets = [0], weights = [2], bias = 0, learning_rate = 0.5

Output: (0.0, 0.0, [2.0], 0.0, [0.0], 0.0)

Hint 1

torch.mv evaluates a matrix-vector product for a batch.

Hint 2

sum along dimension zero combines per-example contributions.

Hint 3

clone can keep updated parameters separate from the supplied tensors.

Requirements

  • Use PyTorch tensor arithmetic to derive gradients manually; automatic differentiation and stored tensor gradients are not allowed.
  • Do not modify the supplied tensors.
  • Preserve the shared float16, float32, or float64 dtype and device; examples and custom Run inputs use float64.

Constraints

  • inputs has shape (B, n), targets has shape (B,), weights has shape (n,), and bias is zero-dimensional.
  • B may be zero; learning_rate is a finite nonnegative Python number.
  • All tensors contain finite values and share one floating-point dtype and device; noncontiguous tensors are valid.
  • All predictions, losses, gradients, and updated parameters are finite.
Try Similar Problems
Train a Tiny Micrograd MLPHardApply a Gradient-Descent StepMediumRecover an IsoFLOP Compute-Optimal FrontierHardApply One Policy-Value Head Training StepHardL-BFGS Two-Loop RecursionHardFit Learning-Rate and Batch-Size ScalingHardCalculate Squared-Error Loss GradientsMediumLogistic Regression Training LoopMediumGRPO Group-Relative AdvantageHardHungarian MatchingHardGradient ClippingHardL-BFGSHard

Accepts: array

Accepts: array

Accepts: array

Accepts: number

Accepts: number

You must run your code first.