Free online math practice

Machine-learning mathematical forms Practice

Frequently used loss, activation, probability, and optimization formulas.

Start practicingMachine-learning mathematical forms

  • Free to practice.
  • No account required.
  • Progress stays on this device.

What you’ll practice

Identify the role of a loss, activation or update rule before recalling its expression. Formula recall does not establish that a model is appropriate for a data set.

Where this helps

Reading and implementing machine-learning models with less formula lookup.

Before you start

Try the examples to see what you’ll practice. Check the answer form and any stated conditions.

Explainers

Take a step against the gradient

For J(θ) = θ² at θ = 3, the gradient is 6. With step size η = 0.1: θ next = 3 − 0.1 × 6 = 2.4.

Use θ next = θ − η∇J(θ), where J is a differentiable loss, ∇J is its gradient, and η > 0 is the step size. In one dimension, a positive derivative means decrease θ, and a negative derivative means increase θ. A sufficiently small move that way lowers the loss. A zero derivative alone gives no decrease direction. A large step can overshoot.

Add weighted inputs and the intercept

Let x = (2, 3), w = (4, −1), and b = 5. ŷ = 4 × 2 + (−1) × 3 + 5 = 10.

The model is ŷ = w · x + b. Pair each input with its weight, add the products, then add the intercept b once. The vectors w and x must have the same number of entries.

Average the squared errors

Observed values are 2 and 5. Predictions are 3 and 3. MSE = ((2 − 3)² + (5 − 3)²)/2 = (1 + 4)/2 = 2.5.

Use MSE = Σ(yᵢ − ŷᵢ)²/n for n > 0 pairs. Here yᵢ is an observed value, ŷᵢ is its prediction, and Σ means add all n terms. Squaring keeps opposite errors from canceling.

Turn one score into a value between zero and one

At z = 0, σ(z) = 1/(1 + e⁰) = 1/2.

The sigmoid formula is σ(z) = 1/(1 + exp(−z)), where exp(u) means e raised to the power u. For a finite real score z, the result is strictly between 0 and 1. Positive scores give values above 1/2, and negative scores give values below 1/2. Keep the minus sign in the exponent.

Turn several scores into shares

Suppose two scores have exponentials 2 and 3. Their softmax values are: 2/(2 + 3) = 2/5 3/(2 + 3) = 3/5

For K scores, softmax(z)ᵢ = exp(zᵢ)/Σⱼ₌₁ᴷ exp(zⱼ). Here exp(u) means e raised to the power u. Divide the chosen score’s exponential by the sum for every score. The outputs are positive and add to 1.

Try a few examples

Real practice. Real progress.

Example 1 of 4

J is differentiable. θ is the parameter vector and η > 0 is the step size.

Choose the exact expression.

Grades and courses that include this topic

Related practice

Start practicingMachine-learning mathematical forms

  • Free to practice.
  • No account required.
  • Progress stays on this device.