PhysSandbox
Classical MechanicsWaves & SoundElectricity & MagnetismOptics & LightGravity & OrbitsLabs
🌙Astronomy & The Sky🌡️Thermodynamics🌍Biophysics, Fluids & Geoscience📐Math Visualization🔧Engineering🧪Chemistry

Related simulators

Continue with similar topics in this category — or all 85 in Math Visualization.

View category →
NewSchool

Gradient Descent (2D)

Launch Simulator

Level sets of f(x,y) and path (x,y) ← (x,y) − η∇f; bowl or elliptic well.

NewUniversity / research

Heat Equation: Finite Differences

Launch Simulator

1D heat equation u_t = αu_xx with explicit FTCS and implicit backward Euler; tune CFL r = αΔt/Δx², watch explicit blow-up for r > 1/2, and compare numerical diffusion.

NewUniversity / research

Conjugate Gradient Solver

Launch Simulator

SPD system Ax=b as quadratic minimization: contour geometry, CG vs steepest descent path, residual norm, and condition number.

NewUniversity / research

Monte Carlo Integration & Variance Reduction

Launch Simulator

Compare plain Monte Carlo, importance sampling, and stratified sampling for ∫f(x)dx, with convergence curves, standard error, and the 1/√N rate.

NewUniversity / research

Power Iteration Eigenvalue Convergence

Launch Simulator

Visualize dominant eigenvector convergence: spectral gap ratio, Rayleigh quotient, eigen residual, and normalized power iterates on the unit circle.

NewUniversity / research

Newton-Raphson Basins in 2D Systems

Launch Simulator

Map Newton basins for nonlinear F(x,y)=0 systems: initial-guess sensitivity, iteration counts, root attraction, and singular-Jacobian failures.

PhysSandbox

Interactive physics, chemistry, and engineering simulators for students, teachers, and curious minds.

Physics

  • Classical Mechanics
  • Waves & Sound
  • Electricity & Magnetism

Science

  • Optics & Light
  • Gravity & Orbits
  • Astronomy & The Sky

More

  • Thermodynamics
  • Biophysics, Fluids & Geoscience
  • Math Visualization
  • Engineering
  • Chemistry

© 2026 PhysSandbox. Free interactive science simulators.

PrivacyTermsContact
Home/Math Visualization/Gradient Descent Optimizers

Gradient Descent Optimizers

Compare SGD, momentum, and Adam on a curved loss landscape; tune learning rate, curvature, stability, and iteration count.

Optimizer settings

0.12
5
0.82
36

Measured values

SGD loss0.0067
Momentum loss0.0039
Adam loss0.3050

High curvature makes learning-rate choice fragile; adaptive and momentum methods often take different paths.

Live graphs

About this model

This visualizer compares SGD, momentum, and Adam on the same two-dimensional loss landscape. A high curvature ratio makes plain gradient descent sensitive to the learning rate, while momentum accumulates velocity and Adam rescales steps using running first and second moments.

Who it's for: Machine learning, optimization, numerical analysis, deep learning, and data science courses.

Key terms

  • Gradient descent
  • SGD
  • Momentum
  • Adam
  • Learning rate
  • Loss landscape

How it works

Compare SGD, momentum, and Adam trajectories on a curved loss landscape.

Key equations

SGD: x_{k+1}=x_k−η∇f(x_k)
Momentum/Adam smooth gradients and rescale per-coordinate steps

Frequently asked questions

Why can a large learning rate diverge?
On steep directions, a step that is too large overshoots the valley and can bounce outward. The stability threshold gets smaller as curvature increases.
Does Adam always win?
No. Adam often helps on poorly scaled coordinates, but the best optimizer depends on the problem, noise, regularization, and generalization behavior.