PhysSandbox
Classical MechanicsWaves & SoundElectricity & MagnetismOptics & LightGravity & OrbitsLabs
🌙Astronomy & The Sky🌡️Thermodynamics🌍Biophysics, Fluids & Geoscience📐Math Visualization🔧Engineering🧪Chemistry

More from Math Visualization

Other simulators in this category — or see all 84.

View category →
NewUniversity / research

Gaussian Mixture EM Algorithm

Expectation-maximization for a two-component Gaussian mixture: responsibilities, E/M steps, likelihood, and covariance ellipses.

Launch Simulator
NewUniversity / research

Support Vector Machine Margin

Hard/soft-margin SVM sketch with C penalty, hinge loss, support vectors, decision boundary, and margin bands.

Launch Simulator
NewUniversity / research

Bayesian Updating / Conjugate Priors

Beta-binomial Bayesian updating with prior/posterior curves, posterior predictive probability, and credible intervals.

Launch Simulator
NewUniversity / research

Markov Chain Mixing

Three-state Markov chain with transition matrix, stationary distribution, total variation distance, detailed-balance cue, and spectral-gap proxy.

Launch Simulator
NewUniversity / research

LMS / NLMS Adaptive Noise Cancellation

Primary p = s + v with v a fixed unknown FIR of Gaussian reference x[n]. Watch an L-tap FIR adapt by LMS or NLMS so error e = p − wᵀx → s; running MSE and ‖w − h‖.

Launch Simulator
NewUniversity / research

DCT & JPEG Quantization (8×8)

64×64 luma: 8×8 DCT with −128 shift, ISO luminance quant table scaled by JPEG quality, or zigzag AC truncation (keep K). Click a block for coefficient heatmaps and zigzag trace.

Launch Simulator
PhysSandbox

Interactive physics, chemistry, and engineering simulators for students, teachers, and curious minds.

Physics

  • Classical Mechanics
  • Waves & Sound
  • Electricity & Magnetism

Science

  • Optics & Light
  • Gravity & Orbits
  • Astronomy & The Sky

More

  • Thermodynamics
  • Biophysics, Fluids & Geoscience
  • Math Visualization
  • Engineering
  • Chemistry

© 2026 PhysSandbox. Free interactive science simulators.

PrivacyTermsContact
Home/Math Visualization/Gradient Descent Optimizers

Gradient Descent Optimizers

This visualizer compares SGD, momentum, and Adam on the same two-dimensional loss landscape. A high curvature ratio makes plain gradient descent sensitive to the learning rate, while momentum accumulates velocity and Adam rescales steps using running first and second moments.

Who it's for: Machine learning, optimization, numerical analysis, deep learning, and data science courses.

Key terms

  • Gradient descent
  • SGD
  • Momentum
  • Adam
  • Learning rate
  • Loss landscape

High curvature makes learning-rate choice fragile; adaptive and momentum methods often take different paths.

Live graphs

Optimizer settings

0.12
5
0.82
36

Measured values

SGD loss0.0067
Momentum loss0.0039
Adam loss0.3050

How it works

Compare SGD, momentum, and Adam trajectories on a curved loss landscape.

Key equations

SGD: x_{k+1}=x_k−η∇f(x_k)
Momentum/Adam smooth gradients and rescale per-coordinate steps

Frequently asked questions

Why can a large learning rate diverge?
On steep directions, a step that is too large overshoots the valley and can bounce outward. The stability threshold gets smaller as curvature increases.
Does Adam always win?
No. Adam often helps on poorly scaled coordinates, but the best optimizer depends on the problem, noise, regularization, and generalization behavior.