Why can’t the tree learn a diagonal boundary with one split?

Each split is **parallel to an axis** (either x ≤ t or y ≤ t). A single oblique line like x + y = 0 requires **staircasing** with many axis cuts or a different model class (linear SVM, oblique trees).

When do Gini and entropy pick different first splits?

Both reward purer children, but the **curvature** differs: entropy penalizes 50/50 mixes more sharply, so it sometimes prefers a slightly different threshold that buys more purity early. On many toy clouds the first split matches; on borderline ties our tie-break prefers **x** over **y**, then a smaller threshold.

Does this match sklearn’s `DecisionTreeClassifier` exactly?

The **split search and impurity formulas** mirror the teaching description, but sklearn adds tie-breaking details, optional `class_weight`, `max_features`, and floating policies. Expect **near-identical** trees on the same data with matching hyperparameters, not bit-identical dumps.

Decision Tree Classifier (2D toy)

Greedy axis-aligned splits on a click-labeled scatter: compare Gini vs entropy impurity, max depth, and min-samples-per-leaf; shaded rectangles show leaf decisions, dashed lines show recursive partitions.

About this model

A binary classification tree built in the plane treats each feature x and y as a coordinate axis. This simulator implements the textbook greedy CART recipe used in scikit-learn’s `DecisionTreeClassifier` for axis-aligned splits: at every node, enumerate every threshold between consecutive sorted values along x and along y that separates the current training subset into nonempty left/right child sets respecting min_samples_leaf. Each candidate split scores the weighted impurity of the children using either Gini (probability of misclassification under random label draws) or binary entropy in bits. The split with largest information gain (parent impurity minus weighted children) is chosen; the process recurses until leaves are pure, depth hits max_depth, no positive gain exists, or there are too few points to split further. Leaves predict the majority class; the canvas shades each leaf’s rectangle and overlays dashed split lines so the recursive partition of ℝ² is visible. Training error is the in-sample misclassification rate (white ring on a point = wrong class inside its leaf).

Who it's for: Introductory machine-learning students comparing impurity criteria, depth limits, and the geometry of axis-aligned decision boundaries before moving to random forests or gradient boosting.

Key terms

CART
Gini impurity
Entropy
Information gain
Axis-aligned split
Min samples per leaf
Max depth
Majority vote

How it works

Train a binary axis-aligned CART tree on a toy 2D point cloud. At each node the learner searches every vertical and horizontal threshold between sorted coordinates, scoring splits by impurity drop (parent minus weighted child impurity) using either Gini or binary entropy. The chosen split recursively partitions the plane into rectangles; leaves vote majority class. Drag points or change max depth and min samples per leaf to see overfitting vs underfitting, and compare how entropy favors purer children slightly earlier than Gini on some layouts.

Frequently asked questions

Why can’t the tree learn a diagonal boundary with one split?: Each split is parallel to an axis (either x ≤ t or y ≤ t). A single oblique line like x + y = 0 requires staircasing with many axis cuts or a different model class (linear SVM, oblique trees).
When do Gini and entropy pick different first splits?: Both reward purer children, but the curvature differs: entropy penalizes 50/50 mixes more sharply, so it sometimes prefers a slightly different threshold that buys more purity early. On many toy clouds the first split matches; on borderline ties our tie-break prefers x over y, then a smaller threshold.
Does this match sklearn’s `DecisionTreeClassifier` exactly?: The split search and impurity formulas mirror the teaching description, but sklearn adds tie-breaking details, optional `class_weight`, `max_features`, and floating policies. Expect near-identical trees on the same data with matching hyperparameters, not bit-identical dumps.

Continue with similar topics in this category — or all 85 in Math Visualization.

View category →

NewSchool

Toy 2-Layer MLP + Backprop (XOR / spiral)

Launch Simulator

Click-labeled 2D data; **tanh** hidden layer + **logistic** output trained by **full-batch** gradient descent on **binary cross-entropy**. Heatmap shows **P(class 1)** evolving across epoch blocks — watch the **0.5 decision contour** wrap XOR or untangle spirals.

NewSchool

Linear Regression: OLS, Ridge, Lasso & R²

Launch Simulator

Click/drag scatter points; fit y = β₀ + β₁x with OLS, Ridge (L2 on slope), or Lasso (L1 on slope). Spike Δy on the largest |x| point to see outlier sensitivity; compare SSE and R².

NewSchool

Convex Hull (Graham & QuickHull)

Launch Simulator

Click to add points, drag to move; Graham scan with step playback or QuickHull divide-by-farthest; compare vertex sets.

NewSchool

Newton's Method (1D)

Launch Simulator

Graph f(x), click x₀, iterate x − f/f′ with numeric derivative; pan and presets. Complex basins: separate Newton fractal sim.

NewSchool

PCA in 2D (principal components & 1D projection)

Launch Simulator

Click-built cloud: covariance eigenvectors as PC1/PC2 arrows from the mean, optional orthogonal drops to the PC1 line, and a bottom strip of PC1 scores — the standard rank-one projection coordinate.

NewSchool

K-Means Clustering (Lloyd)

Launch Simulator

Click to add points, choose k, randomize centroids, then step Lloyd iterations (assign to nearest centroid, update means). Optional Gaussian-mixture demo; watch within-cluster SSE decrease.

Decision Tree Classifier (2D toy)

About this model

How it works

Frequently asked questions

Related simulators

Toy 2-Layer MLP + Backprop (XOR / spiral)

Linear Regression: OLS, Ridge, Lasso & R²

Convex Hull (Graham & QuickHull)

Newton's Method (1D)

PCA in 2D (principal components & 1D projection)

K-Means Clustering (Lloyd)

Decision Tree Classifier (2D toy)

About this model

How it works

Frequently asked questions