Caltech-256 - Object Confusion Map
A TMAP of the Caltech-256 dataset (30,607 images, 257 categories) built from DINOv2 ViT-B/14 CLS embeddings (cosine metric). 96.5% of tree edges connect images from the same broad object group (e.g. sports, clothing, vehicle), with subtree purity averaging 0.875 by group and 0.738 by fine category. The most-connected cross-group pair is sneaker (clothing) <-> tennis-shoes (sports), sharing 132 edges. Some visual bridges reveal genuine ambiguity between fine categories - butterfly and hummingbird share 16 edges, bear and teddy-bear share 8. Tree paths trace how the model gets from one concept to a visually related but semantically distant one: dog reaches teddy-bear in 213 hops via greyhound, horse, and a run of cartoon characters (homer-simpson, cartman).
This demo is hosted externally
The dataset is too large to embed inline. Opens in a new tab.
Open Interactive DemoHow it was made
from tmap import TMAP
from tmap.graph.analysis import boundary_edges, subtree_purity, path_properties
model = TMAP(metric="cosine", n_neighbors=15, seed=42).fit(dinov2_embeddings)
# Object-group boundary edges (e.g. "sports" vs "clothing")
be = boundary_edges(model.tree_, object_groups)
print(f"Same-group edges: {len(model.tree_.edges) - len(be)} / {len(model.tree_.edges)}")
# Subtree purity by broad group vs fine category
group_purity = subtree_purity(model.tree_, object_groups, min_size=10)
category_purity = subtree_purity(model.tree_, categories, min_size=10)
# Why the model confuses "dog" with "teddy-bear"
path = model.path(dog_idx, teddy_bear_idx)
route = path_properties(model.tree_, dog_idx, teddy_bear_idx, categories)