Oxford Pets - Breed Classifier Audit

3,669 imagesMLxAIcomputer-vision

A TMAP built from ResNet-50 avgpool embeddings (2048-d, cosine metric) of the Oxford-IIIT Pets test set. A linear probe achieves 90.4% accuracy across 37 breeds. Tree analysis reveals 302 boundary edges (8.2%) where neighboring points have different predicted breeds. The most confused pairs are American Bulldog <-> Staffordshire Bull Terrier (24 edges) and British Shorthair <-> Russian Blue (24 edges). Subtree purity (predicted labels) has mean 0.67 and median 0.82. Failure paths trace from misclassified images to their nearest correctly classified same-breed neighbor, showing how model confidence changes along the tree. High-confidence misclassifications (e.g., Birman predicted as Siamese) are mislabel candidates.

Open Interactive Demo

How it was made

generate.pypython

from tmap import TMAP
from tmap.graph.analysis import (
    boundary_edges, confusion_matrix_from_tree,
    edge_delta, subtree_purity, path_properties,
)

model = TMAP(metric="cosine", n_neighbors=15).fit(embeddings)

# Boundary edges: where predicted breed changes
be = boundary_edges(model.tree_, predicted_breeds)
print(f"Boundary edges: {len(be)} / {len(model.tree_.edges)} ({len(be)/len(model.tree_.edges):.1%})")

# Confusion matrix from tree structure
cmat, classes = confusion_matrix_from_tree(model.tree_, predicted_breeds)

# Confidence gradients across edges
deltas = edge_delta(model.tree_, confidences)

# Failure path: misclassified -> nearest correct same-class
conf_along = path_properties(model.tree_, wrong_idx, correct_idx, confidences)