Jersey Number Recognition Pipeline

Overview

COSC 419B team project at UBC (team of 5). Built a spatio-temporal jersey number recognition pipeline for the SoccerNet challenge. The system processes raw player tracklets through quality-based keyframe selection, MediaPipe Pose-based torso cropping (TorsoCropper), two-stage recognition (MobileNetV3-Large CNN and a ResNet-18 + Bi-LSTM with attention), and confidence-weighted tracklet consolidation. TorsoCropper isolates tight jersey-region crops per frame, removing background, head, and legs to improve downstream accuracy. Final predictions are aggregated via MC Dropout test-time augmentation and confidence-weighted majority vote across each tracklet.

Key Highlights

• MobileNetV3-Large single-image CNN — 67.9% per-frame val accuracy on 100-class jersey number recognition
• Spatio-temporal Bi-LSTM with ResNet-18 features, attention mechanism, and dual digit heads — 60.4% joint-digit accuracy
• MediaPipe Pose-based TorsoCropper isolating tight jersey crops, removing background, head, and legs
• Quality-based keyframe filtering using sharpness, brightness, and contrast scoring heuristics
• MC Dropout test-time augmentation with confidence-weighted majority vote across tracklets
• Dual digit prediction heads (tens/units) for per-digit decomposition of jersey numbers 0–99