Multi-Object Tracking (MOT) with YOLOv8 Detection and BoT-SORT Tracking
Real-time object detection and tracking system using YOLOv8 for detection and BoT-SORT with CLIP-based ReID for robust multi-object tracking.
- YOLOv8 Detection — Fast and accurate object detection
- BoT-SORT Tracking — State-of-the-art multi-object tracking
- ReID Integration — Re-identification for handling occlusions
- CLIP-based ReID — Using CLIP ViT models for robust appearance matching
- Real-time Processing — Live webcam and video file support
- MOT17 Evaluation — Built-in evaluation scripts for benchmarking
https://github.com/user-attachments/assets/357b8515-b4bc-4ee9-8b61-037c824737d1
(https://github.com/user-attachments/assets/f76395c7-b3ff-482d-a41a-e095012ac4a2)
Video showing real-time detection + tracking with ReID
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ │ │ │ │ │ │ │
│ Input Video │───▶│ YOLOv8 │───▶│ BoT-SORT │───▶│ Output │
│ │ │ Detection │ │ Tracking │ │ Video │
│ │ │ │ │ │ │ │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
│ │
▼ ▼
┌─────────────┐ ┌─────────────┐
│ Bounding │ │ CLIP │
│ Boxes │ │ ReID │
│ + Confidence│ │ Matching │
└─────────────┘ └─────────────┘
| Step | Component | Description |
|---|---|---|
| 1️) | Input | Video file or webcam stream |
| 2️) | Detection | YOLOv8 detects objects (persons) |
| 3️) | Tracking | BoT-SORT assigns track IDs |
| 4️) | ReID | CLIP model handles occlusions |
| 5️) | Output | Video with tracked bounding boxes |
Tracker: BoT-SORT + CLIP ViT-B/16
| Metric | Score | Description |
|---|---|---|
| HOTA | 43.3% | Higher Order Tracking Accuracy |
| MOTA | 37.0% | Multi-Object Tracking Accuracy |
| IDF1 | 47.2% | ID F1 Score |
| Precision | 93.9% | Correct detections / Total predictions |
| Recall | 39.9% | Correct detections / Total ground truth |
| Metric | Value |
|---|---|
| ID Switches (IDSW) | 364 |
| False Positives (FP) | 2,815 |
| False Negatives (FN) | 64,861 |
| Type | Count |
|---|---|
| Ground Truth IDs | 141 |
| Predicted IDs | 255 |
- High Precision (93.9%) — Very few false detections
- Low Recall (39.9%) — Many objects missed (due to occlusion/crowd)
- ID Switches (364) — Some ID swapping during occlusions
robonomous/
├── assets/ # Demo images and videos
│ ├── detection_demo. jpg
│ └── tracking_demo.gif
├── configs/ # Configuration files
│ └── configs.yaml
├── configs_loader/ # Config loading utilities
│ └── config_loader.py
├── demos/ # Demo scripts
│ └── robotics_demo.py
├── evaluation/ # Evaluation scripts
│ ├── evaluation_mot17.py
│ └── hota_evaluation.py
├── predictions/ # Model predictions
│ ├── predictions.txt
│ └── predictions_cleaned.txt
├── src/ # Source code
│ └── trackers/
│ ├── __init__.py
│ ├── botsort_tracker.py
│ ├── botsort_clipVitb16_tracker.py (i have used)
│ └── botsort_clipVitH14_tracker.py
├── utils/ # Utility functions
│ ├── data_cleaning.py
│ └── yolo_frames_to_video.py
├── . gitignore
├── environment.yaml
├── LICENSE
├── README.md
├── requirements.txt
└── pyproject.toml
# Clone the repository
git clone https://github.com/ynyetname/robonomous.git
cd robonomous
# Create conda environment
conda env create -f environment.yaml
# Activate environment
conda activate mot_env# Clone the repository
git clone https://github.com/ynyetname/robonomous.git
cd robonomous
# Create virtual environment (optional)
python -m venv venv
source venv/bin/activate # Linux/Mac
# or
venv\Scripts\activate # Windows
# Install dependencies
pip install -r requirements.txt| Package | Version | Description |
| ultralytics | 8.3.233 | YOLOv8 implementation | | boxmot | latest | BoT-SORT tracker | | opencv-python | 4.12.0.88 | Image processing | | numpy | 2.2.6 | Numerical computing | | torch | latest | Deep learning framework | | torchvision | latest | Vision utilities |
Edit configs/configs.yaml:
python evaluation/evaluation_mot17.py
python evaluation/hota_evaluation.pyEdit configs/configs.yaml to customize:
# Detection Settings
detection:
model: "yolov8m. pt" # YOLOv8 model (n/m/l)
confidence_threshold: 0.01 # Detection confidence
target_classes: [0] # 0 = person
# Tracker Settings
tracker:
reid_weights: "clip_market1501.pt" # ReID model
track_high_thresh: 0.05
track_low_thresh: 0.01
match_thresh: 0.8
appearance_thresh: 0.15
track_buffer: 1200
# Video Settings
video:
fps: 30
codec: "mp4v"
video_file_path: 0 # 0 for webcam, or path to video| Model | Description | Performance |
| clip_market1501.pt | CLIP ViT-B/16 trained on Market1501 | Best (i have used) |
| osnet_x1_0_msmt17.pt | OSNet trained on MSMT17 | Good |
| osnet_x0_25_msmt17.pt | Lightweight OSNet | Fast |
| Tracker | File | Description |
| BoT-SORT | botsort_tracker.py | Basic tracker |
| BoT-SORT + CLIP ViT-B/16 | botsort_clipVitb16_tracker. py | Enhanced with CLIP (i have used) |
| BoT-SORT + CLIP ViT-H/14 | botsort_clipVitH14_tracker.py | High-performance |
| Metric | What It Measures | Good Score |
|---|---|---|
| HOTA | Overall tracking accuracy | > 50% |
| MOTA | Detection + tracking accuracy | > 50% |
| IDF1 | How well IDs are maintained | > 55% |
| Precision | False positive rate | > 90% |
| Recall | False negative rate | > 70% |
Tracking output follows MOT format:
frame_id, track_id, x, y, width, height, confidence, -1, -1, -1
Example:
1, 1, 100, 200, 50, 120, 0.95, -1, -1, -1
1, 2, 300, 150, 45, 110, 0.89, -1, -1, -1
2, 1, 102, 198, 50, 120, 0.94, -1, -1, -1
- Improve tracking accuracy during occlusions and camera angle changes
- Enhance real-time webcam detection and tracking performance
- Add YOLOv9/YOLOv10 support
- Implement DeepSORT tracker for comparison
- Support for multiple camera inputs
- Docker container for easy deployment
- Add more ReID models (OSNet, MGN)
- Improve recall with better detection confidence tuning
Contributions are welcome! Please feel free to submit a Pull Request.
- Fork the repository
- Create your feature branch (
git checkout -b feature/AmazingFeature) - Commit your changes (
git commit -m 'Add some AmazingFeature') - Push to the branch (
git push origin feature/AmazingFeature) - Open a Pull Request
This project is open source and available under the MIT License.
- Ultralytics YOLOv8 — Object detection
- BoxMOT — BoT-SORT implementation
- MOT17 Dataset — Benchmark dataset
- CLIP — ReID feature extraction
Author: ynyetname
GitHub: @ynyetname