"See it. Sort it. Sustain it."
An explainable, confidence-aware computer vision system and intelligent waste-sorting assistant powered by MobileNetV2 transfer learning, Grad-CAM visual attention, and actionable disposal guidance.
Municipal solid waste sorting is heavily prone to consumer misclassification, causing widespread recycling batch contamination. EcoSort AI is an end-to-end computer vision solution engineered to bridge this gap. Rather than serving as a basic "black-box label classifier," EcoSort AI is built on principles of responsible, transparent, and actionable AI:
-
Computer Vision & Transfer Learning: Employs an ImageNet pre-trained MobileNetV2 architecture fine-tuned across 6 household waste classes (
cardboard,glass,metal,paper,plastic,trash), achieving ~79.52% validation accuracy (+30.4% over baseline CNN). -
Explainable AI (Grad-CAM): Dynamically computes class activation heatmaps on the final convolutional feature layer (
out_relu), visually demonstrating what image regions drove the classification. -
Confidence-Aware Inference: Evaluates softmax distributions against configured thresholds (High
$\ge 75%$ , Medium$50-74%$ , Uncertain$< 50%$ ) and alerts users when predictions are ambiguous or close-call. - Actionable Disposal Guidance: Translates material labels into practical municipal bin instructions, contamination warnings (e.g., greasy pizza box contamination), and quantified environmental metrics.
- Rigorous ML Diagnostics: Documents genuine dataset challenges, class imbalance (140 trash samples vs ~500 for other categories), and failure modes.
┌─────────────────┐ HTTP Multipart ┌────────────────────────┐
│ User / Client │ ─────────────────────────► │ FastAPI Backend │
│ (Upload / Cam) │ │ (Validation & Preproc)│
└─────────────────┘ └───────────┬────────────┘
▲ │ (1, 224, 224, 3)
│ ▼
│ ┌────────────────────────┐
│ │ MobileNetV2 Pipeline │
│ │ (Frozen Base + Head) │
│ └───────────┬────────────┘
│ │ Softmax Probabilities
│ ▼
│ ┌────────────────────────┐
│ │ Grad-CAM Explainer │
│ │ (Gradients at out_relu)│
│ └───────────┬────────────┘
│ │ Heatmap Overlay
│ ▼
│ JSON Response + Base64 Visualizer ┌────────────────────────┐
└──────────────────────────────────── │ Disposal Rule Engine │
│ (Action & Impact Map) │
└────────────────────────┘
Trained on 2,527 images categorized across six discrete household waste streams:
| Category | Images | Dataset % | Balanced Class Weight | Primary Stream |
|---|---|---|---|---|
| Cardboard | 403 | 15.9% | 1.045 | Dry Mixed Recyclables / Cardboard |
| Glass | 501 | 19.8% | 0.842 | Glass Bottle Banks / Curbside Glass |
| Metal | 410 | 16.2% | 1.029 | Metals & Aluminum Recycling |
| Paper | 594 | 23.5% | 0.709 | Clean Paper Recycling |
| Plastic | 482 | 19.1% | 0.875 | Rigid Plastic Containers (#1, #2, #5) |
| Trash | 140 | 5.5% | 3.015 | General Non-Recyclable Residuals |
The trash category is heavily underrepresented (only 5.5% of the dataset) and visually heterogeneous (composite packaging, broken items, soiled wrappers). In unweighted training, the model exhibited poor recall on trash (F1 ~0.18). By applying balanced loss weighting during fine-tuning (penalizing errors on trash 3.015x more heavily), trash F1 increased to 0.56.
Baseline CNN (From Scratch) ────────► ~49.1% Validation Accuracy (Trash F1: 0.18)
│
▼ Transfer Learning
MobileNetV2 (Frozen Base) ──────────► ~78.3% Validation Accuracy (Trash F1: 0.51)
│
▼ Stage-2 Fine-Tuning + Class Weights
MobileNetV2 (Final Model) ──────────► 79.52% Validation Accuracy (Trash F1: 0.56)
precision recall f1-score support
cardboard 0.98 0.74 0.84 80
glass 0.71 0.85 0.77 100
metal 0.81 0.80 0.81 82
paper 0.84 0.92 0.88 118
plastic 0.75 0.70 0.72 96
trash 0.61 0.52 0.56 27
accuracy 0.80 503
macro avg 0.78 0.76 0.76 503
weighted avg 0.80 0.80 0.79 503
EcoSort AI visually explains why it classified an object using Gradient-weighted Class Activation Mapping (Grad-CAM):
-
Target Feature Layer: MobileNetV2's final convolutional layer (
out_relu, shape:(None, 7, 7, 1280)). -
Gradient Computation: Tracks output class logits back to spatial feature maps using
tf.GradientTape. - Global Average Pooling: Computes importance weights for each feature channel.
-
Heatmap Generation: Positive linear combination passed through ReLU, scaled to
$[0, 1]$ , colormapped (Jet/Turbo), and blended onto the original image at$\alpha = 0.45$ .
- Python 3.10+ (tested on Python 3.13)
- Virtual environment (recommended)
# Clone repository
git clone https://github.com/your-username/EcoSort.git
cd EcoSort
# Install dependencies
pip install -r backend/requirements.txt# Start FastAPI backend and serve web frontend
uvicorn backend.app.main:app --host 127.0.0.1 --port 8000 --reloadOpen your browser and navigate to:
http://127.0.0.1:8000/
Uploads a waste image file and receives top predictions, confidence tier, Grad-CAM heatmap, and disposal recommendations.
Request:
- Content-Type:
multipart/form-data - Body:
image: Binary file (JPEG/PNG/WEBP),top_k: Integer (default: 3)
Response:
{
"success": true,
"filename": "plastic_bottle.jpg",
"inference_time_ms": 28.4,
"prediction": "plastic",
"confidence": 0.874,
"percentage": 87.4,
"confidence_level": "high",
"confidence_message": "The model is highly confident this item belongs to the Plastic category.",
"alert_type": "success",
"top_predictions": [
{"class": "plastic", "confidence": 0.874, "percentage": 87.4},
{"class": "glass", "confidence": 0.082, "percentage": 8.2},
{"class": "trash", "confidence": 0.044, "percentage": 4.4}
],
"recommendation": {
"title": "Plastic Packaging & Containers",
"primary_stream": "Rigid Plastic Recycling",
"action_steps": [
"Inspect resin identification code (#1, #2, #5 most common).",
"Empty liquids and rinse out remnants.",
"Crush bottle to conserve bin volume."
],
"contamination_warning": "Polystyrene (Styrofoam) and flexible film pouches require specialized drop-offs.",
"environmental_impact": "Recycling plastic prevents persistent microplastic accumulation in marine ecosystems."
},
"gradcam": {
"heatmap_url": "data:image/png;base64,...",
"overlay_url": "data:image/png;base64,...",
"target_layer": "out_relu"
}
}Returns system status, loaded model architecture, and class count.
Returns full quantitative evaluation metrics, class distributions, and baseline comparison.
EcoSort/
├── backend/
│ ├── app/
│ │ ├── __init__.py
│ │ ├── main.py # FastAPI server & route handlers
│ │ ├── model.py # Model loading, caching & warm-up
│ │ ├── preprocessing.py # Image normalization & validation
│ │ ├── prediction.py # Top-k scoring & confidence tiers
│ │ ├── explainability.py # Grad-CAM implementation
│ │ └── recommendations.py # Actionable sorting protocols
│ └── requirements.txt
├── frontend/
│ ├── index.html # Semantic, accessible UI
│ ├── css/
│ │ └── styles.css # Premium sustainability design system
│ └── js/
│ └── app.js # Camera, drag-and-drop, API controller
├── models/
│ ├── cnn_baseline.keras # Baseline CNN (~49.1% accuracy)
│ ├── EcoSort_MobileNetV2_Finetuned.keras # Fine-tuned model (~79.52% accuracy)
│ └── class_names.json # Category mapping
├── notebooks/
│ ├── EcoSortAI_BaselineCNN.ipynb
│ └── EcoSort_MobileNetV2_TransferLearning.ipynb
├── reports/
│ ├── confusion_matrix.png # Validation confusion matrix
│ ├── training_curves.png # Loss and accuracy progression
│ ├── classification_report.txt# Precision, recall, F1 per class
│ └── model_analysis.md # In-depth technical report
├── README.md # Project documentation
└── .gitignore
EcoSort AI — Explainable AI Waste Classification System
- Built and deployed an explainable computer-vision system using MobileNetV2 transfer learning to classify household waste across six categories, achieving ~79.52% validation accuracy (+30.4% over a custom baseline CNN).
- Engineered an Explainable AI (Grad-CAM) pipeline targeting feature layer
out_relualongside confidence-aware inference and top-3 ranking to visualize model attention regions and communicate uncertainty.- Developed a FastAPI backend with real-time inference (<35ms), webcam capture integration, and class-specific waste-sorting protocols and contamination warnings.
- Conducted rigorous class imbalance and error analysis, implementing balanced loss weights to improve F1 score on the underrepresented
trashclass from 0.18 to 0.56.
This project is open-source and available under the MIT License.