Can DeepSORT Realtime Run on Mobile Devices?

Short Answer: Yes, but with significant optimizations and trade-offs since mobile devices lack dedicated GPUs.

Challenges for Mobile Deployment

1. GPU Dependency:
   • DeepSORT relies on Re-ID models (CNNs) for appearance features, which are optimized for GPU parallelism.
   • Mobile CPUs struggle with real-time inference due to limited compute power.
2. Memory Constraints:
   • nn_budget stores appearance features in GPU VRAM. Mobiles share CPU/GPU memory, risking bottlenecks.
3. Thermal/Power Limits:
   • Sustained high compute (e.g., YOLO + DeepSORT) can throttle performance or drain batteries quickly.

Solutions to Run DeepSORT on Mobile

1. Optimize Models for Edge Devices

• Replace Re-ID Model: Use lightweight models like:
  • MobileNetV3 (TensorFlow Lite)
  • OSNet (for Re-ID, paper)
  • Quantized Models (INT8 precision to reduce compute).
• Framework: Deploy via:
  • TensorFlow Lite
  • PyTorch Mobile
  • ONNX Runtime (with hardware acceleration).

2. Reduce Computational Load

• Lower nn_budget (e.g., 20 instead of 100) to limit feature storage.
• Decrease input resolution (e.g., 320x320 instead of 640x640).
• Use CPU-friendly trackers (e.g., SORT without Re-ID) if ID switches are acceptable.

3. Hardware Acceleration

• Mobile GPUs (e.g., Qualcomm Adreno, Apple Neural Engine) via:
  • CoreML (iOS)
  • Android NNAPI
• Edge TPUs (Google Coral) for quantized models.

4. Cloud Offloading

• Offload Re-ID to a server (latency trade-off).