Samaresh Kumar Singh

As AI adoption accelerates, many organizations discover that cloud-only inference cannot meet the latency, bandwidth, privacy, and availability requirements of real-time systems. This talk presents a practical, production-oriented blueprint for moving inference to the edge running AI where data is generated while keeping the cloud for training, governance, and continuous improvement. We introduce a three-tier cloud–edge–device architecture and discuss how to orchestrate heterogeneous hardware (CPU/GPU/NPU/TPU/DPU) using modern scheduling patterns: centralized control, distributed consensus, hierarchical federation, and increasingly, agent-based coordination for autonomous edge deployments. The session dives into low-latency strategies (quantization, distillation, hardware-specific runtimes, caching, dynamic batching, early-exit inference, and pipeline parallelism) and resilience-by-design techniques (offline-first operation, local persistence, eventual consistency, and graceful degradation using MQTT/NATS, CRDT-inspired state handling, and circuit-breaker patterns). Finally, we map these ideas to real-world smart-city and industrial scenarios predictive maintenance, healthcare monitoring, autonomous perception, and smart-building optimization highlighting the trade-offs and implementation decisions that enable dependable edge AI at scale. Attendees will leave with actionable architectural patterns, optimization tactics, and reliability practices to build secure, portable, low-latency AI systems across the cloud–edge continuum.