Exploring GPU sharing techniques for edge AI smart city applications

초록

The growing adoption of edge AI in smart city applications such as trafficmanagement, surveillance, and environmental monitoring necessitates effi-cient computational strategies to satisfy the requirements for low latency andhigh accuracy. This study investigated GPU sharing techniques to improveresource utilization and throughput when running multiple AI applicationssimultaneously on edge devices. Using the NVIDIA Jetson AGX Orin platformand object detection workloads with the YOLOv8 model, we explored the per-formance tradeoffs of the threading and multiprocessing approaches. Our find-ings reveal distinct advantages and limitations. Threading minimizes memoryusage by sharing CUDA contexts, whereas multiprocessing achieves higherGPU utilization and shorter inference times by leveraging independent CUDAcontexts. However, scalability challenges arise from resource contention andsynchronization overheads. This study provides insights into optimizing GPUsharing for edge AI applications, highlighting key tradeoffs and opportunitiesfor enhancing performance in resource-constrained environments.

키워드