Cloud-Based MLOps & AI-Ready Aerial Data Pipeline for Autonomous Systems
Key Challenges
The project faced limited local computing capacity, the need for highly accurate aerial image annotations, complex data standardization, and strict GDPR-driven data security and residency requirements.
Key Results
A secure, scalable AWS-based MLOps pipeline enabled high-quality annotation and automated data processing for thousands of 4K aerial images, achieving 99.99% annotation accuracy while reducing manual data handling and supporting continuous ML model training.
Overview
A leading European research institute, Fraunhofer Institute for Transportation and Infrastructure Systems IVI specializing in transportation and infrastructure systems required advanced solutions to accelerate the development of vision algorithms for autonomous systems. The project focused on building high-quality, densely annotated aerial drone datasets for panoptic segmentation, and transitioning their machine learning (ML) training framework from constrained local hardware resources to a scalable, secure cloud infrastructure on Amazon Web Services (AWS).
Challenges
- Computational Bottlenecks: The research team was constrained by local infrastructure, relying on a fixed set of approximately 10 A100 instances to train complex ML models, which restricted operational scalability.
- Annotation Accuracy and Volume: Training reliable autonomous algorithms demanded exceptionally precise panoptic segmentation annotations (distinguishing countable objects like vehicles and pedestrians from background elements like roads and buildings) across large volumes of high-resolution aerial images.
- Pipeline Standardization: The team faced the challenge of converting heterogeneous annotated data structures into the standardized COCO Panoptic format while preserving data integrity and maintaining compatibility with existing training pipelines.
- Data Residency and Governance: Strict adherence to data privacy regulations (GDPR) was mandatory, necessitating a secure architecture that prevented data proliferation outside the cloud account during remote annotation processes.
Solution
- Secure Remote Annotation Workflow: Integration was established between a dedicated AWS S3 storage bucket and Labelbox, a GDPR-compliant annotation platform. Data residency was maintained within the cloud environment, enabling a remote team to perform virtual annotations without copies ever being retained on external machines.
- Automated Data Conversion Pipeline: A robust MLOps pipeline was deployed on Amazon SageMaker. It automatically pulled completed annotations via the Labelbox API, executed custom Python transformation scripts to handle unique category mapping and instance tracking, and converted mask PNGs and JSON files directly into the COCO Panoptic standard.
- Cloud Infrastructure Governance: Multi-tiered IAM user groups (Administrator, External, Research Associate, Student) were configured to enforce strict least-privilege access across SageMaker, EC2, S3, and VPC resources. Proactive cloud financial governance was instituted via AWS Budget alerts and automated tagging policies preventing untagged resource consumption.
- Secure Networking: A dedicated Site-to-Site VPN with an AWS Virtual Private Gateway was provisioned to ensure seamless and encrypted communication between the on-premises network and the cloud environment.
Business Outcome
- Unmatched Annotation Quality: Achieved a 99.99% annotation accuracy rate through rigorous manual review and quality assurance, establishing a gold-standard dataset optimized for immediate ML model training.
- High-Throughput Processing: Successfully annotated, standardized, and uploaded 1000s of high-resolution 4K images divided across multiple rolling batches within tight project timeline constraints.
- Optimized Operational Workflow: Eliminated manual data handling and physical media transfers through direct S3-to-Labelbox integration, saving significant operational overhead and ensuring rolling batch availability for continuous training.
For more details about the OccuFly paper and the official publication page, please refer to GitHub
