Enterprise ecosystem orchestration
[Integration · Grupo Diveco]
The portal as a serverless integration hub. Design and development of Serverless orchestration Hub (Lambda/API Gateway) integrating critical systems (SAP ERP/BW, ADF, external providers) with Medallion Architecture for data quality. Eliminated expert dependency by providing self-management UI with centralized real-time visibility.
Results
- 100%[Result]Autonomy (no SQL/SAP experts needed)
- Real-time[Result]Centralized visibility
The problem
Integration of heterogeneous platforms (portal, SAP ERP/BW, Azure Data Factory, external providers) with high complexity. Lack of autonomy where process execution and integration management required specialized SQL/SAP personnel, creating bottlenecks and delays in decision-making.
The solution
I designed a centralized orchestration architecture where the portal functions as a Serverless Hub. Used Lambda and API Gateway as elastic middleware. Orchestrated complex ETL pipelines in ADF applying Medallion Architecture for data quality. Established secure connections with SAP (ERP/BW) and external providers.
The challenge
The integration labyrinth and expert dependency
- Heterogeneity and complexity
- Integrating disparate platforms like the portal (data capture), internal systems (SAP ERP and BW), data pipelines (ADF), and external provider systems (Reborn).
- Lack of autonomy
- Process execution and integration management historically required specialized SQL or SAP personnel, creating bottlenecks and delays in decision-making.
The approach
The portal as serverless hub
- Cloud orchestration layer
- I used AWS Lambda and API Gateway to create a lightweight, elastic layer that functions as middleware, mediating communication between the portal and target/source systems.
- Data ingestion and transformation (ETL)
- Orchestrated complex data pipelines on platforms like ADF (Azure Data Factory), ensuring data quality by applying Medallion Architecture before final data movement.
- SAP and provider integration
- Secure connections were established to deliver clean data to critical systems like SAP (ERP/BW) and consume data from external providers.
The impact
Self-management and strategic visibility
- Operational autonomy
- I created a centralized, user-friendly interface that allows the process owner (without being a SQL, SAP, or database expert) to self-manage process execution, testing, and validations.
- Process visibility
- The portal offers complete control with centralized visibility of each pipeline and data process status in real-time, ensuring traceability.
- Guaranteed data flow
- The Serverless architecture and rigorous ingestion approach ensured that data is captured, transformed, and moved to the correct systems with integrity, eliminating manual errors.
Technologies
AWS Lambda · API Gateway · SAP ERP/BW · Azure Data Factory · Medallion Architecture · ETL · Serverless