Services
Cloud Solutions, DevOps
Industries
Financial Services
Product
A leading global exchange organization providing comprehensive solutions in securities, derivatives trading, settlement, and market data management, sought to enhance its service offerings and operational efficiency. The client required a critical service’s development and cloud migration, aiming for scalability, reliability, and faster deployment times.
To achieve this, they partnered with Svitla Systems for our expertise in cloud technology. This partnership enabled the client to efficiently process and annotate vast amounts of financial data from multiple sources, leveraging advanced ETL processes and microservices. Svitla Team’s efforts ultimately enhanced the value delivered to their FinTech and financial institution customers.
Business needs
- Upgrade the current solution to enhance functionality and performance.
- Address the gap in capacity and expertise to develop a sustainable roadmap and technical design.
- Complete the development of a crucial new application.
- Transition data processing to Azure data services for scalability and reliability.
- Implement automated CI/CD processes and provide continuous DevOps support for optimal operation.
Suggested solutions from svitla
- Migrated the application to Azure, emphasizing code efficiency, configuration-based processes, generic solutions, reusability, reduced maintenance, and simple architectures.
- Coordinated closely with other teams, managing cloud resources and CI/CD processes.
- Deployed Azure Data Factory with a main orchestration pipeline, three scenario-specific pipelines, and additional aggregate tools.
- Developed Python Azure Functions for advanced operations, handling incremental file processing and integration with Azure Databricks.
- Implemented ETL computations in Azure Databricks, utilizing two main notebooks for business logic execution and common processing steps.
- Utilized Azure SQL Database for final data storage, employing dynamic queries and generic stored procedures for data merging.
- Scripted the processing and preparation of historical data for migration.
Technologies
Cloud Infrastructure and Networking: Azure Storage Accounts, Azure Function Apps, Azure SQL Database, Azure Data Factory, Azure Databricks, Azure SQL Database
DevOps Tools: PowerShell, Terraform, GitHub Enterprise
Tools and Services: Python, React
Value delivered
- A significant improvement was achieved as the data components and backend successfully migrated to the cloud.
- The transformation and final exposure of raw data streamlined backend operations.
- Migrating historical data to the new application ensured seamless continuity and access to essential historical insights.
- Unifying heavily parametrized pipelines in Azure DataFactory allowed for efficient management of 30 different data sources, negating the need for individual pipelines per source.
- Streamlining Azure Functions for verifications, normalization, and identifications significantly reduced code complexity and maintenance requirements.
- Simplifying the Azure SQL Database structure, with processing handled by a generic stored procedure managed by end-users, markedly enhanced system usability and flexibility.
See more cases
Automation for Drone Data Analysis
Discover how we developed an automated information system for a global conglomerate, streamlining data analysis from drone-captured media through microservices architecture and AWS cloud services, resulting in enhanced efficiency, reduced manual effort, and improved data visualization for Data Analysts.
NLP Enhanced Recruitment Solution
Explore how we transformed recruitment for an AI technology company by implementing a sophisticated user communication system and enhancing interview processes with real-time natural language processing, significantly boosting efficiency and enhancing candidate experiences.
SaaS Platform For Solar PV Provider
Discover how we upgraded a leading US manufacturer's SaaS platform for solar PV products, which operates across over 35 countries. Our team transitioned the platform to a serverless architecture with microservices, automated critical QA processes, and utilized advanced AWS cloud services.