Building a Modern Enterprise Data Platform for Lenskart
Lenskart partnered with BluePi to design a cloud-native enterprise analytics platform on Google Cloud, supporting 2 TB+ daily ingestion, 6000+ MySQL tables, and near real-time CDC latency.
Lenskart
Client
Retail and E-commerce
Industry
Data Platform, Business Intelligence, Data Governance, Data Strategy
Services
2 TB+ daily data ingestion supported
Key Result
The Challenge
Complex AWS-based analytics ecosystem
Lenskart had grown a complex analytics landscape across AWS DMS, Redshift, S3, MySQL, MongoDB, VM-based CRON jobs, custom API integrations, and Power BI hosted on Azure.
The platform needed to consolidate multi-cloud analytics operations and replace legacy OLTP reporting patterns with a stronger OLAP foundation.
Operational gaps in governance and delivery
Manual ingestion processes, VM-based orchestration, missing CI/CD and version control, limited observability, and limited business ownership of transformation logic constrained scale.
The existing environment also needed stronger auditability, lineage, PII discovery, and data quality monitoring.
Near real-time business reporting requirements
Business users needed dashboard refresh and analytics availability fast enough for retail operations across supply chain, warehouse, finance, marketing, CRM, logistics, and sales.
The target platform had to support more than 2 TB of daily ingestion, 6000+ MySQL tables, 50 MongoDB collections, 600+ integrations, and 250+ Power BI dashboards.
Our Solution
- 1
Cloud-native analytics platform on Google Cloud
BluePi designed an end-to-end modern data platform using Debezium on GKE for enterprise-scale CDC, Pub/Sub for event-driven ingestion, and BigQuery Raw, Silver, Gold, and Reporting layers.
The platform used Dataform-based SQL transformations to standardize business logic and support a domain-oriented analytics architecture.
- 2
Enterprise governance, quality, and security foundation
Dataplex, Cloud DLP, column lineage, data catalog, IAM, VPC Service Controls, auditability, and automated data quality monitoring were built into the target architecture.
The design gave Lenskart a stronger governance foundation for sensitive, cross-domain retail analytics.
- 3
Platform engineering and SmartMigrate acceleration
BluePi used Terraform, Cloud Build, CI/CD pipelines, Cloud Monitoring, Cloud Logging, and automated alerting to create an engineered operating model for the platform.
SmartMigrate accelerated workload assessment, migration planning, SQL modernization strategy, inventory analysis, transformation lineage analysis, and best-practice architecture generation.
Business Impact
2 TB+ daily data ingestion supported
The platform was designed to ingest more than 2 TB of daily data across operational databases, MongoDB collections, APIs, analytics sources, and flat files.
Near real-time analytics with sub-minute CDC latency
The architecture supported near real-time analytics with less than one minute CDC latency and under five minutes end-to-end latency.
6000+ operational tables and 250+ dashboards
The target platform accounted for 6000+ MySQL tables, 50 MongoDB collections, 600+ API and file integrations, and 250+ Power BI dashboards.
Governed self-service analytics foundation
The platform established standardized business data models, faster report development, simplified operations, improved scalability, and a future-ready foundation for AI and ML initiatives.
Customer testimonial
BluePi demonstrated deep expertise in enterprise data platform modernization and cloud architecture, including BigQuery ecosystem expertise and the ability to redesign legacy code and pipelines.
BluePi demonstrated deep expertise in enterprise data platform modernization and cloud architecture. Their team did the majority of heavy lifting not just to lift and shift the data - but to rewrite legacy code and pipelines. They architected the new platform that now ingests more than 2 TB of daily data, has 6000+ operational tables, hundreds of integrations, and 250+ Power BI dashboards. Their expertise in the BigQuery ecosystem is commendable. The proposed architecture delivers near real-time data availability, enterprise-grade governance, and a scalable foundation for analytics across our business.
Lenskart Data Platform Team
Enterprise Data Platform Leadership, Lenskart