ArchitectureConference50min
From HDInsight to Spark on Kubernetes: Lessons Learned at Société Générale
This session details Société Générale CIB’s migration from HDInsight to Apache Spark on Kubernetes, covering motivations, architectural decisions, technical challenges, and strategies for secure, scalable, and cost-effective data processing. Attendees will gain practical insights and best practices for modernizing large-scale workloads in financial or data-driven industries.
OMAR ZEROUALSociete Generale Africa Technologies And Services
talkDetail.whenAndWhere
Friday, November 14, 11:00-11:50
Mimosa 2
In this session, we will share the real-world journey of Société Générale CIB in migrating large-scale data processing workloads from Microsoft HDInsight to Apache Spark on Kubernetes (SoK8s). The talk highlights the key drivers behind the move, from cost efficiency to scalability and flexibility, and provides an inside look at the challenges we faced along the way.
We will discuss the architectural choices made, the technical hurdles overcome, and the strategies adopted to ensure enterprise-grade performance, security, and reliability in a financial context. Attendees will walk away with practical lessons, actionable insights, and best practices that can help guide their own modernization efforts — whether in financial services or any other data-intensive industry.
We will discuss the architectural choices made, the technical hurdles overcome, and the strategies adopted to ensure enterprise-grade performance, security, and reliability in a financial context. Attendees will walk away with practical lessons, actionable insights, and best practices that can help guide their own modernization efforts — whether in financial services or any other data-intensive industry.
OMAR ZEROUAL
A seasoned software engineer specializing in large-scale data platforms and distributed architectures. With expertise in Java, Kafka, Apache Spark, Azure Data Lake, and Kubernetes, he has played a pivotal role in designing and deploying secure and scalable data solutions. At Société Générale Africa Technologies & Services, Omar focuses on modernizing enterprise data ecosystems, notably contributing to a cost-efficient transition from HDInsight to Spark-on-Kubernetes environments. Passionate about data-driven systems, distributed architectures, and cloud engineering, he actively shares his experience and best practices with the tech community, helping others accelerate their journey toward resilient and efficient platforms
comments.speakerNotEnabledComments