Lead Data Engineer — Big Data Platforms, ML Data Pipelines & AI-Powered Data Tools
12 years building production data systems at telecom scale. Currently at Airtel X Labs, where my work supports India's first AI-powered spam detection solution, protecting 300M+ users.
01 / about
I'm a data engineer who has spent 12 years turning messy, high-volume telecom and enterprise data into reliable, governed systems that are ready for analytics and ML. I've led migrations from Oracle to Hadoop data lakes and built CDC pipelines from GoldenGate and Solace queues into Hive. Today I lead the DARTS BI reporting team at Airtel X Labs and work on the data platform behind Airtel's AI-powered spam detection.
Lately I've been combining that foundation with AI. I completed a PGP in AI & ML from Great Lakes and UT Austin. I also built a PySpark framework that monitors data drift across ML pipelines, and an LLM-powered agent that lets my team query spam-detection data in plain English. I care about reliability, about automation that removes manual toil, and about data you can trust.
02 / experience
Dec 2021 – Present · 4 yrs 11 mos
Airtel Digital India Pvt. Ltd. · India
Aug 2018 – Dec 2021 · 3 yrs 5 mos
Pune
Jul 2017 – Aug 2018 · 1 yr 2 mos
Pune
Aug 2014 – Jun 2017 · 2 yrs 11 mos
Gurugram
03 / projects
Selected work across AI tooling, ML data platforms, pipelines and automation. Each has a short case study.
Showing 12 projects
A conversational agent that turns plain-English questions into Hive SQL, runs them on the cluster and replies with results, charts or files.
A reusable PySpark framework that compares feature distributions between time windows and raises early alerts, across multiple spam ML pipelines.
The data platform behind India's first AI-powered spam detection solution, protecting 300M+ Airtel users from spam and fraud calls and SMS.
A continuous change-data-capture pipeline from Oracle GoldenGate and Solace queues into Hive, feeding daily postpaid and Telemedia reports.
Ab Initio pipelines that reintegrate reports into GCP, and automated SMS Insights reporting on the bulk SMS aggregator market, delivered to AWS S3.
Led the migration of Oracle source systems into a Hadoop data lake, with generated DMLs, test data and table-by-table validation.
Re-implemented an Oracle PL/SQL account-hierarchy function (CONNECT BY) as a distributed, iterative PySpark job with checkpointing.
Automates onboarding new business attributes into multiple Ab Initio DMLs, an XFR and a JSON config, with developer approval and auto-tagging.
Detects stale snapshot tables, auto-remediates Hive/HDFS metadata drift, and alerts only when upstream data is genuinely missing.
PII data encryption across the DARTS ecosystem, and the AIO Reporting framework to standardise and automate enterprise reporting.
An Ab Initio regression utility comparing AIX and Linux graph outputs and table loads, behind a zero-data-loss migration.
A side project: a tool-using agent built from scratch on a local LLM, including a portfolio-analysis tool that reads holdings from Excel.
04 / skills
primary skill
05 / achievements
Key contributor to a platform protecting 300M+ Airtel users.
Reusable framework adopted across multiple ML pipelines.
Built the automated test suite and hardened the agent to a full pass rate.
Implemented enterprise-wide PII protection.
Plus the regression utility that verified it.
Recognised for work at Accenture.
Great Lakes & The University of Texas at Austin.
06 / education
2023–2024
Great Lakes Executive Learning & The University of Texas at Austin
CGPA 3.79/4.0
2010–2014
Sinhgad Academy of Engineering, University of Pune
07 / contact
Open to Lead / Senior Data Engineering roles — Bengaluru and other locations, on-site or hybrid.