Ravi Shankar

Data engineer. I build Databricks and Azure Data Factory pipelines that move health-insurance data out of legacy systems and into production, then make them run faster.

Based in Melbourne, Australia.

Production jobs I've tuned

Runtimes of Databricks jobs before and after I optimised their SQL and PySpark notebooks.

Job A

Before
2 hrs
After
45 min

Job B

Before
~90 min
After
35–40 min

Experience

Data & Insights Analyst (data engineering)

Jan 2026 – present

Bupa (via contract), Melbourne

  • Build SCD2 dimension and fact tables in Databricks for an actuarial data migration program, in a star-schema layout.
  • Built a daily incremental load that went from development through pre-production and is now live in production.
  • Design parameterised Azure Data Factory pipelines that move data from SQL Server into Databricks, and set and monitor their schedules.
  • Have 10+ tables and jobs live in production, ranging from about 1 million to over 700 million rows.
  • Reconcile migrated history against legacy reports, and traced premium and claims values back to legacy COBOL logic.
  • Deploy with Databricks Asset Bundles and Azure DevOps (Repos, Pipelines, Test Plans), with Unity Catalog for governance.
  • Onboarded 5–10 colleagues across two teams and ran knowledge transfer on my work.

Java Developer

May 2021 – Aug 2022

Tata Consultancy Services (TCS)

  • Wrote stored procedures and SQL across MySQL, PostgreSQL and Oracle.
  • Improved MySQL query performance by 20%.
  • Resolved 80% of user queries.

Team Member

Part-time

Officeworks, Melbourne

  • Reduced out-of-stocks by 20%.
  • Named Team Member of the Month.

Skills

Platforms
Databricks (serverless, Unity Catalog, Asset Bundles), Azure Data Factory, Azure DevOps
Languages
SQL, Python (PySpark, pandas), Java, R
Modelling
SCD2 dimensions, fact tables, star schema, medallion architecture, incremental loads
Databases
SQL Server, PostgreSQL, Oracle, MySQL
Reporting
Power BI

Education and certification

  • Databricks Certified Data Engineer Associate

    Databricks

  • Master of Data Science

    Deakin University, 2024

  • Bachelor of Engineering, Computer Science

    Sri Krishna College of Technology, India, 2021

Earlier projects on GitHub

Get in touch

I'm open to data engineering roles. Email is the quickest way to reach me.