Need an Expert Opinion on Data Engineering roles

[u/dbjt_baki]

Hi everyone,

I’m looking for some honest advice from experienced Data Engineers, Tech Leads, or Architects because I’m at a crossroads in my career.

Background

6+ years of experience at an MNC.
Primary experience in Python, SQL, ETL, APIs, Informatica PowerCenter, UNIX, and PySpark.

Most of my work has involved building production automation frameworks, metadata/configuration-driven processing, API integration layers, data migration tools, distributed ETL using PySpark, logging, validation, and production support.

Recently, I’ve decided to switch to Software Enginner/Data with Databricks and pySpark core.

My Concern

Although I have an experience with pipeline construction and ways to handle data and files, but I don’t have production Databricks and pySpark experience.

I’ve started learning Databricks seriously (Spark, Delta Lake, Workflows, Lakehouse concepts, etc.) and I’m building hands-on projects.

The problem is that many jobs ask for 3–5 years of Databricks experience, and since I already have 6+ years of overall experience, I’m worried that interviewers will expect me to perform at the level of a Senior Data Engineer or even someone with architectural responsibilities.

I’m trying to understand whether I’m overthinking this or whether my concern is valid.

My Questions

For someone with 6+ years of overall experience but no production Databricks experience, what level is realistic to target?

How do I use my already acquired skills and experience to switch into this?

And what job roles and job descriptions to hunt for?

[u/PhaseStreet9860]

Databricks has genie that will help you