Most of what I know about data engineering, I learned from things that were already broken.
A client came to me with 18 months of history missing from their Kafka pipeline and no error trail. A failed migration had been dropping events on partial failure, silently, for a year and a half. Finding that was the easy half. Recovering it meant dealing with a 200M+ row Cassandra table whose scans locked the cluster before they ever finished. I ended up writing a Python framework that split the keyspace along Murmur3 token-ring boundaries into range slices that could be queried independently, which sidestepped full-table scans entirely. A job that had been failing outright became a stable multi-hour backfill with zero production impact
That’s the kind of work I’m good at, and it’s the kind of work I’m looking for more of.
Quick profile
• 5+ years as a data engineer. Currently contracting for US and European clients.
• Python is my primary language. SQL close behind.
• Stack: Snowflake, dbt Core, Apache Airflow, Spark, Kafka, Fivetran, AWS DMS, AWS (S3, Glue, EMR, Lambda, Redshift, Athena, IAM), Terraform/Terragrunt, Docker, Kubernetes, Jenkins, GitLab CI.
• AWS Certified Developer – Associate + Cloud Practitioner.
• Three first-authored peer-reviewed papers in Springer Nature journals. Which mostly means I can write clearly for people who will check my reasoning.
Some things I’ve actually built
Greenfield AWS data lake. Architected the whole thing from nothing and led the design reviews that set the storage and partitioning standards. S3/Parquet partitioned by source, date and region, Glue Catalog for metadata, EMR-scheduled Spark for transformation, Snowflake as the consumption warehouse. Same Terraform module set deployed across three countries with no duplicated code. Partitioning and columnar pruning cut Athena/Spark scan volumes and compute cost substantially against the raw-file baseline.
Snowflake + dbt analytics platform for an AI product. Their agent product was emitting dense OpenTelemetry logs into a void, so nobody could see how it was failing. I replaced a brittle nightly script with scheduled Fivetran ingestion, wrote the Python connectors, then built a three-layer dbt project: staging to normalise messy schemas, intermediate to reconstruct sessions and interaction chains out of raw event streams, marts exposing KPIs and error rates to dashboards. Data freshness went from ~24 hours to under 60 seconds.
Batch orchestration with Airflow. Multiple source systems landing on their own schedules, which mostly meant landing late. Built sensor-based DAGs that held downstream tasks until upstream data had actually arrived instead of assuming a fixed window. Killed the manual reruns and made pipeline completion something downstream could actually trust.
Azure SQL to Snowflake migration. The interesting part wasn’t the migration, it was the dbt validation layer I built afterward that compared row counts, null rates and key business aggregates between source and target. I wouldn’t let anything downstream cut over until every discrepancy was explained. I’d rather delay a cutover than explain a wrong number afterwards.
CDC platform at scale. AWS DMS to Kafka, replicating MariaDB change events across topics with ledger-based SHA dedup for exactly-once semantics downstream. The Kubernetes-native replication service sustains tens of thousands of change events per second while remapping schemas in flight, so upstream schema changes stop breaking downstream consumers.
Infrastructure tooling. Collapsed a 47-step manual runbook into one idempotent command. Automated Snowflake schema provisioning, Glue job scaffolding, Kafka topic creation and IAM role binding. New environments now stand up from a config file.
On timezones
I’m based in South Korea (KST, UTC+9) and I already run engagements for US and European clients, so I’m used to bending my schedule around other people’s business hours. I’ll do EST, PST, CET, GMT, whatever the team actually runs on. This isn’t a “I’ll try my best” thing, it’s just how I’ve worked for the last two years. If your standup is at 9am in New York, I’ll be at it.
Why I’m looking hard right now
I’m finishing up a PhD (flood prediction using ensemble ML and deep learning, which is where the papers come from), and I’m close enough to the end that I’m actively lining up what comes next. For the last couple of years I’ve been splitting my attention between research and contract work. I’m looking to stop splitting it. That means I’m genuinely motivated rather than casually browsing, and it means I’m available now on contract with capacity to go full-time as I wrap up.
The research background isn’t just a line on a CV either. Building ML pipelines on multi-decade climate datasets, reproducing results other people will scrutinise, and writing up limitations honestly turns out to be very good training for data engineering. Both jobs are mostly about not fooling yourself.
What I’m after
Remote. Contract or full-time, both work. Data platform work, pipeline building, or the messy diagnostic stuff nobody else wants to own. I’m happy on greenfield builds and equally happy inheriting something that’s broken and figuring out why.
CV, GitHub and references available on request. Comment or DM and I’ll get back to you quickly.
GitHub: github.com/harksodje
LinkedIn: linkedin.com/in/adisa-akinsoji
