Data Engineering Track
Real production problems, not toy puzzles
102 hand-picked scenarios across SQL, modeling, databases, batch pipelines, streaming, storage, cloud cost, system design and people work, plus a senior interview track that walks through the everyday Monday-morning situations and senior-level improvements. Each one is short enough to solve in hours but real enough to prepare you for production and for senior interviews.
Start here
Data Engineering Roadmap: Beginner to Expert
A staged learning path from SQL fluency to running data systems at scale. Six months, i...
Curated for seniors
Senior Interview Scenarios
12 Monday-morning situations with the senior-level fix walked through
Interview prep
Interview Must-Haves
37 problems that show up in almost every senior interview loop
#15 Medium
Teaching SQL Performance to a Junior
SQL & Querying
EXPLAINperformancementoringoptimization
#18 Medium
CTE vs Subquery
SQL & Querying
CTEsubquerymaterializationrecursion
#20 ⭐ Medium
Window Functions vs GROUP BY
SQL & Querying
window functionsGROUP BYrunning totalsranking
#16 Medium
SELECT DISTINCT Hiding Join Bugs
SQL & Querying
DISTINCTjoinsgrainsemi-join
#17 ⭐ Medium
Reading an EXPLAIN Plan
SQL & Querying
EXPLAINquery planjoinssort spill
#19 Medium
Same Query Different Answers
SQL & Querying
time zonesRLSsession settingsdebugging
#8 ⭐ Easy
OLTP vs OLAP
Databases & Operations
OLTPOLAPcolumn storerow store
#75 Medium
SQL vs NoSQL
Databases & Operations
SQLNoSQLKVdocument +2
#67 ⭐ Easy
Transactions and ACID
Databases & Operations
transactionsACIDdurabilityatomicity
#68 ⭐ Medium
Isolation Levels in Plain Words
Databases & Operations
isolationsnapshotanomaliesMVCC
#69 Medium
Normalization and When to Denormalize
Databases & Operations
normalization3NFdenormalizationstar schema
#66 ⭐ Easy
Indexes When to Add and When They Hurt
Databases & Operations
indexesB-treewrite costEXPLAIN
#70 ⭐ Medium
B-Tree vs Hash vs LSM Tree
Databases & Operations
B-treehashLSMstorage engines
#73 Medium
Database Connection Pooling
Databases & Operations
connection poolPgBouncersizingPostgres
#74 Medium
Deadlocks and Lock Escalation
Databases & Operations
deadlockslocksretrieslock escalation
#71 Medium
Read Replicas and Replication Lag
Databases & Operations
replicasreplication lagread after write
#72 ⭐ Hard
Sharding and Picking a Shard Key
Databases & Operations
shardingshard keyhot shardshash
#44 ⭐ Easy
Explaining Fact Table Grain
Data Modeling
grainfactsdimensionsaggregations
#43 Medium
Mixing Facts and Dimensions
Data Modeling
star schemaSCD2viewshistory
#10 ⭐ Medium
Slowly Changing Dimensions
Data Modeling
SCDdimensionshistorydbt snapshot
#41 ⭐ Medium
Tables for an Airbnb Like App
Data Modeling
star schemaSCD2multi-currencyreviews
#42 Medium
Tracking Subscription Plan History
Data Modeling
historyvalid_from/tobillingSCD2
#45 Medium
Current State and Full History
Data Modeling
event sourcingprojectionsMVaudit
#7 ⭐ Easy
ETL vs ELT and Why ELT Won
Batch Pipelines & Orchestration
ETLELTdbtwarehouse
#9 ⭐ Medium
Idempotency in Data Pipelines
Batch Pipelines & Orchestration
idempotencyretriesMERGEpartitions
#11 Medium
Data Contracts in Plain Words
Batch Pipelines & Orchestration
data contractsschema registryownership
#1 Easy
Log File Error Analysis
Batch Pipelines & Orchestration
file streamingcounterstop-NIoT logs
#3 Medium
Transform and Clean Raw Data for Analytics
Batch Pipelines & Orchestration
CSVvalidationregexdate checks
#5 Medium
Merging Messy CSVs from Multiple Partners
Batch Pipelines & Orchestration
CSVcolumn mappingdate parsingfile walk
#12 ⭐ Easy
Parquet vs CSV vs JSON
Storage & Lakehouse
ParquetCSVJSONcolumnar storage
#13 ⭐ Medium
Data Lake vs Warehouse vs Lakehouse
Storage & Lakehouse
lakewarehouselakehouseIceberg +1
#6 Easy
Partitioning vs Clustering in BigQuery
Storage & Lakehouse
BigQuerypartitioningclusteringcost
#55 Easy
Partitioning Clustering Materialized Views
Storage & Lakehouse
partitioningclusteringMVBigQuery
#38 Easy
Store Partner Files in S3 or Warehouse
Cloud & Cost
S3raw layerauditschema evolution
#35 Medium
Lambda vs Cloud Function vs Cloud Run
Cloud & Cost
serverlessAWSGCPruntime limits
#36 Easy
Scheduled Pipeline Pay Only When Run
Cloud & Cost
scheduled jobsCloud Run JobsAWS Batch
#37 Medium
BigQuery vs Snowflake for New Team
Cloud & Cost
BigQuerySnowflakepricing model
#39 Medium
Managed Airflow vs Self Hosted
Cloud & Cost
AirflowMWAAComposerAstronomer +1
#40 Medium
BigQuery Access Control for 50 Person Company
Cloud & Cost
IAMdatasetsgroupsRLS +1
#2 Easy
Rolling Average of Sensor Readings
Streaming
rolling windowdequeIoT sensorsreal-time
#4 Medium
Schema Evolution and Validation for Streaming Events
Streaming
JSONschema evolutiontype coercionpydantic
#56 ⭐ Medium
Watermarks in Plain Words
Streaming
watermarksevent timeallowed lateness
#57 ⭐ Medium
Kafka Ordering Guarantee
Streaming
Kafkapartition keyorderingidempotent producer
#14 ⭐ Medium
Exactly Once Delivery
Streaming
exactly onceidempotencyKafkastreaming
#58 ⭐ Medium
Streaming Consumer Lag Diagnosis
Streaming
lagback-pressureskewFlink UI
#22 ⭐ Hard
Banking App Monthly Spending Widget
System Design
streamingCDCserving storelow latency
#24 ⭐ Hard
Spotify Minutes Listened This Week
System Design
streaming aggregationKV storewatermarks
#23 Hard
Ride Hailing Surge Pricing
System Design
streamingH3real-timepricing
#26 Hard
Delivery Idle Driver Tracking
System Design
streamingH3TTLgeospatial
#27 Medium
Year in Review Recap
System Design
batchKV storeCDNimage render
#28 Medium
Low Balance Notification Pipeline
System Design
batchidempotencytime zonesnotifications
#25 Hard
Smart Meter to Monthly Bill PDF
System Design
billingSCD2idempotencyaudit
#21 Hard
Data Platform for an Electricity Retailer
System Design
smart meterIoTwarehousebatch
#49 Easy
User Says Data Is Wrong
Batch Pipelines & Orchestration
commsvague reportstriage
#31 Easy
The Dashboard is Wrong
Batch Pipelines & Orchestration
trustcommsvague reports
#46 ⭐ Medium
Region Suddenly Shows Zero Revenue
Batch Pipelines & Orchestration
dashboardjoinsSCDtime zones
#47 ⭐ Medium
Airflow Green but Output Empty
Batch Pipelines & Orchestration
silent successidempotencyanomaly checks
#29 ⭐ Medium
Daily Report Quietly Wrong for Two Weeks
Batch Pipelines & Orchestration
incidentpostmortemcommsdata quality
#50 Medium
Partition Always Ten Percent Smaller
Batch Pipelines & Orchestration
anomalybaselinespatternsjudgement
#48 ⭐ Medium
Query Suddenly 80x Slower
Databases & Operations
EXPLAINstatisticsplan flipjoin strategy
#34 ⭐ Hard
Three Days of Data Lost
Batch Pipelines & Orchestration
Kafka retentionreplayrecoverypostmortem
#65 Medium
4000 DAG Airflow at 90 Percent CPU
Batch Pipelines & Orchestration
Airflowschedulerparsingscale-out
#54 Medium
Just Throw More Memory At It
Cloud & Cost
upsizeplan inspectionoptimization
#53 Easy
Hourly Scan on Daily Data
Cloud & Cost
summary tablesMVrefreshBI tool
#51 ⭐ Medium
BigQuery Bill Eight Times Higher
Cloud & Cost
INFORMATION_SCHEMAtop queriesslot reservation
#30 ⭐ Medium
Warehouse Cost Doubled in Two Months
Cloud & Cost
costgovernancecommsINFORMATION_SCHEMA
#52 Medium
Four Hour Spark Job Under One Hour
Cloud & Cost
Spark UIskewAQEbroadcast joins
#59 Easy
Onboarding a New Analyst
People & Process
onboardingmentoringpairing
#60 Easy
Metric by Tomorrow vs Doing It Right
People & Process
commsprioritizationmetrics
#33 Medium
Executive Needs a Number Tomorrow
People & Process
commsexeccaveatsprioritization
#61 Medium
Two Teams Disagree on Active User
People & Process
metric ownershipcommsmetrics layer
#32 Medium
Inheriting a Pipeline No One Owns
People & Process
ownershipjudgementrewrite-or-not
#63 Medium
Inherited Pipeline No Docs No Tests
People & Process
ownershipdocstestsexpectations
#64 Medium
Breaking Change in dbt Model 200 Consumers
People & Process
dbtdeprecationcommsrollout
#62 ⭐ Medium
Postmortem After a Bad Day
People & Process
postmortemblamelessaction items
#87 Easy
DuckDB for Single-Machine Analytics
SQL & Querying
DuckDBanalyticsOLAPsingle-node
#88 Easy
Polars vs Pandas for ETL
SQL & Querying
PolarsPandasArrowETL
#77 Medium
dbt Incremental vs Full Refresh
Batch Pipelines & Orchestration
dbtincrementalMERGEmaterialisation
#78 Medium
dbt Snapshots for Slowly Changing Dimensions
Data Modeling
dbtsnapshotSCD type 2history
#76 ⭐ Medium
Change Data Capture, Log-Based vs Query-Based
Batch Pipelines & Orchestration
CDCDebeziumbinlogreplication
#83 ⭐ Medium
PII, Masking, and Right-to-be-Forgotten
Data Modeling
PIIGDPRmaskingtokenisation
#80 ⭐ Medium
Iceberg, Schema Evolution and Time Travel
Storage & Lakehouse
Iceberglakehouseschema evolutiontime travel
#81 Medium
Delta vs Iceberg vs Hudi
Storage & Lakehouse
Delta LakeIcebergHudilakehouse
#86 Medium
Event Tracking Schema Design
Streaming
event trackingtelemetryschemaanalytics
#79 ⭐ Hard
Spark Shuffle, Skew, and the Small File Problem
Cloud & Cost
Sparkshuffleskewpartitioning
#89 Medium
Great Expectations vs dbt Tests
Batch Pipelines & Orchestration
data testingdbt testsGreat ExpectationsCI
#82 Medium
Column-Level Lineage in Practice
Batch Pipelines & Orchestration
lineageOpenLineagedbtimpact analysis
#84 ⭐ Medium
Data Observability, Freshness, Volume, and Drift
System Design
observabilitySLOfreshnessdrift
#90 Medium
OpenLineage and Data Discovery
System Design
OpenLineageDataHubOpenMetadatacatalog
#85 Hard
Feature Store, Online vs Offline
System Design
feature storeMLOpsonlineoffline
#91 Medium
Partner CSV, Schema Drift on Monday Morning
Batch Pipelines & Orchestration
schema driftcontractsingestionpartner data
#92 Medium
Late Events After the Partition Closed
Streaming
late datawatermarksevent timereprocessing
#93 Medium
Dashboard Stale Despite a Healthy Job
Batch Pipelines & Orchestration
stale datafreshnesscacheSLA
#94 ⭐ Hard
Versioning a Breaking Grain Change
Data Modeling
versioninggraindbtdeprecation +1
#95 ⭐ Medium
Duplicate Rows From a Successful Pipeline
Batch Pipelines & Orchestration
duplicatesidempotencyMERGEunique key
#96 Medium
The 20-Minute Query That Should Be 2 Seconds
SQL & Querying
EXPLAINpartitioningclusteringoptimization
#97 Hard
Backfill 90 Days Without Blowing the Budget
Batch Pipelines & Orchestration
backfillidempotencypartition replacethrottling
#98 ⭐ Medium
Same Metric, Three Different Numbers
Data Modeling
semantic layersingle source of truthdefinitionsgovernance
#99 Medium
No Staging Layer, Everything Touches Raw
Data Modeling
staginglayered architecturedbt structurerefactoring
#100 Medium
Timezones Silently Wrong Across the Warehouse
SQL & Querying
timezonesUTCtimestampsdaylight saving
#101 Medium
No Dev Environment, Everyone Tests in Prod
People & Process
environmentsdbt targetsschema-per-PRCI
#102 Hard
Critical Model, No Tests, Nobody Dares Touch It
Batch Pipelines & Orchestration
testingrefactoringcharacterization testslegacy
No problems match these filters
Try removing a filter or clearing your search.