Which Data Engineering Tool Is Worth Learning?

Databricks or Snowflake, AWS or Azure or GCP: the answers usually come from whoever replies first. These rankings come from 36,543 live data engineering openings and the pay at the companies behind them, covering 54 tools. The short version: the tools employers ask for most pay about the market rate, and the ones that pay a premium are asked for least.

Last updated: September 9, 2026Proudly published by: Jeff Wahl7 min readVerified against 36,543 live job openings

Reach

The tools in the most job ads pay about the market rate.

AWS is named in 47.75% of openings and still lands -6.3% against the $205K median. Employers treat this tier as the price of entry, so knowing it opens doors without lifting an offer.

AWS 47.75%CI/CD 36.84%Kubernetes 32.61%Azure 29.4%

Premium

The tools that pay above market show up in few ads.

Iceberg carries the highest median at $247K, +20.5% over the market, on 1.62% of openings. Employers running these tools are a narrower group, and they pay more for people who have.

Iceberg +20.5%Flink +19.5%Hadoop +13.7%Looker +10.5%

The trade-off, plotted

Requested often, or paid well. Rarely both.

Every tool with a published pay figure, placed by how often employers ask for it and what those employers pay at senior level. The horizontal axis is logarithmic, because AWS is named 29x more often than the best-paying tool.

Dots above the dashed line pay over the market median. Almost all of them sit on the left, where employers ask for the tool least. Nothing occupies the top right.

The question everyone asks

Databricks or Snowflake

The framing assumes one slot and two candidates for it. The job ads disagree.

Databricks

In job ads10.13%

Open roles3,700

Median pay$176K

Against market-14.1%

Employers reporting69

Snowflake

In job ads9.86%

Open roles3,603

Median pay$198K

Against market-3.6%

Employers reporting79

Most employers want both. 38.12% of Databricks openings also ask for Snowflake, and 38.48% of Snowflake openings also ask for Databricks, roughly 3.13x what you would see if the two were unrelated. The pay gap of $21K says more about who adopted each one than about the products. Databricks has spread through large enterprises, insurers and consultancies that pay under the tech median, while Snowflake skews toward employers above it. Learn whichever your target companies run, and expect to meet the other.

AWS, Azure or GCP

Employers are not asking you to pick one

These three add up well past 100% because most openings name more than one. The skill being checked is whether you have run production work in a cloud, not which logo was on it.

AWS47.75% of openings · $192K median · -6.3%

The default. Named in nearly half of all openings, which makes it the safest single answer and the least differentiating one.

Azure29.4% of openings · $174K median · -14.9%

Concentrated in enterprise and government work. Pay runs under the market median, and the Microsoft stack tends to travel with it.

GCP27.01% of openings · $180K median · -12.2%

The smallest of the three by openings, and heavily weighted toward companies already running BigQuery.

Rank them yourself

All 54 tools, on your priorities

Move the slider to weight openings against pay, sort any column, or plot how each tool's share of job ads has moved.

What matters more to you?

50% openings · 50% pay

Pay onlyBalancedOpenings only

#	Tool
1	Kubernetes	72.1	32.61%(11,915)	$205K	0%	-0.94 pts
2	CI/CD	68.5	36.84%(13,463)	$200K	-2.6%	+4.88 pts
3	Docker	68.3	22.61%(8,264)	$205K	0%	-1.04 pts
4	Flink	67.7	2.21%(809)	$245K	+19.5%	-0.61 pts
5	Hadoop	65.6	3.38%(1,234)	$233K	+13.7%	-0.79 pts
6	PyTorch	65.6	7.22%(2,637)	$217K	+5.9%	-2.86 pts
7	AWS	64.6	47.75%(17,451)	$192K	-6.3%	-2.19 pts
8	Airflow	63.4	7.54%(2,756)	$214K	+4.4%	-0.96 pts
9	Iceberg	63.2	1.62%(592)	$247K	+20.5%	-0.02 pts
10	TensorFlow	62.7	4.71%(1,723)	$220K	+7.3%	-2.53 pts
11	Kafka	62.3	12.78%(4,670)	$205K	0%	-0.72 pts
12	Spark	61.9	12.37%(4,519)	$205K	0%	-2.25 pts
13	Terraform	61.5	18.12%(6,621)	$200K	-2.6%	-0.22 pts
14	Looker	60.1	3.46%(1,264)	$227K	+10.5%	-0.13 pts
15	BigQuery	58	5.07%(1,854)	$214K	+4.4%	-0.02 pts
16	Tableau	55.1	6.48%(2,369)	$205K	0%	-0.7 pts
17	dbt	54	5.21%(1,905)	$207K	+1%	+1.13 pts
18	Snowflake	53.6	9.86%(3,603)	$198K	-3.6%	-1.12 pts
19	MongoDB	52.5	5.03%(1,837)	$205K	0%	-0.15 pts
20	MySQL	51	6.2%(2,264)	$200K	-2.6%	-0.24 pts
21	Elasticsearch	51	3.95%(1,444)	$207K	+1%	+0.01 pts
22	Pandas	50.1	3.11%(1,135)	$210K	+2.4%	-0.83 pts
23	Fabric	49.9	2.06%(754)	$217K	+5.9%	-0.06 pts
24	PostgreSQL	49.7	14.94%(5,458)	$187K	-8.8%	+0.7 pts
25	Redis	49.4	5.3%(1,937)	$200K	-2.6%	-0.23 pts
26	S3	49.1	5.93%(2,167)	$198K	-3.6%	-0.35 pts
27	GCP	48.7	27.01%(9,871)	$180K	-12.2%	-3.2 pts
28	Redshift	48.4	3.05%(1,116)	$207K	+1%	+0.01 pts
29	MLflow	48.3	2.16%(791)	$214K	+4.4%	-0.79 pts
30	DynamoDB	47.3	2.91%(1,065)	$206K	+0.5%	-0.55 pts
31	Hive	45.1	1.56%(570)	$214K	+4.4%	-0.07 pts
32	Azure	44.9	29.4%(10,743)	$174K	-14.9%	-0.97 pts
33	Cassandra	44.8	1.51%(553)	$214K	+4.4%	-0.1 pts
34	SQS	43.5	2.15%(786)	$205K	0%	-0.02 pts
35	scikit-learn	42.2	2.65%(967)	$200K	-2.6%	-0.77 pts
36	Delta Lake	41.7	1.81%(660)	$205K	0%	-0.17 pts
37	Athena	40.7	1.02%(373)	$220K	+7.3%	+0.22 pts
38	Presto	40.6	1.11%(404)	$214K	+4.4%	-0.42 pts
39	Kinesis	40.3	1.58%(576)	$205K	0%	-0.42 pts
40	Databricks	37.7	10.13%(3,700)	$176K	-14.1%	+0.39 pts
41	Lambda	36.5	4.36%(1,593)	$182K	-11.2%	-0.02 pts
42	EMR	36.4	1.43%(522)	$200K	-2.6%	+0.42 pts
43	Glue	36.2	1.53%(559)	$198K	-3.6%	+0.21 pts
44	Power BI	34	6.85%(2,504)	$176K	-14.1%	+1.26 pts
45	NumPy	29.3	1.92%(700)	$176K	-14.1%	-0.7 pts
46	Informatica	18.2	0.68%(249)	$165K	-19.5%	-0.11 pts
47	Synapse	17.4	0.88%(320)	$154K	-24.8%	-0.09 pts
48	Data Factory	15	1.26%(462)	$150K	-26.8%	-0.05 pts
49	Dagster	12.5	1.35%(495)	n/a	n/a	+0.03 pts
50	Palantir Foundry	10.6	1.12%(410)	n/a	n/a	-0.51 pts
51	Fivetran	10.4	1.1%(402)	n/a	n/a	+0.11 pts
52	Prefect	7.4	0.83%(304)	n/a	n/a	-0.16 pts
53	Beam	6.5	0.76%(276)	n/a	n/a	-0.21 pts
54	NiFi	0	0.41%(151)	n/a	n/a	-0.03 pts

Score blends openings and pay at the weighting you set. “In % of jobs” is how often data engineering job ads name the tool. “vs market” compares its median against the $205K market median for senior roles. A dash means too few employers report pay for that tool to show a figure.

Where the premium sits

The tools worth adding once you have the basics

Median pay at senior level among the employers that ask for each one, against a $205K market median. Each of these is asked for in a small share of openings, so treat them as a second move rather than a first.

01Iceberg1.62% of ads · 32 employers$247K +20.5%

02Flink2.21% of ads · 39 employers$245K +19.5%

03Hadoop3.38% of ads · 42 employers$233K +13.7%

04Looker3.46% of ads · 32 employers$227K +10.5%

05TensorFlow4.71% of ads · 60 employers$220K +7.3%

06Athena1.02% of ads · 15 employers$220K +7.3%

07PyTorch7.22% of ads · 71 employers$217K +5.9%

08Fabric2.06% of ads · 36 employers$217K +5.9%

09Airflow7.54% of ads · 64 employers$214K +4.4%

10BigQuery5.07% of ads · 40 employers$214K +4.4%

Any two tools

Check whether they compete for the same slot

Two tools that appear on the same openings are a stack you will learn both halves of. Two that never co-occur are a genuine choice.

Tool A

Tool B

In job adsHow often data engineering job ads name it	10.13%	9.86%
Open roles	3,700	3,603
Median pay at the employers hiring for itEvery employer counted once	$176K	$198K
Against the market median	-14.1%	-3.6%
Employers reporting payHow much pay data stands behind the figure	69	79
Change in job ads	+0.39 pts	-1.12 pts
Overall score	37.7	53.5

Most employers want both. 38.12% of Databricks openings also ask for Snowflake, and 38.48% of Snowflake openings also ask for Databricks, about 3.13x what you would see if the two were unrelated. Employers asking for one commonly expect the other.

Your market

One national ranking cannot answer for every city

Openings in 29 US states and 51 countries carry enough volume to rank on their own. These are the tools each market asks for most, measured against how often the country as a whole asks for them.

California

5,484 openings

Flink3.79% · 1.7x

PyTorch11.85% · 1.6x

TensorFlow6.53% · 1.4x

Texas

950 openings

Hadoop6.63% · 2.0x

Power BI11.26% · 1.6x

Tableau9.37% · 1.4x

New York

1,993 openings

dbt10.94% · 2.1x

Looker6.17% · 1.8x

Redshift5.12% · 1.7x

A figure of 2.0x means employers in that state name the tool twice as often as employers nationally. Pick any state or country in the location view to re-rank all 54 tools against local openings.

If you want a single answer

Reach first, premium second

Openings that name a premium tool almost always name a reach tool as well, so the order matters more than the choice.

Step 1

One cloud, deeply

AWS appears in 47.75% of openings. Depth in one cloud beats a shallow pass over three, and the concepts move with you when an employer runs a different one.

Step 2

Orchestration and transformation

Airflow (7.54%, $214K) and dbt (5.21%, $207K) both clear the market median and show up together on the same openings. This is the cheapest step from market rate to above it.

Step 3

Match the stack you are targeting

CI/CD is gaining ground faster than anything else, up 4.88 points of share. Beyond that, let your target openings decide: filter this page to your state or country and learn what those employers list.

Common questions

Databricks or Snowflake: which has more demand?+

Databricks is named in 10.13% of data engineering openings and Snowflake in 9.86%, so demand is close. Pay runs the other way: $176K against $198K at senior level. They also turn up together often, with 38.12% of Databricks openings asking for Snowflake as well. Pick the one your target employers run.

AWS, Azure, or GCP: which cloud should I learn?+

AWS leads at 47.75% of openings, Azure follows at 29.4% and GCP at 27.01%. Most openings name more than one, so employers are checking that you have shipped in a cloud rather than which one. Go deep in whichever your target market uses.

Which tool pays the most?+

Iceberg, at a $247K median across 32 employers, +20.5% against the market. That reflects which employers have adopted it rather than a raise you collect for learning it, and it appears in only 1.62% of openings.

Which tool is growing fastest?+

CI/CD has gained the most ground, up 4.88 points of share among openings. The window here is months rather than years, so read it as current direction.

Why are Python and SQL not ranked here?+

They are languages, and close to universal in data engineering openings, so ranking them would put two rows at the top and settle nothing. This page covers the platforms and tools where the choice is open.

How are these numbers calculated?+

Demand is the share of data engineering openings that name each tool, taken from 36,543 live roles. Pay is the median total compensation at senior level among the employers hiring for that tool, with every employer counted once regardless of how many salary reports it has, and published only where at least 15 employers report. Since pay is measured per employer, these figures show what companies using a tool pay. Treat them as a signal about who hires for it rather than a raise you collect for learning it.

Keep going

Data engineer salary by level

Base, bonus and equity from junior through staff, with company-tier comparisons.

Data engineer jobs

The live openings behind this page, filtered by tool, level and location.

Data engineer roadmap

What to learn in what order, from first pipeline to production systems.

Data engineering projects

15 builds that put these tools on your resume, each with a public repo.

Data engineer interview questions

The loop these roles hire on, round by round.

The weekly data challenge

Dirty, production-shaped data, scored blind each week.

02 / Why practice

The tool gets you the screen. The loop is still the loop.

01
Reading a solution is not the same as writing one
Every engineer who has frozen on a query they had read a dozen times knows the gap. The only preparation that closes it is producing the answer yourself, under time, before the interview does it for you
02
76% of hiring managers reject on the coding task, not the resume
From HackerRank's 2024 Developer Skills Report. Candidates who look strong on paper still fail the live screen if they haven't done timed, executable practice
03
System design comes down to the calls you defend out loud
Ingestion, batch vs streaming, the bronze/silver/gold layers, idempotency, backfill and replay. Sketching the pipeline and naming the failure modes is the signal, not the boxes

Start practising