Discussion · Interview

Appeal of Data Engineer (DE) jobs?

←

Appeal of Data Engineer (DE) jobs?

royal_flint_3282·

I read articles that DE is hot and in demand. but I also see that plenty of companies are paying data engineers less than backend SWEs and less than data scientists. So is this just an artificial shortage since the pay is too low? Like McDonald's "can't bring on" enough people... at $15/hr in a HCOL area.

Those of you who are data engineers, do you love the gig? Is it more interesting than being a backend SWE?

or is it that the interviews are easier or more specialized? Without the stats and modeling case studies required for a ds, and Can you get a DE role without the leetcode and system design required for a SWE?

TC 480, L7, YoE: 15. I'm a DS at a company which pays SWE > DS > DE; I do DS and not backend SWE since I enjoy it more even though it pays less.

10

4 comments

I ditched on DS, and while my DE job is more BI orientated. I am finally making strides internally on how key data modeling is prior to ingestion.

While I make 🥜 compared to the rest of people here I have a pretty damn enjoyable work environment.

4

I wouldn't disagree with that logic.
They operate on a much lower level of the pyramid, and you are quite right, without them we'd all be dead within 120 days.
We received said demonstration about two months or so ago when you couldn't get normal volume of food at stores, and many folks did go without food because of that combined with price hikes.
Thankfully it was only a partial shortage, since a full shortage would be catastrophic on the level of what many cities in Ukraine or Yemen are currently experiencing.

1

My simple take:
DE is superior to both SWE and DS, the reasoning also simple:
No Data, No Dice. Everything sounds excellent in theory, and you can "refactor" code until you are blue in the face. Push comes to shove, rubber meets the road, data is everything. The data itself is the money.
That's why DEs are the most critical engineers in the data stack.

0

To this day, have no idea what data engineers do. Because in my opinion folks who use all those big data technologies like spark, beam etc are software engineers.

0