Job hunting is always a hassle. It’s a brutal game, where you need to stand

4 min readDec 6, 2020

The causal mechanism behind disruption that Grove so quickly understood was that even if a disruptive innovation started off as inferior, by virtue of it dramatically expanding the market, it would improve at a far greater rate than the incumbent. It was what enabled Intel (and Microsoft) to win the computing market in the first place: even though personal computers were cheaper, selling something that sat in every home and on every desk ends up funding a lot more R&D spend than selling a few very expensive servers that only existed in server rooms.Similarly, Apple’s initial foray into chips didn’t produce anything that special in terms of silicon. But it didn’t need to — people were happy to just have a computer that they could keep in their pocket. Apple has gone on to sell a lot of iPhones, and all those sales have funded a lot of R&D. The silicon inside them has kept improving, and improving, and improving. And their fab partner, TSMC, has gone along with them for the ride.In the world of High-Frequency Trading, automated applications process hundreds of millions of market signals every day and send back thousands of orders on various exchanges around the globe.

Let’s start with the most general role, data scientist. Being a data scientist entails, you will deal with all aspects of the project. Starting from the business side to data collecting and analyzing, and finally visualizing and presting.

What about this chart is interesting? Well, it turns out, it bears a striking resemblance to one drawn before — actually, 25 years ago. Take a look at this chart drawn by Clayton Christensen, back in 1995 — in his very first article on disruptive innovation:

Data engineers are responsible for designing, building, and maintaining data pipelines. They need to test ecosystems for the businesses and prepare them for data scientists to run their algorithms.

Those normalised messages are then sent to algorithmic servers, statistics engines, user interfaces, logs servers, and databases of all kind (in-memory, physical, distributed).

Often, in big companies, team leaders in charge of people with specialized skills are data scientists; their skill set allows them to overlook a project and guide them from start to finish.

Considering the rising popularity of the filed — that is not slowing down any time soon — I decided to write this article to simply explain the difference between the roles and eliminate any confusion anyone on the look of a new job may have.

In a typical architecture, financial exchange signals will be converted into a single internal market data format (exchanges use various protocols such as TCP/IP, UDP Multicast and multiple formats such as binary, SBE, JSON, FIX, etc.).

He might not have realized it at the time, but when Grove was reading Christensen’s work, he wasn’t just reading about how Intel would go on to conquer the personal computer market. He was also reading about what would eventually befall the company he co-founded, 25 years before it happened.

Before we start, I must say that these titles are not fixed and may change in the future. Also, some roles may overlap and have more or fewer responsibilities based on the company hiring. However, this article should help you explore the top 10 data science roles for the most part.

A data scientist knows a bit of everything; every step of the project, because of that, they can offer better insights on the best solutions for a specific project and uncover patterns and trends. Moreover, they will be in charge of researching and developing new algorithms and approaches.

Data analysts are responsible for different tasks such as visualizing, transforming, and manipulating the data. Sometimes they are also responsible for web analytics tracking and A/B testing analysis.

They need to maintain these database systems, both from the functionality perspective and the administrative one. So, they need to keep track of the data and decide who can view, use, and manipulate different sections of the data.
If you enjoyed this article, you might like my free newsletter here — I’ll email you (very occasionally!) when there’s a new post. You might also you might enjoy Exponent, the podcast I co-host with Ben Thompson of Stratechery. We talk about this article on episode 190.
Finally, today marks the day where, for Intel, those two lines on the graph intersect. Unlike the last time the two lines intersected in the personal computer market, Intel is not the one doing the disrupting. And now, it’s just a matter of time before the performance of ARM-based chips continues its march upmarket into Intel’s last refuge: the server business.

Data engineers also work on batch processing of collected data and match its format to the stored data. In short, they make sure that the data is ready to be processed and analyzed.If you enjoyed this article, you might like my free newsletter here — I’ll email you (very occasionally!) when there’s a new post. You might also you might enjoy Exponent, the podcast I co-host with Ben Thompson of Stratechery. We talk about this article on episode 190.The second most known role is a data analyst. Data scientist and data analysis and somewhat sometimes overlapped a company will hire you, and you will be called a “data scientist” when most of the job you will be doing is data analytics.

Job hunting is always a hassle. It’s a brutal game, where you need to stand

Written by Andrew R. Joseph