The Past and Future of Data Science || Unlabelled Unstructured
Like all industries, the field of data science has rapidly changed over the last few decades, leading to the development of new technologies, ideas, and jobs. To explore the past and the future of data science, the Unlabelled Unstructured team invited Jonathan Talmi, a Data Platform Lead and Senior Analytics Engineer at Snapcommerce.
Meet Jonathan Talmi
Let me introduce you to Jonathan first. Jonathan graduated from McGill University with a Bachelor’s degree in Honours Economics. Before working at Snapcommerce, he worked in Analytics at Instacart, Shopify, and the Bank of Canada. Now, he is working as a Senior Analytics Engineer at a fast-growing Canadian Startup called Snapcommerce. Now, you may have heard of an analyst or an engineer, but what is an analytics engineer? Here is what I learned about analytics engineering, a new and emerging role in the tech industry.
Data Engineer vs Data Analyst
To understand how analytics engineering started, we need to first understand the difference between a data engineer and a data analyst. Data engineers are responsible for bringing data into the ecosystem and require full stack skill sets. On the other hand, analysts are responsible for cleaning, preparing, and transforming data for analysis. An analyst’s goal is to gather insights about a business or an organization based on data. However, with technology developing, some of the roles of a data engineer or an analyst have changed.
What is an ETL? What has changed?
Around a decade ago, data engineers were required to write a lot of ETL(Extract, Transform, Load) and API connections to fetch, insert, and ship the data to the operational systems. ETL is a type of data integration, used by many companies 15–20 years ago. Organizations would essentially be running their own analytics database or an analytics data warehouse on-premises. The process of bringing data into your own data warehouse was called ETL: extract the data from a source system, transform that data within the sort of pipeline, apply any sorts of cleaning and transformations, and finally load that into the analytics data warehouse. However, in the modern data stack, a lot of this is solved by companies like Stitch, Fivetran, and Census. Thus, writing ETL is already automated away and outsourced to major companies. This allowed data engineers to build data-intensive applications instead, such as setting up core infrastructure or core aspects of the platform that directly solve problems that people in the data team and the wider company are having. Simply put, engineers were able to do the more “interesting” part of the job rather than solving the same problem that hundreds of other companies have already solved.
The Emergence of Analytics Engineering
With technology developing, the line between the roles of an analyst and an engineer started to blur. The analytics engineering position started taking on a lot of responsibility on modern data teams. In other words, they are like analysts who are more technically minded and data engineers who can handle all of the data ingestion. To give a real-life example of what analytic engineers do, Jonathan’s role at his job is essentially to build pipelines, bring data, clean data, and find meaningful trends.
In all, I think that analytics engineer exemplifies adaptability. The world of data science, technology as a larger picture, is growing faster than ever. This means that we must be aware of changes and adapt accordingly. When this does not happen, it can stunt growth and development. The way I interpret this role is that technological advancement has allowed us to work more efficiently, thus being able to cover more roles. This ultimately gives us room for the development and exploration of new areas of data science.
To learn more about the future of data science, check out our full episode below!
