How will data science change in the next five years? originally appeared on Quora - the knowledge sharing network where compelling questions are answered by people with unique insights.
Answer by Anthony Goldbloom, co-founder and CEO of Kaggle, on Quora.
How will data science change in the next five years? In answering this question, I'm going to focus less on what I expect to happen at the cutting edge of data science and more on how data science continues its progression towards becoming mainstream and ubiquitous.
When thinking about where data science is going in the next five years, it's useful to reflect on how data science has evolved over the past five years. When Kaggle started in 2010, the term "data science" wasn't common yet. Members of our community referred to themselves as doing advanced analytics, statistics, machine learning, bioinformatics, econometrics, or one of the various other disciplines that involve working with data and statistical techniques. Companies also referred to the departments that did data-related work by their functions: marketing analytics, risk, underwriting, chemical informatics, etc.
The word data science really took off after O'Reilly's Strata Conference in 2011. That conference brought fifteen hundred "data scientists" together. It gave individuals with different job titles a single way to refer to their skill set. It told senior management that data professionals in different departments actually have the same skill set.
So if O'Reilly's Strata Conference was the first inning, I believe we've now moved into the second inning (for those not in the U.S., there are nine innings in a baseball game). We're now seeing many companies consolidating their data scientists into a single large data science organization. The most effective structures involve the data science organization seconding data scientists out to the business units (marketing, risk, etc.). This structure works well because the data science organization learns how to attract and recruit data science teams, but allows data scientists to work closely with those who have context on the problems they're working on. Airbnb is a great example of a company using this structure effectively.
As companies derive more value from their existing data science teams, those teams will continue to grow. Ultimately, I think the central data science organization will go away and each business unit will have large dedicated data science teams.
Data science is really succeeding when it becomes the primary decision-making tool inside organizations. When there's a decision to be made and management's first instinct is to ask, "What does data science say?" then we've succeeded.
Tackling this question from a different direction, I believe data science will be bigger than software engineering in the next decade. If we define a data scientist as somebody using R or the Python Data tools, there are probably 1.5-3 million* data scientists in world (compared with 20 million software engineers). Meanwhile, there are ~8 million SAS users and ~120 million Excel users. I believe that SAS will slowly decline as SAS-heavy jobs adopt R and Python, and that many jobs that require heavy Excel use will also switch to using R and Python.
*Triangulating around Kaggle's userbase (650,000) and Jupyter Project users (they estimate 3 million).
This question originally appeared on Quora. - the knowledge sharing network where compelling questions are answered by people with unique insights. You can follow Quora on Twitter, Facebook, and Google+.