Data Scientist is the top job in the USA. And this is the case for a few years in a row according to Glassdoor’s annual report. Demand continues to outpace available resources. A quick search of LinkedIn shows over 11.5K open Data Scientist roles and over 7.4K Data Engineering roles in the USA alone. Incredible.

Separating Data Science from Analytics-Enabled Jobs

PwC’s research[1] in this area clearly separates analytics-enabled jobs from data science jobs.  Common analytics-enabled jobs require hands-on experience with reporting and visualization software to aid in the collection and examination of data. In other words, the business user. These decision makers may learn to apply some predictive modeling techniques to their business domain, however typically lack a strong-enough foundation to build robust end-to-end data science analyses themselves.

PwC contends – and we do agree – that “Competencies and skills needed for data science jobs are different. (…) Candidates for these roles have strong credentials in programming and applied data science. Competition for these candidates is fierce now, and it is not likely to ease, as more and more companies become digital, and change their operating models and talent needs.” Data science professionals have a strong Statistics foundation; are familiar with techniques to prepare data before applying any algorithm; carefully compare multiple algorithms performance before choosing the most appropriate and cost effective one, and can easily discuss the results with business stakeholders.

The Business Risk is Real  

It’s the data science category that is most interesting and directly tied to a business’s ability to grow through technology innovation around data analysis, advanced analytics, and artificial intelligence.

In our conversations with technology executives, the challenge of hiring data science and data engineers creates business risk to maintaining their competitive advantage and ultimately top-line growth. Most of these companies represent industries that are known as early adopters of artificial intelligence—financial services (FinTech), technology, and communication & collaboration—which I discussed in a prior blog.

In today’s business climate, executives know the importance of innovating in rapid cycles. And resource limitations are always a challenge, especially in Silicon Valley. But now the pressure of both—accelerating innovation & data science engineering resources—exacerbates their ability to keep up.

Building a Good Data Science Team Takes Domain Expertise

Adding a data science team to the organization creates its own unique challenges. When you don’t currently have data science expertise capable of vetting candidates, how do you really understand skill sets beyond resume credentials? It’s difficult.

We began building data science skills within our organization a while ago and over time have honed how we evaluate candidates into a rigorous process of skill and competency assessment.

In addition, we want to understand who they are as a person, because as with any other position at Daitan Group, an individual’s principles and values are as important as their skills. Principles and values are fundamental to how we operate as an organization and our culture.

In the area of data science skills, we look for degreed credentials, real hands-on and/or published experience; and then we pose several questions designed to reveal the sophistication levels of skills, as well as the competency to adapt with change. For example, we start by probing their thinking process when faced with a hypothetical data science problem; analyzing the questions they ask to better frame or reframe the problem; which data preparation techniques they would consider and why; how they assess a model’s performance, and so on.

The stakes are high. Some algorithms are quite sensitive to the input data quality, and are only appropriate to certain types of problems. Making decisions based on low-quality models is the equivalent of relying on a broken compass to navigate a ship. In business, it means moving in the wrong direction, jeopardizing growth—or worse.

How Do Tech Executives Adapt to Meet Demand for Data Science Skills? Some Look to Outsourcing to Accelerate.

When a company has never used outside engineering resources, how do they cut through the cycles it takes to determine what good companies are candidates—especially when they’re looking for advanced analytics and data science skills?

I asked my circle of executives, and learned that the process comes down to prior experience and who they (and their engineers) know. They rely on trusted advise from an experienced, trusted source. We get that. In fact, it makes sense, as Daitan’s business is all word-of-mouth from one client to another—usually executives discussing business challenges seeking peer advise.

With many of our existing clients, we have forged their data science initiatives side-by-side with them and helped augment current teams for data science programs.  The skill sets range from full stack engineers and DevOps, to data engineers and data scientists. And, what does DevOps have to do with it? Everything if you want to operationalize your data science projects into production-released products and services.

It’s the Questions People Ask Us That Bridge Technology and Knowledge—and Build Trust

Having an active innovation program also helps. Our view of innovation is twofold: first, help clients understand market dynamics around new technologies; anticipate change that will affect their competitive position, and transform from legacy systems to modern architectures by leveraging the tools, frameworks and methodologies best suited for their needs.

Second, drive our own innovation initiatives including research and thought leadership that transfers knowledge across the organization and with clients’ engineering teams. This often results in an active dialogue on technical topics that generate Tech Talks, white papers, Meet Up presentations and (like-minded) technical communities.  Here are a few examples of questions that we have been discussing with our engineers, clients, user communities, influencers and executives, particularly in the context of data science and advanced analytics:

  • How to design a high volume, high velocity data pipeline that can handle billions of events per day?
  • When do you use an event-driven architecture?
  • What are the approaches to compare classification/prediction algorithms’ performance?
  • For today’s data architectures, how should we approach technology and tool selection to implement?
  • What is a practical framework to approach an AI project that can evolve around understanding intention and preemptive incidence response?
  • How can we migrate legacy data silos into a Data Lake without disrupting business?
  • How to present data in analytics dashboards to facilitate decision making?
  • What is AI bias and how can we avoid it?
  • What are the right technical skills to include on a data science team?
  • What drives progression from Machine Learning to Deep Learning?

Changing to grow is normal for every tech business. And when change is moving at high speed, business risk increases and every day becomes a valuable learning experience.  Regardless of your industry, data science and Artificial Intelligence will affect your future. There is much to learn as all of us move ahead. So, if you have your own questions, or would like to know more about one of the questions above—feel free to ping Daitan Group. It’s what we do best.

[1]  PwC What’s Next for the 2017 Data Science and Analytics Job Market?  Sept 2017