- Posted by Daitan
- On June 11, 2018
- Agile, Data Engineer, Data Science, DevOps
There’s lots of talk today about Data Science, and as a resource, Data Scientists are definitely in high demand. Data Scientists bring a critical set of skills companies need to capitalize on big data and advanced analytics. But, let’s be pragmatic about it, Data Science is just one set of skills that also must be supported by other skills in order for a company to truly capture business benefit from the work. In the famous words of DJ Patil, “Data Science is a team sport.” At Daitan Group, we agree.
In our experience, it’s the adjacent and complementary roles surrounding the Data Scientist that enable companies to effectively operationalize the value of Data Science projects.
How a Data Science Team Should Work
Data Science projects are complex and usually result in producing pivotal technology tied to a company’s strategic growth. So, when we build a team, we start with foundational skills like Software Development, Data Engineering and (of course) Data Scientists; and we add DevOps Engineering and Quality Engineering—to ensure the net result is a product the company can use.
There are inherent similarities between Software Development and Data Science in terms of how they work. Data Science is all about exploration, experimentation, iterative modeling, failing and retrying in order to deliver. Sound familiar? A Lean Software Development environment operates the same—always working toward continuous integration, continuous delivery and continuous feedback. The DevOps approach supports both scenarios with frequent, incremental changes to code versions, which means frequent testing and deployment that culminates in feedback into the team.
When you approach the team with this mindset, you build one that can do Data Science and bring its business value up to management by releasing products that adhere to normal operational models—hence our emphasis on skills in Agile and DevOps.
In reality, many Data Scientists may be brilliant at coalescing the business analysis and Data Engineering into a strong “story” that can be understood by executives and other corporate stakeholders; but they may be new to Lean or Agile methods. And Software Developers are great with Agile and Data Engineering, but not necessarily have the visionary perspective of how data patterns can predict outcomes in order to bring important—and unique—business value. But, when you bring it all together in a DevOps environment, the practical flow looks something like this: Data Scientists focus on the science and conceptualizing the model into a business vision, while Data Engineers curate and transform the data, bridging to Software Developers that productize it through services and APIs; for Quality Engineers to test and validate.
All of this functions within a DevOps framework that provides operational infrastructure and processes. It works well because each role complements the others; and combined, they enable the full realization of a product. There is one more aspect to this team that is an important part of successful execution—and that is, bringing together the right expertise, in the right technologies, so the MVP will not only meet initial requirements, but scale to future needs.
It’s been interesting to see the impact of Data Science really gain momentum over the past few years. Increasingly, it drives our conversations with senior executives—across any industry—looking to leverage big data they generate or data they need to consume. If you’re working on big data, advanced analytics with Machine Learning or Deep Learning attributes, let’s talk. We would be interested in your thoughts on building the right team structure for Data Science.