n a discussion with a colleague recently, I learned about an interesting paradox. De

n a discussion with a colleague recently, I learned about an interesting paradox. Despite the massive rise in the amount of data generated, captured, stored, and analyzed (IDC Claims we will have 175 trillion gigabytes in 5 years), and the multi-trillion-dollar analytics valuations from Gartner & McKinsey, every year business leaders claim that their organizations are less and less data-driven. Executives consistently cite people and process issues as the primary blocker, with only a small percentage citing technology. This begs the question: what is stopping organizations from using technology to enable people and process changes that make organizations more data-driven?
Your data & analytics team is probably running like a recycling plant
Image for post
Image Source: US Air Force

https://www.epicmountainsports.com/tup/video-san-antonio-spurs-vs-la-clippers-live04.html
https://www.epicmountainsports.com/tup/video-san-antonio-spurs-vs-la-clippers-live05.html
https://www.epicmountainsports.com/tup/video-san-antonio-spurs-vs-la-clippers-live06.html
https://www.epicmountainsports.com/tup/video-sevilla-vs-linares-en-vivo-partido-01.html
https://www.epicmountainsports.com/tup/video-sevilla-vs-linares-en-vivo-partido-02.html
https://www.epicmountainsports.com/tup/video-sevilla-vs-linares-en-vivo-partido-03.html
https://www.epicmountainsports.com/tup/video-sevilla-vs-linares-en-vivo-partido-04.html
https://www.epicmountainsports.com/tup/video-sevilla-vs-linares-en-vivo-partido-05.html
https://www.epicmountainsports.com/tup/video-sevilla-vs-linares-en-vivo-partido-06.html
https://www.epicmountainsports.com/tup/video-timberwolves-vs-nuggets-line-01.html
https://www.epicmountainsports.com/tup/video-timberwolves-vs-nuggets-line-02.html
https://www.epicmountainsports.com/tup/video-timberwolves-vs-nuggets-line-03.html
https://www.epicmountainsports.com/tup/video-timberwolves-vs-nuggets-line-04.html
https://www.epicmountainsports.com/tup/video-timberwolves-vs-nuggets-line-05.html
https://www.epicmountainsports.com/tup/video-timberwolves-vs-nuggets-line-06.html
https://www.epicmountainsports.com/tup/video-villarreal-vs-zamora--ver-en-directo-01.html
https://www.epicmountainsports.com/tup/video-villarreal-vs-zamora--ver-en-directo-02.html
https://www.epicmountainsports.com/tup/video-villarreal-vs-zamora--ver-en-directo-03.html
https://www.epicmountainsports.com/tup/video-villarreal-vs-zamora--ver-en-directo-04.html
The challenges facing organizations as they seek to become data driven are in many ways similar to the problems that have faced the recycling industry in recent years. The costs of producing something usable are extremely high due to high cost of cleaning up the mixed mess of bagged plastic, cardboard, trash, and metals that is dumped onto the recycling plant. When you assume that a business can take whatever you give to it and be consistently profitable, you are setting yourself up for failure. The outcome in recycling has been a sharp decline in municipal recycling, combined with a re-thinking of the ways we can control the inputs.
Looking back at the data side, the conditions are actually a little worse than in recycling, because data teams aren’t just being asked to produce raw materials. A closer analogy would be to ask a recycling plant to take their mess of inputs and produce an ever-expanding array of finished products in on the other side. I think that it is this bind that has led to the continued decline of organizations being data-driven. Doing a little root cause analysis, I’ve identified the following 3 factors:
People who control what data is generated within an organization, typically by building or selecting operational systems, don’t realize what it can be used for, don’t have any incentives to care, and therefore don’t optimize for data capture.
Because of #1, people who build data products are handcuffed, causing those that use the data products to guide decisions don’t trust the underlying data.
No one within the organization is charged with estimation of the impact of 1 & 2, making data quality a hidden cost (to the tune of $15M / year for the average organization) that goes unaddressed over time.
These are structural issues that combine to ensure that data is not a first-class-citizen, leading the data organization to struggle to pull together insights from unclean, missing, and misunderstood data sets. This difficulty is often cited in articles as “Data Scientists spend 80% of their time doing data preparation and cleanup.” However, the data organization at most companies is much larger than the data science team, and with often an order of magnitude more analysts, with a lower degree of technical skill to handle the data cleanup and preparation, leveraging data than the data science team. This leads leadership to think that the data organization is underperforming, when in reality they are stuck with limited resources. Left alone, this can create a data-death-spiral as turnover increases, taking away the resources that really understand the data, and further slowing the development of data products.
If we agree that these problems exist, then what should we do to solve for them?
How to Create a Data Owner Role so you can work like a manufacturer, not a recycler
Image for post
Image Source: Wikimedia
In contrast to recycling, most manufacturing organizations have whole segments of their business dedicated to controlling their supply chain. This group focuses on ensuring that the materials that they are assembling into their products are fit for purpose, providing their plants with everything they need to make high quality products.
The best solution to this problem in the data space, as identified by my colleague Sean McCall, is to create and fill a missing role in most organizations: the Data Owner. In this post, I’m hoping to dramatically expand on the role to help you get started with it in your organization.
Data Owners are assigned to a Data Domain (i.e. Customer, Sales, etc.) and specifically charged with the goal of maximizing the value derived from the data within that domain. More than that, they are empowered to control the data supply chain by demanding changes and adding requirements to the products that generate data, so that they can maximize the value they can deliver from the data. This new role requires a blend of skills from Product Ownership and Data Analysis that likely don’t exist in many people within a typical enterprise. This means that organizations that create the role must invest in cross-training to blend these skills.
What does a Data Owner do?
The top level responsibility of a Data Owner is clear: maximize the value that is derived from the data in their domain. Below are some of the key responsibilities that allow them to achieve this value:
Defining and building the business case for the data products that will leverage data from their domain, often in collaboration with the Data Owners from other domains
Image for post
A sample business case for a customer acquisition
Advocating for changes in data sources by providing requirements related to the capture and modeling of data during product development or selection
Image for post
Designing product instrumentation to answer questions and validate intuitions about their domain.
Image for post
Questions, Exploration, and Intuition should be leveraged to develop Facts that are displayed to users.
Prioritizing data issue resolution within their domain against other work
Image for post
Interfacing with consumers of data in their domain to provide additional context on the usage of the data and ensuring that metadata & documentation for their domain accurately reflects the real world and is helpful to self-service consumers of the data
Image for post
Who is not a Data Owner?
There are many roles within an organization that might sound similar to a Data Owner, but there are key differences or gaps that keep them from achieving the goals of a Data Owner.
Data Steward: The tone of the role says it all. The Data Steward role often has the charge to maximize value within their data domain, but in a passive way after the data has already been generated. They are not advocates for capturing the right data, but simply stewards massaging and directing it after it has been generated.
Product Owner: Unless a product is truly a data product, in that it is built to provide value from analytics rather than from interactions with customers, then this role typically lacks the data and analytics understanding to ensure that data products are considered as a first-class portion of their role. Having a product owner that does understand data can be a very positive differentiator for an organization.
Data Analysts/Scientists: These roles are focused on the technical aspects of delivering value from data, which is a complicated task. Therefore, they don’t typically have the time to focus on the broad business implications and design elements that come with the Data Owner role.
Image for post
Where do Data Owners Come From?
As your organization starts to roll out DataOps practices, Data Owners can be sourced from the Business, Analytics, and Product organizations. Below are some of the capabilities that Data Owners should have:
Should have deep domain knowledge (understanding of the real-world conditions of the business) within their assigned data domain
Should have strong user-centered design skills and be able to practice empathy with data consumers
Should be strategy oriented, and capable of valuing and advocating for their ideas
Should understand the complexities that need to be controlled for in data products based on a solid understanding of statistics and their domain knowledge
Should be capable of communicating with technical resources (software developers, Data Architects, data engineers) and business resources (often their peers)
Does not have to have deep expertise in data modeling best practices, as this can be delegated to a Data Architect, but needs to have a basic u