Published on the 16/02/2021 | Written by Heather Wright

Share and share alike…

From geolocation, weather and supplier data to government economic and labour information, online news postings and social media opinions, there’s a wealth of data out there. But while companies are making great strides in collecting and analysing data, few are taking advantage of data generated outside their walls, according to McKinsey, which says overlooking that external data is a missed opportunity.

It’s an opportunity to track the externalities that matter – from predicting how major weather events will impact your supply chain and reducing operational risks based on real-time analysis of news and social media data for raw material suppliers to identifying fast growing consumer trends and marketing opportunities using search data, social media analysis, and transaction and receipt panels.

Using external data has the potential to be game changing across a variety of business functions and sectors.

Strategic analysis can also be bolstered, identifying product improvement opportunities by analysing reviews across social media and eCom platforms for example, as can operations and forecasting.

Postings on job websites, social media data and government economic and labour sources can be harnessed to better predict and improve employee retention rates, while a grocer might use weather data, data from suppliers and economic data and forecasts to improve demand forecasting and reduce stockouts.

“Organisations that stay abreast of the expanding external-data ecosystem and successfully integrate a broad spectrum of external data into their operations can outperform other companies by unlocking improvements in growth, productivity, and risk management,” the management consultancy says.

Deloitte is also an advocate for external data. It says analysing external data can help companies see risks and opportunities that they would miss with inputs limited to data generated from internal operations, customers and first-tier suppliers.

“Analysing external data can illuminate how factors such as shifting consumer behaviours, competitor initiatives, or geopolitical events can affect a business.”

That’s something that the Covid-19 pandemic has illustrated clearly, McKinsey says.

In just a few short months, consumer purchasing habits, activities and digital behaviour changed rapidly, making pre-existing research, forecast and predictive models obsolete and with companies discovering little of use in their internal data to guide them.

“Meanwhile, a wealth of external data could – and still can – help organisations plan and respond at a granular level.”

So what’s holding companies back from harnessing external data?

According to McKinsey there are several practical challenges, including the effort required to understand what’s actually available in a fragmented and rapidly expanding market.

“Thousands of data products can be obtained through a multitude of channels – including data brokers, data aggregators, and analytics platforms – and the number grows every day,” McKinsey’s Mohammed Aaser and Doug McElhaney say.

“Analysing the quality and economic value of data products also can be difficult. Moreover, efficient usage and operationalisation of external data may require updates to the organisation’s existing data environment, including changes to systems and infrastructure. Companies also need to remain cognisant of privacy concerns and consumer scrutiny when they use some types of external data.”

McKinsey says using external data has the potential to be game changing across a variety of business functions and sectors. But harnessing that data successfully means a journey for businesses.

McKinsey’s recommendations?

Establish a dedicated team for external data sourcing, with a dedicated data scout or strategist partnering with the data analytics team and business functions to identify operational, cost and growth improvements that could be powered by external data.

Develop relationships with data marketplaces and aggregators. Online searches to find individual data sets are not necessarily effective, with McKinsey instead advocating the use of data marketplace and aggregation platforms that specialise in building relationships with hundreds of data sources, often in specific data domains, such as consumer, real estate or government.

Once the team has identified a potential data set, its data engineers should work directly with business stakeholders and data scientists to evaluate the data and determine the degree to which the data will improve business outcomes.

Prepare the data architecture for new external data streams. Reaping the returns from external data requires up-front planning, a flexible data architecture and ongoing quality assurance testing, McKinsey says.

“Modifications should be designed to ensure that the data architecture is flexible enough to support the integration of a continuous ‘conveyor belt’ of incoming data from a variety of data sources – for example, by enabling API calls from external sources along with entity-resolution capabilities to intelligently link the external data to internal data.

“In other cases, it may require tooling to support large-scale data ingestion, querying, and analysis. Data architecture and underlying systems can be updated over time as needs mature and evolve.”

Going even further – sharing data with competitors
Gartner goes a step further in its call for companies to embrace external data. It’s advocating for more collaborative data sharing among organisations – even competitors – using digital trust technologies, including blockchain, to share the data without compromising confidential or commercial secrets.

The research house notes how ahead of the Covid-19 pandemic 10 large pharmaceutical companies, including Johnson & Jonson and AstraZeneca, undertook collaborative efforts to train their machine learning algorithms on each other’s data to help accelerate and reduce the cost of the discovery of drugs.

“This rare example shows that organisations can deliver more value when they collaobrate in sharing data externally, even with competitors, yielding comparatively increased value through efficiency and cost savings for each organisation,” Gartner says.

Lyda Clougherty Jones, a senior director analyst with Gartner, says there should be more collaborative data sharing unless there is a vetted reason not to, as not sharing data often frustrates business outcomes and can be detrimental.

So confident is Gartner that sharing data is the way to go, that it’s forecasting that organisations who promote data sharing will outperform peers ‘on most business value metrics’ come 2023. It’s a bold statement, given Gartner is also predicting that through 2022, less than five percent of data sharing programs will correctly identify trusted data and locate trusted data sources.

It’s a move that requires a big mind change, Gartner admits, with companies needing to foster a data sharing culture, rather than the current data ownership culture.