The McKinsey Quarterly

  • Recommend (72)
  • Text Size
  • Print
  • Download PDF
  • Link to This

The challenge—and opportunity—of ‘big data’

A new report explores the explosive growth of digital information and its potential uses.

opportunity for big data article, make big data more accessibleautomate decision making, Energy, Resources, Materials

In This Article

Audio

audio MP3 The challenge—and opportunity—of ‘big data’

To use the audio player, please install the Adobe Flash Player plugin version 9 or greater.

Download MP3

The proliferation of data has always been part of the impact of information and communications technology. Now, as computers and cell phones continue to pervade our daily activities and as millions of networked sensors are being embedded in these devices (as well as in automobiles, “smart” meters, and other machines), the amount of data available for analysis is exploding. The scale and scope of the changes that such “big data” are bringing about have reached an inflection point. Companies capture trillions of bytes of information about customers, suppliers, and operations. Many citizens look with suspicion at the amount of data collected on every aspect of their lives. Can big data play a useful role?

New research from the McKinsey Global Institute (MGI) finds that collecting, storing, and mining big data for insights can create significant value for the world economy, enhancing the productivity and competitiveness of companies and the public sector and creating a substantial economic surplus for consumers. The report Big data: The next frontier for innovation, competition, and productivity explores the state of digital data, how different domains can use large data sets to create value, and the implications for the leaders of private-sector companies and public-sector organizations, as well as for policy makers. The report’s analysis is supplemented by a detailed examination of five domains—health care, retailing, the public sector, manufacturing, and personal-location data.

MGI’s analysis shows that companies and policy makers must tackle significant hurdles to fully capture big data’s potential. The United States alone faces a shortage of 140,000 to 190,000 people with analytical and managerial expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the study of big data (exhibit). Companies and policy makers must also tackle misaligned incentives around issues such as privacy and security, access to data, and technology deployment.

The report identified five broadly applicable ways to leverage big data:

  1. Make big data more accessible and timely. Transparency, enabled by big data, can unlock a great deal of value. In the public sector, increasing access to data across separate departments can sharply reduce search and processing times. In manufacturing, integrating data from R&D, engineering, and manufacturing units to facilitate concurrent engineering can cut time to market.
  2. Use data and experiments to expose variability and raise performance. As organizations create and store more transactional data in digital form, they can collect more accurate and detailed performance information on everything from product inventories to sick days.
  3. Segment populations to customize. Big data allow organizations to create ever-narrowing segmentations and to tailor services precisely to meet customer needs. This approach is well known in marketing and risk management but can be revolutionary in areas such as the public sector.
  4. Use automated algorithms to replace and support human decision making. Sophisticated analytics can substantially improve decision making, minimize risks, and unearth valuable insights that would otherwise remain hidden. Such analytics have applications from tax agencies to retailers.
  5. Innovate with new business models, products, and services. To improve the development of next-generation offerings and to create innovative after-sales services, manufacturers are leveraging data obtained from the use of products. The emergence of real-time location data has created a new set of location-based mobile services from navigation to people tracking.

Read the executive summary or download the full report on the McKinsey & Company Web site.

Keep the conversation going on Twitter

Interested in big data? Use the #McKBigData and #BigData hashtags to follow the conversation on Twitter. We’ll be responding to your comments via our McKinsey Quarterly Twitter account, @McKQuarterly.

Recommend (72)
  • 21 MAY 2011
    Saf Elmansour
    Partner and Sr. Consultant
    Shokwav Marketing
    Washington, DC USA

    ...Business intelligence and data warehousing are poised to grow and become more closely integrated with the sales and marketing departments....

    .
    Saf Elmansour
    Partner and Sr. Consultant
    Shokwav Marketing
    Washington, DC USA

    Having worked for years in telecom, which is a data-intensive industry, I am used to data mining, prediction models, and clustering. I can clearly see this trend spreading as more of our actions are moving to the digital—and quantifiable—world. Business intelligence and data warehousing are poised to grow and become more closely integrated with the sales and marketing departments. We can already see the rise of BI cloud software ready with application performance interfaces for a seamless integration with enterprise software.

    .
  • 20 MAY 2011
    Abhisek Bhowmik
    Consultant
    Infosys
    Phoenix, AZ USA

    ...We can be very smart with our analysis tools, but if the data itself is wrong or false, it will always be “Garbage In/Garbage Out.”

    .
    Abhisek Bhowmik
    Consultant
    Infosys
    Phoenix, AZ USA

    Epic work! I agree that we need the right kind of analysis of the huge amount of data the world generates, but I want to bring your attention to an often ignored area—data quality. We can be very smart with our analysis tools, but if the data itself is wrong or false, it will always be “Garbage In/Garbage Out.”

    .
  • 17 MAY 2011
    Johanna Wesnigk
    Dr
    NWVM
    Bremen, Germany

    I deal with environmental and genomic data, the amount of which is just growing exponentially, that we collect with sensors and next-generation sequencing facilities....

    .
    Johanna Wesnigk
    Dr
    NWVM
    Bremen, Germany

    I deal with environmental and genomic data, the amount of which is just growing exponentially, that we collect with sensors and next-generation sequencing facilities. You can apply these insights to the public sector as well, and to spin-off companies from universities and research institutes that can create knowledge from “big data.”

    .
  • 15 MAY 2011
    Mitch Zhu
    Business Support (Report, Web)
    Telstra
    Sydney, Australia

    ...This new dot-com boom is driven by the new Web-based corporate data storage and analysis systems...

    .
    Mitch Zhu
    Business Support (Report, Web)
    Telstra
    Sydney, Australia

    In my view, we are welcoming a new dot-com boom, or the “dot-com MK II,” which is not the simple replication of the last boom that we experienced a decade ago.

    This new dot-com boom is driven by the new Web-based corporate data storage and analysis systems—such as SAP BW, Microsoft SharePoint, and Oracle APEX—that are changing the landscape of corporate business management and that enhance productivity, speed, and accuracy.

    In my opinion, Apple (with the iPhone and iPad) is leading the way, followed by Google, Samsung, and Nokia. They are truly revolutionizing the way people live, and the level of changes they are catalyzing can only be matched by that of the personal computer (in the 1980s) and the Internet (in the 1990s), both of which changed our way of life on such a large scale. Today, iPhone users hardly use their desktop PCs now since everything is now in their palms.

    Also in my view, the current dot-com boom is a natural progression from the first one, which was characterized by the isolated database (Oracle, SAP R/3), the Internet, and text-based mobile phones. When all this was linked together, it became the dot-com MK II.

    .
  • 14 MAY 2011
    Charlie Tombazian
    President
    Innovative Strategies
    Scottsdale, AZ USA

    ...I wonder if there exists the “desire” to do this work among the Generation Y’s who would likely be tapped as our future analysts?...

    .
    Charlie Tombazian
    President
    Innovative Strategies
    Scottsdale, AZ USA

    Given the data explosion, the need for analysts and managers who can extract value and make decisions based on the information is clear. But I wonder if there exists the “desire” to do this work among the Generation Y’s who would likely be tapped as our future analysts? Stereotypically, they don’t appear to be interested in detailed, disciplined work. So unless it can be made fun and compelling (like a crime scene investigation), I doubt whether the need for analysts will be filled by Gen Y. This may be a better role for the “semi-retired” boomers.

    .
  • 14 MAY 2011
    Muhammad Usman Gilani
    HR Manager
    Pakistan Today
    Lahore, Pakistan

    ...Analysts or managers with good analytical brains must also have the acumen or be trained in the art of using data for employee development....

    .
    Muhammad Usman Gilani
    HR Manager
    Pakistan Today
    Lahore, Pakistan

    As our ways of collecting data improve every day, it is no longer the real challenge; that lies in the collection and use of data when also managing human capital.

    Data are rational statistics, without the capacity to display human emotions. Analysts or managers with good analytical brains must also have the acumen or be trained in the art of using data for employee development. Employees view data collection with a suspicious eye the same way they constantly weigh the benefits and threats of new technologies when they are introduced at their companies. It takes considerable effort and time to conduct trainings and resolve issues to make sure your employees get into their comfort zone with the data or technology. A culture of two-way communication throughout the hierarchy and cross-functional harmonization are important for any organization to establish before it starts to collect and use data to benefit itself and its employees.

    .
  • 14 MAY 2011
    Vaidy Baal
    Retired private investor
    Self-employed
    Edmonton, AB Canada

    ...I wonder how this increased data collection is going to increase profitability#&8212;has anyone done any computer modeling to know how likely this is to mean something positive for profits?

    .
    Vaidy Baal
    Retired private investor
    Self-employed
    Edmonton, AB Canada

    Your earlier article said that 70 percent returns are based on market variability, and only 30 percent is due to management style and strategy. Management of more data does not change this puzzle, as the 70 percent deals with chaos and fractals that are operating under the principles of uncertainity. I wonder how this increased data collection is going to increase profitability—has anyone done any computer modeling to know how likely this is to mean something positive for profits?

    .
  • 14 MAY 2011
    Firoz Shroff
    President
    Social Entrepreneurship Consortium Inc
    Toronto, Canada

    After reading your article, it seems a business-intelligence application that relies on big data could become a good tool for deal-designing and deal-making.

    .
    Firoz Shroff
    President
    Social Entrepreneurship Consortium Inc
    Toronto, Canada

    After reading your article, it seems a business-intelligence application that relies on big data could become a good tool for deal-designing and deal-making.

    .
  • 13 MAY 2011
    John Galavan
    Sales Director
    QL2 Software
    Seattle, WA

    ...The impact of data to track competition, product development, and distribution feeds the strategies that you have noted. Today, this topic is absolutely crucial....

    .
    John Galavan
    Sales Director
    QL2 Software
    Seattle, WA

    As a pioneer in data extraction on a global basis, we see not only the tip of the iceberg but also the greater mass below the surface. The impact of data to track competition, product development, and distribution feeds the strategies that you have noted. Today, this topic is absolutely crucial. We find that data can be old, aging, and stale—and the volume can be so large as to become useless. We approach these issues with a hosted on-demand service, and our clients share a common request: real-time data in usable formats and volume. I hope to read the second installment in what I hope will be a series.

    .
Submit Your Comments

The user information you enter into this form will not update your site profile. To update your profile, please visit your profile page.

Subject The challenge—and opportunity—of ‘big data’

*Required

We may publish your comments online and in the print edition of McKinsey Quarterly. Those chosen, which may be edited for length and clarity, will appear along with your name and details, but not your e-mail address. We will use your e-mail address only to send you a confirmation copy of your comments and to notify you if we publish them online.

We value your feedback and will consider it carefully. Nonetheless, we receive so many comments that we cannot acknowledge all of them.

See also:
Preview

Related Audio
In a recent interview, McKinsey Global Institute senior fellow Michael Chui discusses how the scale and scope of companies’ access to data is changing the way they do business—and what business leaders and policy makers need to do to make sure they’re on top of the trend. To listen, use the audio tool in the box to the left.
Embed E-mail