IDC: More data, more problems
Between 2009 and 2020, the information in the “Digital Universe” (DU) will grow by a factor of 44; the number of “files” in it to be managed will grow by a factor of 67, storage capacity will grow by a factor of 30, yet the staffing and investment to manage the DU will grow by a factor of 1.4, according to an EMC-sponsored report from research firm IDC, of Framingham, Mass.
“This should make it clear to CIOs and business executives that much of the next 10 years of their careers will be spent dealing with the challenge of the mismatch of these growth rates,” IDC stated.
Since 2007, IDC has been sizing up what it calls the Digital Universe, or the amount of digital information created and replicated in a year. According to the report, the DU grew by 62 percent--to nearly 800,000 petabytes--last year. A petabyte is 1 million gigabytes.
Between 2010 and 2020, the amount of digital information created and replicated in the world will grow to an almost “inconceivable 35 trillion gigabytes as all major forms of media – voice, TV, radio print – complete the journey from analog to digital.” This explosive growth means that by 2020, the DU will be 44 times as big as it was in 2009, the report stated.
This influx of consumer technologies into the workplace will create stresses and strains on the organizations that must manage, store, protect and dispose of all this electronic content, according to the IDC report.
“By 2020, almost 50 percent of the information in the DU will require a level of IT-based security beyond a baseline level of virus protection…and while the portion of that part of the DU that needs the highest level of security is small…that portion will grow by a factor of 100,” stated the report.
In creating the model for the DU, IDC has discovered that the gigabytes a person may create through his or her own actions – taking photos, blogging, sending emails – is less than 10 percent of the information about that person in the DU. The other 90 percent is comprised of credit records, surveillance photos, analytics on behavior, web-use histories and so on.
CIOs need access to tools and expertise to protect this burgeoning quantity of data that needs protection in both the physical and, increasingly, the virtual world, according to the report. “Without another regulatory driver like a breach, it is often difficult for CIOs to get the full attention of management regarding non-technical aspects of information security.”
According to IDC, the issues can be narrowed down to:
In 2009, the world spent nearly $4 trillion on hardware, software, services networks and IT staff to manage the DU, IDC stated. That spending is expected to grow modestly between now and 2020, which means the cost of managing each byte in the DU will drop steadily.
While the cost per byte drops, so does the investment in IT staff per byte. The investment in IT staff per information container, or file, drops even faster, IDC stated. This falling investment ratio, while spurring the growth of the DU, also means that the tools for managing it will have to change. According to IDC:
“This should make it clear to CIOs and business executives that much of the next 10 years of their careers will be spent dealing with the challenge of the mismatch of these growth rates,” IDC stated.
Since 2007, IDC has been sizing up what it calls the Digital Universe, or the amount of digital information created and replicated in a year. According to the report, the DU grew by 62 percent--to nearly 800,000 petabytes--last year. A petabyte is 1 million gigabytes.
Between 2010 and 2020, the amount of digital information created and replicated in the world will grow to an almost “inconceivable 35 trillion gigabytes as all major forms of media – voice, TV, radio print – complete the journey from analog to digital.” This explosive growth means that by 2020, the DU will be 44 times as big as it was in 2009, the report stated.
This influx of consumer technologies into the workplace will create stresses and strains on the organizations that must manage, store, protect and dispose of all this electronic content, according to the IDC report.
“By 2020, almost 50 percent of the information in the DU will require a level of IT-based security beyond a baseline level of virus protection…and while the portion of that part of the DU that needs the highest level of security is small…that portion will grow by a factor of 100,” stated the report.
In creating the model for the DU, IDC has discovered that the gigabytes a person may create through his or her own actions – taking photos, blogging, sending emails – is less than 10 percent of the information about that person in the DU. The other 90 percent is comprised of credit records, surveillance photos, analytics on behavior, web-use histories and so on.
CIOs need access to tools and expertise to protect this burgeoning quantity of data that needs protection in both the physical and, increasingly, the virtual world, according to the report. “Without another regulatory driver like a breach, it is often difficult for CIOs to get the full attention of management regarding non-technical aspects of information security.”
According to IDC, the issues can be narrowed down to:
- Developing tools for search and discovery of information as the DU expands, including finding ways to add structure to unstructured data through metadata, automatic content tagging and pattern recognition.
- Deploying tools for new levels of information management and prioritized storage.
- Deploying tools and expertise for security and privacy protection for a growing portion of the DU in hybrid physical/virtual environments.
- Getting ready for some level of conversion to cloud-based services to start equipping IT staff with the new skills required for providing IT as a service and to obtain some economy of scale for ever-scarcer IT talent.
- Obtaining support from top management and from business units to implement the non technical aspects of dealing with the DU, including setting policies on social media, training end users on information security and classifying information in an effort to set up storage.
In 2009, the world spent nearly $4 trillion on hardware, software, services networks and IT staff to manage the DU, IDC stated. That spending is expected to grow modestly between now and 2020, which means the cost of managing each byte in the DU will drop steadily.
While the cost per byte drops, so does the investment in IT staff per byte. The investment in IT staff per information container, or file, drops even faster, IDC stated. This falling investment ratio, while spurring the growth of the DU, also means that the tools for managing it will have to change. According to IDC:
- The increased complexity of managing digital information will be an incentive to migrate to cloud service;
- Within data centers, expect continued pressure for data center automation, consolidation and virtualization;
- For the management of end-user and customer business transactions, look for more end-user self-service;
- Expect bottlenecks in key specialties such as security, information management, advanced content management and real-time processing; and
- This manage-more-with-less situation will put ever increasing stress on IT organizations. Those that manage this stress better than competitors will have an advantage.