IBM Software Analyzing Tens-of-Petabytes of Data in Native Format

As companies seek to gain real-time insight from diverse types of data, IBM Corp. unveiled software and services to help clients more effectively gain competitive insight, optimize infrastructure and better manage resources to address Internet-scale data. Organizations can integrate and analyze tens-of-petabytes of data in its native format and gain critical intelligence in sub-second response times.

IBM also announced a $100 million investment for continued research on technologies and services that will enable clients to manage and exploit data as it continues to grow in diversity, speed and volume. The initiative will focus on research to drive the future of massive scale analytics, through advancing software, systems and services capabilities.

The news comes on the heels of the 2011 IBM Global CIO Study where 83 percent of 3,000 CIOs surveyed said applying analytics and business intelligence to their IT operations is the most important element of their strategic growth plans over the next three to five years.

Today’s news further enables Smarter Computing innovations realized by designing systems that incorporate Big Data for decision making, and optimized systems tuned to the task and managed in a cloud.

According to recent IT industry analyst reports, enterprise data growth over the next five years is estimated to increase by more than 650 percent. Eighty percent of this data is expected to be unstructured.  

The new analytics capabilities pioneered by IBM Research will enable chief information officers (CIOs) to construct specific, fact-based financial and business models for their IT operations. Traditionally, CIOs have had to make decisions about their IT operations without the benefit of tools that can help interpret and model data.

IBM is expanding its portfolio
and furthering its investments in analytics with:

• New, patented software capabilities to analyze massive volumes of streaming data with sub-millisecond response times and Hadoop-based analytics software to offer scalable storage to handle tens-of-petabytes level data.  These capabilities complement and leverage existing IT infrastructure to support a variety of both structured and unstructured data types.
• 20 new services offerings, featuring patented analytical tools for business and IT professionals to infuse predictive analytics throughout their IT operations. The services enable IT organizations to assess, design and configure their operations to address and take advantage of petabytes of data.

"The volume and velocity of information is generated at a record pace. This is magnified by new forms of data coming from social networking and the explosion of mobile devices," said Steve Mills, Senior Vice President and Group Executive, IBM Software & Systems. "Through our extensive capabilities in business and technology expertise, IBM is best positioned to help clients not only extract meaningful insight, but enable them respond at the same rate at which the data arrives."

New Services Address Analytics for IT Infrastructure
Leveraging years of intellectual capital in managing data centers and IT departments, as well as over 30 patented technologies from IBM Research, the new IT services feature dozens of analytical tools to help IT professionals use server, storage and networking technologies more efficiently, improving security and insight into planning major IT investments.  

Examples of services
that help clients with analytics include:

• Cloud Workload Analysis – The new analysis tool maps your IT workload characteristics and current capabilities to prioritize cloud deployment and migrations plans. This allows IT managers to identify cloud opportunities 90 percent faster to reduce costs.
• 
  Server and Storage – New server optimization and analysis tools achieve up to 50 percent reduced transformation costs and up to 80 percent faster implementation time. New storage services help create self-service to provision growth while reducing architects time by 50 percent.
• Data Center Lifecycle Cost Analysis Tool – Identifies how to reduce total data center costs by up to 30 percent by assessing total cost plus including environmental impact over a 10 to 20 year life.
• Security Analytic services – Analytic systems identify known events and automatically handle them; This results in handling of more than 99 percent of critical events without human intervention.

IBM Big Data Software Taps into Hadoop
IBM is making available new InfoSphere BigInsights and Streams software that allows clients to gain fast insight into information flowing in and around their businesses. The software, which incorporates more than 50 patents, analyzes traditional structured data found in databases along with unstructured data – such as text, video, audio, images, social media, click streams – allowing decision makers to act on it at unprecedented speeds.  

BigInsights software is the result of a four-year effort of more than 200 IBM Research scientists and is powered by the open source technology, Apache Hadoop. The software provides a framework for large scale parallel processing and scalable storage for terabyte to petabytes-level data. It incorporates Watson-like technologies, including unstructured text analytics and indexing that allows users to analyze rapidly changing data formats and types on the fly.  

Additional new features include data governance and security, developer tools, and enterprise integration to make it easier for clients to build a new class of Big Data analytics applications. IBM also offers a free downloadable BigInsights Basic Edition for clients to help them explore Big Data integration capabilities.  

Also born at IBM Research, InfoSphere Streams software analyzes data coming into an organization and monitors it for any changes that may signify a new pattern or trend in real time. This capability helps organizations to capture insights and make decisions with more precision, providing an opportunity to respond to events as they happen.

New advancements to Streams software makes it possible to analyze Big Data such as Tweets, blog posts, video frames, EKGs, GPS, and sensor and stock market data up to 350 percent faster than before. BigInsights complements Streams by applying analytics to the organization’s historical data as well as data flowing through Streams. This is an ongoing analytics cycle that becomes increasingly powerful as more data and real-time analytic results are available to be modeled for improvement.

As a long time proponent of open source technology, IBM has chosen the Hadoop project as the cornerstone of its Big Data Strategy. With a continued focus on building advanced analytics solutions for the enterprise, IBM is building upon the power of these open source technologies while adding improved management and security functions, and reliability that businesses demand. Hadoop’s ability to process a broad set of information across multiple computing platforms, combined with IBM’s analytics capabilities, now makes it possible for clients to tackle today’s growing Big Data challenges. IBM’s portfolio of Hadoop-based offerings also include IBM Cognos Consumer Insight which integrates social media content with traditional business analytics, and IBM Coremetrics Explore which segments consumer buying patterns and drills down into mobile data. Additionally, Hadoop is the software framework the IBM Watson computing system uses for distributing the workload for processing information, which supports the systems breakthrough ability to understand natural language and provide specific answers to questions at rapid speeds.

University of Ontario Institute of Technology
Expands Neo-Natal Research to China
Dr. Carolyn McGregor, Research Chair in Health Informatics at the University of Ontario Institute of Technology has been exploring new approaches for the last 12 years to provide specialists in neonatal intensive care units better ways to spot potentially fatal infections in premature babies.  

Changes in streams of real-time data such as respiration, heart rate and blood pressure are closely monitored in her work and now she is expanding her research to China. "Building upon our work in Canada and Australia, we will apply our research to premature babies at hospitals in China. With this new additional data, we can compare the differences and similarities of diverse populations of premature babies across continents," said Dr. McGregor. "In comparing populations, we can set the rules to optimize the system to alert us when symptoms occur in real time, which is why having the streaming capability that the IBM platform offers is critical. The types of complexities that we’re looking for in patient populations would not be accessible with traditional relational database or analytical approaches."

IBM’s Big Data software and services reinforces IBM’s analytics initiatives to deliver Watson-like technologies that help clients address industry specific issues. On the heels of The IBM Jeopardy! Challenge, in which the IBM Watson system demonstrated a capability to understand natural language, advanced analytical capabilities can now be applied on real client challenges ranging from identifying fraud in tax or healthcare systems, to predicting consumer buying behaviors for retail clients.

Over the past five years, IBM has invested more than $14 billion in 24 analytics acquisitions. Today, more than 8,000 IBM business consultants are dedicated to analytics and over 200 mathematicians are developing breakthrough algorithms inside IBM Research. IBM holds more than 22,000 active U.S. patents related to data and information management.

To hear how IBM clients are using analytics to transform their business
More on IBM Big Data initiatives
More on IBM’s set of new analytics services