Most Data Integration Tools Not Meeting Big Data Demand

Syncsort released findings from a study of more than 350 IT and business professionals across the United States and Canada that demonstrates a disconnect between the capabilities of data integration tools and the challenges facing organizations.

The survey, commissioned by Syncsort and conducted independently by BeyeNETWORK, finds that many data integration tools have failed to keep up with the rapidly evolving data performance requirements.

Growing data volumes and shrinking processing windows pose significant challenges to more than 70 percent of the professionals surveyed. More concerning, is that 68 percent of those surveyed point to data integration tools as impeding the organization’s ability to achieve strategic business objectives.
   
Additional findings include:

• The cost of owning and using data integration tools is the number one challenge for 39 percent of survey respondents
• More than 57 percent find their data integration tools require significant tuning by IT staff to achieve suitable performance
• 35 percent consider the need to do hand coding and 31 percent the need to buy/upgrade hardware or software to improve performance a significant challenge with their current data integration tools
• Only 31 percent of respondents indicate that data transformations take place in the ETL tool, with more transformations taking place in the database (44 percent), custom code/in-house solutions (43 percent) or a combination of all three (33 percent)
• More than 45 percent state it takes between one week and one month to develop reports utilizing new data not currently in the data warehouse and an additional 26 percent state it takes them longer than a month
• Nearly 65 percent of respondents estimate that their organization’s data volumes have grown by more than 50 percent in the past 2 years

"We needed to develop a process that would perform change data capture activities for several large database tables, and needed it to run every night in a very limited window," said Wayne Schmidt, President, The Sirocco Group Ltd. "All of the other solutions we tried ran for literally two to three days. Using Syncsort’s DMExpress, we were able to achieve the results we were looking for in just a few hours."

"Organizations are struggling with how to strike a balance between expensive and time-consuming workarounds for dealing with exploding data volumes, and a desire to drive ever-increasing value and insights from this data," said W. Sean Ford, Chief Marketing Officer, Syncsort. "Despite the significant investments that businesses have made in acquiring and managing their data integration tools, this new survey data clearly shows that the majority of tools are failing to keep up with today’s requirements from both a technical and economical perspective."

"Companies need to rethink their strategies to create opportunities to capitalize on the era of ‘Big Data’ and remain competitive," said Flavio Santoni, Chief Executive Officer, Syncsort. "We believe that Syncsort’s heritage in supporting thousands of customers with some of the most complex, data-intensive environments gives us a unique position in the marketplace, and the proven technology that no one else can offer to help companies get more business value from their data than ever before."

About the Survey
Syncsort in conjunction with BeyeNETWORK surveyed 359 business and IT professionals from the United States and Canada. Survey data was collected from March 23, 2011 to March 29, 2011.

More on the survey (you need to register)