Cloudera Proposes to Donate Impala and Kudu

Cloudera, Inc., a provider of data management and analytics platform built on Apache Hadoop, will submit proposals for Impala and Kudu to join the Apache Software Foundation (ASF).

By donating its analytic database and columnar storage projects to the ASF, Cloudera aims to accelerate the growth and diversity of their respective developer communities.

Since its launch in 2012, Impala has been broadly adopted and widely embraced as analytic database for Hadoop, with support across multiple industry vendors and integration with all business intelligence (BI) tools. Initially unlocking interactive SQL performance on Hadoop with core SQL-92 functionality, Impala 2.0 further matured its analytic SQL capabilities – enabling low-latency operational BI and data discovery solutions while supporting multi-user concurrency across analysts and casual users alike. Impala 2.0 became a game changer for many businesses, delivering these popular business use cases and familiar access natively to Hadoop.

A majority of Cloudera customers across all industries rely on Impala for their enterprise analytic and BI workloads, including Epsilon Data Management, LLC, Magnify Analytic Solutions, Premier, Quaero, RelayHealth, and SFR SA. With a continued focus on reliability at massive scale, Impala has also increasingly supported customers running millions of queries, running on clusters ranging from tens to hundreds of nodes, and over a thousand concurrent users.

Impala has been Apache-licensed since its public launch and has since become an open source standard in the Hadoop ecosystem. Since opening up Impala for community contributions earlier this year, there has been increasing development activity, with Google developing integrations between Impala and BigTable, as well as contributions from Arcadia Data, Inc., Intel Corporation, and others. By donating the project to the ASF, this diverse community can further drive the vision of Impala from its well-established foundation.

The beta release of Kudu, the updateable storage engine for Hadoop, brought another milestone for analytics on Hadoop. For the first time, the community has both an interactive query engine with Impala and an updateable storage engine with Kudu – enabling fast analytic use cases on data as it changes. In a short period since its release, Kudu has experienced interest within the open source community. Through its application to join the ASF, Kudu will continue to benefit from the broader development community and the collaboration of these projects will expand the use cases they can serve as more companies look to develop real-time analytic applications.

“We believe that the ASF is the right venue to foster an open source community around both Impala’s and Kudu’s development. We expect that these projects will benefit from more productive collaboration with related Apache projects, and under the auspices of the ASF, will attract talented contributors who will push development forward at pace,” said Doug Cutting, chief architect, Cloudera and co-creator of Hadoop. “With a strong architectural foundation set for both these projects, we believe that the timing is right for development to move wholesale to the ASF.“

Cloudera will continue to develop and support the Impala and Kudu projects, as it has with ASF projects. Cloudera employees have founded and launched several open source projects with the ASF, including Apache Hadoop, Apache Flume, Apache HBase, Apache Parquet, and ZooKeeper. Within the ASF, Cloudera also has 13 company employees who are Apache members, a title that holds the highest merit within the foundation. These members are: Sean Busbey, Jarek Jarcec Cecho, Doug Cutting, Patrick Hunt, Todd Lipcon, Mark Miller, Aaron T. Myers, Sean Owen, Yonik Seeley, Michael Stack, Tom White, and Xuefu Zhang.

“Impala has made a significant impact on the widespread adoption of Hadoop, and its move into the ASF is a signal of the high demand for Hadoop to be accessible to all businesses,” said Marcel Kornacker, founder, Impala and lead architect, Cloudera. “We’ve built a strong foundation for Impala and are excited to continue shaping its vision with the open source community.“

“In the short time since Kudu’s beta release in September, we’ve been thrilled at the community reception from open source developers and users alike across many different organizations. The need for an updateable storage engine in the Hadoop ecosystem has resonated, especially with the increased interest in building real-time analytic applications,” said Todd Lipcon, founder and lead engineer of the Kudu project, Cloudera. “With the submission of Kudu to the Apache Incubator, we are excited to work with the broader community of Hadoop ecosystem developers to continue to build out Kudu integrations with the rest of the Hadoop ecosystem and develop a new open standard storage solution.“