IBM Enterprise Data Analytics With Cloudera

IBM Enterprise Data Analytics with Cloudera, V5.0.4 combines DB2 Big SQL advanced SQL-on-Hadoop engine and Cloudera Enterprise Data Hub V6.2 to enable faster analytics, improved search, and scalability for demanding data requirements

IBM Enterprise Data Analytics with Cloudera, V5.0.4 improves analytics and meets enterprise data lake requirements by combining DB2 Big SQL with Cloudera Enterprise Data Hub.

Db2 Big SQL, a SQL engine on Hadoop, accelerates fast-evolving, open source ecosystems by boosting the power of analytical workloads on data lakes. The core capabilities of Db2 Big SQL focus on data virtualization, SQL compatibility, scalability, performance, and enterprise security and governance. This query engine enables teams to gain insights from disparate data sources, including Hadoop.

Db2 Big SQL offers following capabilities:

ANSI SQL-compliant SQL engine brings the power of SQL and PL/SQL to Apache Hadoop.
Mature and advanced SQL engine is optimized for high performance with high concurrency with efficient resource utilization.
Inherent capability to understand different SQL dialects (such as Oracle, DB2, and IBM Netezza data warehouse appliances) enables seamless application portability when data is moved to Hadoop from traditional data stores. This feature also enables the re-use of prebuilt applications when data is moved to a big data platform.
Hive, HBase, Spark, and other data sources can be concurrently queried through a single database connection or single query by using Federation technology of Db2.
Hadoop data can be enriched and augmented with other data stores for deep analytics with data virtualization capabilities that provide advanced federated access to relational database management system (RDBMS).

Cloudera Enterprise Data Hub is a platform for ML and analytics and capable of scaling to the needs of the enterprise. This offering is purpose-built for unifying data resources and data workloads, and optimizing on-premises, cloud, or hybrid environments.

Version 6.2 of the platform brings a foundation for faster analytics, improved search, and greater scalability for the most demanding data requirements, offering improvements in productivity, innovation, and enterprise quality.

Cloudera Enterprise Data Hub includes the following Cloudera components:

Core Hadoop (HDFS, YARN/Mapreduce, Hive, Pig, Hue, Sentry, Flume, Sqoop, and Kafka)
HBaseImpala
Kudu, Search
Spark
Spark Streaming
Hive on Spark
Cloudera Manager
Cloudera Director
Cloudera Navigator (Audit and Lineage, Encryption, and Key Trustee)

This release complements the capabilities of Db2 Big SQL,
bringing quality enhancements, bug fixes, and other improvements such as:

Support for Spark Structured Streaming and microbatch processing in increments of 100ms.
Support for HDFS Erasure Coding. Hive, MapReduce, Spark, BDR, and Navigator are capable of interacting with Erasure Coded data.
Support for JBOD storage. Kafka no longer requires RAID-compliant storage.
Improvements through a rebased Apache Accumulo to version 1.9.2. Enhancements include better scanning performance and rate limits on compaction.
Expanded platform support and improved security. Version 6.2 supports deployment with OpenJDK 8 and Oracle’s JDK. It also supports AWS CloudHSM for HDFS encryption-at-rest. Several defaults in Kafka, Impala, Sqoop, and Flume are changed to be more secure, adding bi-directional replication (BDR) from insecure to secure (Kerberized) clusters to ease the transition to secure clusters.
Enhanced cloud orchestration and support. With this release, Cloudera Altus Director enhances proxy support for secure environments, improves scripting capabilities (including pretermination scripts), and offers support for Google Cloud subnetworks.

Planned availability date: November 26, 2019