Huawei AI Data Lake Solution Positioned to Accelerate Industry Intelligence

At 4th Huawei Innovative Data Infrastructure (IDI) Forum (Munich, Germany), Huawei Technologies Co., Ltd. launched the AI Data Lake Solution, designed to accelerate AI adoption across industries.

Peter Zhou, VP, Huawei and president, data storage product line, introduced the solution in his keynote speech: Data Awakening, Accelerating Intelligence with AI-Ready Data Infrastructure.

While digital transformation has evolved over decades and brought sweeping change, one thing remains constant: the critical importance of data. This was highlighted in Zhou speech: “To be Al-ready, get data-ready. The continuous deepening of industry digitalization is a process of transforming data into information and knowledge.“

By integrating data storage, data management, resource management, and the AI toolchain, the AI Data Lake Solution delivers a AI corpus and speeds up model training and inference, empowering enterprises to embrace AI.

In his address, Zhou provided details about products and technologies that are part of Data Lake solution:

Oceanstor A800

• Data storage: continuous innovation in performance, capacity, and resilience

  • Accelerated AI model training and inference: The company’s OceanStor A series high-performance AI storage delivers exceptional performance. It, for instance, enabled the AI technologies developer iFlytek, among others, to boost cluster training efficiency. Its advanced inference acceleration technology enhances inference performance, reduces latency, and elevates the application user experience- accelerating the deployment of large-model inference applications in production environments.

  • Storage of mass AI data: The OceanStor Pacific all-flash scale-out storage offers a high capacity density of 4PB/2 U and ultra-low power consumption of 0.25W/TB. Designed to manage exabyte-scale data with ease, it is well-suited for data-intensive workloads across education, scientific research, medical imaging, and media.

  • AI corpus and vector database backup: OceanProtect backup storage provides 10 times higher backup performance than other mainstream options and boasts 99.99% ransomware attack detection accuracy, safeguarding key data of training corpus and vector databases in fields like oil and gas and MSPs.

• Data management: data visibility, manageability, and mobility across regions
  Huawei DME is a data management platform that integrates the Omni-Dataverse to help customers eliminate data silos in geographically dispersed data centers. In addition, DME’s ability to retrieve data from over 100 billion files in seconds helps customers efficiently process data and unlock its full potential.

• Resource management: pooling of diverse xPUs and intelligent scheduling of AI resources
  Powered by virtualization and container technologies, the DCS platform provides efficient xPU resource pooling and intelligent scheduling, boosting resource utilization. In addition, the DataMaster in DME enables all-scenario, AI-powered O&M with AI Copilot, offering a range of AI applications such as intelligent Q&A, O&M assistant, and inspection expert, creating an O&M experience.