What are you looking for ?
Infinidat
Articles_top

Autonomous Driving Technologies Firm WeRide Chooses WekaIO

To manage AI data pipeline

WekaIO, Inc. announced that WeRide, a smart mobility company with autonomous driving technologies, has selected its Weka File System (WekaFS), a fast shared parallel file system to manage its AI data pipeline from the edge to the core to the cloud.

Weride Chooses Wekaio

WeRide implemented WekaFS using a hybrid model to manage compute and storage resources both on-premises using commodity Intel x86-based servers and in the AWS Cloud. WeRide chose it because it presented a hardware-agnostic solution that was also the most cost-efficient, delivering high-bandwidth I/O to the company’s GPU farm for performance with mixed workloads.

WeRide is a multi-faceted AI start-up that works on advanced R&D cycles for Level 4 (L4) autonomous driving vehicles and on partnerships with transportation platform providers that support robotaxi services for commuters. The company processes data at the petabyte level with a daily mix of large video and image files generated from mapping the operational design domain for the robotaxi service.

The images are collected from more than 2 million kilometers of driving distance. The start-up produces millions of high-quality labeling data that is annotated at the core, trained by the AI model on the cloud-based cluster, and fed back to the on-premises AI engine.

It needed a cost-effective solution that would provide high I/O bandwidth to keep the GPU farm saturated with data. It also had to deliver performance for mixed workloads with significant volumes of metadata, enable a hybrid implementation for the organization to maximize its investment, and be hardware-agnostic to allow for flexible and scalable expansion. It selected Weka as it met every one of these requirements and provided technical support expertise and customer references.

We had built a GPU farm, and we needed a high-speed data pipe to feed it without impacting our bottom line. After an extensive cost analysis and evaluation of several alternative solutions, including open-source and HDFS, Weka stood out as the clear leader,” says Paul Liu, engineering operations lead, WeRide. “Weka was the best choice for our hardware procurement model and was ideal for fulfilling our objective to make our storage a utility for our users – completely hardware-agnostic and transparent to the end-user. Weka’s customer references demonstrated product maturity and the technical support team proved invaluable by getting us launched in AWS.”

Beyond delivering high I/O bandwidth to data-hungry GPUs to keep them fully utilized, WekaFS is a fast and scalable file system for data-intensive applications, whether hosted on-premises or in the public cloud. It is a POSIX file system that scales performance linearly as the GPU server farm grows, so WeRide will not have to compromise performance with future expansions. And since the customer is running WekaFS on GPU servers in converged mode, creating a single namespace from all the locally attached NVMe drives, they will not have to invest in expensive hardware for their on-premises cluster.

The Weka software has allowed WeRide to take maximum advantage of its investment in GPUs while achieving the required infrastructure cost efficiencies. In addition, with our file system they get the benefit of workload uniformity and mobility between their on-premises cluster and the public cloud,” said Liran Zvibel, co-founder and CEO, WekaIO. “Weka is solving big problems for customers who require the flexibility of a software-only storage solution that provides all the traditional enterprise features while delivering superior performance at scale. Innovators such as WeRide share my vision to make storage a utility, it’s there and it works, and data available to anyone in the organization who needs it in a predictable time, no matter where they are.

Datacenters are evolving, incorporating accelerated computing technologies and cloud strategies to support new workloads such as AI or ML. WekaFS is cloud-optimized and architected to provide high bandwidth I/O to GPU-enabled compute clusters playing a big role in enabling digital transformation. We are pleased to have been the solution provider of the Weka software licenses for WeRide to drive their AI workflow,” added Chris Saso, CTO Dasher Technologies, Inc., a Weka WIN partner.

Resources:

Articles_bottom
AIC
ATTO
OPEN-E