Komprise Smart Data Workflows to Automate Unstructured Data Discovery

Komprise, Inc. announced Smart Data Workflows, a systematic process to discover relevant file and object data across cloud, edge and on-premises datacenters and feed data in native format to AI and ML tools and data lakes.

Click to enlarge

Industry analysts from Industrial Review predict that at least 80% of the world’s data will be unstructured by 2025. This data is critical for AI and ML-driven applications and insights, yet much of it is locked away in disparate data storage silos. This creates an unstructured data blind spot, resulting in billions of dollars in missed big data opportunities.

The company has expanded Deep Analytics Actions to include copy and confine operations based on Deep Analytics queries, added the ability to execute external functions such as running natural language processing functions via API and expanded global tagging and search to support these workflows. Smart Data Workflows allow to define and execute a process with as many of these steps needed in any sequence, including external functions at the edge, datacenter or cloud. The company’s Global File Index and Smart Data Workflows together reduce the time it takes to find, enrich and move the right unstructured data by up to 80%.

“Komprise has delivered a rapid way to visualize our petabytes of instrument data and then automate processes such as tiering and deletion for optimal savings,” says Jay Smestad, senior director, IT, PacBio. “Now, the ability to automate workflows so we can further define this data at a more granular level and then feed it into analytics tools to help meet our scientists’ needs is a game changer.”

Click to enlarge

Smart Data Workflows are relevant across many sectors. Here’s example from pharmaceutical industry: 

1) Search: Define and execute a custom query across on-prem, edge and cloud data silos to find all data for Project X with Deep Analytics and Global File Index.

2) Execute and enrich: Execute an external function on Project X data to look for a specific DNA sequence for a mutation and tag such data as ‘Mutation XYZ’.

3) Cull and mobilize: Move only Project X data tagged with ‘Mutation XYZ’ to the cloud using Komprise Deep Analytics Actions for central processing.

4) Manage Data Lifecycle: Move the data to a lower storage tier for cost savings once the analysis is complete.

Other Smart Data Workflow use cases include:

• Legal divestiture: Find and tag all files related to a divestiture project and move sensitive data to an object-locked storage bucket and move the rest to a writable bucket.
• Autonomous vehicles: Find crash test data related to abrupt stopping of a specific vehicle model and copy this data to the cloud for further analysis. Execute an external function to identify and tag data with ‘Reason=Abrupt Stop’ and move only the relevant data to the cloud data lakehouse to reduce time and cost associated with moving and analyzing unrelated data.

“Whether it’s massive volumes of genomics data, surveillance data, IoT, GDPR or user shares across the enterprise, Komprise Smart Data Workflows orchestrate the information lifecycle of this data in the cloud to efficiently find, enrich and move the data you need for analytics projects,” says Kumar Goswami, CEO, Komprise. “We are excited to move to this next phase of our product journey, making it much easier to manage and mobilize massive volumes of unstructured data for cost reduction, compliance and business value.”

Resource :
Blog : Komprise Smart Data Workflows: Automate Unstructured Data Discovery