In March 2024, Dell launched Data Lakehouse, a turnkey solution with a powerful query engine based on Starburst technology, providing high-performance, high-concurrency access to distributed data, regardless of the data source and its location. Dell Data Lakehouse is built on Dell S3-compatible storage, which provides a high-performance, highly available storage layer for storing and querying data in open formats such as iceberg.
IT and data leaders face the challenge of accelerating analytics and AI while keeping costs under control. While data lakehouse adoption offers performance benefits at lower costs, IT and data engineering teams must decide which data to optimize and cache and which to leave alone in the data lake.
Data managers at many companies complain that data is growing faster than their budgets.
To meet these needs, Dell Introduces Warp Speed on Dell Data Lakehouse to Optimize Data Storage on Dell S3-compatible storage.
Warp Speed for data storage
Warp Speed is a new feature of Dell Data Lakehouse that autonomously learns query patterns and identifies frequently accessed data to create optimal indexes and caches, while keeping infrequently accessed data where it is located.
Warp Speed accelerates query performance while keeping costs down. With Warp Speed, the same cluster can run queries to the data lake 3-5 times faster without requiring any changes to the query from the end user. It can also help reduce cluster sizes by up to 40%.
Warp Speed improves performance, reduces costs and optimizes data analytics with Dell Data Lakehouse
In simpler terms, organizations they can run more queries on large clusters or run the same volume of queries on smaller clusters.
- Acceleration of queries in the data lakes: It enables you to autonomously index your data lake and accelerate exploratory data sets on demand without involving data engineering.
- Creation of high performance control panels: It enables faster drill-down from TB to PB of data, without any change to the end-user experience. The same queries now run faster.
Warp Speed employs a combination of acceleration technologies to achieve its remarkable performance.
· Indexing autonomous: This feature creates appropriate index types (bitmap, dictionary, tree) tailored to each data block, speeding up operations such as joins, filters, and lookups. Indexes are stored on an SSD on the compute nodes for fast access.
· Storage smart caching: Smart Caching is a patented SSD columnar block caching that optimizes performance based on data usage frequency. Caching eliminates unnecessary table scans and provides greater data reuse between queries, saving processing costs.
Availability
Beginning July 17, Warp Speed will be available to all Dell Data Lakehouse customers and will be supported for those using Dell S3-compatible storage in their data lake. There are no changes to the software license, it is now built-in. Compute node configuration will be modified to include SSDs that have been fully tested and evaluated by Dell to support Warp Speed index and cache.
