Lustre over ZFS


ZFS is a robust, scalable file-system with features not available in other file systems available today. Raid Inc. offers highly-customized Lustre over ZFS solutions to enable cheaper, more reliable storage for Lustre while maintaining high performance .

Today, HPC is a key technology for enterprises of all sizes. Getting maximum performance from HPC and data - intensive applications is a necessity that requires storage infrastructure that scales endlessly and delivers unmatched I/O levels. In partnering with Intel® Enterprise Edition for Lustre software (Intel® EE for Lustre software), RAID Inc. unleashes the performance and scalability of the Lustre parallel file system for HPC workloads, including technical ‘big data’ applications. Lustre over ZFS comes with many added benefits. The below are some concepts that details how Lustre and ZFS interact. Lustre targets run on a Local File-System on Lustre Severs. Object Storage Device (OSD) Layer supported are:


ZFS is the 2nd use of the OSD layer based on OpenZFS implementation. Other file system are possible with non trivial work

Lustre targets can be different types (hybrid Ldiskfs/ZFS is possible)

Allows the replacement of Ldiskfs (EXT4) with ZFS

Lustre Clients are unaffected by the choice of OSD filesystem

Compression and deduplication

Snapshot and replication

Data scrubbing and resilvering

LDISKFS (EXT4) is the commonly used driver

Other key features of Lustre over ZFS are:

Hybrid Storage Pool (ARC+L2ARC+ZIL)

ZFS helps build a software RAID for Lustre

Supports encryption

Checksumming

Copy on Write (COW)

Always consistent on disk

Manageability

PERFORMANCE
Intel EE for Lustre software has been designed to enable fully parallel I/O throughput across thousands of clients, servers, and storage devices. Metadata and data are stored on separate servers to allow optimization of each system for the different workloads they present. Improved metadata scalability using Distributed Namespace (DNE) feature is now integrated in Intel Manager for Lustre. Intel EE for Lustre can also scale down efficiently to provide fast parallel storage for smaller organizations.
MATURITY
Lustre has been in use in the world’s largest data centers for over a decade and hardened in the harshest big data environments. Intel EE for Lustre software is rigorously tested, reliable, and backed by Intel, the leading provider of technical support for Lustre software. Intel EE for Lustre software delivers commercial - ready Lustre in a package that can scale efficiently both up and down to suit your business workloads, with built - in manageability.
INTEGRATED APACHE HADOOP ADAPTER
When organizations operate both Lustre and Apache Hadoop within a shared HPC infrastructure, there is a compelling use case for using Lustre as the file system for Hadoop analytics, as well as HPC storage. Intel Enterprise Edition for Lustre includes an Intel - developed adapter which allows users to run MapReduce applications directly on Lustre. This optimizes the performance of MapReduce operations while delivering faster, more scalable, and easier to manage storage.
CAPACITY
The object - based storage architecture of Intel EE for Lustre software can scale to tens of thousands of clients and petabytes of data.
AFFORDABILITY
Intel EE for Lustre software is based on the community release of Lustre software, and is hardware, server, and network fabric neutral. Enterprises can scale their storage deployments horizontally, yet continue to have simple - to - manage storage.
INTEL® MANAGER FOR LUSTRE
Intel Manager for Lustre software includes simple, but powerful, management tools that provide a unified, consistent view of Lustre storage systems and simplify the installation, configuration, monitoring, and overall management of Lustre. The manager consolidates all Lustre information in a central, browser - accessible location for ease of use.

Learn More about RAID Inc.'s Lustre over ZFS
Talk to an Expert

Storage Systems with Lustre Over ZFS


Enterprises are focused on addressing challenges with data that is expanding at a faster rate like never before. It has become a challenge for industries to perform data utilization at speeds that help faster innovation. Those that can deliver insights faster while managing rapid infrastructure growth are the leaders in their industry. An organization’s underlying storage must support both new-era big data and traditional applications with security, reliability and high-performance. RAID Inc storage systems optimized for Lustre over ZFS will meet your specific HPC needs and challenges. Our high-performance File-system solutions help you manage your data at scale with the distinctive ability to perform IO with faster speeds while being reliable along with options like archiving and analytics in place.

Gain performance with Lustre Over ZFS and RAID Inc. storage.
Request A Quote Now!

Case Studies & Other Resources



So many satisfied
Customers

Our Approach


To deliver our tailored HPC solutions, RAID Inc.'s consultative sales and system design process is customized every step of the way. From the initial sales call to installation day, RAID Inc. works closely with clients to assess their unique computing and storage requirements by examining every facet of their computing needs. To learn more about our
6-Step Holistic Approach, 
click here.

Find

RAID Inc. performs rigorous requirements engineering strategies to develop solutions based on the actual need. It incorporates strategic approaches to find and understand your IO performance requirements when deploying systems and storage while keeping your costs low.

Analyze

RAID Inc. analyzes the best systems and storage solutions based on the requirements elicitation performed. With a wide range of compute and networking products available in the market, we deploy the right servers and networking products along with our storage to meet you actual needs.

Solve

RAID Inc. solves your HPC problems in no time. Being an industry leader, we served 1000s of customers in solving their HPC problems. The areas we covered include but not limited to genomic research, space explorations, weather forecasting, gas and oil explorations, governments, educational institutions, defense, life sciences and many others.