AWS Windows FSx vs Luster FSx vs EFS

Posted in AWS, Cloud by nkcode On February 25, 2021. No comments

AWS EC2 EFS (Elastic File System) vs Windows FSx (Fast Storage File system) vs Luster FSx (Fast Storage File system)

FSx:

Amazon FSx makes it easy and cost-effective to launch and run popular file systems that are fully managed by AWS. With Amazon FSx, you can leverage the rich feature sets and fast performance of widely-used open source and commercially licensed file systems, while avoiding time-consuming administrative tasks such as hardware provisioning, software configuration, patching, and backups. It provides cost-efficient capacity with high levels of reliability and integrates with a broad portfolio of AWS services to enable faster innovation.

Amazon FSx provides you with two file systems to choose from Amazon FSx for Windows File Server for enterprise workloads and Amazon FSx for Lustre for high-performance workloads.

Amazon FSx for Windows File Server

Amazon FSx for Windows File Server provides fully managed, highly reliable, and scalable file storage that is accessible over the industry-standard Server Message Block (SMB) protocol. It is built on Windows Server, delivering a wide range of administrative features such as user quotas, end-user file restore, and Microsoft Active Directory (AD) integration. It offers single-AZ and multi-AZ deployment options, fully managed backups, and encryption of data at rest and in transit. You can optimize cost and performance for your workload needs with SSD and HDD storage options, and you can scale storage and change the throughput performance of your file system at any time. Amazon FSx file storage is accessible from Windows, Linux, and macOS compute instances and devices running on AWS or on-premises.

Amazon FSx for Windows File Server provides simple, fully managed, highly reliable file storage that’s accessible over the industry-standard Server Message Block (SMB) protocol.

Built on Windows Server, providing full SMB support and a wide range of administrative features like user quotas, data deduplication, and end-user file restore.
Provides file storage that is accessible from Windows, Linux, and MacOS compute instances and devices running on AWS or on-premises.
Integrates with Microsoft Active Directory (AD) to support Windows-based environments and enterprises.
Offers single-AZ and multi-AZ deployment options, SSD and HDD storage options, and provides fully managed backups.
All file system data is automatically encrypted at rest and in transit.
You can dynamically scale your file system to fit your storage and throughput needs.

Amazon FSx for Lustre

Amazon FSx for Lustre is a fully managed service that provides cost-effective, high-performance, scalable storage for compute workloads. Many workloads such as machine learning, high performance computing (HPC), video rendering, and financial simulations depend on compute instances accessing the same set of data through high-performance shared storage.

Powered by Lustre, the world’s most popular high-performance file system, FSx for Lustre offers sub-millisecond latencies, up to hundreds of gigabytes per second of throughput, and millions of IOPS. It provides multiple deployment options and storage types to optimize cost and performance for your workload requirements.

FSx for Lustre file systems can also be linked to Amazon S3 buckets, allowing you to access and process data concurrently from both a high-performance file system and from the S3 API.

Amazon FSx for Lustre makes it easy and cost-effective to launch and run the world’s most popular high-performance file system. Use it for workloads where speed matters, such as machine learning, high-performance computing (HPC), video processing, and financial modeling.

Allows your workloads to process data with consistent sub-millisecond latencies, up to hundreds of gigabytes per second of throughput, and up to millions of IOPS.
POSIX-compliant, so you can use your current Linux-based applications without having to make any changes, providing a native file system interface that works as any file system does with your Linux operating system.
Supports multiple deployment options for short-term and long-term data processing.
Seamlessly integrated with Amazon S3 (connect your S3 data sets to your FSx for Lustre file system, run your analyses, write results back to S3, and delete your file system), Amazon SageMaker, Amazon Elastic Kubernetes Service (EKS), and AWS ParallelCluster.
Accessible from on-premises over Direct Connect and VPN connections.
File system data is automatically encrypted at-rest and in-transit.

Elastic File System

Amazon Elastic File System (Amazon EFS) provides a simple, serverless, set-and-forget, elastic file system that lets you share file data without provisioning or managing storage. It can be used with AWS Cloud services and on-premises resources and is built to scale on-demand to petabytes without disrupting applications. With Amazon EFS, you can grow and shrink your file systems automatically as you add and remove files, eliminating the need to provision and manage capacity to accommodate growth.

An Amazon EFS Elastic file system is accessed by EC2 instances running inside one of your VPCs. Instances connect to a file system by using a network interface called a mount target. Each mount target has an IP address, which we assign automatically or you can specify.

Amazon EFS is a fully-managed service that makes it easy to set up, scale, and cost-optimize file storage in the Amazon Cloud. With a few clicks in the AWS Management Console, you can create file systems that are accessible to Amazon EC2 instances via a file system interface (using standard operating system file I/O APIs) and support full file system access semantics (such as strong consistency and file locking).

Amazon EFS file systems can automatically scale from gigabytes to petabytes of data without needing to provision storage. Tens, hundreds, or even thousands of Amazon EC2 instances can access an Amazon EFS file system at the same time, and Amazon EFS provides consistent performance to each Amazon EC2 instance. Amazon EFS is designed to be highly durable and highly available. With Amazon EFS, there is no minimum fee or setup costs, and you pay only for what you use.

References: