AWS Devops : Storage

AWS DEVOPS : Cheat Sheet

Storage

  • AWS storage type
    • Block: ebs 
    • File : EFS, fsx luster [hpc], fsx luster [smb protocol, windows]
    • Object : S3
  • Data Tranfer
    • AWS transfer family : ftp/sftp/ftps : including onpremise
    • AWS data sync : can copy data between : including on premise
      • When copying data to Amazon S3, DataSync automatically converts each file to be a single S3 object in a 1:1 relationship. 

between

  •  Network File System (NFS) shares
  • Server Message Block (SMB) shares
  • Self-managed object storage
  • AWS Snowcone
  • Amazon S3 buckets
  • Amazon EFS file systems
  • Amazon FSx for Windows File Server file systems
  • For[ edge/archiving/transferring/hybrid workload]
  • AWS Snow family : offline data transfer and Using the AWS Snow Family for edge local compute and storage use cases
    • Cone
    • Snowball
    • Snowmobile
  • AWS Application Migration Service
    • for lift-and-shift migrations
    • AWS MGN supports migrations from VMware vSphere, Microsoft Hyper-V, Amazon EC2, and other clouds to AWS
  • Data Protection
    • AWS Backup is a fully managed data protection service that makes it easy to centralize and automate across AWS services, in the cloud, and on premises
      • Cross region, cross account, schedule, lifecyle, retention policies, and can do for multiple service at a single place.

EBS

  • AZ specific, snapshot available regional
    • General purpose SSD
    • PIOPS SSD
    • Optimized HDD
    • COLD HDD
  • You can create point-in-time snapshots of EBS volumes, which are persisted to Amazon S3. 
  •  If the check box for Delete on Termination is checked, the volume(s) will delete upon the termination of the EC2 instance
  • Amazon EBS encryption uses 256-bit Advanced Encryption Standard algorithms (AES-256) and an Amazon-managed key infrastructure. The encryption occurs on the server that hosts the EC2 instance, providing encryption of data-in-transit from the EC2 instance to Amazon EBS storage
  • Snapshots are incremental backups
  • Automating cross-account snapshot copies enables you to copy your Amazon EBS snapshots to specific Regions in an isolated account and encrypt those snapshots with an encryption key. 
  • Force the immediate initialization of the entire volume. For more information, see Initialize Amazon EBS volumes on Linux.
  • Enable fast snapshot restore on a snapshot to ensure that the EBS volumes created from it are fully-initialized at creation and instantly deliver all of their provisioned performance.
  • Virtualization Type : paravirtualized, HVM
  • Amazon EBS Multi-Attach enables you to attach a single Provisioned IOPS SSD ( io1 or io2 ) volume to multiple instances that are in the same Availability Zon : 16 instance
  • There is an option for recycle bin policies Snapshot and AMI
  • Lifecycle policy based on volume/isntance
  • You can add schedule daily /weekly/monthly/yearly/custom cron
  • Retention type : count / age

EFS

-Standard storage classes – EFS Standard and EFS Standard–Infrequent Access (Standard–IA),

One Zone storage classes – EFS One Zone and EFS One Zone–Infrequent Access (EFS One Zone–IA), 

  • AWS DataSync is an online data transfer service that simplifies, automates, and accelerates moving and replicating data between on-premises storage systems, and also between AWS storage services. DataSync can copy data between Network File System (NFS), Server Message Block (SMB) file servers, self-managed object storage, AWS Snowcone, Amazon S3 buckets, Amazon EFS file systems, and FSx for Windows File Server file systems.
    • Hpc: fsx for lustere
    • SMB: fsx for windows
    • HTTP s3
    • NFS : EFS
  • You can also use DataSync to transfer files between two EFS file systems, including file systems in different AWS Regions and file systems owned by different AWS accounts
  • Regional Replication can be setup : regional DR policy
  • Automatic  backup can be setup : resillency
  • Amazon EFS file systems are resilient to one or more Availability Zone failures within an AWS Region. Mount targets themselves are designed to be highly available. 

Storage Gateway

Volume Gateway

  • A Volume Gateway provides cloud-backed storage volumes that you can mount as Internet Small Computer System Interface (iSCSI) devices from your on-premises application servers.
  • Cached : 
  • Stored : asynchronous backup

Amazon S3 File Gateway –Amazon S3 File Gateway supports a file interface into Amazon Simple Storage Service (Amazon S3) and combines a service and a virtual software appliance. By using this combination, you can store and retrieve objects in Amazon S3 using industry-standard file protocols such as Network File System (NFS) and Server Message Block (SMB).

S3

  • Bucket policy
  • ACL
  • Cross region replication needs versioning enabled
  • Default encryption: AWS SSE, need to enable
  • Can get the data related logs in S3 server access logs
  • Object lock, from being, deleted and overwritten
  • Multi-region access point , provides low latency over global access
  • One can configure MFA for deletion
  • AWS Glacier encrypts the data by default
  • EFS : encryption only through KMS while s3  SSE : AWS managed keys, customer managed keys, AWS KMS
AWS DEVOPS : Cheat Sheet

Leave a Reply

Your email address will not be published. Required fields are marked *