πŸ—‚οΈ AWS Object + File Storage Β· Module 04

Amazon S3 + EFS
Theory Notes

S3 (Simple Storage Service) for object storage + EFS (Elastic File System) for shared file storage. Complete guide with real Indian company scenarios.

πŸ’Ύ

Block Storage

VM's virtual hard drive. Attached directly to EC2.

AWS: Amazon EBS β†’ Module 03
πŸ—‚οΈ

Object Storage S3 β€” HERE

Files stored via URL. Unlimited capacity, any file type.

AWS: Amazon S3
πŸ“

File Storage EFS β€” HERE

Shared NFS drive for multiple Linux EC2 instances.

AWS: Amazon EFS (this module)
Topics Covered β€” S3 (01–12) + EFS (13–17)
1

What is Amazon S3?

Amazon S3 (Simple Storage Service) is AWS's object storage service. Unlike EBS (which is a hard drive attached to one EC2), S3 is an internet-accessible storage β€” you can store any file and access it via a URL from anywhere in the world.

S3 is one of AWS's oldest and most-used services. It stores objects (files + metadata) inside buckets (containers). There is no folder structure in S3 β€” only the illusion of folders through "/" in object names.

☁️ Analogy β€” Google Drive for Applications

Think of S3 as Google Drive for your applications.

You upload files. Each file gets a URL. Any application (or person) with permission can download it via that URL. Unlike Google Drive for humans, S3 stores billions of files and serves billions of requests per day β€” at any scale. It never gets "full" β€” storage is unlimited, you only pay for what you use.

♾️

Unlimited Storage

No capacity limit. Store 1 file or 1 billion files. Individual objects up to 5 TB.

🌐

Access via URL

Every object has a unique URL. Access it from any browser, app, or region worldwide.

πŸ›‘οΈ

11 Nines Durability

99.999999999% β€” AWS replicates your data across 3+ AZs automatically. Almost impossible to lose data.

πŸ’°

Pay as You Use

Pay only for storage used + data transferred. No upfront commitment. Prices from $0.023/GB/month.

🏒 Real Company Scenarios

Hotstar (now Disney+ Hotstar) streams to 500 million+ users. Every cricket match, movie, and web series is stored on AWS S3. The video files (each season of a show can be 500 GB+), thumbnail images, subtitle files, and chapter markers are all stored as S3 objects. When you open Hotstar and see a show thumbnail β€” that image is being served from an S3 bucket via Amazon CloudFront (CDN). S3's 11-nines durability ensures Hotstar has never lost a single video file since launch.

Zepto (10-minute grocery delivery app) stores all product images β€” photos of 10,000+ grocery items β€” in S3. Each product has 3-4 high-resolution images. S3 serves these images to the Zepto app and website. When a Zepto engineer updates a product image, they just replace the S3 object β€” the app everywhere automatically shows the new image within minutes.

2

Buckets & Objects

S3 has a simple two-level structure: Buckets contain Objects.

ConceptDetailsExample
Bucket NameGlobally unique, 3-63 chars, lowercase, no underscoreszomato-food-images-prod
Object KeyFull path to object inside bucketrestaurants/delhi/biryani.jpg
Object URLhttps://[bucket].s3.[region].amazonaws.com/[key]https://zomato-food-images-prod.s3.ap-south-1.amazonaws.com/restaurants/delhi/biryani.jpg
Max Object Size5 TB per single object (use multipart upload for >100 MB)A 2-hour HD movie = ~4 GB β€” fine for S3
Bucket RegionChosen at creation, cannot be changedMumbai (ap-south-1) for Indian user data
🏒 Real Company Scenario β€” Zomato's Bucket Structure

Zomato organizes its S3 buckets like this:

β€’ zomato-restaurant-images-prod β†’ Food photos uploaded by restaurants (1M+ images)

β€’ zomato-user-profile-photos β†’ Profile pictures of 100M+ users

β€’ zomato-order-invoices-2026 β†’ PDF invoices for all orders (compliance)

β€’ zomato-analytics-raw-data β†’ Click-stream data for data science team

Inside zomato-restaurant-images-prod, objects are organized: cities/delhi/restaurants/biryani-house/dish-1.jpg. This isn't a real folder β€” it's just "/" in the key name, but the S3 Console shows it as folders for convenience.

3

S3 Storage Classes

S3 has 7 storage classes β€” you pay less for data you access less frequently. Choosing the right class saves significant money. All classes provide 11-nines durability.

S3 Standard

Default β€” Frequently Accessed

Highest availability (99.99%). Instant access. Replicated across 3+ AZs. For active production data.

~$0.023/GB/month
Intelligent-Tiering

Auto-Optimized β€” Unknown Access

AWS automatically moves objects between Frequent and Infrequent tiers. Small monitoring fee per 1,000 objects.

~$0.023/GB + monitoring fee
Standard-IA

Infrequent Access

Lower storage cost, retrieval fee charged. Good for data accessed once a month or less. Minimum 30-day storage.

~$0.0125/GB/month
One Zone-IA

One AZ β€” Even Cheaper

Stored in only ONE AZ (not 3). 20% cheaper than Standard-IA. OK if you can recreate the data if that AZ fails.

~$0.01/GB/month
Glacier Instant Retrieval

Archive β€” ms Retrieval

For archive data accessed once per quarter. Millisecond retrieval time. Minimum 90 days storage.

~$0.004/GB/month
Glacier Flexible Retrieval

Archive β€” Min/Hours Retrieval

Retrieval in 1 min–12 hours. For data accessed once or twice a year. Minimum 90 days storage.

~$0.0036/GB/month
Glacier Deep Archive

Cheapest β€” 12hr Retrieval

Retrieval within 12 hours. Lowest cost storage in all of AWS. For data you almost never access β€” 7-10 year archive. Minimum 180 days.

~$0.00099/GB/month
🏒 Real Company Scenarios β€” Storage Class Selection

BookMyShow stores current movie showtimes, seat maps, and booking confirmations in S3 Standard β€” accessed millions of times per day. But old ticket PDFs from 2020–2023 (accessed maybe once a year when customers need a refund) are stored in Glacier Flexible Retrieval β€” 85% cheaper. BookMyShow saves β‚Ή40+ lakhs per year in storage costs just by choosing the right class for different data ages.

IRCTC must retain all railway booking records for 7 years (government compliance). Records older than 2 years are automatically moved to Glacier Deep Archive β€” at β‚Ή0.066/GB/month ($0.00099) β€” the cheapest possible storage. 7 years of booking data at 50 TB = costs only β‚Ή3,300/month vs β‚Ή1.15 lakhs/month on S3 Standard. That's 97% savings.

4

Access Control (Bucket Policies, ACLs, Public/Private)

By default, S3 buckets are 100% private β€” no one can access anything. You control access through 3 mechanisms.

πŸ“œ

Bucket Policies

JSON policies attached to the bucket. Grant access to specific IAM users, roles, or entire public. Most flexible option.

πŸ‘€

ACLs (Access Control Lists)

Legacy per-object permissions. Grant read/write to specific AWS accounts. Rarely used today β€” Bucket Policies are preferred.

πŸ”’

Block Public Access

Account-level override that blocks ALL public access regardless of bucket policies. Enable this on all buckets that don't need to be public.

🌐

Public Bucket

When you want everyone to read files (e.g., website images). Make bucket public + add bucket policy allowing public GetObject.

Never Leave Customer Data Buckets Public β€” Common Security Mistake
Many AWS data breaches happen when developers accidentally make an S3 bucket public. If you store user data, financial records, or health records β€” never make the bucket public. Use Pre-signed URLs instead (see Section 5). AWS now alerts you if a bucket with suspicious content is made public, and many companies enable "Block Public Access" at the account level.
5

Pre-signed URLs β€” Temporary Access

A Pre-signed URL is a temporary URL that gives anyone who has it time-limited access to a specific S3 object β€” without making the bucket public.

It is generated by your server using your AWS credentials. The URL includes an expiry time (e.g., 15 minutes, 1 hour). After expiry, the URL stops working.

πŸ”‘ Analogy β€” Hotel Key Card

Pre-signed URL = Hotel key card valid for 3 days.

The hotel (S3) is private β€” you can't just walk in. But the receptionist (your server) generates a special key card (pre-signed URL) that lets you enter your specific room (object) for exactly 3 days. After checkout (expiry), the key card stops working. You never gave permanent access β€” just a time-limited pass.

🏒 Real Company Scenarios

Razorpay (India's top payment gateway) generates payment receipts as PDF files stored in a private S3 bucket. When a merchant downloads a receipt, Razorpay's server generates a pre-signed URL valid for 5 minutes. The merchant clicks the link and downloads the PDF. After 5 minutes, the link expires β€” even if they shared it with someone, the URL no longer works. This keeps financial documents secure while allowing legitimate access.

Nykaa (beauty e-commerce) stores product demonstration videos privately on S3. When a user opens a product page, the backend generates a pre-signed URL valid for 2 hours. The video player uses this URL to stream the video. Even if a user copies the URL and tries to share it, after 2 hours it stops working. This prevents unauthorized video sharing/downloading.

6

S3 Versioning

Versioning keeps multiple versions of every object in the same bucket. When you upload a new version, the old version is kept β€” not deleted. Every version gets a unique version ID.

πŸ”„

Rollback Anytime

Accidentally deleted a file? Overwrote it with wrong data? Restore any previous version instantly.

πŸ—‘οΈ

Delete Marker

Deleting an object adds a "delete marker" β€” original versions still exist. Remove the marker to restore.

πŸ’°

Costs More Storage

Every version stored = billed separately. Use lifecycle policies to delete old versions after N days.

Always Enable Versioning on Critical Buckets
For buckets storing user data, configurations, or any data you can't afford to lose β€” always enable versioning. It is especially important for buckets that applications write to automatically β€” a code bug could overwrite millions of objects, and versioning lets you recover all of them.
7

Static Website Hosting

S3 can serve static websites (HTML, CSS, JS files β€” no server-side code) directly. No EC2, no web server needed. Just upload your files and enable "Static Website Hosting" in bucket settings.

Your website URL becomes: http://[bucket-name].s3-website.[region].amazonaws.com

For custom domains (like yourcompany.com), use with Amazon CloudFront (CDN) for HTTPS + speed.

🏒 Real Company Scenarios

Meesho hosts its seller documentation website (how-to guides, API docs, policy pages) on S3 Static Website Hosting. The entire site is HTML/CSS/JS files β€” no backend needed. Hosting on S3 costs β‚Ή500–2,000/month vs running an EC2 (β‚Ή3,000–10,000/month). For any website that doesn't need a database or server-side processing, S3 is the cheapest and most reliable option.

Thousands of Indian startups host their product landing pages, coming-soon pages, and marketing microsites on S3 + CloudFront. Zero server management, infinite scalability, and near-zero cost for low-traffic sites.

8

Lifecycle Policies β€” Automate Storage Management

Lifecycle Policies automatically transition objects to cheaper storage classes or delete them after a set number of days β€” without any manual intervention. Define rules once; AWS handles it forever.

Day 0: Upload

Fresh object β€” user accesses it frequently

S3 Standard: $0.023/GB
↓

Day 30: Auto-move to S3 Standard-IA

Lifecycle rule kicks in β€” infrequent access after 1 month

S3 Standard-IA: $0.0125/GB (45% cheaper)
↓

Day 90: Auto-move to Glacier

Rarely accessed β€” archive it

Glacier Flexible: $0.0036/GB (84% cheaper than Standard)
↓

Day 365: Auto-delete (optional)

If data is no longer needed after 1 year

$0 β€” object deleted
🏒 Real Company Scenarios

OYO Rooms collects hotel check-in photos, maintenance reports, and guest feedback documents. They use lifecycle policies: photos from last 30 days stay in S3 Standard (operations team checks them daily). Photos 30–90 days old move to Standard-IA (referenced occasionally for disputes). Photos older than 90 days move to Glacier (kept 3 years for legal compliance). After 3 years, auto-deleted. OYO saves β‚Ή25+ lakhs per year compared to keeping everything in S3 Standard forever.

Swiggy's delivery partner app generates location GPS logs. Recent 7 days in S3 Standard (real-time analysis). 7–30 days in Standard-IA (monthly reporting). 30 days+ in Glacier Deep Archive (compliance). After 2 years, auto-deleted. This is a standard data lake pattern used by all major Indian tech companies.

9

S3 Replication

S3 Replication automatically copies objects from one bucket to another β€” in the same region or a different region. Two types:

Replication Requirements
Versioning must be enabled on BOTH source and destination buckets for replication to work. Replication only copies new objects after you enable it β€” existing objects are not automatically copied (use S3 Batch Operations to copy existing objects).
10

S3 Encryption

S3 offers multiple encryption options. Since Nov 2023, all new S3 objects are encrypted by default with SSE-S3 β€” no extra configuration needed.

TypeManaged ByBest For
SSE-S3 (default)AWS manages keys automaticallyStandard data β€” zero effort
SSE-KMSAWS KMS β€” you control key rotation and access logsCompliance data β€” audit who accessed keys
SSE-CYou provide your own key with every requestStrict data sovereignty β€” key never leaves your control
Client-SideYou encrypt before uploadingMaximum security β€” encrypted even before reaching AWS
11

AWS S3 vs Azure Blob Storage

ConceptAmazon S3Azure Blob Storage
Service NameAmazon S3Azure Blob Storage
Container typeBucketContainer (inside Storage Account)
Object nameObject (with Key)Blob
Hot storageS3 StandardHot tier
Infrequent accessS3 Standard-IACool tier
ArchiveGlacier Deep ArchiveArchive tier
Static WebsiteS3 Static Website HostingAzure Static Web Apps / $web container
VersioningS3 VersioningBlob Versioning (same concept)
CDNAmazon CloudFrontAzure CDN / Front Door
Temp URLsPre-signed URLsSAS (Shared Access Signature) URLs
Data transfer toolAWS SnowballAzure Data Box
File StorageAmazon EFS (NFS, Linux)Azure Files (SMB + NFS)
12

S3 Summary

Object storage (not block) Globally unique bucket names Default: private 7 storage classes Lifecycle = auto cost savings Pre-signed URL = temp access Versioning = accidental delete protection
πŸ“ AWS File Storage Β· Amazon EFS

Amazon EFS

Elastic File System β€” Shared Network File Storage for Linux EC2 instances. Multiple EC2s read/write the same file system simultaneously.

13

What is Amazon EFS? NEW

EFS = Elastic File System

Amazon EFS is a fully managed, shared NFS (Network File System) for Linux-based EC2 instances. Unlike EBS (one volume β†’ one EC2), EFS allows multiple EC2 instances to read and write the same file system simultaneously.

EFS is elastic β€” it grows and shrinks automatically as you add or remove files. You never provision capacity; you only pay for what you store.

πŸ“ Analogy β€” Company Network Drive

EFS = Company's shared network drive (like an office \\server\files)

Think of EFS like a company's shared network folder that all employees (EC2 instances) can access at the same time. Employee A can edit a document, Employee B can open the same folder and see the change immediately β€” they're all connected to the same storage. EBS would be like each employee having their own private hard drive β€” no sharing. S3 would be like a shared Google Drive β€” accessible via URL but not mountable as a drive letter.

πŸ”—

Multi-Attach

Thousands of EC2 instances can mount and access the same EFS file system simultaneously.

🐧

Linux Only

EFS uses NFS protocol β€” only compatible with Linux EC2 instances. For Windows, use Amazon FSx for Windows.

πŸ“

Auto-Scales

Starts at 0 bytes. Grows automatically as you add files. Shrinks when you delete them. Never run out of space.

🌐

Cross-AZ

EFS file system is accessible from multiple Availability Zones in the same region. EBS is locked to a single AZ.

FeatureEFS (File Storage)EBS (Block Storage)S3 (Object Storage)
TypeShared network file systemBlock device (virtual hard drive)Object store (URL-accessed)
ProtocolNFS (Network File System)Block (transparent to OS)HTTP REST API
AccessThousands of EC2s simultaneouslyOne EC2 at a time (except io2 multi-attach)Anyone with URL/permission
OSLinux only (NFS)Linux and WindowsOS-independent
Pricing~$0.30/GB/month (Standard)~$0.08–0.10/GB/month (gp3)~$0.023/GB/month (Standard)
CapacityAuto-elastic (petabyte scale)Provisioned size (max 64 TB)Unlimited
Use forShared configs, ML models, home dirsOS drive, databases, appsImages, videos, backups, logs
🏒 Real Company Scenarios β€” When Companies Use EFS

A major OTT platform like Hotstar or SonyLIV runs 50+ video transcoding servers simultaneously β€” taking one uploaded video and converting it into 10 different resolutions (240p, 360p, 480p, 720p, 1080p, 4K, etc.). All 50 servers read the same original video file and write their output to the same shared folder. This is a classic EFS use case β€” the file system is mounted on all 50 EC2s, and they all work in parallel on shared storage.

Swiggy's ML team trains machine learning models (delivery time prediction, demand forecasting) using Python on 20 EC2 GPU instances. All 20 servers need access to the same 500 GB training dataset. With EFS, they mount the same file system on all 20 instances. With EBS, they'd need to copy 500 GB to each machine β€” 10 TB total, slow and expensive. EFS = one copy, all servers access it.

Freshdesk (Freshworks product) runs on a fleet of 100+ application servers. All servers need access to the same configuration files, shared libraries, and log directories. EFS is mounted at /app/config and /app/logs β€” when DevOps updates a config file, all 100 servers see the change immediately without any restart or sync process.

14

EFS Performance Modes

Choose Your Performance Mode at Creation

EFS offers two performance modes. Choose based on how many files per second your workload requires. This cannot be changed after creation β€” choose carefully.

General Purpose (default)

Lowest latency per file operation. Best for most workloads. Up to 35,000 IOPS. Suitable for web servers, content management, home directories, dev environments.

βœ… Choose this for 99% of use cases. Use by default.

Max I/O

Highest aggregate throughput and IOPS β€” but slightly higher latency per operation. For workloads with thousands of EC2s all accessing simultaneously. Big data, media processing, analytics.

βœ… Choose only for massive parallel workloads with 100s of EC2 clients.
Also Choose Throughput Mode
Separately from performance mode, also choose: Elastic Throughput (auto-scales throughput, pay per use β€” recommended for most) OR Provisioned Throughput (fixed throughput, you pay for it regardless of use β€” for predictable high-throughput workloads).
15

EFS Storage Classes

EFS also has storage tiers to reduce cost

Similar to S3, EFS has tiered pricing. Files you access less frequently should be in cheaper tiers. Use EFS Intelligent-Tiering to automatically move files between tiers based on access.

Storage ClassPrice (approx.)Use For
EFS Standard~$0.30/GB/monthActively accessed files (hot data)
EFS Standard-Infrequent Access~$0.025/GB/month + retrieval feeFiles not accessed for 30+ days
EFS One Zone~$0.16/GB/monthSingle AZ (cheaper, less resilient) β€” dev/test workloads
EFS One Zone-Infrequent Access~$0.013/GB/month + retrieval feeSingle AZ + rarely accessed β€” cheapest EFS option
Use EFS Intelligent-Tiering
Enable EFS Intelligent-Tiering β€” it automatically moves files that haven't been accessed for 30+ days to Standard-IA. Files accessed again automatically move back to Standard. This is the easiest way to cut EFS costs by 30–70% without changing any application code.
16

EBS vs S3 vs EFS β€” The Full Comparison

This is the most important table in this module. Know this cold β€” exam questions always compare these three.

QuestionUse EBS πŸ’ΎUse S3 πŸ—‚οΈUse EFS πŸ“
What do I store?OS disk, database files, app binariesImages, videos, backups, logs, documentsShared configs, shared datasets, home directories
How many EC2s access it?1 (exclusive)∞ (via URL)Thousands simultaneously
Can it run a database?βœ… Yes β€” fast IOPS❌ No β€” not a file system⚠️ Possible but not recommended
Works on Windows?βœ… Yesβœ… Yes (REST API)❌ No β€” NFS = Linux only
Static website hosting?❌ Noβœ… Yes❌ No
Storage grows automatically?❌ No β€” manually provisionedβœ… Yes β€” unlimitedβœ… Yes β€” elastic
AWS Pricing (approx.)$0.08–0.10/GB/mo (gp3)$0.023/GB/mo (Standard)$0.30/GB/mo (Standard)
Azure equivalentAzure Managed DisksAzure Blob StorageAzure Files
Quick Decision Rule
VM needs a hard drive? β†’ EBS  |  Store files accessible via URL? β†’ S3  |  Multiple Linux servers need shared files? β†’ EFS  |  Windows shared file share? β†’ Amazon FSx for Windows (not EFS)
17

EFS Summary

EFS = shared NFS Linux only Auto-elastic Multi-EC2 access Cross-AZ Intelligent-Tiering saves cost Azure Files equivalent
AWS Official Docs

Official Documentation

Service Topic AWS Docs URL
S3S3 OverviewS3/Welcome.html
S3Storage Classesstorage-class-intro.html
S3Pre-signed URLsShareObjectPreSignedURL.html
S3Lifecycle Policiesobject-lifecycle-mgmt.html
EFSEFS Overviewefs/whatisefs.html
EFSPerformance Modesefs/performance.html
EFSEFS Storage Classesefs/storage-classes.html