AWS Glue
Serverless data integration service that discovers your data's schema, catalogs it, and runs ETL jobs without any servers to manage.
Verified against AWS documentation · 2026-09-09
Key facts
- 01Two halves, tested separately: the Data Catalog (metadata — what your data is) and ETL jobs (serverless Spark that moves and transforms it)
- 02Crawlers scan S3 and JDBC sources, infer schema and partitions, and write table definitions into the Data Catalog — the data itself never moves
- 03Athena, Redshift Spectrum, and EMR all read the same Data Catalog — crawl once, query from anywhere
- 04No-code paths exist: Glue Studio (visual ETL canvas) and DataBrew (point-and-click cleaning for analysts who don't write Spark)
Exam triggers
Phrases and scenarios that point here.
- "Serverless ETL", "no infrastructure to manage", "prepare and transform data for analytics"
- Files already sitting in S3 with no schema defined → Glue crawler + Data Catalog, then Athena
- "Central metadata repository shared across analytics services" → Glue Data Catalog
- Transform + load → Glue; query in place, no transform → Athena; long-running tuned cluster → EMR
Not this service
- Run SQL over S3 without moving or reshaping anything → Athena
- Persistent Hadoop/Spark cluster you tune, patch, and keep warm → EMR
Use cases
- 01A partner drops CSVs into S3 every night — a scheduled crawler picks up the new partitions and the data is queryable in Athena by morning, with nobody hand-writing a schema
- 02Converting sprawling raw CSV/JSON into partitioned Parquet, so every downstream Athena query scans a fraction of the bytes and costs a fraction as much
- 03Nightly job joining a database export with clickstream files into one flat reporting table that QuickSight points at
- 04One-off backfill over years of archived logs — the Spark capacity spins up, chews through it, and disappears, leaving no cluster to remember to shut down
- 05An analyst dedupes rows, fixes date formats, and standardizes country codes in DataBrew instead of filing a ticket with the data team
How it's charged
- 01Two separate meters: DPU-hours for compute (jobs, crawlers) and a monthly metadata fee for the Data Catalog — a catalog nobody queries still shows up on the bill
- 02Jobs bill per second on allocated DPUs (1 DPU = 4 vCPU / 16 GB) with a 1-minute minimum; Spark jobs need at least 2 DPUs and default to 10, which is the usual source of bill shock
- 03Crawlers bill at the same DPU rate but with a 10-minute minimum per run — an hourly crawler on a schema that changes weekly is almost entirely waste
- 04Data Catalog free tier covers the first million stored objects and first million requests per month; past that you pay per 100,000 objects and per million requests, and every partition counts as an object
- 05Glue never charges for the data itself — S3 storage, CloudWatch logs, and the target warehouse are billed by those services
