Home / Spark / Shuffle and joins
Spark: MetadataFetchFailedException: Missing an output location for shuffle
org.apache.spark.shuffle.MetadataFetchFailedExceptionMissing an output location for shuffle 3 partition 47A reducer asked for map output that no longer has a known location, usually because the executor that wrote it was lost. Spark retries the stage, but repeated losses fail the job. Look for why executors died.
Common causes
- Executors were killed for memory (YARN or Kubernetes limits) or by preemption, taking their shuffle files with them.How to confirm: Look for "Container killed by YARN" or lost executor messages in the driver log just before this error.
- Dynamic allocation removed idle executors that still held shuffle data, without an external shuffle service.How to confirm: spark.dynamicAllocation.enabled is true while spark.shuffle.service.enabled is false and shuffle tracking is off.
How to fix it
Stop executors from dying: give them more memory or overhead, or reduce per-task memory with more partitions.
With dynamic allocation, enable the external shuffle service or shuffle tracking so shuffle data outlives executors.
| Setting | Default | Note |
|---|---|---|
spark.shuffle.service.enabled | false | |
spark.dynamicAllocation.shuffleTracking.enabled | true |
Where the message comes from
MapOutputTracker.scalainapache/sparkatv4.1.3- Message checked in Spark 4.1.3
Message template in the source: Missing an output location for shuffle %d partition %d