Shuffle: why it dominates your job runtime
When data crosses the network between executors, you are paying for it.
This concept is being written.
The detailed explanation, examples, and diagrams for this topic are not published yet. In the meantime, here is what you can do:
- Browse the rest of the System Design Concepts library.
- Follow the System Design Roadmap for a structured path through these topics.
- Try a practice problem that uses this concept in context.