Add Checkpoints to a Long-Running Job
Added 2026-11-06
A three-hour job that dies should not restart from zero.
I have a long-running batch job: [task, batch size, how it fails]. Design resume-from-checkpoint: 1) choose the checkpoint granularity — per batch, per file, per interval — and the cost trade-off; 2) define the state record: done, in progress, failed items, idempotency key; 3) say how "in progress" items are judged and redone after a crash; 4) prevent double processing under concurrency and out-of-order completion; 5) handle checkpoints incompatible with a code change — versioning or invalidation; 6) give the acceptance test: kill it midway and confirm the result matches a clean run.
Site original
Chinese original
Replace these placeholders 1
Swap the bracketed parts for your own details before sending.
[task, batch size, how it fails]
How to use
Checkpoints must carry an idempotency key, or resuming becomes reprocessing.
Try tweaking it
- Have it restate requirements and constraints before writing code.
- Ask for the smallest change plus a rollback path, not a full-file rewrite.
What you should get
- A minimal, runnable change with copy-ready code
- Impact list: which files and call sites must follow
Common mistakes
- Asking it to touch ten files at once, then losing track of what broke what.
- Pasting only the error, with no code or version, and getting an unfixable answer.
How to check the output
- The change runs, can be rolled back, and touches nothing unrelated.
- Edge cases (empty, oversized, concurrent) are covered or explained.
Where it works
- Good for: having it restate requirements and constraints, then propose the smallest runnable change.
- Watch out: include versions and a minimal reproduction with the error, or the fix may look plausible but not fit.