•
Accelerate v1.15.0 (9 Sep 2026): FSDP2 activation checkpointing wraps the matched transformer layer (not each child); new fsdp_activation_checkpointing_offload to pinned CPU; tied-embedding fix on torch>=2.13; DTensor grad clip / prepare_model. Hook is Accelerate. Not investment advice.
Hugging Face Accelerate v1.15.0 (PyPI accelerate 1.15.0, 9 September 2026) is the after-work hook if you fine-tune with FSDP2 and feel activation memory pressure—not a full PyTorch rewrite. The product hook is Accelerate; light mention of PyTorch FSDP2 / Transformers ecosystem is OK. Focus: activation checkpointing wraps the matched transformer layer itself (not each child attn / mlp / norm), plus FSDP2-only fsdp_activation_checkpointing_offload (checkpointed layer inputs → pinned CPU). Also: tied-embedding fix on torch ≥ 2.13, DTensor grad-clip / prepare_model improvements. Not investment advice.
The only course CTA is Introduction to Python Programming.
# fsdp2.yaml (illustrative)
fsdp_config:
fsdp_version: 2
fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP
fsdp_activation_checkpointing: true
fsdp_activation_checkpointing_offload: true
accelerate launch --config_file fsdp2.yaml train.py
Why it matters: wrapping children of the matched layer kept inter-child activations alive for backward; wrapping the layer drops that waste. Offload then moves remaining checkpoint inputs to pinned CPU—grads match plain activation checkpointing per the release notes. Concept guide: FSDP1 vs FSDP2.
pip install -U "accelerate==1.15.0"
python -c "import accelerate; print(accelerate.__version__)"
1. Set fsdp_version: 2 + fsdp_activation_checkpointing: true on a small job you already run.
2. Flip fsdp_activation_checkpointing_offload: true only if GPU RAM is still tight after (1).
3. If you train tied-embedding models, note the torch ≥ 2.13 fix (same fully_shard group for output embedding).
Stop after one of three.
Upgrade Accelerate, enable layer-level FSDP2 checkpointing on one config, measure peak activation memory. Stuck on env basics? Introduction to Python Programming.