fsdp
Fully Sharded Data Parallel
FYI
https://huggingface.co/docs/accelerate/usage_guides/fsdpFully Sharded Data Parallel
https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/Introducing PyTorch Fully Sharded Data Parallel (FSDP) API
https://huggingface.co/docs/accelerate/main/en/concept_guides/fsdp1_vs_fsdp2fsdp1-vs-fsdp2
https://huggingface.co/docs/accelerate/main/en/concept_guides/fsdp_and_deepspeed#fsdp-vs-deepspeedfsdp-vs-deepspeed