DoRA
https://arxiv.org/abs/2402.09353DoRA: Weight-Decomposed Low-Rank Adaptation
Sebastian Raschka(@rasbt)
While everyone is talking about Sora, there's a potential successor to LoRA (low-rank adaptation) called DoRA. Here's a closer look at the "DoRA: Weight-Decomposed Low-Rank Adaptation" paper: https://arxiv.org/abs/2402.09353
LoRA is probably the most widely used parameter-efficient…
https://pbs.twimg.com/media/GGd015TW0AEnOlI.jpg