Spaces:

nanotron
/

ultrascale-playbook

Running

App Files Files Community

Resources

View closed (96)

Questions?

#84 opened about 1 year ago by

More ressources

#73 opened about 1 year ago by

Error in section A3

#126 opened 3 months ago by

Little error in A0 section

#125 opened 3 months ago by

Update ultra_blog.md

#124 opened 3 months ago by

Is there any error in the illustrations of the "Interleaving Stages" ？

#123 opened 4 months ago by

Clarification Needed: Description of Gradient Accumulation's Peak Memory Impact Seems Incorrect

#122 opened 4 months ago by

fix typo & a QUESTION about 16-bit training

#121 opened 5 months ago by

TP self attention figure

#120 opened 5 months ago by

typos

#119 opened 7 months ago by

Fix: Link Error on Page 17

#117 opened 7 months ago by

Potential Link Error on Page 17 of the Ultra-Scale Playbook

#116 opened 7 months ago by

Fix typo in memory footprint variable name

#115 opened 8 months ago by

sharing results on trained networks

#114 opened 9 months ago by

TP Question

#113 opened 9 months ago by

Question on the "Summarizing it all" figure

#111 opened 10 months ago by

How to understand the graph "Tensor parallelism with column linear + row Linear"

#109 opened 11 months ago by

Incorrect link in Data Parallelism?

#108 opened 11 months ago by

Thoughts on adding Hybrid Sharded Data Parallel to the guide

#107 opened 11 months ago by

Typo in Sequence Parallelism TO -> TP

#106 opened 11 months ago by

Wrong section title for FSDP?

#105 opened 12 months ago by

A mistake ? Weights/grads/optimizer stats memory for mixed precision

#104 opened 12 months ago by

Questions about pipeline parallelism

#103 opened about 1 year ago by

Widget does not take TP into account for Parameter / Gradient / Optimizer State Sharding

#98 opened about 1 year ago by

Am I misunderstanding Zero-1 and Zero-2?

#94 opened about 1 year ago by

Fix description of Zero-1

#93 opened about 1 year ago by

Few Errors

#86 opened about 1 year ago by

How can the following figure be obtained, and is there a way to tag the name of each tensor during profiling?

#83 opened about 1 year ago by

Thanks for sharing. Was looking for similar research to get to know about compute(AI+GPU)

#79 opened about 1 year ago by

Make it easier to import into reader applications

#77 opened about 1 year ago by