Distributed Tensor (DTensor) in PyTorch: Overview
A brief note on DTensor's workings.
Distributed Tensor (DTensor) in PyTorch: Overview
Note: Will not find anything new if you have already read DTensor RFC and DTensor Status.
What is DTensor?
DTensor is a distributed tensor representation designed to efficiently train large deep learning models by reducing memory redundancy across multiple devices.
This post is licensed under CC BY 4.0 by the author.