2025

• 15 Jun Parameter Efficient Fine Tuning Notes
• 13 Jun Basic_principles_distributed_training
• 13 May Notes on PyTorch's Distributed Data Parallel (DDP)
• 22 Jan Different Transformers Attention Variants

2024

• 28 Sep Context Parallelism in Transformers: A Brief Overview
• 28 Sep Distributed Tensor (DTensor) in PyTorch: Overview
• 28 Sep Zero Redunduncy Optimizer (ZeRO): Paper Summary
• 30 Aug Distributed training technologies for Transformers: Overview

2021

• 20 Aug Named Entity Recognition (NER) as Machine Reading Comprehension (MRC)
• 14 Jan Train BERT for Question Answering Task

2020

• 15 Aug Abstractive Text Summarization with GPT2
• 20 May How to Improve YOLOv3