Lecture about Swin-Unet

Summary

1. Attention-based networks
   1. Background
      1. Structured output problems
      2. Encoder-decoder framework
   2. Attention Mechanisms
   3. Image Captioning with Attention Mechanisms
2. Transformers
   1. Background: Machine Translation
   2. Self-attention
   3. Scaled dot-product attention
   4. Multi-head Self Attention module
3. Vision domain - DETR
   1. Background: Object detection & set prediction problem
   2. Detection Transformer (DETR)
4. Vision Transformer - ViT
   1. Background: Transformers + CNN or RNN
   2. Vision Transformers - ViT
5. Optimizing ViT
   1. Background: Data-hungry transformers
   2. Distillation
6. Swin Transformer
   1. Background: Problems using Transformers with images
   2. Swin Transformer architecture
   3. Swin Transformer block
   4. Shifted Window based Self-attention
7. Swin-Unet
   1. Background: How to improve Unets segmentation
   2. Swin-Unet architecture
   3. Patch merging and Patch expanding layers