This video talks about RTDETR v2 – a short list of updates to the original RTDETR model that was first of its kind to offer high-requests-per-second, performant object detection based on transformers.
The video dives into Deformable Attention, 2 small additions to its algorithm made in RTDETR v2 – variable number of sampling points + discrete sampling.
Second part of the video shows the actual source code of the model, goes over the same concepts in code (deformable attention in the decoder), and then explains other 2 additions in RTDETR v2 – Dynamic Data Augmentation and Scale-Adaptive Hyperparameters Customization
Important links:
– Jupyter Notebook shown in the video: https://github.com/adensur/blog/blob/main/computer_vision_zero_to_hero/32_rtdetr_v2/sandbox.ipynb
– Installation instructions: https://github.com/adensur/blog/blob/main/computer_vision_zero_to_hero/32_rtdetr_v2/Install.md
– Paper: https://arxiv.org/pdf/2407.17140
– Source code: https://github.com/lyuwenyu/RT-DETR/tree/main/rtdetrv2_pytorch
– My previous video about RTDETR v1: https://youtu.be/nfd7pJsG1rk

00:00 – Intro
02:10 – Deformable Attention in Decoder
07:02 – Bilinear and Discrete Sampling
09:33 – RTDETR v2 Deformable Attention Modifications
10:18 – Source Code Intro and Setup
10:44 – Code Read: Dataloaders, model inputs and outputs
16:25 – Code Read: RTDETRv2 Model Code
20:45 – Code Read: Decoder & Deformable Attention
41:59 – Dynamic Data Augmentation
45:54 – Results

source