A step-by-step guide to self-attention with illustrations and code — The illustrations are best viewed on the Desktop. A Colab version can be found here (thanks to Manuel Romero!). Changelog:
12 Jan 2022 — Improve clarity
5 Jan 2022 — Fix typos and improve clarity What do BERT, RoBERTa, ALBERT, SpanBERT, DistilBERT, SesameBERT, SemBERT, SciBERT, BioBERT, MobileBERT, TinyBERT and CamemBERT…