atdot.fyi / alcanzai
Academic paper knowledge organizer
Star
Fork 0
Watch
alcanzai sample_note.md
2.4 kB
Markdown
at claude

title: "Attention Is All You Need" authors: ["Vaswani, Ashish", "Shazeer, Noam", "Parmar, Niki", "Uszkoreit, Jakob"] year: 2017 venue: "Advances in Neural Information Processing Systems" volume: "30" pages: "5998-6008" arxiv: "1706.03762" type: "paper" status: "unread" added: "2026-02-26" tags:

  • transformers
  • attention-mechanism
  • self-attention
  • neural-machine-translation
  • encoder-decoder

Attention Is All You Need #

Vaswani, Ashish et al. • 2017

[!quote] Memorable Quote "The Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output."

Quick Refresh #

This paper introduces the Transformer architecture, which relies entirely on attention mechanisms without recurrence or convolution. The model achieves state-of-the-art results on machine translation tasks while being more parallelizable and requiring less training time.

Why You Cared #

This is the foundational paper for modern language models like GPT and BERT. Understanding the Transformer architecture is essential for working with contemporary NLP systems.

Key Concepts #

#transformers #attention-mechanism #self-attention #neural-machine-translation #encoder-decoder

Cites (Key Papers) #

  • [[Bahdanau, D., Cho, K. & Bengio, Y. (2014) - Neural machine translation by jointly learning to align and ...]]
  • [[Bahdanau, D., Cho, K. & Bengio, Y. (2014) - Neural machine translation by jointly learning to align and ...]]
  • [[Bahdanau, D., Cho, K. & Bengio, Y. (2014) - Neural machine translation by jointly learning to align and ...]]

Cited By #

This section will be populated as you process papers that cite this one.

Details #

Published: Advances in Neural Information Processing Systems, Vol. 30, pp. 5998-6008 arXiv: 1706.03762 PDF: [[arxiv_1706.03762.pdf]]

Abstract #

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks...

Full Citation List #

  1. Bahdanau, D., Cho, K. & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate.
  2. Bahdanau, D., Cho, K. & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate.
  3. Bahdanau, D., Cho, K. & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate.