Explain transformer architecture and variants

Explain the Transformer architecture in detail. Include: the encoder/decoder stack structure; self-attention, cross-attention, and position-wise feed-forward networks; the scaled dot-product attention equation (key/query/value shapes) and multi-head attention. Describe positional encodings (sinusoidal vs learned, relative positions) and their impact on order. Contrast encoder-only, decoder-only, and encoder–decoder models and discuss masking for autoregressive decoding. Analyze computational/memory complexity O(n²) and methods to scale to long sequences (sparse/linear attention variants and their trade-offs). Discuss LayerNorm placement (pre-LN vs post-LN), residual connections, stability considerations, and initialization. Finally, outline how you would adapt Transformers to molecular data such as SMILES strings or molecular graphs, including tokenization, stereochemistry handling, data augmentation, and suitable training objectives (masked LM, autoregressive LM, contrastive pretraining).