Hugging Face released transformers v5.21.0 on October 26, 2026 (tag v5.21). The release adds four new model architectures. Read the release notes.
New model support
- MiniCPM-OCR (OpenBMB): a document-understanding model from the MiniCPM-V family keeping the 4.6 architecture — a SigLIP vision encoder and window-attention merger paired with a Qwen3.5 language model — specialized for text-dense images such as scanned pages, screenshots, receipts and tables.
- Flint (Recursive): native support for the mixture-of-experts language model already distributed through Ollama. Compared with a standard SwiGLU MoE transformer, it uses hybrid attention — sliding-window attention with rotary embeddings over the current token and the previous 512 tokens in most layers, full attention without positional encoding in the rest — plus learned attention sinks with per-head RMS-normalized queries and keys. Each sparse layer routes a token to 6 of 256 experts by sigmoid score with an ungated shared expert; the first two layers are dense.
- UltraVBert: extends ModernVBert by pairing the UltraBERT text encoder with a SigLIP vision encoder for visual document understanding and retrieval. Pages are split into 512-pixel tiles plus a global view, each 4×4 block of vision patches merges into one token, and
max_image_tiles(27 by default) caps tiles per page. - Gradium Phonon TTS: a family of on-device text-to-speech models trained with the Pocket TTS recipe. A causal transformer reads a voice prompt and text, emits one continuous 32-dimensional audio latent per step at 12.5 Hz through a one-step flow head, and an end-of-speech head decides when the utterance ends; latents decode to 24 kHz audio. Voices ship as precomputed 512-dimensional latents from a separate speaker codec and cannot be cloned from a recording.
This report covers the October 26 release notes. Recent releases are covered in our reports on v5.20.1 and v5.20.0; Ollama's Flint rollout is in our v0.40.8 report.