0.8.1

This release contains important bug fixes for v0.8.0. We highly recommend upgrading!

• V1 Fixes

  • Ensure using int64 for sampled token ids (#15065)
  • Fix long dtype in topk sampling (#15049)
  • Refactor Structured Output for multiple backends (#14694)
  • Fix size calculation of processing cache (#15114)
  • Optimize Rejection Sampler with Triton Kernels (#14930)
  • Fix oracle for device checking (#15104)

• TPU

  • Fix chunked prefill with padding (#15037)
  • Enhanced CI/CD (#15054, 14974)

• Model

  • Re-enable Gemma3 for V1 (#14980)
  • Embedding model support LoRA (#14935)
  • Pixtral: Remove layer instantiation duplication (#15053)

What's Changed

• [Bugfix] Fix interface for Olmo2 on V1 by @ywang96 in https://github.com/vllm-project/vllm/pull/14976
• [CI/Build] Use AutoModelForImageTextToText to load image models in tests by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/14945
• [V1] Guard Against Main Thread Usage by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/14972
• [V1] TPU - Fix CI/CD runner for V1 and remove V0 tests by @alexm-redhat in https://github.com/vllm-project/vllm/pull/14974
• [Bugfix] Fix bnb quantization for models with both HF-format and Mistral-format weights by @tristanleclercq in https://github.com/vllm-project/vllm/pull/14950
• [Neuron] trim attention kernel tests to fit trn1.2x instance by @liangfu in https://github.com/vllm-project/vllm/pull/14988
• [Doc][V1] Fix V1 APC doc by @shen-shanshan in https://github.com/vllm-project/vllm/pull/14920
• [Kernels] LoRA - Retire SGMV and BGMV Kernels by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/14685
• [Mistral-Small 3.1] Update docs and tests by @patrickvonplaten in https://github.com/vllm-project/vllm/pull/14977
• [Misc] Embedding model support LoRA by @jeejeelee in https://github.com/vllm-project/vllm/pull/14935
• [Bugfix] torchrun compatibility by @hiyouga in https://github.com/vllm-project/vllm/pull/14899
• [Bugfix][Frontend] Fix validation of logprobs in ChatCompletionRequest by @schoennenbeck in https://github.com/vllm-project/vllm/pull/14352
• [Misc][Docs] fix the comments of KV_T and CACHE_T in CALL_RESHAPE_AND_CACHE_XX macros by @yangsijia-serena in https://github.com/vllm-project/vllm/pull/14347
• [Bugfix] Loosen type check to avoid errors in V1 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15021
• [Bugfix] Register serializers for V0 MQ Engine by @simon-mo in https://github.com/vllm-project/vllm/pull/15009
• [TPU][V1][Bugfix] Fix chunked prefill with padding by @NickLucche in https://github.com/vllm-project/vllm/pull/15037
• MI325 configs, fused_moe_kernel bugfix by @ekuznetsov139 in https://github.com/vllm-project/vllm/pull/14987
• [MODEL] Add support for Zamba2 models by @yury-tokpanov in https://github.com/vllm-project/vllm/pull/13185
• [Bugfix] Fix broken CPU quantization due to triton import by @Isotr0py in https://github.com/vllm-project/vllm/pull/15038
• [Bugfix] Fix LoRA extra vocab size by @jeejeelee in https://github.com/vllm-project/vllm/pull/15047
• [V1] Refactor Structured Output for multiple backends by @russellb in https://github.com/vllm-project/vllm/pull/14694
• [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels by @WoosukKwon in https://github.com/vllm-project/vllm/pull/14930
• [V1] TPU - CI/CD use smaller model by @alexm-redhat in https://github.com/vllm-project/vllm/pull/15054
• fix long dtype in topk sampling by @chujiezheng in https://github.com/vllm-project/vllm/pull/15049
• [Doc] Minor v1_user_guide update by @JenZhao in https://github.com/vllm-project/vllm/pull/15064
• [Misc][V1] Skip device checking if not available by @comaniac in https://github.com/vllm-project/vllm/pull/15061
• [Model] Pixtral: Remove layer instantiation duplication by @juliendenize in https://github.com/vllm-project/vllm/pull/15053
• [Model] Remove duplicated message check in Mistral chat completion request by @b8zhong in https://github.com/vllm-project/vllm/pull/15069
• [Core] Update dtype detection and defaults by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/14858
• [V1] Ensure using int64 for sampled token ids by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15065
• [Bugfix] Re-enable Gemma3 for V1 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/14980
• [CI][Intel GPU] update XPU dockerfile and CI script by @jikunshang in https://github.com/vllm-project/vllm/pull/15109
• [V1][Bugfix] Fix oracle for device checking by @ywang96 in https://github.com/vllm-project/vllm/pull/15104
• [Misc] Avoid unnecessary HF do_rescale warning when passing dummy data by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15107
• [Bugfix] Fix size calculation of processing cache by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15114
• [Doc] Update tip info on using latest transformers when creating a custom Dockerfile by @MarcCote in https://github.com/vllm-project/vllm/pull/15070
• [Misc][Benchmark] Add support for different tokenizer_mode by @aarnphm in https://github.com/vllm-project/vllm/pull/15040
• [Bugfix] Adjust mllama to regional compilation by @jkaniecki in https://github.com/vllm-project/vllm/pull/15112
• [Doc] Update the "the first vLLM China Meetup" slides link to point to the first page by @imkero in https://github.com/vllm-project/vllm/pull/15134
• [Frontend] Remove custom_cache_manager by @fulvius31 in https://github.com/vllm-project/vllm/pull/13791
• [V1] Minor V1 async engine test refactor by @andoorve in https://github.com/vllm-project/vllm/pull/15075

Full Changelog: https://github.com/vllm-project/vllm/compare/v0.8.0...v0.8.1