vLLM 0.8.0 release notes

v0.8.0 featured 523 commits from 166 total contributors (68 new contributors)!

Highlights

V1

We have now enabled V1 engine by default (#13726) for supported use cases. Please refer to V1 user guide for more detail. We expect better performance for supported scenarios. If you'd like to disable V1 mode, please specify the environment variable VLLM_USE_V1=0, and send us a GitHub issue sharing the reason!

Support variety of sampling parameters (#13376, #10980, #13210, #13774)
Compatability prompt logprobs + prefix caching (#13949), sliding window + prefix caching (#13069)
Stability fixes (#14380, #14379, #13298)
Pluggable scheduler (#14466)
SupportsV0Only protocol for model definitions (#13959)
Metrics enhancements (#13299, #13504, #14695, #14082)
V1 user guide (#13991) and design doc (#12745)
Support for Structured Outputs (#12388, #14590, #14625, #14630, #14851)
Support for LoRA (#13705, #13096, #14626)
Enhance Pipeline Parallelism (#14585, #14643)
Ngram speculative decoding (#13729, #13933)

DeepSeek Improvements

We observe state of the art performance with vLLM running DeepSeek model on latest version of vLLM:

MLA Enhancements:
- FlashMLA integration (#13747, #13867, #14451)
- MLA support for V1 (#13789, #14253, #14384, #14540, #14921)
- MLA with chunked prefill (#12639)
- Holistic memory and performance optimization (#14769, #14770,#14842)
- Support MLA for CompressedTensorsWNA16 (#13725)
Distributed Expert Parallelism (EP) and Data Parallelism (DP)
- EP Support for DeepSeek Models (#12583)
- Add enable_expert_parallel arg (#14305)
- EP/TP MoE + DP Attention (#13931)
- Set up data parallel communication (#13591)
MTP: Expand DeepSeek MTP code to support k > n_predict (#13626)
Pipeline Parallelism:
- DeepSeek V2/V3/R1 only place lm_head on last pp rank (#13833)
- Improve pipeline partitioning (#13839)
GEMM
- Add streamK for block-quantized CUTLASS kernels (#12978)
- Add benchmark for DeepGEMM and vLLM Block FP8 Dense GEMM (#13917)
- Add more tuned configs for H20 and others (#14877)

New Models

Gemma 3 (#14660)
- Note: You have to install transformers from main branch (pip install git+https://github.com/huggingface/transformers.git) to use this model. Also, there may be numerical instabilities for float16/half dtype. Please use bfloat16 (preferred by HF) or float32 dtype.
Mistral Small 3.1 (#14957)
Phi-4-multimodal-instruct (#14119)
Grok1 (#13795)
QwQ-32B and toll calling (#14479, #14478)
Zamba2 (#13185)

NVIDIA Blackwell

Support nvfp4 cutlass gemm (#13571)
Add cutlass support for blackwell fp8 gemm (#13798)
Update the flash attn tag to support Blackwell (#14244)
Add ModelOpt FP4 Checkpoint Support (#12520)

Breaking Changes

The default value of seed is now None to align with PyTorch and Hugging Face. Please explicitly set seed for reproduciblity. (#14274)
The kv_cache and attn_metadata arguments for model's forward method has been removed; as the attention backend has access to these value via forward_context. (#13887)
vLLM will now default generation_config from model for chat template, sampling parameters such as temperature, etc. (#12622)
Several request time metrics (vllm:time_in_queue_requests, vllm:model_forward_time_milliseconds, vllm:model_execute_time_milliseconds) has been deprecated and subject to removal (#14135)

Updates

Update to PyTorch 2.6.0 (#12721, #13860)
Update to Python 3.9 typing (#14492, #13971)
Update to CUDA 12.4 as default for release and nightly wheels (#12098)
Update to Ray 2.43 (#13994)
Upgrade aiohttp to incldue CVE fix (#14840)
Upgrade jinja2 to get 3 moderate CVE fixes (#14839)

Features

Frontend API

API Server
- Support return_tokens_as_token_id as a request param (#14066)
- Support Image Emedding as input (#13955)
- New /load endpoint for load statistics (#13950)
- New API endpoint /is_sleeping (#14312)
- Enables /score endpoint for embedding models (#12846)
- Enable streaming for Transcription API (#13301)
- Make model param optional in request (#13568)
- Support SSL Key Rotation in HTTP Server (#13495)
Reasoning
- Support reasoning output (#12955)
- Support outlines engine with reasoning outputs (#14114)
- Update reasoning with stream example to use OpenAI library (#14077)
CLI
- Ensure out-of-tree quantization method recognize by cli args (#14328)
- Add vllm bench CLI (#13993)
Make LLM API compatible for torchrun launcher (#13642)

Disaggregated Serving

Support KV cache offloading and disagg prefill with LMCache connector (#12953)
Support chunked prefill for LMCache connector (#14505)

LoRA

Add LoRA support for TransformersModel (#13770)
Make the deviceprofilerinclude LoRA memory. (#14469)
Gemma3ForConditionalGeneration supports LoRA (#14797)
Retire SGMV and BGMV Kernels (#14685) (#14685)

VLM

Generalized prompt updates for multi-modal processor (#13964)
Deprecate legacy input mapper for OOT multimodal models (#13979)
Refer code examples for common cases in dev multimodal processor (#14278)

Quantization

BaiChuan SupportsQuant (#13710)
BartModel SupportsQuant (#14699)
Bamba SupportsQuant (#14698)
Deepseek GGUF support (#13167)
GGUF MoE kernel (#14613)
Add GPTQAllSpark Quantization (#12931)
Better performance of gptq marlin kernel when n is small (#14138)

Structured Output

xgrammar: Expand list of unsupported jsonschema keywords (#13783)

Hardware Support

AMD

Faster Custom Paged Attention kernels (#12348)
Improved performance for V1 Triton (ROCm) backend (#14152)
Chunked prefill/paged attention in MLA on ROCm (#14316)
Perf improvement for DSv3 on AMD GPUs (#13718)
MoE fp8 block quant tuning support (#14068)

TPU

Integrate the new ragged paged attention kernel with vLLM v1 on TPU (#13379)
Support start_profile/stop_profile in TPU worker (#13988)
Add TPU v1 test (#14834)
TPU multimodal model support for ragged attention (#14158)
Add tensor parallel support via Ray (#13618)
Enable prefix caching by default (#14773)

Neuron

Add Neuron device communicator for vLLM v1 (#14085)
Add custom_ops for neuron backend (#13246)
Add reshape_and_cache (#14391)
Vectorize KV cache load in FlashPagedAttention to maximize DMA bandwidth (#13245)

CPU

Upgrade CPU backend to torch-2.6 (#13381)
Support FP8 KV cache in CPU Backend(#14741)

s390x

Adding cpu inference with VXE ISA for s390x architecture (#12613)
Add documentation for s390x cpu implementation (#14198)

Plugins

Remove cuda hard code in models and layers (#13658)
Move use allgather to platform (#14010)

Bugfix and Enhancements

Illegal memory access for MoE On H20 (#13693)
Fix FP16 overflow for DeepSeek V2 (#13232)
Illegal Memory Access in the blockwise cutlass fp8 GEMMs (#14396)
Pass all driver env vars to ray workers unless excluded (#14099)
Use xgrammar shared context to avoid copy overhead for offline engine (#13837)
Capture and log the time of loading weights (#13666)

Developer Tooling

Benchmarks

Consolidate performance benchmark datasets (#14036)
Update benchmarks README (#14646)

CI and Build

Add RELEASE.md (#13926)
Use env var to control whether to use S3 bucket in CI (#13634)

Documentation

Add RLHF document (#14482)
Add nsight guide to profiling docs (#14298)
Add K8s deployment guide (#14084)
Add developer documentation for torch.compile integration (#14437)

What's Changed

Update pre-commit's isort version to remove warnings by @hmellor in https://github.com/vllm-project/vllm/pull/13614
[V1][Minor] Print KV cache size in token counts by @WoosukKwon in https://github.com/vllm-project/vllm/pull/13596
fix neuron performance issue by @ajayvohra2005 in https://github.com/vllm-project/vllm/pull/13589
[Frontend] Add backend-specific options for guided decoding by @joerunde in https://github.com/vllm-project/vllm/pull/13505
[Bugfix] Fix max_num_batched_tokens for MLA by @mgoin in https://github.com/vllm-project/vllm/pull/13620
[Neuron][Kernel] Vectorize KV cache load in FlashPagedAttention to maximize DMA bandwidth by @lingfanyu in https://github.com/vllm-project/vllm/pull/13245
Add llmaz as another integration by @kerthcet in https://github.com/vllm-project/vllm/pull/13643
[Misc] Adding script to setup ray for multi-node vllm deployments by @Edwinhr716 in https://github.com/vllm-project/vllm/pull/12913
[NVIDIA] Fix an issue to use current stream for the nvfp4 quant by @kaixih in https://github.com/vllm-project/vllm/pull/13632
Use pre-commit to update requirements-test.txt by @hmellor in https://github.com/vllm-project/vllm/pull/13617
[Bugfix] Add mm_processor_kwargs to chat-related protocols by @ywang96 in https://github.com/vllm-project/vllm/pull/13644
[V1][Sampler] Avoid an operation during temperature application by @njhill in https://github.com/vllm-project/vllm/pull/13587
Missing comment explaining VDR variable in GGUF kernels by @SzymonOzog in https://github.com/vllm-project/vllm/pull/13290
[FEATURE] Enables /score endpoint for embedding models by @gmarinho2 in https://github.com/vllm-project/vllm/pull/12846
[ci] Fix metrics test model path by @khluu in https://github.com/vllm-project/vllm/pull/13635
[Kernel]Add streamK for block-quantized CUTLASS kernels by @Hongbosherlock in https://github.com/vllm-project/vllm/pull/12978
[Bugfix][CPU] Fix cpu all-reduce using native pytorch implementation by @Isotr0py in https://github.com/vllm-project/vllm/pull/13586
fix typo of grafana dashboard, with correct datasource by @johnzheng1975 in https://github.com/vllm-project/vllm/pull/13668
[Attention] MLA with chunked prefill by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/12639
[Misc] Fix yapf linting tools etc not running on pre-commit by @Isotr0py in https://github.com/vllm-project/vllm/pull/13695
docs: Add a note on full CI run in contributing guide by @terrytangyuan in https://github.com/vllm-project/vllm/pull/13646
[HTTP Server] Make model param optional in request by @youngkent in https://github.com/vllm-project/vllm/pull/13568
[Bugfix][API Server] Fix invalid usage of 'ge' and 'le' in port valid… by @WangErXiao in https://github.com/vllm-project/vllm/pull/13672
[Misc] Capture and log the time of loading weights by @waltforme in https://github.com/vllm-project/vllm/pull/13666
[ROCM] fix native attention function call by @gongdao123 in https://github.com/vllm-project/vllm/pull/13650
[Bugfix][Model] OLMo 2: split qkv correctly for GQA and MQA by @2015aroras in https://github.com/vllm-project/vllm/pull/13687
[Misc] Bump compressed-tensors by @dsikka in https://github.com/vllm-project/vllm/pull/13619
[Bugfix] Fix benchmark script bug: inaccurate stats for vllm backend when max_model_len < input_len + output_len by @WangErXiao in https://github.com/vllm-project/vllm/pull/13691
[v1] Support allowed_token_ids in v1 Sampler by @houseroad in https://github.com/vllm-project/vllm/pull/13210
[Bugfix] V1 Memory Profiling: V0 Sampler Integration without Rejection Sampler by @JenZhao in https://github.com/vllm-project/vllm/pull/13594
Correction to TP logic for Mamba Mixer 2 when Num Groups not divisible by TP Size by @fabianlim in https://github.com/vllm-project/vllm/pull/13660
[V1][Metrics] Support vllm:cache_config_info by @markmc in https://github.com/vllm-project/vllm/pull/13299
[Metrics] Add --show-hidden-metrics-for-version CLI arg by @markmc in https://github.com/vllm-project/vllm/pull/13295
[Misc] Reduce LoRA-related static variable by @jeejeelee in https://github.com/vllm-project/vllm/pull/13166
[CI/Build] Fix pre-commit errors by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/13696
[core] set up data parallel communication by @youkaichao in https://github.com/vllm-project/vllm/pull/13591
[ci] fix linter by @youkaichao in https://github.com/vllm-project/vllm/pull/13701
Support SSL Key Rotation in HTTP Server by @youngkent in https://github.com/vllm-project/vllm/pull/13495
[NVIDIA] Support nvfp4 cutlass gemm by @kaixih in https://github.com/vllm-project/vllm/pull/13571
[V1][Kernel] Refactor the prefix_prefill kernel so that the caller no longer has to pass in the context lengths by @SageMoore in https://github.com/vllm-project/vllm/pull/13095
[ROCm] Apply FP8 weights padding to values not divisible by 512 bytes on ROCm by @gshtras in https://github.com/vllm-project/vllm/pull/13231
[Doc] Dockerfile instructions for optional dependencies and dev transformers by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/13699
[Bugfix] Fix boolean conversion for OpenVINO env variable by @helena-intel in https://github.com/vllm-project/vllm/pull/13615
[XPU]fix setuptools version for xpu by @yma11 in https://github.com/vllm-project/vllm/pull/13548
[CI/Build] fix uv caching in Dockerfile by @dtrifiro in https://github.com/vllm-project/vllm/pull/13611
[CI/Build] Fix pre-commit errors from #13571 by @ywang96 in https://github.com/vllm-project/vllm/pull/13709
[BugFix] Minor: logger import in attention backend by @andylolu2 in https://github.com/vllm-project/vllm/pull/13706
[ci] Use env var to control whether to use S3 bucket in CI by @khluu in https://github.com/vllm-project/vllm/pull/13634
[Quant] BaiChuan SupportsQuant by @kylesayrs in https://github.com/vllm-project/vllm/pull/13710
[LMM] Implement merged multimodal processor for whisper by @Isotr0py in https://github.com/vllm-project/vllm/pull/13278
[Core][Distributed] Use IPC (domain socket) ZMQ socket for local comms by @njhill in https://github.com/vllm-project/vllm/pull/13688
[Misc] Deprecate --dataset from benchmark_serving.py by @ywang96 in https://github.com/vllm-project/vllm/pull/13708
[v1] torchrun compatibility by @youkaichao in https://github.com/vllm-project/vllm/pull/13642
[V1][BugFix] Fix engine core client shutdown hangs by @njhill in https://github.com/vllm-project/vllm/pull/13298
Fix some issues with benchmark data output by @huydhn in https://github.com/vllm-project/vllm/pull/13641
[ci] Add logic to change model to S3 path only when S3 CI env var is on by @khluu in https://github.com/vllm-project/vllm/pull/13727
[V1][Core] Fix memory issue with logits & sampling by @ywang96 in https://github.com/vllm-project/vllm/pull/13721
[model][refactor] remove cuda hard code in models and layers by @MengqingCao in https://github.com/vllm-project/vllm/pull/13658
[Bugfix] fix(logging): add missing opening square bracket by @bufferoverflow in https://github.com/vllm-project/vllm/pull/13011
[CI/Build] add python-json-logger to requirements-common by @bufferoverflow in https://github.com/vllm-project/vllm/pull/12842
Expert Parallelism (EP) Support for DeepSeek Models by @cakeng in https://github.com/vllm-project/vllm/pull/12583
[BugFix] Illegal memory access for MoE On H20 by @Abatom in https://github.com/vllm-project/vllm/pull/13693
[Misc][Docs] Raise error when flashinfer is not installed and VLLM_ATTENTION_BACKEND is set by @NickLucche in https://github.com/vllm-project/vllm/pull/12513
[V1] V1 engine implements parallel sampling (AsyncLLM and LLMEngine) by @afeldman-nm in https://github.com/vllm-project/vllm/pull/10980
Revert "[V1][Core] Fix memory issue with logits & sampling" by @ywang96 in https://github.com/vllm-project/vllm/pull/13775
Fix precommit fail in fused_moe intermediate_cache2 chunking by @mgoin in https://github.com/vllm-project/vllm/pull/13772
[Misc] Clean Up EngineArgs.create_engine_config by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/13734
[Misc][Chore] Clean Up AsyncOutputProcessing Logs by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/13780
Remove unused kwargs from model definitions by @hmellor in https://github.com/vllm-project/vllm/pull/13555
[Doc] arg_utils.py: fixed a typo by @eli-b in https://github.com/vllm-project/vllm/pull/13785
[Misc] set single whitespace between log sentences by @cjackal in https://github.com/vllm-project/vllm/pull/13771
[Bugfix][Quantization] Fix FP8 + EP by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/13784
[Misc][Attention][Quantization] init property earlier by @wangxiyuan in https://github.com/vllm-project/vllm/pull/13733
[V1][Metrics] Implement vllm:lora_requests_info metric by @markmc in https://github.com/vllm-project/vllm/pull/13504
[Bugfix] Fix deepseek-v2 error: "missing 1 required positional argument: 'residual'" by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/13802
[Bugfix] Support MLA for CompressedTensorsWNA16 by @mgoin in https://github.com/vllm-project/vllm/pull/13725
Fix CompressedTensorsWNA16MoE with grouped scales by @mgoin in https://github.com/vllm-project/vllm/pull/13769
[Core] LoRA V1 - Add add/pin/list/remove_lora functions by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/13705
[Misc] Check that the model can be inspected upon registration by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/13743
[Core] xgrammar: Expand list of unsupported jsonschema keywords by @russellb in https://github.com/vllm-project/vllm/pull/13783
[Bugfix] Modify modelscope api usage in transformer_utils by @shen-shanshan in https://github.com/vllm-project/vllm/pull/13807
[misc] Clean up ray compiled graph type hints by @ruisearch42 in https://github.com/vllm-project/vllm/pull/13731
[Feature] Support KV cache offloading and disagg prefill with LMCache connector. by @YaoJiayi in https://github.com/vllm-project/vllm/pull/12953
[ROCm][Quantization][Kernel] Using HIP FP8 header by @gshtras in https://github.com/vllm-project/vllm/pull/12593
[CI/Build] Fix V1 LoRA failure by @jeejeelee in https://github.com/vllm-project/vllm/pull/13767
[Misc]Clarify Error Handling for Non-existent Model Paths and HF Repo IDs by @Chen-0210 in https://github.com/vllm-project/vllm/pull/13724
[Bugfix] Initialize attention bias on the same device as Query/Key/Value by @edwardzjl in https://github.com/vllm-project/vllm/pull/13468
[Bugfix] Flush TunableOp results before worker processes are destroyed. by @naromero77amd in https://github.com/vllm-project/vllm/pull/13623
[Bugfix] Fix deepseek-vl2 inference with more than 2 images by @Isotr0py in https://github.com/vllm-project/vllm/pull/13818
Fix /v1/audio/transcriptions Bad Request Error by @HermitSun in https://github.com/vllm-project/vllm/pull/13811
[Bugfix] Revert inspection code in #13743 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/13832
Fix string parsing error by @Chen-0210 in https://github.com/vllm-project/vllm/pull/13825
[Neuron] Add custom_ops for neuron backend by @liangfu in https://github.com/vllm-project/vllm/pull/13246
Fix failing MyGemma2Embedding test by @hmellor in https://github.com/vllm-project/vllm/pull/13820
[Model] Support Grok1 by @mgoin in https://github.com/vllm-project/vllm/pull/13795
DeepSeek V2/V3/R1 only place lm_head on last pp rank by @hmellor in https://github.com/vllm-project/vllm/pull/13833
[misc] Show driver IP info when Ray fails to allocate driver worker by @ruisearch42 in https://github.com/vllm-project/vllm/pull/13858
[V1][Spec Decode] Change Spec Decode Rejection Sampling API by @LiuXiaoxuanPKU in https://github.com/vllm-project/vllm/pull/13729
[Misc]Code Cleanup by @noemotiovon in https://github.com/vllm-project/vllm/pull/13859
[Kernel][Build/CI] Bump CUTLASS to 3.8 and add initializers for cutlass epilogues by @henrylhtsang in https://github.com/vllm-project/vllm/pull/13797
Improve pipeline partitioning by @hmellor in https://github.com/vllm-project/vllm/pull/13839
[Doc] fix the incorrect module path of tensorize_vllm_model by @tianyuzhou95 in https://github.com/vllm-project/vllm/pull/13863
[ROCm] Disable chunked prefill/prefix caching when running MLA on non-cuda platforms by @SageMoore in https://github.com/vllm-project/vllm/pull/13844
[v0][Core] Use xgrammar shared context to avoid copy overhead for offline engine by @sethkimmel3 in https://github.com/vllm-project/vllm/pull/13837
[Misc] Improve LoRA spelling by @jeejeelee in https://github.com/vllm-project/vllm/pull/13831
[Misc] Fix input processing for Ultravox by @ywang96 in https://github.com/vllm-project/vllm/pull/13871
[Bugfix] Add test example for Ultravox v0.5 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/13890
Add comments on accessing kv_cache and attn_metadata by @hmellor in https://github.com/vllm-project/vllm/pull/13887
[Bugfix] Handle None parameters in Mistral function calls. by @fgreinacher in https://github.com/vllm-project/vllm/pull/13786
[Misc]: Add support for goodput on guided benchmarking + TPOT calculation refactor by @b8zhong in https://github.com/vllm-project/vllm/pull/13736
[Bugfix] Do not crash V0 engine on input errors by @joerunde in https://github.com/vllm-project/vllm/pull/13101
[Bugfix] Update expected token counts for Ultravox tests by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/13895
[TPU] use torch2.6 with whl package by @Chenyaaang in https://github.com/vllm-project/vllm/pull/13860
[Misc] fixed qwen_vl_utils parameter error by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/13906
[Bugfix] Backend option to disable xgrammar any_whitespace by @wallashss in https://github.com/vllm-project/vllm/pull/12744
[BugFix] Make FP8 Linear compatible with torch.compile by @WoosukKwon in https://github.com/vllm-project/vllm/pull/13918
[Kernel] FlashMLA integration by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/13747
[ROCm][Quantization][Kernel] Use FP8 FNUZ when OCP flag is 0 or undefined by @HollowMan6 in https://github.com/vllm-project/vllm/pull/13851
Use CUDA 12.4 as default for release and nightly wheels by @mgoin in https://github.com/vllm-project/vllm/pull/12098
[misc] Rename Ray ADAG to Compiled Graph by @ruisearch42 in https://github.com/vllm-project/vllm/pull/13928
[ROCm][V1] Update reshape_and_cache to properly work with CUDA graph padding by @SageMoore in https://github.com/vllm-project/vllm/pull/13922
[V1][Metrics] Handle preemptions by @markmc in https://github.com/vllm-project/vllm/pull/13169
[CI/Build] Add examples/ directory to be labelled by mergify by @b8zhong in https://github.com/vllm-project/vllm/pull/13944
[Misc] fixed 'required' is an invalid argument for positionals by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/13948
[PP] Correct cache size check by @zhengy001 in https://github.com/vllm-project/vllm/pull/13873
Fix test_block_fp8.py test for MoE by @mgoin in https://github.com/vllm-project/vllm/pull/13915
[VLM] Support multimodal inputs for Florence-2 models by @Isotr0py in https://github.com/vllm-project/vllm/pull/13320
[Model] Deepseek GGUF support by @SzymonOzog in https://github.com/vllm-project/vllm/pull/13167
Update quickstart.md by @observerw in https://github.com/vllm-project/vllm/pull/13958
Deduplicate .pre-commit-config.yaml's exclude by @hmellor in https://github.com/vllm-project/vllm/pull/13967
[bugfix] Fix profiling for RayDistributedExecutor by @ruisearch42 in https://github.com/vllm-project/vllm/pull/13945
Update LMFE version to v0.10.11 to support new versions of transforme… by @noamgat in https://github.com/vllm-project/vllm/pull/13930
[Bugfix] Fix qwen2.5-vl overflow issue by @Isotr0py in https://github.com/vllm-project/vllm/pull/13968
[VLM] Generalized prompt updates for multi-modal processor by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/13964
[Attention] MLA support for V1 by @chenyang78 in https://github.com/vllm-project/vllm/pull/13789
Bump azure/setup-helm from 4.2.0 to 4.3.0 by @dependabot in https://github.com/vllm-project/vllm/pull/13742
[VLM] Deprecate legacy input mapper for OOT multimodal models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/13979
[ROCm] Fix the Kernels, Core, and Prefix Caching AMD CI groups by @SageMoore in https://github.com/vllm-project/vllm/pull/13970
[V1][Minor] Minor cleanup for GPU Model Runner by @WoosukKwon in https://github.com/vllm-project/vllm/pull/13983
[core] Perf improvement for DSv3 on AMD GPUs by @qli88 in https://github.com/vllm-project/vllm/pull/13718
[Attention] Flash MLA for V1 by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/13867
[Model][Speculative Decoding] Expand DeepSeek MTP code to support k > n_predict by @benchislett in https://github.com/vllm-project/vllm/pull/13626
[Misc] Print FusedMoE detail info by @jeejeelee in https://github.com/vllm-project/vllm/pull/13974
[V1]SupportsV0Only protocol for model definitions by @ywang96 in https://github.com/vllm-project/vllm/pull/13959
[Bugfix] Check that number of images matches number of <|image|> tokens with mllama by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/13911
[Doc] Move multimodal Embedding API example to Online Serving page by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/14017
[Bugfix][Disaggregated] patch the inflight batching on the decode node in SimpleConnector to avoid hangs in SimpleBuffer (nccl based) by @hasB4K in https://github.com/vllm-project/vllm/pull/13987
Use smaller embedding model when not testing model specifically by @hmellor in https://github.com/vllm-project/vllm/pull/13891
[Hardware][Intel-Gaudi] Regional compilation support by @Kacper-Pietkun in https://github.com/vllm-project/vllm/pull/13213
[V1][Minor] Restore V1 compatibility with LLMEngine class by @Ryp in https://github.com/vllm-project/vllm/pull/13090
Update AutoAWQ docs by @hmellor in https://github.com/vllm-project/vllm/pull/14042
[Bugfix] Fix MoeWNA16Method activation by @jeejeelee in https://github.com/vllm-project/vllm/pull/14024
[VLM][Bugfix] Enable specifying prompt target via index by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/14038
[Bugfix] Initialize attention bias on the same device as Query/Key/Value for QwenVL Series by @LouieYang in https://github.com/vllm-project/vllm/pull/14031
[Doc] Fix ROCm documentation by @b8zhong in https://github.com/vllm-project/vllm/pull/14041
Fix entrypoint tests for embedding models by @hmellor in https://github.com/vllm-project/vllm/pull/14052
[V1][TPU] Integrate the new ragged paged attention kernel with vLLM v1 on TPU by @vanbasten23 in https://github.com/vllm-project/vllm/pull/13379
[v1] Cleanup the BlockTable in InputBatch by @heheda12345 in https://github.com/vllm-project/vllm/pull/13977
Add RELEASE.md by @atalman in https://github.com/vllm-project/vllm/pull/13926
[v1] Move block pool operations to a separate class by @heheda12345 in https://github.com/vllm-project/vllm/pull/13973
[core] Bump ray to 2.43 by @ruisearch42 in https://github.com/vllm-project/vllm/pull/13994
[torch.compile] Fix RMSNorm

These notes run past the length kept in the archive. The rest is on the publisher’s page.