vLLM 0.9.1 release notes

Highlights

This release features 274 commits, from 123 contributors (27 new contributors!)

Progress in large scale serving
- DP Attention + Expert Parallelism: CUDA graph support (#18724), DeepEP dispatch-combine kernel (#18434), batched/masked DeepGEMM kernel (#19111), CUTLASS MoE kernel with PPLX (#18762)
- Heterogeneous TP (#18833), NixlConnector Enable FlashInfer backend (#19090)
- DP: API-server scaleout with many-to-many server-engine comms (#17546), Support DP with Ray (#18779), allow AsyncLLMEngine.generate to target a specific DP rank (#19102), data parallel rank to KVEventBatch (#18925)
- Tooling: Simplify EP kernels installation (#19412)
RLHF workflow: Support inplace model weights loading (#18745)
Initial full support for Hybrid Memory Allocator (#17996), support cross-layer KV sharing (#18212)
Add FlexAttention to vLLM V1 (#16078)
Various production hardening related to full cuda graph mode (#19171, #19106, #19321)

Model Support

Support Magistral (#19193), LoRA support for InternVL (#18842), minicpm eagle support (#18943), NemotronH support (#18863, #19249)
Enable data parallel for Llama4 vision encoder (#18368)
Add DeepSeek-R1-0528 function call chat template (#18874)

Hardware Support & Performance Optimizations

Add H20-3e fused MoE kernel tuning configs for DeepSeek-R1/V3 (#19205), Qwen3-235B-A22B (#19315)
Blackwell: Add Cutlass MLA backend (#17625), Tunings for SM100 FP8 CUTLASS kernel (#18778), Use FlashInfer by default on Blackwell GPUs (#19118), Tune scaled_fp8_quant by increasing vectorization (#18844)
FP4: Add compressed-tensors NVFP4 support (#18312), FP4 MoE kernel optimization (#19110)
CPU: V1 support for the CPU backend (#16441)
ROCm: Add AITER grouped topk for DeepSeekV2 (#18825)
POWER: Add IBM POWER11 Support to CPU Extension Detection (#19082)
TPU: Initial support of model parallelism with single worker using SPMD (#18011), Multi-LoRA Optimizations for the V1 TPU backend (#15655)
Neuron: Add multi-LoRA support for Neuron. (#18284), Add Multi-Modal model support for Neuron (#18921), Support quantization on neuron (#18283)
Platform: Make torch distributed process group extendable (#18763)

Engine features

Add Lora Support to Beam Search (#18346)
Add rerank support to run_batch endpoint (#16278)
CLI: add run batch (#18804)
Server: custom logging (#18403), allowed_token_ids in ChatCompletionRequest (#19143)
LLM API: make use_tqdm accept a callable for custom progress bars (#19357)
perf: [KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437)

API Deprecations

Disallow pos-args other than model when initializing LLM (#18802)
Remove inputs arg fallback in Engine classes (#18799)
Remove fallbacks for Embeddings API (#18795)
Remove mean pooling default for Qwen2EmbeddingModel (#18913)
Require overriding get_dummy_text and get_dummy_mm_data (#18796)
Remove metrics that were deprecated in 0.8 (#18837)

Documentation

Add CLI doc (#18871)
Update SECURITY.md with link to our security guide (#18961), Add security warning to bug report template (#19365)

What's Changed

[CI/Build] [TPU] Fix TPU CI exit code by @CAROLZXYZXY in https://github.com/vllm-project/vllm/pull/18282
[Neuron] Support quantization on neuron by @aws-satyajith in https://github.com/vllm-project/vllm/pull/18283
Support datasets in vllm bench serve and sync with benchmark_[serving,datasets].py by @mgoin in https://github.com/vllm-project/vllm/pull/18566
[Bugfix] Disable prefix caching by default for benchmark by @cascade812 in https://github.com/vllm-project/vllm/pull/18771
[Build] Fixes for CMake install by @ProExpertProg in https://github.com/vllm-project/vllm/pull/18570
[Core] Improve Tensor serialisation by @lgeiger in https://github.com/vllm-project/vllm/pull/18774
[rocm] Fix wrong attention log by @fxmarty-amd in https://github.com/vllm-project/vllm/pull/18764
[Bugfix] Fix nomic max_model_len by @noooop in https://github.com/vllm-project/vllm/pull/18755
[Bugfix]: correctly propagate errors message caught at the chat_templating step to the client by @gcalmettes in https://github.com/vllm-project/vllm/pull/18769
[V1] fix torch profiling for V1 offline scenarios by @divakar-amd in https://github.com/vllm-project/vllm/pull/18445
[V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) by @RonaldBXu in https://github.com/vllm-project/vllm/pull/18781
[Bugfix][FailingTest]Fix test_model_load_with_params.py by @rabi in https://github.com/vllm-project/vllm/pull/18758
[Deprecation] Require overriding get_dummy_text and get_dummy_mm_data by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18796
[Deprecation] Remove unused sync methods in async_timeout by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18792
[Deprecation] Remove fallbacks for Embeddings API by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18795
[CI] improve embed testing by @noooop in https://github.com/vllm-project/vllm/pull/18747
Fix PiecewiseCompileInterpreter by @zou3519 in https://github.com/vllm-project/vllm/pull/17338
[BugFix] FA2 MLA Accuracy Issue by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/18807
[Platform][Dist] Make torch distributed process group extendable by @MengqingCao in https://github.com/vllm-project/vllm/pull/18763
Enable Pydantic mypy checks and convert configs to Pydantic dataclasses by @hmellor in https://github.com/vllm-project/vllm/pull/17599
[Frontend] add run batch to CLI by @reidliu41 in https://github.com/vllm-project/vllm/pull/18804
decrement server_load on listen for disconnect by @daniel-salib in https://github.com/vllm-project/vllm/pull/18784
[Core] Add Lora Support to Beam Search by @alex-jw-brooks in https://github.com/vllm-project/vllm/pull/18346
[Chore] update ty configuration by @aarnphm in https://github.com/vllm-project/vllm/pull/18839
[Misc] fix olmoe model layer for TP > 1 by @lengrongfu in https://github.com/vllm-project/vllm/pull/18828
[V1][Metrics] Remove metrics that were deprecated in 0.8 by @markmc in https://github.com/vllm-project/vllm/pull/18837
[Chore][Spec Decode] Update check NoneType instead of assigning variables by @aarnphm in https://github.com/vllm-project/vllm/pull/18836
[Hardware][TPU][V1] Multi-LoRA Optimisations for the V1 TPU backend by @Akshat-Tripathi in https://github.com/vllm-project/vllm/pull/15655
Remove checks for None for fields which should never be None by @hmellor in https://github.com/vllm-project/vllm/pull/17985
[Core] Enable CUDA graphs for DP + All2All kernels by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/18724
[Bugfix][ROCm] fix the power of 2 exception from triton_unified_attention.py when running llama4 models and unit test fix by @hongxiayang in https://github.com/vllm-project/vllm/pull/18100
Prevent the cross-encoder logic from being applied to classification tasks by @maxdebayser in https://github.com/vllm-project/vllm/pull/18838
Add ability to use CUDAGraphs with use_inductor=False by @zou3519 in https://github.com/vllm-project/vllm/pull/17345
[Bugfix][TPU] fix moe custom kernel import by @yaochengji in https://github.com/vllm-project/vllm/pull/18853
[Doc][Neuron] Update documentation for Neuron by @elaineyz in https://github.com/vllm-project/vllm/pull/18868
Skip device and quant Pydantic validation to make plugin device work by @Yikun in https://github.com/vllm-project/vllm/pull/18843
Fixes a dead link in nightly benchmark readme by @nerdalert in https://github.com/vllm-project/vllm/pull/18856
[Neuron] Add multi-LoRA support for Neuron. by @aws-satyajith in https://github.com/vllm-project/vllm/pull/18284
[LoRA] Add LoRA support for InternVL by @jeejeelee in https://github.com/vllm-project/vllm/pull/18842
[Doc] Remove redundant spaces from compatibility_matrix.md by @windsonsea in https://github.com/vllm-project/vllm/pull/18891
[doc] add CLI doc by @reidliu41 in https://github.com/vllm-project/vllm/pull/18871
[Bugfix] Fix misleading information in the documentation by @jeejeelee in https://github.com/vllm-project/vllm/pull/18845
[Misc] Replace TODO in serving transcription by @NickLucche in https://github.com/vllm-project/vllm/pull/18895
[Bugfix] Ensure tensors are contiguous during serialisation by @lgeiger in https://github.com/vllm-project/vllm/pull/18860
[BugFix] Update pydantic to fix error on python 3.10 by @ProExpertProg in https://github.com/vllm-project/vllm/pull/18852
Fix an error in dummy weight loading for quantization models by @Chenyaaang in https://github.com/vllm-project/vllm/pull/18855
[Misc][Tools][Benchmark] Add benchmark_serving supports for llama.cpp. by @Duyi-Wang in https://github.com/vllm-project/vllm/pull/18692
[Doc] Fix codeblocks formatting in LoRA adapters documentation by @Zerohertz in https://github.com/vllm-project/vllm/pull/18907
[Bugfix] Fix the failing gte embedding test by @Isotr0py in https://github.com/vllm-project/vllm/pull/18720
[Attention][V1] Toggle for v1 attention backend by @gshtras in https://github.com/vllm-project/vllm/pull/18275
[ROCm][V0][Attention] Revert to the previous FA triton kernel by @gshtras in https://github.com/vllm-project/vllm/pull/18226
[Deprecation] Disallow pos-args other than model when initializing LLM by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18802
[Misc] Remove duplicate init for self.vllm_config by @googs1025 in https://github.com/vllm-project/vllm/pull/18896
[V1] Allocate kv_cache with stride order for V1 by @NickLucche in https://github.com/vllm-project/vllm/pull/18775
[BugFix] Make DP work with connector-delayed new requests by @njhill in https://github.com/vllm-project/vllm/pull/18559
[P/D] NixlConnector DP fixes by @wseaton in https://github.com/vllm-project/vllm/pull/18903
Use standalone_compile by default in torch >= 2.8.0 by @zou3519 in https://github.com/vllm-project/vllm/pull/18846
[TPU] remove transpose ops in moe kernel by @yaochengji in https://github.com/vllm-project/vllm/pull/18923
[Bugfix] Fix PP default fallback behavior for V1 by @mgoin in https://github.com/vllm-project/vllm/pull/18915
[Misc] Update type annotation for rotary embedding base by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18914
[TPU][CI/CD] Clean up docker for TPU tests. by @CAROLZXYZXY in https://github.com/vllm-project/vllm/pull/18926
improve the robustness of parsing vlms config in AutoRound by @wenhuach21 in https://github.com/vllm-project/vllm/pull/18894
[Bugfix] Consistent ascii handling in tool parsers by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/18883
[Model] Use AutoWeightsLoader for mamba2 by @jinyouzhi in https://github.com/vllm-project/vllm/pull/18918
[docs] fix: fix markdown syntax by @eric-haibin-lin in https://github.com/vllm-project/vllm/pull/18927
[ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. by @vllmellm in https://github.com/vllm-project/vllm/pull/18938
[Bugfix] Remove NVFP4 scales assertions to fix load_format=dummy by @mgoin in https://github.com/vllm-project/vllm/pull/18861
[Deprecation] Remove mean pooling default for Qwen2EmbeddingModel by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18913
[Misc]Fix benchmarks/README.md for speculative decoding by @rabi in https://github.com/vllm-project/vllm/pull/18897
[doc] add mkdocs doc by @reidliu41 in https://github.com/vllm-project/vllm/pull/18930
[Model] Use in-place adds in SigLIP by @lgeiger in https://github.com/vllm-project/vllm/pull/18922
[Bugfix][Failing Test] Fix test_vllm_port.py by @rabi in https://github.com/vllm-project/vllm/pull/18618
[Misc]Fix typo by @Always-Naive in https://github.com/vllm-project/vllm/pull/18947
[Bugfix][TPU] Fix tpu model runner testcase failure by @CAROLZXYZXY in https://github.com/vllm-project/vllm/pull/18810
[CI/Build] remove regex from build dependencies by @dtrifiro in https://github.com/vllm-project/vllm/pull/18945
[Feature] minicpm eagle support by @huangyuxiang03 in https://github.com/vllm-project/vllm/pull/18943
[doc] show the count for fork and watch by @reidliu41 in https://github.com/vllm-project/vllm/pull/18950
[Docs] Update SECURITY.md with link to our security guide by @russellb in https://github.com/vllm-project/vllm/pull/18961
Improve "failed to get the hash of the compiled graph" error by @zou3519 in https://github.com/vllm-project/vllm/pull/18956
[Perf] API-server scaleout with many-to-many server-engine comms by @njhill in https://github.com/vllm-project/vllm/pull/17546
Benchmark script for fp8 vs bf16 gemm by @mgoin in https://github.com/vllm-project/vllm/pull/17126
[VLM] Add PP support and fix GPTQ inference for Ovis models by @Isotr0py in https://github.com/vllm-project/vllm/pull/18958
[Misc] add group_size is -1 in awq quantization by @lengrongfu in https://github.com/vllm-project/vllm/pull/18910
Tool parser regex timeout handling by @wseaton in https://github.com/vllm-project/vllm/pull/18960
[Docs] Correct multiprocessing design doc by @lgeiger in https://github.com/vllm-project/vllm/pull/18964
create util function for batched arange by @yuguo68 in https://github.com/vllm-project/vllm/pull/18937
[Frontend] Add rerank support to run_batch endpoint by @pooyadavoodi in https://github.com/vllm-project/vllm/pull/16278
[Misc] Fix estimated max model len msg by @sarckk in https://github.com/vllm-project/vllm/pull/18966
[Bugfix]: Fix the incompatibility issue with Structured Outputs when Thinking is disabled by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/18879
fix security issue of logging llm output by @luccafong in https://github.com/vllm-project/vllm/pull/18980
[Neuron] Add Multi-Modal model support for Neuron by @aws-satyajith in https://github.com/vllm-project/vllm/pull/18921
[doc] fix the list rendering issue - security.md by @reidliu41 in https://github.com/vllm-project/vllm/pull/18982
[BugFix] Pydantic part 2 by @ProExpertProg in https://github.com/vllm-project/vllm/pull/18911
[FEAT][ROCm] Add AITER grouped topk for DeepSeekV2 by @vllmellm in https://github.com/vllm-project/vllm/pull/18825
[Bugfix] Fix for issue 17396 by @frreiss in https://github.com/vllm-project/vllm/pull/18773
[ROCm][Kernel] Add gfx950 support for skinny gemms by @charlifu in https://github.com/vllm-project/vllm/pull/18010
[P/D] NixlConnector use cache device index for memory registration by @ptarasiewiczNV in https://github.com/vllm-project/vllm/pull/18969
[BugFix] Fix multi-node offline data-parallel by @njhill in https://github.com/vllm-project/vllm/pull/18981
[Misc] add return token strs for tokenize by @reidliu41 in https://github.com/vllm-project/vllm/pull/18941
[Misc][Benchmark] Add support for CustomDataset by @ekagra-ranjan in https://github.com/vllm-project/vllm/pull/18511
[Bugfix] Fix EAGLE3 broken logits by @benchislett in https://github.com/vllm-project/vllm/pull/18909
[Core] Rework dtype resolution by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18751
[LoRA] Support dynamically initialize packed_modules_mapping for VLM with arbitrary components by @Isotr0py in https://github.com/vllm-project/vllm/pull/18987
[doc] small fix - mkdocs by @reidliu41 in https://github.com/vllm-project/vllm/pull/18996
Let max_num_batched_tokens use human_readable_int for large numbers by @mgoin in https://github.com/vllm-project/vllm/pull/18968
[BugFix] fix data parallel construct ipv6 url addres by @lengrongfu in https://github.com/vllm-project/vllm/pull/18991
[BugFix] Fix incorrect metrics shutdown error log message by @njhill in https://github.com/vllm-project/vllm/pull/18992
[doc] wrong output by @reidliu41 in https://github.com/vllm-project/vllm/pull/19000
[Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp all reduce in set_forward_context by @izhuhaoran in https://github.com/vllm-project/vllm/pull/18935
[Bugfix][Nixl] Fix DP Metadata Handshake by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/19008
[Core] Support inplace model weights loading by @22quinn in https://github.com/vllm-project/vllm/pull/18745
[doc] add pytest tips by @reidliu41 in https://github.com/vllm-project/vllm/pull/19010
[Model] enable data parallel for Llama4 vision encoder by @jennyyyyzhen in https://github.com/vllm-project/vllm/pull/18368
[Frontend] enable custom logging for the uvicorn server (OpenAI API server) by @fpaupier in https://github.com/vllm-project/vllm/pull/18403
[Bugfix][Model] Attempt to fix eagle in V0. by @gshtras in https://github.com/vllm-project/vllm/pull/18978
add an absolute path for run.sh by @calvin0327 in https://github.com/vllm-project/vllm/pull/18258
[Hardware][TPU] Initial support of model parallelism with single worker using SPMD by @lsy323 in https://github.com/vllm-project/vllm/pull/18011
[Doc] Remove duplicate TOCs during MkDocs migration by @Zerohertz in https://github.com/vllm-project/vllm/pull/19021
[Bugfix][EP+DP] Use pplx-kernel internode instead of intranode by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/19034
Adding "LoRA Test %N" to AMD production tests by @Concurrensee in https://github.com/vllm-project/vllm/pull/18929
[CPU][CI] Re-enable the CPU CI tests by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/19046
[ROCm][Build] Clean up the ROCm build by @gshtras in https://github.com/vllm-project/vllm/pull/19040
[V1] Support DP with Ray by @ruisearch42 in https://github.com/vllm-project/vllm/pull/18779
Add tarsier model support by @princepride in https://github.com/vllm-project/vllm/pull/18985
[bugfix] small fix logic issue by @reidliu41 in https://github.com/vllm-project/vllm/pull/18999
Reduce logs in CLI scripts and plugin loader by @mgoin in https://github.com/vllm-project/vllm/pull/18970
[Bugfix] Use cmake 3.26.1 instead of 3.26 to avoid build failure by @houseroad in https://github.com/vllm-project/vllm/pull/19019
[v1][KVCacheManager] Rename BlockHashType to BlockHash by @heheda12345 in https://github.com/vllm-project/vllm/pull/19015
Update docker docs with ARM CUDA cross-compile by @mgoin in https://github.com/vllm-project/vllm/pull/19037
[Doc] Add InternVL LoRA support by @jeejeelee in https://github.com/vllm-project/vllm/pull/19055
[Misc] Update WeightsMapper for qwen2-vl/qwen2.5-vl by @Isotr0py in https://github.com/vllm-project/vllm/pull/19054
[Doc] Update V1 user guide for embedding and enc-dec models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/19060
[doc] clarify windows support by @youkaichao in https://github.com/vllm-project/vllm/pull/19088
[CI/Build] Remove V0 LoRA test by @jeejeelee in https://github.com/vllm-project/vllm/pull/19066
Fix underscores in dict keys passed via CLI by @hmellor in https://github.com/vllm-project/vllm/pull/19030
[Bugfix] disable processor cache by @zucchini-nlp in https://github.com/vllm-project/vllm/pull/19068
[Doc] Improve the Pull Request template with key components by @houseroad in https://github.com/vllm-project/vllm/pull/19086
[Misc] Add missing _Backend enums by @NickLucche in https://github.com/vllm-project/vllm/pull/19081
[Misc] fix: add miss best_of param validation by @googs1025 in https://github.com/vllm-project/vllm/pull/18555
[Misc] Add SPDX-FileCopyrightText by @simon-mo in https://github.com/vllm-project/vllm/pull/19100
[Doc] Readme standardization by @SorenDreano in https://github.com/vllm-project/vllm/pull/18695
[doc] update docker version by @reidliu41 in https://github.com/vllm-project/vllm/pull/19074
[Kernel] DeepEP dispatch-combine kernel integration by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/18434
[V1] Support cross-layer KV sharing by @sarckk in https://github.com/vllm-project/vllm/pull/18212
[Perf] Tune scaled_fp8_quant by increasing vectorization by @mgoin in https://github.com/vllm-project/vllm/pull/18844
Fix interaction between Optional and Annotated in CLI typing by @hmellor in https://github.com/vllm-project/vllm/pull/19093
[v1] Re-init input batch for multiple kv cache groups by @heheda12345 in https://github.com/vllm-project/vllm/pull/18654
[V1][Spec Decode][Ngram] 1.35x gain -> 1.95x gain on InstructCoder with prompt fix by @ekagra-ranjan in https://github.com/vllm-project/vllm/pull/18971
[Bugfix] get_num_blocks_to_allocate with null_block by @heheda12345 in https://github.com/vllm-project/vllm/pull/19031
[Bugfix]: Fix the incompatibility issue with tool_choice 'required' when Thinking is enabled by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/19075
[Bugfix][P/D] Fix Prefix Cache Bug by @NickLucche in https://github.com/vllm-project/vllm/pull/18411
[Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for models with sliding window layers by @heheda12345 in https://github.com/vllm-project/vllm/pull/19029
feat: add data parallel rank to KVEventBatch by @PeaBrane in https://github.com/vllm-project/vllm/pull/18925
[Misc] Fix path and python alias errors in disagg_prefill exmaples by @Jeffwan in https://github.com/vllm-project/vllm/pull/18919
[Docs] Add developer doc about CI failures by @russellb in https://github.com/vllm-project/vllm/pull/18782
[CPU] V1 support for the CPU backend by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/16441
[Core] Cast multimodal input in hf processor by @lgeiger in https://github.com/vllm-project/vllm/pull/18862
[KERNEL] Sampler. CUDA kernel for applying repetition penalty by @vadiklyutiy in https://github.com/vllm-project/vllm/pull/18437
[Cleanup][v1]:remote guided-decoding-backend for example by @calvin0327 in https://github.com/vllm-project/vllm/pull/19059
[NVIDIA] Add Cutlass MLA backend by @kaixih in https://github.com/vllm-project/vllm/pull/17625
[Bugfix] Fix FA3 full cuda graph correctness by @WoosukKwon in https://github.com/vllm-project/vllm/pull/19106
Fix #19130 by @princepride in https://github.com/vllm-project/vllm/pull/19132
[TPU] Skip hanging tests by @lsy323 in https://github.com/vllm-project/vllm/pull/19115
Fix ValueError: Missing value for tag key(s): model_name,engine. by @eicherseiji in https://github.com/vllm-project/vllm/pull/19113
[Misc] Add packages for benchmark as extra dependency by @Isotr0py in https://github.com/vllm-project/vllm/pull/19089
Improve the output precision of embedding models by @noooop in https://github.com/vllm-project/vllm/pull/19092
[CI/Build][Bugfix] Ensure compatibility with transformers 4.52 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18678
Add DeepSeek-R1-0528 function call chat template by @Xu-Wenqing in https://github.com/vllm-project/vllm/pull/18874
Sm100 blockwise fp8 swap ab by @IwakuraRein in https://github.com/vllm-project/vllm/pull/18564
[Doc] Update V1 Guide for embedding models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/19141
Allow AsyncLLMEngine.generate to target a specific DP rank by @jmswen in https://github.com/vllm-project/vllm/pull/19102
[Bugfix][EP+DP] Fix internode check by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/19112
[Perf] Tunings for SM100 FP8 CUTLASS kernel by @mgoin in https://github.com/vllm-project/vllm/pull/18778
[TPU] Update dynamo dump file name in compilation test by @lsy323 in https://github.com/vllm-project/vllm/pull/19108
[Bugfix] fix v1 cpu worker fails on macOS by @kebe7jun in https://github.com/vllm-project/vllm/pull/19121
[Kernel] Integrate batched/masked deepgemm kernel by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/19111
[Misc] refactor: simplify EngineCoreClient.make_async_mp_client in AsyncLLM by @googs1025 in https://github.com/vllm-project/vllm/pull/18817
[P/D] Heterogeneous TP by @NickLucche in https://github.com/vllm-project/vllm/pull/18833
[doc] small fix by @reidliu41 in https://github.com/vllm-project/vllm/pull/19167
[Bugfix][Nixl] Fix full prefix cache hit bug by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/18632
[Bugfix] Fix port handling in make_zmq_path by @mgoin in https://github.com/vllm-project/vllm/pull/19117
[Torch Nightly]add missing dependency by @yangw-dev in https://github.com/vllm-project/vllm/pull/18770
Handle non-serializable objects when dumping benchmark results by @huydhn in https://github.com/vllm-project/vllm/pull/19114
[BugFix][Minor] Fix full cuda graph bug when max_num_seqs < 512 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/19171
[Bugfix]: Fix the incompatibility issue with stream when Thinking is disabled by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/19135
[Build] Annotate wheel and container path for release workflow by @simon-mo in https://github.com/vllm-project/vllm/pull/19162
[Misc] Remove unnecessary fallback to prefill-decode attention by @vllmellm in https://github.com/vllm-project/vllm/pull/19138
[Misc] Do not override NCCL_CUMEM_ENABLE if set explicitly by @22quinn in https://github.com/vllm-project/vllm/pull/19105
[Frontend] improve vllm run-batch --help display by @reidliu41 in https://github.com/vllm-project/vllm/pull/19187
[Bugfix] properly catch PIL-related errors for vision models when incorrect data urls are provided by @gcalmettes in https://github.com/vllm-project/vllm/pull/19202
[mistral_common] Add v11 tokenizer by @patrickvonplaten in https://github.com/vllm-project/vllm/pull/19193
Add H20-3e fused MoE kernel tuning configs for DeepSeek-R1/V3 by @Xu-Wenqing in https://github.com/vllm-project/vllm/pull/19205
[Hardware][NVIDIA] FP4 MoE kernel optimization by @dubcyfor3 in https://github.com/vllm-project/vllm/pull/19110
[MISC][Bugfix] Use less CPU when message queue has been empty for some time by @p12tic in

These notes run past the length kept in the archive. The rest is on the publisher’s page.