vLLM 0.7.0 release notes

Highlights

vLLM's V1 engine is ready for testing! This is a rewritten engine designed for performance and architectural simplicity. You can turn it on by setting environment variable VLLM_USE_V1=1. See our blog for more details. (44 commits).
New methods (LLM.sleep, LLM.wake_up, LLM.collective_rpc, LLM.reset_prefix_cache) in vLLM for the post training frameworks! (#12361, #12084, #12284).
torch.compile is now fully integrated in vLLM, and enabled by default in V1. You can turn it on via -O3 engine parameter. (#11614, #12243, #12043, #12191, #11677, #12182, #12246).

This release features

400 commits from 132 contributors, including 57 new contributors.
- 28 CI and build enhancements, including testing for nightly torch (#12270) and inclusion of genai-perf for benchmark (#10704).
- 58 documentation enhancements, including reorganized documentation structure (#11645, #11755, #11766, #11843, #11896).
- more than 161 bug fixes and miscellaneous enhancements

Features

Models

New generative models: CogAgent (#11742), Deepseek-VL2 (#11578, #12068, #12169), fairseq2 Llama (#11442), InternLM3 (#12037), Whisper (#11280)
New pooling models: Qwen2 PRM (#12202), InternLM2 reward models (#11571)
VLM: Merged multi-modal processor is now ready for model developers! (#11620, #11900, #11682, #11717, #11669, #11396)
- Any model that implements merged multi-modal processor and the get_*_embeddings methods according to this guide is automatically supported by V1 engine.

Hardwares

Apple: Native support for macOS Apple Silicon (#11696)
AMD: MI300 FP8 format for block_quant (#12134), Tuned MoE configurations for multiple models (#12408, #12049), block size heuristic for avg 2.8x speedup for int8 models (#11698)
TPU: support for W8A8 (#11785)
x86: Multi-LoRA (#11100) and MoE Support (#11831)
Progress in out-of-tree hardware support (#12009, #11981, #11948, #11609, #12264, #11516, #11503, #11369, #11602)

Features

Distributed:
- Support torchrun and SPMD-style offline inference (#12071)
- New collective_rpc abstraction (#12151, #11256)
API Server: Jina- and Cohere-compatible Rerank API (#12376)
Kernels:
- Flash Attention 3 Support (#12093)
- Punica prefill kernels fusion (#11234)
- For Deepseek V3: optimize moe_align_block_size for cuda graph and large num_experts (#12222)

Others

Benchmark: new script for CPU offloading (#11533)
Security: Set weights_only=True when using torch.load() (#12366)

What's Changed

[Docs] Document Deepseek V3 support by @simon-mo in https://github.com/vllm-project/vllm/pull/11535
Update openai_compatible_server.md by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11536
[V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling by @WoosukKwon in https://github.com/vllm-project/vllm/pull/11394
[V1] Fix yapf by @WoosukKwon in https://github.com/vllm-project/vllm/pull/11538
[CI] Fix broken CI by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11543
[misc] fix typing by @youkaichao in https://github.com/vllm-project/vllm/pull/11540
[V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11534
[BugFix] Deepseekv3 broke quantization for all other methods by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11547
[Platform] Move model arch check to platform by @MengqingCao in https://github.com/vllm-project/vllm/pull/11503
Update deploying_with_k8s.md with AMD ROCm GPU example by @AlexHe99 in https://github.com/vllm-project/vllm/pull/11465
[Bugfix] Fix TeleChat2ForCausalLM weights mapper by @jeejeelee in https://github.com/vllm-project/vllm/pull/11546
[Misc] Abstract out the logic for reading and writing media content by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11527
[Doc] Add xgrammar in doc by @Chen-0210 in https://github.com/vllm-project/vllm/pull/11549
[VLM] Support caching in merged multi-modal processor by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11396
[MODEL] Update LoRA modules supported by Jamba by @ErezSC42 in https://github.com/vllm-project/vllm/pull/11209
[Misc]Add BNB quantization for MolmoForCausalLM by @jeejeelee in https://github.com/vllm-project/vllm/pull/11551
[Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix by @Isotr0py in https://github.com/vllm-project/vllm/pull/11566
[Bugfix] Fix for ROCM compressed tensor support by @selalipop in https://github.com/vllm-project/vllm/pull/11561
[Doc] Update mllama example based on official doc by @heheda12345 in https://github.com/vllm-project/vllm/pull/11567
[V1] [4/N] API Server: ZMQ/MP Utilities by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11541
[Bugfix] Last token measurement fix by @rajveerb in https://github.com/vllm-project/vllm/pull/11376
[Model] Support InternLM2 Reward models by @Isotr0py in https://github.com/vllm-project/vllm/pull/11571
[Model] Remove hardcoded image tokens ids from Pixtral by @ywang96 in https://github.com/vllm-project/vllm/pull/11582
[Hardware][AMD]: Replace HIPCC version with more precise ROCm version by @hj-wei in https://github.com/vllm-project/vllm/pull/11515
[V1][Minor] Set pin_memory=False for token_ids_cpu tensor by @WoosukKwon in https://github.com/vllm-project/vllm/pull/11581
[Doc] Minor documentation fixes by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11580
[bugfix] interleaving sliding window for cohere2 model by @youkaichao in https://github.com/vllm-project/vllm/pull/11583
[V1] [5/N] API Server: unify Detokenizer and EngineCore input by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11545
[Doc] Convert list tables to MyST by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11594
[v1][bugfix] fix cudagraph with inplace buffer assignment by @youkaichao in https://github.com/vllm-project/vllm/pull/11596
[Misc] Use registry-based initialization for KV cache transfer connector. by @KuntaiDu in https://github.com/vllm-project/vllm/pull/11481
Remove print statement in DeepseekScalingRotaryEmbedding by @mgoin in https://github.com/vllm-project/vllm/pull/11604
[v1] fix compilation cache by @youkaichao in https://github.com/vllm-project/vllm/pull/11598
[Docker] bump up neuron sdk v2.21 by @liangfu in https://github.com/vllm-project/vllm/pull/11593
[Build][Kernel] Update CUTLASS to v3.6.0 by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/11607
[CI/Build][CPU] Fix CPU CI by lazy importing triton FP8 kernels by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/11618
[platforms] enable platform plugins by @youkaichao in https://github.com/vllm-project/vllm/pull/11602
[VLM] Abstract out multi-modal data parsing in merged processor by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11620
[V1] [6/N] API Server: Better Shutdown by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11586
[Bugfix] Validate and concatenate image embeddings in MiniCPMVBaseModel by @whyiug in https://github.com/vllm-project/vllm/pull/11631
[benchmark] Remove dependency for H100 benchmark step by @khluu in https://github.com/vllm-project/vllm/pull/11572
[Model][LoRA]LoRA support added for MolmoForCausalLM by @ayylemao in https://github.com/vllm-project/vllm/pull/11439
[Bugfix] Fix OpenAI parallel sampling when using xgrammar by @mgoin in https://github.com/vllm-project/vllm/pull/11637
[Misc][LoRA] Support Rank Stabilized LoRA (RSLoRA) by @JohnGiorgi in https://github.com/vllm-project/vllm/pull/6909
[Bugfix] Move the _touch(computed_blocks) call in the allocate_slots method to after the check for allocating new blocks. by @sakunkun in https://github.com/vllm-project/vllm/pull/11565
[V1] Simpify vision block hash for prefix caching by removing offset from hash by @heheda12345 in https://github.com/vllm-project/vllm/pull/11646
[V1][VLM] V1 support for selected single-image models. by @ywang96 in https://github.com/vllm-project/vllm/pull/11632
[Benchmark] Add benchmark script for CPU offloading by @ApostaC in https://github.com/vllm-project/vllm/pull/11533
[Bugfix][Refactor] Unify model management in frontend by @joerunde in https://github.com/vllm-project/vllm/pull/11660
[VLM] Add max-count checking in data parser for single image models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11661
[Misc] Optimize Qwen2-VL LoRA test by @jeejeelee in https://github.com/vllm-project/vllm/pull/11663
[Misc] Replace space with - in the file names by @houseroad in https://github.com/vllm-project/vllm/pull/11667
[Doc] Fix typo by @serihiro in https://github.com/vllm-project/vllm/pull/11666
[V1] Implement Cascade Attention by @WoosukKwon in https://github.com/vllm-project/vllm/pull/11635
[VLM] Move supported limits and max tokens to merged multi-modal processor by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11669
[VLM][Bugfix] Multi-modal processor compatible with V1 multi-input by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11674
[mypy] Pass type checking in vllm/inputs by @CloseChoice in https://github.com/vllm-project/vllm/pull/11680
[VLM] Merged multi-modal processor for LLaVA-NeXT by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11682
According to vllm.EngineArgs, the name should be distributed_executor_backend by @chunyang-wen in https://github.com/vllm-project/vllm/pull/11689
[Bugfix] Free cross attention block table for preempted-for-recompute sequence group. by @kathyyu-google in https://github.com/vllm-project/vllm/pull/10013
[V1][Minor] Optimize token_ids_cpu copy by @WoosukKwon in https://github.com/vllm-project/vllm/pull/11692
[Bugfix] Change kv scaling factor by param json on nvidia gpu by @bjmsong in https://github.com/vllm-project/vllm/pull/11688
Resolve race conditions in Marlin kernel by @wchen61 in https://github.com/vllm-project/vllm/pull/11493
[Misc] Minimum requirements for SageMaker compatibility by @nathan-az in https://github.com/vllm-project/vllm/pull/11576
Update default max_num_batch_tokens for chunked prefill by @SachinVarghese in https://github.com/vllm-project/vllm/pull/11694
[Bugfix] Check chain_speculative_sampling before calling it by @houseroad in https://github.com/vllm-project/vllm/pull/11673
[perf-benchmark] Fix dependency for steps in benchmark pipeline by @khluu in https://github.com/vllm-project/vllm/pull/11710
[Model] Whisper model implementation by @aurickq in https://github.com/vllm-project/vllm/pull/11280
[V1] Simplify Shutdown by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11659
[Bugfix] Fix ColumnParallelLinearWithLoRA slice by @zinccat in https://github.com/vllm-project/vllm/pull/11708
[V1] Improve TP>1 Error Handling + Stack Trace by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11721
[Misc]Add BNB quantization for Qwen2VL by @jeejeelee in https://github.com/vllm-project/vllm/pull/11719
Update requirements-tpu.txt to support python 3.9 and 3.11 by @mgoin in https://github.com/vllm-project/vllm/pull/11695
[V1] Chore: cruft removal by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11724
log GPU blocks num for MultiprocExecutor by @WangErXiao in https://github.com/vllm-project/vllm/pull/11656
Update tool_calling.md by @Bryce1010 in https://github.com/vllm-project/vllm/pull/11701
Update bnb.md with example for OpenAI by @bet0x in https://github.com/vllm-project/vllm/pull/11718
[V1] Add RayExecutor support for AsyncLLM (api server) by @jikunshang in https://github.com/vllm-project/vllm/pull/11712
[V1] Add kv cache utils tests. by @xcnick in https://github.com/vllm-project/vllm/pull/11513
[Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture by @yanburman in https://github.com/vllm-project/vllm/pull/11233
[VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11717
[Bugfix] Fix precision error in LLaVA-NeXT feature size calculation by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11735
[Model] Remove unnecessary weight initialization logic by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11736
[Bugfix][V1] Fix test_kv_cache_utils.py by @jeejeelee in https://github.com/vllm-project/vllm/pull/11738
[MISC] Replace c10::optional with std::optional by @houseroad in https://github.com/vllm-project/vllm/pull/11730
[distributed] remove pynccl's redundant stream by @cennn in https://github.com/vllm-project/vllm/pull/11744
fix: [doc] fix typo by @RuixiangMa in https://github.com/vllm-project/vllm/pull/11751
[Frontend] Improve StreamingResponse Exception Handling by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11752
[distributed] remove pynccl's redundant change_state by @cennn in https://github.com/vllm-project/vllm/pull/11749
[Doc] [1/N] Reorganize Getting Started section by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11645
[Bugfix] Remove block size constraint by @comaniac in https://github.com/vllm-project/vllm/pull/11723
[V1] Add BlockTable class by @WoosukKwon in https://github.com/vllm-project/vllm/pull/11693
[Misc] Fix typo for valid_tool_parses by @ruisearch42 in https://github.com/vllm-project/vllm/pull/11753
[V1] Refactor get_executor_cls by @ruisearch42 in https://github.com/vllm-project/vllm/pull/11754
[mypy] Forward pass function type hints in lora by @lucas-tucker in https://github.com/vllm-project/vllm/pull/11740
k8s-config: Update the secret to use stringData by @surajssd in https://github.com/vllm-project/vllm/pull/11679
[VLM] Separate out profiling-related logic by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11746
[Doc][2/N] Reorganize Models and Usage sections by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11755
[Bugfix] Fix max image size for LLaVA-Onevision by @ywang96 in https://github.com/vllm-project/vllm/pull/11769
[doc] explain how to add interleaving sliding window support by @youkaichao in https://github.com/vllm-project/vllm/pull/11771
[Bugfix][V1] Fix molmo text-only inputs by @jeejeelee in https://github.com/vllm-project/vllm/pull/11676
[Kernel] Move attn_type to Attention.init() by @heheda12345 in https://github.com/vllm-project/vllm/pull/11690
[V1] Extend beyond image modality and support mixed-modality inference with Llava-OneVision by @ywang96 in https://github.com/vllm-project/vllm/pull/11685
[Bugfix] Fix LLaVA-NeXT feature size precision error (for real) by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11772
[Model] Future-proof Qwen2-Audio multi-modal processor by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11776
[XPU] Make pp group initilized for pipeline-parallelism by @ys950902 in https://github.com/vllm-project/vllm/pull/11648
[Doc][3/N] Reorganize Serving section by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11766
[Kernel][LoRA]Punica prefill kernels fusion by @jeejeelee in https://github.com/vllm-project/vllm/pull/11234
[Bugfix] Update attention interface in Whisper by @ywang96 in https://github.com/vllm-project/vllm/pull/11784
[CI] Fix neuron CI and run offline tests by @liangfu in https://github.com/vllm-project/vllm/pull/11779
fix init error for MessageQueue when n_local_reader is zero by @XiaobingSuper in https://github.com/vllm-project/vllm/pull/11768
[Doc] Create a vulnerability management team by @russellb in https://github.com/vllm-project/vllm/pull/9925
[CI][CPU] adding build number to docker image name by @zhouyuan in https://github.com/vllm-project/vllm/pull/11788
[V1][Doc] Update V1 support for LLaVa-NeXT-Video by @ywang96 in https://github.com/vllm-project/vllm/pull/11798
[Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calculation by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11800
[doc] add doc to explain how to use uv by @youkaichao in https://github.com/vllm-project/vllm/pull/11773
[V1] Support audio language models on V1 by @ywang96 in https://github.com/vllm-project/vllm/pull/11733
[doc] update how pip can install nightly wheels by @youkaichao in https://github.com/vllm-project/vllm/pull/11806
[Doc] Add note to gte-Qwen2 models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11808
[optimization] remove python function call for custom op by @youkaichao in https://github.com/vllm-project/vllm/pull/11750
[Bugfix] update the prefix for qwen2 by @jiangjiadi in https://github.com/vllm-project/vllm/pull/11795
[Doc]Add documentation for using EAGLE in vLLM by @sroy745 in https://github.com/vllm-project/vllm/pull/11417
[Bugfix] Significant performance drop on CPUs with --num-scheduler-steps > 1 by @DamonFool in https://github.com/vllm-project/vllm/pull/11794
[Doc] Group examples into categories by @hmellor in https://github.com/vllm-project/vllm/pull/11782
[Bugfix] Fix image input for Pixtral-HF by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11741
[Misc] sort torch profiler table by kernel timing by @divakar-amd in https://github.com/vllm-project/vllm/pull/11813
Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… by @WangErXiao in https://github.com/vllm-project/vllm/pull/11824
Fixed docker build for ppc64le by @npanpaliya in https://github.com/vllm-project/vllm/pull/11518
[OpenVINO] Fixed Docker.openvino build by @ilya-lavrenov in https://github.com/vllm-project/vllm/pull/11732
[Bugfix] Add checks for LoRA and CPU offload by @jeejeelee in https://github.com/vllm-project/vllm/pull/11810
[Docs] reorganize sponsorship page by @simon-mo in https://github.com/vllm-project/vllm/pull/11639
[Bug] Fix pickling of ModelConfig when RunAI Model Streamer is used by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11825
[misc] improve memory profiling by @youkaichao in https://github.com/vllm-project/vllm/pull/11809
[doc] update wheels url by @youkaichao in https://github.com/vllm-project/vllm/pull/11830
[Docs] Update sponsor name: 'Novita' to 'Novita AI' by @simon-mo in https://github.com/vllm-project/vllm/pull/11833
[Hardware][Apple] Native support for macOS Apple Silicon by @wallashss in https://github.com/vllm-project/vllm/pull/11696
[torch.compile] consider relevant code in compilation cache by @youkaichao in https://github.com/vllm-project/vllm/pull/11614
[VLM] Reorganize profiling/processing-related code by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11812
[Doc] Move examples into categories by @hmellor in https://github.com/vllm-project/vllm/pull/11840
[Doc][4/N] Reorganize API Reference by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11843
[CI/Build][Bugfix] Fix CPU CI image clean up by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/11836
[Bugfix][XPU] fix silu_and_mul by @yma11 in https://github.com/vllm-project/vllm/pull/11823
[Misc] Move some model utils into vision file by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11848
[Doc] Expand Multimodal API Reference by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11852
[Misc]add some explanations for BlockHashType by @WangErXiao in https://github.com/vllm-project/vllm/pull/11847
[TPU][Quantization] TPU W8A8 by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11785
[Kernel][Triton][AMD] Use block size heuristic for avg 2.8x speedup for int8 models by @rasmith in https://github.com/vllm-project/vllm/pull/11698
[Docs] Add Google Cloud Meetup by @simon-mo in https://github.com/vllm-project/vllm/pull/11864
[CI] Turn on basic correctness tests for V1 by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/10864
treat do_lower_case in the same way as the sentence-transformers library by @maxdebayser in https://github.com/vllm-project/vllm/pull/11815
[Doc] Recommend uv and python 3.12 for quickstart guide by @mgoin in https://github.com/vllm-project/vllm/pull/11849
[Misc] Move print_*_once from utils to logger by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11298
[Doc] Intended links Python multiprocessing library by @guspan-tanadi in https://github.com/vllm-project/vllm/pull/11878
[perf]fix current stream by @youkaichao in https://github.com/vllm-project/vllm/pull/11870
[Bugfix] Override dunder methods of placeholder modules by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11882
[Bugfix] fix beam search input errors and latency benchmark script by @yeqcharlotte in https://github.com/vllm-project/vllm/pull/11875
[Doc] Add model development API Reference by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11884
[platform] Allow platform specify attention backend by @wangxiyuan in https://github.com/vllm-project/vllm/pull/11609
[ci]try to fix flaky multi-step tests by @youkaichao in https://github.com/vllm-project/vllm/pull/11894
[Misc] Provide correct Pixtral-HF chat template by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11891
[Docs] Add Modal to deployment frameworks by @charlesfrye in https://github.com/vllm-project/vllm/pull/11907
[Doc][5/N] Move Community and API Reference to the bottom by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11896
[VLM] Enable tokenized inputs for merged multi-modal processor by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11900
[Doc] Show default pooling method in a table by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11904
[torch.compile] Hide KV cache behind torch.compile boundary by @heheda12345 in https://github.com/vllm-project/vllm/pull/11677
[Bugfix] Validate lora adapters to avoid crashing server by @joerunde in https://github.com/vllm-project/vllm/pull/11727
[BUGFIX] Fix UnspecifiedPlatform package name by @jikunshang in https://github.com/vllm-project/vllm/pull/11916
[ci] fix gh200 tests by @youkaichao in https://github.com/vllm-project/vllm/pull/11919
[optimization] remove python function call for custom activation op by @cennn in https://github.com/vllm-project/vllm/pull/11885
[platform] support pytorch custom op pluggable by @wangxiyuan in https://github.com/vllm-project/vllm/pull/11328
Replace "online inference" with "online serving" by @hmellor in https://github.com/vllm-project/vllm/pull/11923
[ci] Fix sampler tests by @youkaichao in https://github.com/vllm-project/vllm/pull/11922
[Doc] [1/N] Initial guide for merged multi-modal processor by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11925
[platform] support custom torch.compile backend key by @wangxiyuan in https://github.com/vllm-project/vllm/pull/11318
[Doc] Rename offline inference examples by @hmellor in https://github.com/vllm-project/vllm/pull/11927
[Docs] Fix docstring in get_ip function by @KuntaiDu in https://github.com/vllm-project/vllm/pull/11932
[Doc] Docstring fix in benchmark_long_document_qa_throughput.py by @KuntaiDu in https://github.com/vllm-project/vllm/pull/11933
[Hardware][CPU] Support MOE models on x86 CPU by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/11831
[Misc] Clean up debug code in Deepseek-V3 by @Isotr0py in https://github.com/vllm-project/vllm/pull/11930
[Misc] Update benchmark_prefix_caching.py fixed example usage by @remimin in https://github.com/vllm-project/vllm/pull/11920
[Bugfix] Check that number of images matches number of <|image|> tokens with mllama by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/11939
[mypy] Fix mypy warnings in api_server.py by @frreiss in https://github.com/vllm-project/vllm/pull/11941
[ci] fix broken distributed-tests-4-gpus by @youkaichao in https://github.com/vllm-project/vllm/pull/11937
[Bugfix][SpecDecode] Adjust Eagle model architecture to align with intended design by @llsj14 in https://github.com/vllm-project/vllm/pull/11672
[Bugfix] fused_experts_impl wrong compute type for float32 by @shaochangxu in https://github.com/vllm-project/vllm/pull/11921
[CI/Build] Move model-specific multi-modal processing tests by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11934
[Doc] Basic guide for writing unit tests for new models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/11951
[Bugfix] Fix RobertaModel loading by @NickLucche in https://github.com/vllm-project/vllm/pull/11940
[Model] Add cogagent model support vLLM by @sixsixcoder in https://github.com/vllm-project/vllm/pull/11742
[V1] Avoid sending text prompt to core engine by @ywang96 in https://github.com/vllm-project/vllm/pull/11963
[CI/Build] Add markdown linter by @rafvasq in https://github.com/vllm-project/vllm/pull/11857
[Model] Initialize support for Deepseek-VL2 models by @Isotr0py in https://github.com/vllm-project/vllm/pull/11578
[Hardware][CPU] Multi-LoRA implementation for the CPU backend by @Akshat-Tripathi in https://github.com/vllm-project/vllm/pull/11100
[Hardware][TPU] workaround fix for MoE on TPU by @avshalomman in https://github.com/vllm-project/vllm/pull/11764
[V1][Core][1/n] Logging and Metrics by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11962
[Model] Support GGUF models newly added in transformers 4.46.0 by @Isotr0py in https://github.com/vllm-project/vllm/pull/9685

These notes run past the length kept in the archive. The rest is on the publisher’s page.