vllm: Assertion `!(srcMmaLayout && dstMmaLayout) && "Unexpected mma -> mma layout conversion"' failed.
When executing script examples/offline_inference_with_prefix.py, it will call context_attention_fwd from vllm.model_executor.layers.triton_kernel.prefix_prefill, which triggered the following error
python: /project/lib/Analysis/Allocation.cpp:40: std::pair<llvm::SmallVector<unsigned int>, llvm::SmallVector<unsigned int> > mlir::triton::getCvtOrder(mlir::Attribute, mlir::Attribute): Assertion `!(srcMmaLayout && dstMmaLayout) && "Unexpected mma -> mma layout conversion"' failed.
Platform :
- V100
- CUDA 12.0
- python 3.11.6
- vllm 0.3.0+cu120
- triton 2.1.0
- torch 2.1.2
related to #1669
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 2
- Comments: 27 (4 by maintainers)
Adding the following flag worked for me:
same issue on V100, any update to support V100?
same problem with Nvidia-v100.