hip: [Issue] Incorrect mapping `nvcc` flags to `hipcc` flags
Problem Description
When there is an obvious nvcc flag that has 1-to-1 mapping to hipcc flag, hipify should do the right thing and convert the flags to hip flag. I think hipify does the correct job most of the time but I have found a case where it doesn’t, causing cryptic and strange compile errors and putting the burden on the cuda devs who have no idea what hipify is doing for the most part.
nvcc flags in question:
# nvcc
"-U__CUDA_NO_HALF_OPERATORS__", # <-- NVCC
"-U__CUDA_NO_HALF_CONVERSIONS__", # <-- NVCC
Expectation from hipify
# expected correct mapping, 1-to-1
"-U__HIP_NO_HALF_OPERATORS__", # <-- ROCm/HIP FIX
"-U__HIP_NO_HALF_CONVERSIONS__", # <-- ROCm/HIP FIX
Actual mapping: Not performed. Causing the following to be injected for hipcc
"-D__HIP_NO_HALF_OPERATORS__", # <-- ROCm/HIP BUG
"-D__HIP_NO_HALF_CONVERSIONS__", # <-- ROCm/HIP BUG
Full Output
/opt/rocm-6.3.3/bin/hipcc -I/root/miniconda3/lib/python3.12/site-packages/torch/include -I/root/miniconda3/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/root/miniconda3/lib/python3.12/site-packages/torch/include/THH -I/opt/rocm-6.3.3/include -I/root/miniconda3/lib/python3.12/site-packages/nvidia/cuda_runtime/include -I/root/miniconda3/include/python3.12 -I/root/miniconda3/include/python3.12 -c rock_kernel/q_gemm.hip -o build/temp.linux-x86_64-cpython-312/rock_kernel/q_gemm.o -fPIC -D__HIP_PLATFORM_AMD__=1 -DUSE_ROCM=1 -DHIPBLAS_V2 -DCUDA_HAS_FP16=1 -D__HIP_NO_HALF_OPERATORS__=1 -D__HIP_NO_HALF_CONVERSIONS__=1 -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -D_GLIBCXX_USE_CXX11_ABI=1 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1016\" -DTORCH_EXTENSION_NAME=rockthem_kernel -D_GLIBCXX_USE_CXX11_ABI=1 --offload-arch=gfx900 --offload-arch=gfx906 --offload-arch=gfx908 --offload-arch=gfx90a --offload-arch=gfx942 --offload-arch=gfx1030 --offload-arch=gfx1100 --offload-arch=gfx1101 --offload-arch=gfx1102 -fno-gpu-rdc
Operating System
Ubuntu 22.04
CPU
AMD ZEN5 EPYC
GPU
MI300X
ROCm Version
6.3.3
ROCm Component
HIP
Steps to Reproduce
I have pushed reproducing code into a repo: https://github.com/ModelCloud/rockthem
git clone https://github.com/ModelCloud/rockthem
cd rockthem
pip install -e . --no-build-isolation -v
Fix is quite simple: Correctly map nvcc to hip flags when 1-to-1 match is found or whenever applicable to alleviate cuda migration pain.
(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
No response
Additional Information
No response
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 3
- Comments: 16 (9 by maintainers)
Commits related to this issue
- [ROCm] simple hipify of nvcc flags for CUDAExtension Fixes https://github.com/ROCm/hip/issues/3764. A user convenience. — committed to ROCm/pytorch by naromero77amd 2 years ago
- [ROCm] simple hipify of nvcc flags for CUDAExtension Fixes https://github.com/ROCm/hip/issues/3764. A user convenience. — committed to ROCm/pytorch by naromero77amd 2 years ago
- [ROCm] Fixes and improvements to CUDA->HIP flag conversion for CPP extensions (#149245) Fixes https://github.com/ROCm/hip/issues/3764. Fixes and improvements to CUDA->HIP flag conversion for CPP ext... — committed to pytorch/pytorch by naromero77amd 2 years ago
- [ROCm] Fixes and improvements to CUDA->HIP flag conversion for CPP extensions (#149245) Fixes https://github.com/ROCm/hip/issues/3764. Fixes and improvements to CUDA->HIP flag conversion for CPP ext... — committed to pytorch/pytorch by naromero77amd 2 years ago
- [ROCm] Fixes and improvements to CUDA->HIP flag conversion for CPP extensions (#149432) [ROCm] Fixes and improvements to CUDA->HIP flag conversion for CPP extensions (#149245) Fixes https://github.c... — committed to pytorch/pytorch by pytorchbot 2 years ago
So installing
ubuntu 22.04clangand doing rocm compilation absolutely destroyed my pip env. This likely separate bug unrelated to this.Found that amd/rocm bundle it’s own clang.
Correct, hipify does src -> src translation; it does not translate build flags; we currently do not provide a tool that translates the nvcc flags, however, we have started discussions around that.
hipify is both a tool and a process . When referring to the tool, it refers to one of hipify-clang, hipify-perl, or hipify-torch. However, it is often referred to simply as hipify. When referring to the process, it references the process of taking CUDA code and translating - or hipify’ing it - into HIP code.
hipcc is a thin compiler driver; it invokes clang or nvcc depending on the environment. It also may add include/library options for the target compiler . In the early days, hipcc did more than it does today. In the early days, clang itself was missing functionality and so it was baked into hipcc. By and large, that functionality has been lifted out of hipcc.
I understand your confusion. I often see see hipcc referred to as a compiler. It is not a compiler. E.g., hipcc unto itself cannot compile anything; it formulates the clang, or nvcc, invocation and then invokes the compiler. You mentioned that you did not have clang installed. Because your environment is ROCm 6.3.3 + mi300x, I assumed that you would have clang via that ROCm installation.
If interested hipify - https://github.com/ROCm/HIPIFY ; https://github.com/ROCm/hipify_torch hipcc - https://github.com/ROCm/llvm-project/tree/amd-staging/amd/hipcc
Attempting to get this into release/2.7. https://github.com/pytorch/pytorch/issues/149044#issuecomment-2734637320.
K / thanks much / I’ll take a look
@searlmc1
For a little context, the kernel I was testing and used in the reproducer is a kernel from the vLLM project and in-directly SGLang, since SGlang imports vLLM kernels. I also contribute to these projects.
This context is important becaue the kernel compiles correctly using vllm setup as they do not use pytorch’s cudaExension hook for compilation but
cmakeinstead. This was another headache that I had where I was asking myself “vLLM” compiles correctly for pytroch, but I can’t?Now the answer is becoming clear. Beause I, and most pytorch projects use
from torch.utils import cpp_extensionfor cuda kernel compilation, the env generate by ROCm by pytorch is different vscmake: the three flags you pointed out in pytorch commit:https://github.com/pytorch/pytorch/blob/main/torch/utils/cpp_extension.py#L279
I think it will great if default flags in the cmake vs pytorch build cpp extension are sycned, as much as possible for the rocm eco system. This way, any compiler issues can be correctly pushed down to the developers. Otherwise, us devs are like headless chickens since hunting for compiler flag difference is the last thing on our minds or frankly, good at.
Using the reproducer, with a few extra setup steps and a tweak to workaround a compiler minimum version check, I am able to produce the hipcc cmd containing …-D__HIP_NO_HALF_OPERATORS__=1 -D__HIP_NO_HALF_CONVERSIONS__=1… as depicted in the screenshot above
Hi @Qubitium - thanks for reporting the issue. @emankov - can you look into this?