hip: [Issue] Incorrect mapping `nvcc` flags to `hipcc` flags

Problem Description

When there is an obvious nvcc flag that has 1-to-1 mapping to hipcc flag, hipify should do the right thing and convert the flags to hip flag. I think hipify does the correct job most of the time but I have found a case where it doesn’t, causing cryptic and strange compile errors and putting the burden on the cuda devs who have no idea what hipify is doing for the most part.

nvcc flags in question:

# nvcc
"-U__CUDA_NO_HALF_OPERATORS__", # <-- NVCC
"-U__CUDA_NO_HALF_CONVERSIONS__", # <-- NVCC

Expectation from hipify

# expected correct mapping, 1-to-1 
"-U__HIP_NO_HALF_OPERATORS__", # <-- ROCm/HIP FIX
"-U__HIP_NO_HALF_CONVERSIONS__", # <-- ROCm/HIP FIX

Actual mapping: Not performed. Causing the following to be injected for hipcc

"-D__HIP_NO_HALF_OPERATORS__", # <-- ROCm/HIP BUG
"-D__HIP_NO_HALF_CONVERSIONS__", # <-- ROCm/HIP BUG

Full Output

/opt/rocm-6.3.3/bin/hipcc -I/root/miniconda3/lib/python3.12/site-packages/torch/include -I/root/miniconda3/lib/python3.12/site-packages/torch/include/torch/csrc/api/include -I/root/miniconda3/lib/python3.12/site-packages/torch/include/THH -I/opt/rocm-6.3.3/include -I/root/miniconda3/lib/python3.12/site-packages/nvidia/cuda_runtime/include -I/root/miniconda3/include/python3.12 -I/root/miniconda3/include/python3.12 -c rock_kernel/q_gemm.hip -o build/temp.linux-x86_64-cpython-312/rock_kernel/q_gemm.o -fPIC -D__HIP_PLATFORM_AMD__=1 -DUSE_ROCM=1 -DHIPBLAS_V2 -DCUDA_HAS_FP16=1 -D__HIP_NO_HALF_OPERATORS__=1 -D__HIP_NO_HALF_CONVERSIONS__=1 -O3 -std=c++17 -U__CUDA_NO_HALF_OPERATORS__ -U__CUDA_NO_HALF_CONVERSIONS__ -D_GLIBCXX_USE_CXX11_ABI=1 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1016\" -DTORCH_EXTENSION_NAME=rockthem_kernel -D_GLIBCXX_USE_CXX11_ABI=1 --offload-arch=gfx900 --offload-arch=gfx906 --offload-arch=gfx908 --offload-arch=gfx90a --offload-arch=gfx942 --offload-arch=gfx1030 --offload-arch=gfx1100 --offload-arch=gfx1101 --offload-arch=gfx1102 -fno-gpu-rdc

Operating System

Ubuntu 22.04

CPU

AMD ZEN5 EPYC

GPU

MI300X

ROCm Version

6.3.3

ROCm Component

HIP

Steps to Reproduce

I have pushed reproducing code into a repo: https://github.com/ModelCloud/rockthem

git clone https://github.com/ModelCloud/rockthem
cd rockthem
pip install -e . --no-build-isolation -v 

Fix is quite simple: Correctly map nvcc to hip flags when 1-to-1 match is found or whenever applicable to alleviate cuda migration pain.

Image

Image

Image

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 3
  • Comments: 16 (9 by maintainers)

Commits related to this issue

Most upvoted comments

Hi @Qubitium - can you share the output of clang --version? While I am able to reproduce the hipcc cmd line, I initially hit an error where a compiler version check in py_envs/lib/python3.12/site-packages/torch/utils/cpp_extension.py failed. My clang’s version looks roughly like so: AMD clang version 19.0.0git ; the ‘0git’ tripped up the check as it expected an integer value. I assume you did not hit this, hence the question. Thanks,

So installing ubuntu 22.04 clang and doing rocm compilation absolutely destroyed my pip env. This likely separate bug unrelated to this.

Found that amd/rocm bundle it’s own clang.

(base) root@gpu-xl:~/rockthem# /opt/rocm-6.3.3/bin/amdclang --version
AMD clang version 18.0.0git (https://github.com/RadeonOpenCompute/llvm-project roc-6.3.3 25012 e5bf7e55c91490b07c49d8960fa7983d864936c4)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/rocm-6.3.3/lib/llvm/bin
Configuration file: /opt/rocm-6.3.3/lib/llvm/bin/clang.cfg

Correct, hipify does src -> src translation; it does not translate build flags; we currently do not provide a tool that translates the nvcc flags, however, we have started discussions around that.

hipify is both a tool and a process . When referring to the tool, it refers to one of hipify-clang, hipify-perl, or hipify-torch. However, it is often referred to simply as hipify. When referring to the process, it references the process of taking CUDA code and translating - or hipify’ing it - into HIP code.

hipcc is a thin compiler driver; it invokes clang or nvcc depending on the environment. It also may add include/library options for the target compiler . In the early days, hipcc did more than it does today. In the early days, clang itself was missing functionality and so it was baked into hipcc. By and large, that functionality has been lifted out of hipcc.

I understand your confusion. I often see see hipcc referred to as a compiler. It is not a compiler. E.g., hipcc unto itself cannot compile anything; it formulates the clang, or nvcc, invocation and then invokes the compiler. You mentioned that you did not have clang installed. Because your environment is ROCm 6.3.3 + mi300x, I assumed that you would have clang via that ROCm installation.

If interested hipify - https://github.com/ROCm/HIPIFY ; https://github.com/ROCm/hipify_torch hipcc - https://github.com/ROCm/llvm-project/tree/amd-staging/amd/hipcc

K / thanks much / I’ll take a look

-D__HIP_NO_HALF_OPERATORS__=1 is not mapped from the nvcc flag per se. It is explicitly added by https://github.com/pytorch/pytorch/blob/main/torch/utils/cpp_extension.py#L279 ; now, the next question is why is it explicitly added. I will loop in ROCm-Pytorch folks

@searlmc1

For a little context, the kernel I was testing and used in the reproducer is a kernel from the vLLM project and in-directly SGLang, since SGlang imports vLLM kernels. I also contribute to these projects.

This context is important becaue the kernel compiles correctly using vllm setup as they do not use pytorch’s cudaExension hook for compilation but cmake instead. This was another headache that I had where I was asking myself “vLLM” compiles correctly for pytroch, but I can’t?

Now the answer is becoming clear. Beause I, and most pytorch projects use from torch.utils import cpp_extension for cuda kernel compilation, the env generate by ROCm by pytorch is different vs cmake: the three flags you pointed out in pytorch commit:

https://github.com/pytorch/pytorch/blob/main/torch/utils/cpp_extension.py#L279

I think it will great if default flags in the cmake vs pytorch build cpp extension are sycned, as much as possible for the rocm eco system. This way, any compiler issues can be correctly pushed down to the developers. Otherwise, us devs are like headless chickens since hunting for compiler flag difference is the last thing on our minds or frankly, good at.

Using the reproducer, with a few extra setup steps and a tweak to workaround a compiler minimum version check, I am able to produce the hipcc cmd containing …-D__HIP_NO_HALF_OPERATORS__=1 -D__HIP_NO_HALF_CONVERSIONS__=1… as depicted in the screenshot above

Hi @Qubitium - thanks for reporting the issue. @emankov - can you look into this?