rattler-build: rattler doesn't set rpath for binaries

The following recipe fails on linux-64

# yaml-language-server: $schema=https://raw.githubusercontent.com/prefix-dev/recipe-format/main/schema.json

context:
  name: mssql-tools
  version: '18.2.1.1'

package:
  name: ${{ name }}
  version: ${{ version }}

source:
  - url: https://packages.microsoft.com/rhel/7/prod/${{ name }}18-${{ version }}-1.x86_64.rpm
    sha256: 98758f29f1b1aad13c5ce32a5cf1e849d35a97054d030ee5dccdab6aefd2aef9

build:
  number: 0
  script:
    - bsdtar xf mssql-tools18-${{ version }}-1.x86_64.rpm
    - mkdir -p $PREFIX/bin
    - cp opt/mssql-tools18/bin/* $PREFIX/bin

requirements:
  build:
    - libarchive

tests:
  - package_contents:
      bin:
        - sqlcmd
        - bcp
  - script:
      - patchelf --print-rpath $CONDA_PREFIX/bin/bcp | grep -q \$ORIGIN/../lib
      - patchelf --print-rpath $CONDA_PREFIX/bin/sqlcmd | grep -q \$ORIGIN/../lib
    requirements:
      run:
        - patchelf

The corresponding conda-build recipe succeeds with conda-build

package:
  name: mssql-tools
  version: '18.2.1.1'

source:
  - url: https://packages.microsoft.com/rhel/7/prod/mssql-tools18-18.2.1.1-1.x86_64.rpm
    sha256: 98758f29f1b1aad13c5ce32a5cf1e849d35a97054d030ee5dccdab6aefd2aef9

build:
  number: 0
  script:
    # bsdtar not relevant because conda-build extracts it automatically
    - mkdir -p $PREFIX/bin
    - cp opt/mssql-tools18/bin/* $PREFIX/bin

test:
  commands:
  - patchelf --print-rpath $CONDA_PREFIX/bin/bcp | grep -q \$ORIGIN/../lib
  - patchelf --print-rpath $CONDA_PREFIX/bin/sqlcmd | grep -q \$ORIGIN/../lib
  requires:
    - patchelf

Would be nice to have this example as a unit test

About this issue

  • Original URL
  • State: closed
  • Created 5 months ago
  • Comments: 16 (5 by maintainers)

Most upvoted comments

I think it is closely related to the rpaths option mentioned in #527:

  # settings for shared libraries and executables
  dynamic_linking:
    # linux only, list of rpaths (was rpath)
    rpaths: [path] (defaults to ['lib/'])

But looks like we need to force the $PREFIX/lib as an RPATH into the binary if no RPath is set to make these work.

Absolutely.

It was already on my list so I will follow up with a PR.

Argh, ok, so these binaries don’t have an rpath and that’s why our patching doesn’t work… and also won’t work with our current Rust implementation (not such a big deal since we can still use patchelf).

But looks like we need to force the $PREFIX/lib as an RPATH into the binary if no RPath is set to make these work.

@orhun See the failing tests in https://github.com/prefix-dev/rattler-build/actions/runs/7642259861/job/20821467331?pr=532 #532

my local logs: (from osx-arm64 as build-platform)

rattler-build build -r r/recipe.yaml --target-platform=linux-64
[...]
✔ Successfully updated the environment
Work dir: "/private/tmp/rattler-build-rpath/output/bld/rattler-build_mssql-tools_1706108677/work"
Build script: "/private/tmp/rattler-build-rpath/output/bld/rattler-build_mssql-tools_1706108677/work/conda_build.sh"
+ bsdtar xf mssql-tools18-18.2.1.1-1.x86_64.rpm
+ mkdir -p /private/tmp/rattler-build-rpath/output/bld/rattler-build_mssql-tools_1706108677/host_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/bin
+ cp opt/mssql-tools18/bin/bcp opt/mssql-tools18/bin/sqlcmd /private/tmp/rattler-build-rpath/output/bld/rattler-build_mssql-tools_1706108677/host_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/bin
Copying done!
2024-01-24T15:04:40.303967Z  WARN rattler_build::linux::link: 

builtin relink failed for /var/folders/rz/q1r4tv0n6ll38q18fbk419mc0000gn/T/mssql-toolsokjg22/bin/bcp: rpath not found in dynamic section. Please file an issue on Github!


patchelf for "/var/folders/rz/q1r4tv0n6ll38q18fbk419mc0000gn/T/mssql-toolsokjg22/bin/bcp": ""
2024-01-24T15:04:40.328363Z  WARN rattler_build::linux::link: 

builtin relink failed for /var/folders/rz/q1r4tv0n6ll38q18fbk419mc0000gn/T/mssql-toolsokjg22/bin/sqlcmd: rpath not found in dynamic section. Please file an issue on Github!


patchelf for "/var/folders/rz/q1r4tv0n6ll38q18fbk419mc0000gn/T/mssql-toolsokjg22/bin/sqlcmd": ""
Relink done!

Files in package:

  - bin/bcp
  - bin/sqlcmd
  - info/about.json
  - info/hash_input.json
  - info/index.json
  - info/paths.json
  - info/recipe/recipe.yaml
  - info/recipe/rendered_recipe.yaml
  - info/recipe/variant_config.yaml
  - info/tests/1/run_test.sh
  - info/tests/1/test_time_dependencies.json
Creating target folder "/private/tmp/rattler-build-rpath/output/linux-64"

Output: mssql-tools-18.2.1.1-hb0f4dca_0

Variant configuration:
┌─────────────────┬──────────┐
│ Variant         ┆ Version  │
╞═════════════════╪══════════╡
│ target_platform ┆ linux-64 │
└─────────────────┴──────────┘

Build dependencies:
╭─────────────────┬────────────┬────────────┬───────────────┬─────────────┬────────────╮
│ Package         ┆ Spec       ┆ Version    ┆ Build         ┆ Channel     ┆       Size │
╞═════════════════╪════════════╪════════════╪═══════════════╪═════════════╪════════════╡
│ libarchive      ┆ libarchive ┆ 3.7.2      ┆ hcacb583_1    ┆ conda-forge ┆ 765.44 KiB │
│ bzip2           ┆            ┆ 1.0.8      ┆ h93a5062_5    ┆ conda-forge ┆ 119.46 KiB │
│ ca-certificates ┆            ┆ 2023.11.17 ┆ hf0a4a13_0    ┆ conda-forge ┆ 150.82 KiB │
│ icu             ┆            ┆ 73.2       ┆ hc8870d7_0    ┆ conda-forge ┆  11.44 MiB │
│ libcxx          ┆            ┆ 16.0.6     ┆ h4653b0c_0    ┆ conda-forge ┆   1.11 MiB │
│ libiconv        ┆            ┆ 1.17       ┆ h0d3ecfb_2    ┆ conda-forge ┆ 660.61 KiB │
│ libxml2         ┆            ┆ 2.12.4     ┆ h0d0cfa8_1    ┆ conda-forge ┆ 574.75 KiB │
│ libzlib         ┆            ┆ 1.2.13     ┆ h53f4e23_5    ┆ conda-forge ┆  46.97 KiB │
│ lz4-c           ┆            ┆ 1.9.4      ┆ hb7217d7_0    ┆ conda-forge ┆ 137.88 KiB │
│ lzo             ┆            ┆ 2.10       ┆ h642e427_1000 ┆ conda-forge ┆ 153.55 KiB │
│ openssl         ┆            ┆ 3.2.0      ┆ h0d3ecfb_1    ┆ conda-forge ┆   2.72 MiB │
│ xz              ┆            ┆ 5.2.6      ┆ h57fd34a_0    ┆ conda-forge ┆ 230.17 KiB │
│ zstd            ┆            ┆ 1.5.5      ┆ h4f39d0f_0    ┆ conda-forge ┆ 391.12 KiB │
╰─────────────────┴────────────┴────────────┴───────────────┴─────────────┴────────────╯




Running tests
Removing previously cached package "/Users/pavel/Library/Caches/rattler/cache/pkgs/mssql-tools-18.2.1.1-hb0f4dca_0"
Creating test environment in "/private/tmp/rattler-build-rpath/output/bld/rattler-build_mssql-tools_1706108677/work/test"
Collecting tests from "/Users/pavel/Library/Caches/rattler/cache/pkgs/mssql-tools-18.2.1.1-hb0f4dca_0"
test "/Users/pavel/Library/Caches/rattler/cache/pkgs/mssql-tools-18.2.1.1-hb0f4dca_0/info/tests/1"

Resolving environment for:

  Platform: linux-64
  Channels: 
   - /private/tmp/rattler-build-rpath/output
   - conda-forge
  Specs:
   - patchelf
   - mssql-tools ==18.2.1.1 hb0f4dca_0


✔ /private/tmp/rattler [00:00:00] Using cache

┌───────────────┬──────────┬─────────────┬─────────────┬────────────┐
│ Package       ┆ Version  ┆ Build       ┆ Channel     ┆ Size       │
╞═══════════════╪══════════╪═════════════╪═════════════╪════════════╡
│ mssql-tools   ┆ 18.2.1.1 ┆ hb0f4dca_0  ┆ output      ┆ 243.34 KiB │
│ patchelf      ┆ 0.17.2   ┆ h58526e2_0  ┆ conda-forge ┆ 91.84 KiB  │
│ libgcc-ng     ┆ 13.2.0   ┆ h807b86a_3  ┆ conda-forge ┆ 755.50 KiB │
│ _libgcc_mutex ┆ 0.1      ┆ conda_forge ┆ conda-forge ┆ 2.50 KiB   │
│ _openmp_mutex ┆ 4.5      ┆ 2_gnu       ┆ conda-forge ┆ 23.07 KiB  │
│ libgomp       ┆ 13.2.0   ┆ h807b86a_3  ┆ conda-forge ┆ 411.95 KiB │
│ libstdcxx-ng  ┆ 13.2.0   ┆ h7e041cc_3  ┆ conda-forge ┆ 3.66 MiB   │
└───────────────┴──────────┴─────────────┴─────────────┴────────────┘
✔ Successfully updated the environment
Testing commands:
+ patchelf --print-rpath /private/tmp/rattler-build-rpath/output/bld/rattler-build_mssql-tools_1706108677/work/test/bin/bcp
+ grep -q '$ORIGIN/../lib'
/var/folders/rz/q1r4tv0n6ll38q18fbk419mc0000gn/T/rattler-test-C0Wkfw.sh: line 19: /private/tmp/rattler-build-rpath/output/bld/rattler-build_mssql-tools_1706108677/work/test/bin/patchelf: cannot execute binary file
Error:   × failed to run test