openenclave: LVI mitigation build under Nix fails to find cstdlib

I’m attempting to package up openenclave for NixOS. I managed to get it building and running fine without LVI mitigations, but enabling them I’m running into some issues with clang.

openenclave-sgx> /build/source/tools/oeedger8r-cpp/src/parser.cpp:4:10/build/source/tools/oeedger8r-cpp/src/main.cpp:: 4:10fatal error::  'algorithm' file not foundfatal error:
openenclave-sgx> 'cstdlib' file not found
openenclave-sgx> #include <cstdlib>#include <algorithm>
openenclave-sgx>          ^~~~~~~~~         ^~~~~~~~~~~
openenclave-sgx> In file included from /build/source/tools/oeedger8r-cpp/src/lexer.cpp:4:
openenclave-sgx> /build/source/tools/oeedger8r-cpp/src/lexer.h:7:10: fatal error: 'cctype' file not found
openenclave-sgx> #include <cctype>

It seems like it cannot find the C++ standard headers, even though they should be included by default. I’m curious if there is something that is being done to modify the search path in the wrappers, because I can’t find anything to that note.

Nix derivation: https://gist.github.com/CitadelCore/17bd121b74c451737d2da24608f1b4ca Build log: https://gist.github.com/CitadelCore/5a17e3c2fac336316f241d452f116353

About this issue

Original URL
State: open
Created 3 years ago
Comments: 15 (8 by maintainers)

Most upvoted comments

What’s going on is that the invoke_compiler script has been generated to hard-code the compiler for the case when LVI mitigation flags are not passed in:

    if [ $lvi_mitigation == 0 ]; then
        /nix/store/d7c8whpgvxj7lzjjfwk96sr9fpqk0znh-clang-wrapper-8.0.1/bin/clang $@
    else

This causes clang to be invoked when /lvi_mitigation_bin/clang++ wrapper is called, resulting in missing include errors.

Retaining the the theme of the original snippet would fix the compile error: https://github.com/openenclave/openenclave/blob/a381bfe51de0178b1989fb0d2abbc7b8e7179fbe/scripts/lvi-mitigation/invoke_compiler#L26-L28

We need to invoke the actual compiler, and not the wrapper. The challenge is that under nix, the compilers won’t exist in /usr/bin.

anakrish on Feb 11, 2021

Yes, I would be fine with contributing my changes. Will open a pull request once I’ve got my derivation cleaned up.

RealityAnomaly on Feb 18, 2021

I attempted to handle that via substituteInPlace in my derivation:

substituteInPlace scripts/lvi-mitigation/invoke_compiler \
            --replace '/usr/bin/"$compiler"' '${clang}/bin/clang' \
            --replace 'export PATH=/bin:/usr/local:"$lvi_bin_path"' 'export PATH=$PATH:"$lvi_bin_path"'

That should just append $lvi_bin_path, and not do anything funky with /bin or /usr/local.

RealityAnomaly on Feb 11, 2021

You should be able to build it like this:

{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
    nativeBuildInputs = [(
        pkgs.callPackage (import ./default.nix { type = "sgx"; }) { }
    )];
}

Save my default.nix and that as shell.nix in a folder, and then run nix-shell. Note that I have included breakpointHook in my derivation for testing, so it will pause on an error instead of exiting, use Ctrl+C to exit.

RealityAnomaly on Feb 11, 2021