conan: [bug] % in profiles

Describe the bug

Not sure if this is a bug or something you can fix but here we go:

I set NINJA_STATUS in profile:

[buildenv]
NINJA_STATUS=[%f/%t %p :: %es]->

It displays correctly in log:

======== Input profiles ========
Profile host:
...
[buildenv]
CMAKE_EXPORT_COMPILE_COMMANDS=ON
NINJA_STATUS=[%f/%t %p :: %es]->

But then, during build its broken

message(STATUS ------------NINJA_STATUS-----$ENV{NINJA_STATUS}-----------------)

becomes:

-- ------------NINJA_STATUS-----[t : es]->-----------------

And all the logs look like this:

[t : es]->Building CXX ....

This happens on Windows (not on Linux).

So my suggestion is, that the % characters are used for env variable resolutions like in cmd (e.g. %PATH%).

Workaround: Escape the % chars.

[buildenv]
NINJA_STATUS=[%%f/%%t %%p :: %%es]->

How to reproduce it

No response

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Comments: 18 (15 by maintainers)

Most upvoted comments

The problem seems to be more common than previously known. It also exists on Linux in a similar way. If you add something like this

[buildenv]
FOO_LOCATION=$HOME/foo

to your profile, the variable will be /home/<user>/foo when doing conan build ., but the generated CMakePresets.json contains:

            "environment": {
                "FOO_LOCATION": "$HOME/foo"
            },

This leads to the same effect, that you get different env variables depending on how you use conan.

Should Conan assume that env-vars being written to CMake presets environment have been escaped and de-escape them?

I don’t think that’s possible in a reliable way. If you get escaped sequences like %%e, you could “deescape” and put %e into the CMakePresets.json, but what do you do for something that’s really interpreted like %PATH%? You would need to manually expand this for CMake. And it would not adapt automatically if it changes, different to the Windows Cmd behaviour.

Is this even documented somewhere, that Windows Cmd and Bash syntax is used here as a special case? For me, this was completely unexpected when I first saw this. I expected that the env variables just get passed as-is. For me, that Windows Cmd is used seems like an implementation detail and it could also be directly set from Python together with subprocess or Windows PowerShell or similar.

For Linux and bash, these variables only get expanded as they are inside double quotes, single quotes would not have this behaviour.

Even though this not solved anytime soon, would you consider reopening it? Just to let everyone know there still are problems here.

Sure, lets re-open. To be clear, I haven’t really given up, I have been trying to find other possible approaches to this, but nothing reasonable so far.

I am labeling it as a “question” though, this is not necessarily a bug, but expected behavior from Conan, but an unfortunate conflict of technologies, the format of the Ninja status var, and Windows terminal expansion

To complete the information for other readers too:

It is possible to do in tool-requires recipe:

def package_info(self):
        # This gets passed correctly on Linux, but not on Windows...
        v =  "[%f/%t %p :: %es]->" if self.settings.os != "Windows" else  "[%%f/%%t %%p :: %%es]->"
        self.buildenv_info.define("NINJA_STATUS", v)

It is also possible to do in profiles something like:

[buildenv]
{% if platform.system() == "Windows" %}
NINJA_STATUS=[%%f/%%t %%p :: %%es]->
{% else %}
NINJA_STATUS=[%f/%t %%p :: %es]->
{% endif %}

To have something that works in Windows and other platforms.

But it seems this wouldn’t be a bug, but something intrinsic to the env-vars processing by the OS, and Conan cannot do anything, it cannot automatically escape env-vars contents, as it would very easily be breaking and problematic (recall that it might also need to escape $ in non-windows systems).

I am closing this ticket then as not a bug. Thanks for the feedback!

Thanks for the clarification @aagor

yes, it is the buildenv.bat environment scripts, that are executed in a subprocess (which will be a cmd one). The replacement indeed happens there, and this is why Conan can’t do much about it.

It could be argued that Conan could escape the % when creating a xxxxx.bat file, but I am afraid this could easily be breaking, as it is a valid use case to define vars with some value such as SOMETHING;%SOMEVAR%, which relies exactly on the substitution of %SOMEVAR% env-var happening, and it seems impossible to know and tell the difference.

This is why I don’t see a clear fix here, it seems an unfortunate alignment of different tools such as Ninja and Windows bat files.