awkward: In dependent project, ak.layout fails to convert to ak::Content. Update pybind11!
I’ve been working on some fastjet bindings with an awkward1 interface based on pybind11 (I’m aware of pyjet and the SWIG bindings, but for various reasons, this was a better approach for me. I’m not linking them here because they’re early and ugly and not really ready for anyone else, but the repo can be found on my profile if it’s somehow helpful). I have an early version working on macOS, but when I moved to linux (ubuntu 18.04 for what it’s worth, but I’m working in a virtualenv), it throws a TypeError whenever I try to pass an array layout to c++. For a trivial function which just returns what it receives, on linux I get
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import awkward1 as ak
>>> import pybind11_awkward as pba
>>> pba.awkward_test(ak.Array([1,2,3]).layout)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: awkward_test(): incompatible function arguments. The following argument types are supported:
1. (arr: awkward::Content) -> awkward::Content
Invoked with: <NumpyArray format="l" shape="3" data="1 2 3" at="0x000001be97b0"/>
On macOS, I get:
Python 3.8.5 (default, Jul 21 2020, 10:48:26)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import awkward1 as ak
>>> import pybind11_awkward as pba
>>> pba.awkward_test(ak.Array([1,2,3]).layout)
In function
<NumpyArray format="l" shape="3" data="1 2 3" at="0x7ff17b283a00"/>
Other functions seem to work fine - it’s apparently something about passing awkward arrays. I’m using awkward 0.3.1.
I’m a bit stuck on how to proceed. I recognize there’s a huge phase space where I could have gone wrong, so I made a quick repository that reproduces the issue, and I tried to keep as much consistent as possible. It’s available here. You can install it with poetry install
, and then run test_simple.py
. (In principle, it should work with pip using PEP 517, but my build script isn’t quite right, sorry). I’m not 100% sure that this is actually an awkward1 issue (there are still some differences in the setups), but it seems like a reasonable place to start.
As always, help or suggestions are greatly appreciated! Thanks!
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 19 (19 by maintainers)
Commits related to this issue
- Update to pybind11 2.5.0 Fixes #483 — committed to raymondEhlers/awkward-1.0 by raymondEhlers 4 years ago
- Update to pybind11 2.5.0 (#485) Fixes #483 — committed to scikit-hep/awkward by raymondEhlers 4 years ago
Oh, okay—a platform-dependent error (worked in CI, failed on your system). I’m glad the new pybind11 fixes that because it would be tough to debug.
The next version will likely come out when Python 3.9 can be tested, which will be as soon as it’s added to Azure’s pipelines. (c.f. microsoft/azure-pipelines-tool-lib#77).
Sorry, I think I wasn’t super clear on this point: this issue was triggered just by passing arrays from python -> cpp. It also occurred trying to compile and run
dependent.cpp
on our cluster. So I don’t see any obvious additional functionality to add todependent.cpp
that will trigger this issue. Which makes this especially puzzling to me. In any case, if I think of a test as I keep developing my code, I’ll definitely open a PR.Sounds good - I look forward to the next release!
Thanks again to you and @veprbl for all of the help!
Absolutely. Is that the latest version? I can update it to whatever is necessary, as long as I don’t run into errors in CI (which would have to be addressed anyway—newer versions were supposed to be better, so if Awkward can’t be compiled against it, that’s something wrong with Awkward…)
Does updating pybind11 completely solve the problem? Everything’s fine after that?
The issue you faced, is that something that could be added to the dependent project test? The dependent project runs in CI, so if this is an issue that wasn’t fully treated before, I’d like to add it.
Thanks for looking into this, @veprbl!