ctypesgen: String seems incorrect
I ran into some problems, and need help with them (version 1.0.1-1.0.2 on python 3.7.4):
File "/home/alanr/monitor/src/cma/AssimCtypes.py", line 1323, in <module>
proj_class_new.argtypes = [gsize, String]
TypeError: item 2 in _argtypes_ passes a union by value, which is unsupported
The C source in question looks like this:
gpointer
proj_class_new(gsize objsize, ///< Size of object to be allocated
const char * static_classname) ///< Static string giving name of class
The corresponding header file looks like this:
WINEXPORT gpointer proj_class_new(gsize objsize, const char * static_classname);
The second argument is clearly a pointer, not a call by value - of anything. The generated Python looks like this:
1320 # /home/alanr/monitor/src/include/proj_classes.h: 28
1321 if hasattr(_libs['libassimilationclientlib.so'], 'proj_class_new'):
1322 proj_class_new = _libs['libassimilationclientlib.so'].proj_class_new
1323 proj_class_new.argtypes = [gsize, String]
1324 proj_class_new.restype = gpointer
String is a generated class, which looks like this:
339 class String(MutableString, Union):
340
341 _fields_ = [("raw", POINTER(c_char)), ("data", c_char_p)]
342
343 def __init__(self, obj=""):
344 if isinstance(obj, (bytes, UserString)):
345 self.data = bytes(obj)
346 else:
347 self.raw = obj
As an experiment, I removed the Union from the String definition, and then it seemed to work a bit better…
A few milliseconds later, I got this error: File “/home/alanr/monitor/src/cma/AssimCtypes.py”, line 1028, in <module> (‘priv’, POINTER(GSourcePrivate)), TypeError: second item in fields tuple (index 11) must be a C type
The code in question looks like this:
1015 struct__GSource._fields_ = [
1016 ('callback_data', gpointer),
1017 ('callback_funcs', POINTER(GSourceCallbackFuncs)),
1018 ('source_funcs', POINTER(GSourceFuncs)),
1019 ('ref_count', guint),
1020 ('context', POINTER(GMainContext)),
1021 ('priority', gint),
1022 ('flags', guint),
1023 ('source_id', guint),
1024 ('poll_fds', POINTER(GSList)),
1025 ('prev', POINTER(GSource)),
1026 ('next', POINTER(GSource)),
1027 ('name', String),
1028 ('priv', POINTER(GSourcePrivate)),
1029 ]
It appears that String is not a legal field type. So, this is a problem…
– Alan
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 22 (2 by maintainers)
Links to this issue
Commits related to this issue
- Locked in on Python 3.6 for release 2.0.x. This is because ctypesgen generates Python code which Python 3.7.x (for x greater than some value), and Python 3.8 doesn't think works correctly. You can re... — committed to assimilation/assimilation-official by Alan-R 7 years ago
- bpo-16575: Add checks for unions passed by value to functions. (GH-16799) — committed to python/cpython by vsajip 7 years ago
- Downgrade Python to avoid https://github.com/davidjamesca/ctypesgen/issues/77 — committed to BlockSigner/wowpng by betatim 7 years ago
- Changes to allow running under Python 3.6.9. The capture_output paramter was introduced in Python 3.7. There is a PPA for Ubuntu 18 that allows 3.7.9 to be installed, however this has other issues d... — committed to dazzag24/wowpng by dazzag24 7 years ago
Strange. Just tried this with Py 3.7.5 and it seems to have had no problems, including the return value being mutable as expected.
On the other hand: Any thoughts on replacing this interface with something less wonky?
I propose the following changes:
These changes should make the generated code:
Update: In case this isn’t already apparent, I was hoping for a comment from at least some ctypesgen devs/users: @Alan-R , @neteler , @betatim , @dazzag24 , …
The GRASS GIS project is using ctypesgen for a long time (thanks!) but now facing problems as it fails with Python 3.7.6+.
While we could work-around it would be nice to see this fixed here. Please see the related bug report:
https://trac.osgeo.org/grass/ticket/4018
I have actually wondered about this bit of string handling done by ctypesgen. I am not entirely sure if this special handling is currently necessary or if the original purpose it served is warranted.
Taking a closer look at this, it seems that, as an argument, the “String” class at best helps to automatically convert various things into a byte array. I personally think that the bytes type should be required to be used at the c-interface when a “const char *” parameter is required – let ctypes handle the automatic conversion to the c_char_p; there is no real sense in trying to reimplement the automatic type conversion already happening in ctypes. Other auto-conversion I think are misplaced, especially in light of the very specific differences between str and bytes. (It is a one-line change to move to my suggested pattern)
As a return value, it appears that the intent was to allow non-const char* types to be returned as mutable strings. Playing around with this—it does not seem that this actually works correctly. Replacing this with a simpler ctypes.Union appears to work. Since I don’t currently have >py36, I’d appreciate someone else trying this for me for newer python versions to see what happens: