arrow: [CI][Python] Conda Python 3.10 jobs fail with UnicodeDecodeError due to gdb issue
Describe the bug, including details regarding any error messages, version, and platform.
The CI job AMD64 Conda Python 3.10 Without Pandas has started failing on test_gdb.py with several UnicodeDecodeError invalid continuation byte.
Example of job failures from different PRs and main:
- https://github.com/apache/arrow/actions/runs/14878658706/job/41781302772?pr=46261
- https://github.com/apache/arrow/actions/runs/14878500696/job/41780789650
- https://github.com/apache/arrow/actions/runs/14876650269/job/41775137379
The log is quite long so I am adding only one of the test failures but there are several so please check the output of the jobs:
______________________________ test_scalars_heap _______________________________
gdb_arrow = <pyarrow.tests.test_gdb.GdbSession object at 0x7f722c8add10>
def test_scalars_heap(gdb_arrow):
> check_heap_repr(gdb_arrow, "heap_null_scalar", "arrow::NullScalar")
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_gdb.py:728:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_gdb.py:242: in check_heap_repr
s = gdb.print_value(f"*{expr}")
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_gdb.py:143: in print_value
out = self.run_command(f"p {expr}")
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_gdb.py:137: in run_command
return self.wait_until_ready()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pyarrow.tests.test_gdb.GdbSession object at 0x7f722c8add10>
def wait_until_ready(self):
"""
Record output until the gdb prompt displays. Return recorded output.
"""
# TODO: add timeout?
while (not self.last_stdout_line.startswith(b"(gdb) ") and
self.proc.poll() is None):
block = self.proc.stdout.read(4096)
if self.verbose:
sys.stdout.buffer.write(block)
sys.stdout.buffer.flush()
block, sep, last_line = block.rpartition(b"\n")
if sep:
self.last_stdout.append(self.last_stdout_line)
self.last_stdout.append(block + sep)
self.last_stdout_line = last_line
else:
assert block == b""
self.last_stdout_line += last_line
if self.proc.poll() is not None:
raise IOError("gdb session terminated unexpectedly")
> out = b"".join(self.last_stdout).decode('utf-8')
E UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 46: invalid continuation byte
opt/conda/envs/arrow/lib/python3.10/site-packages/pyarrow/tests/test_gdb.py:122: UnicodeDecodeError
----------------------------- Captured stdout call -----------------------------
p *heap_null_scalar
_______________________________ test_array_data ________________________________
Component(s)
Continuous Integration, Python
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (15 by maintainers)
Commits related to this issue
- GH-46343: [CI] Avoid installing gdb 16.3 on python 3.10 jobs to fix CI — committed to raulcd/arrow by raulcd 2 years ago
- GH-46343: [CI] Avoid installing gdb 16.3 on python 3.10 jobs to fix CI (#46511) ### Rationale for this change The Python 3.10 jobs are currently failing due to conda gdb==16.3 being installed and ha... — committed to apache/arrow by raulcd 2 years ago
- GH-46343: [CI][Python] Remove workaround for gdb packaging issue In https://github.com/apache/arrow/pull/46511 we skipped installing gdb on conda-based Python 3.10 CI jobs, to work around https://git... — committed to pitrou/arrow by pitrou a year ago
- GH-46343: [CI][Python] Remove workaround for gdb packaging issue (#46848) ### Rationale for this change In https://github.com/apache/arrow/pull/46511 we skipped installing gdb on conda-based Python ... — committed to apache/arrow by pitrou a year ago
Never mind… I think I might have messed up some steps in the process. I followed your steps exactly and managed to reproduced it.
This is due to the libstdcxx printer (comes along with gcc) not being registered in the conda environment’s gdb.
I’ve been able to reproduce with:
Not entirely sure what is the error you were facing, could you share more details?