sunpy: HECClient invalid token
Description
When using the HECClient, trying to list the names of the available tables raises a ValueError regarding an invalid token. Below is the traceback in the hopes it will be of some use:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-48-d2712761b82a> in <module>
----> 1 hc.select_table()
~/.bin/miniconda3/envs/presage/lib/python3.9/site-packages/sunpy/net/helio/hec.py in select_table(self)
257 >>> hc.select_table() # doctest: +SKIP
258 """
--> 259 tables = self.get_table_names()
260 table_list = [t[0] for t in tables if len(t[0]) > 0]
261 table_list.sort()
~/.bin/miniconda3/envs/presage/lib/python3.9/site-packages/sunpy/net/helio/hec.py in get_table_names(self)
234 """
235 results = self.hec_client.service.getTableNames()
--> 236 tables = votable_handler(etree.tostring(results))
237 return tables.array
238
~/.bin/miniconda3/envs/presage/lib/python3.9/site-packages/sunpy/net/helio/hec.py in votable_handler(xml_table)
47 fake_file = io.BytesIO()
48 fake_file.write(xml_table)
---> 49 votable = parse_single_table(fake_file)
50 for i in range(len(votable.array)):
51 item = votable.array[i][0]
~/.bin/miniconda3/envs/presage/lib/python3.9/site-packages/astropy/io/votable/table.py in parse_single_table(source, **kwargs)
183 kwargs['table_number'] = 0
184
--> 185 votable = parse(source, **kwargs)
186
187 return votable.get_first_table()
~/.bin/miniconda3/envs/presage/lib/python3.9/site-packages/astropy/utils/decorators.py in wrapper(*args, **kwargs)
533 warnings.warn(message, warning_type, stacklevel=2)
534
--> 535 return function(*args, **kwargs)
536
537 return wrapper
~/.bin/miniconda3/envs/presage/lib/python3.9/site-packages/astropy/io/votable/table.py in parse(source, columns, invalid, verify, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)
164 source,
165 _debug_python_based_parser=_debug_python_based_parser) as iterator:
--> 166 return tree.VOTableFile(
167 config=config, pos=(1, 1)).parse(iterator, config)
168
~/.bin/miniconda3/envs/presage/lib/python3.9/site-packages/astropy/io/votable/tree.py in parse(self, iterator, config)
3570 'GROUP': self._add_group}
3571
-> 3572 for start, tag, data, pos in iterator:
3573 if start:
3574 tag_mapping.get(tag, self._add_unknown_tag)(
ValueError: 395:1: not well-formed (invalid token)
Steps to Reproduce
from sunpy.net.helio import hec
hc = hec.HECClient()
print(hc.get_table_names())
System Details
General
#######
OS: Debian GNU/Linux 11 bullseye (Linux 5.10.0-9-amd64)
Arch : 64bit, ()
SunPy : 3.0.1
Required Dependices
###################
numpy: 1.20.3
parfive: 1.5.0
astropy: 4.2.1
Optional Dependices
###################
asdf: Missing, need asdf>=2.6.0; extra == "all"
beautifulsoup4: 4.10.0
dask: 2021.8.1
drms: 0.6.2
glymur: 0.9.4
h5netcdf: 0.11.0
h5py: 3.2.1
matplotlib: 3.4.2
pandas: 1.3.3
python-dateutil: 2.8.2
scikit-image: 0.18.1
scipy: 1.7.1
sqlalchemy: 1.4.22
tqdm: 4.62.2
zeep: 4.1.0
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 44 (30 by maintainers)
Commits related to this issue
- Fix #5633 Part of the XML response contains dangling tags at the end of text. Once these are removed, the helio response can then be correctly parsed. — committed to jaypmorgan/sunpy by jaypmorgan 3 years ago
Yeah, not a bug with SunPy as the following code also produces the inconsistent behaviour (
</helio:queryResponse></S:Body></S:Envelope>missing) on conda vs. pip:I’ll see if I can find the source!
@nabobalis Okay, so I’ve found one package that changes the result. It didn’t seem like zeep in the end.
In the environment that doesn’t work, I have a package called `requests-file @ file:///Users/ktietz/demo/mc3/conda-bld/requests-file_1629455781986/work’.
When I manually install `requests-file==1.5.1’. It resolves the problem.
Aha! That indeed works @Cadair, the bug no longer occurs.
@nabobalis I was able to replicate this (your list of URLS returns lots of “not well-formed”) on Linux and macOS. I created the environment with
conda create -n sunpy-latest python=3.8 sunpy. I was not able to replicate it on my development environment, which I keep up-to-date with the SunPy GitHub repository. Although, I suspect this issue is due to bug in a dependency. (I should probably be updating the Conda environment also as you see most of the dependencies are old!)Linux
General
OS: Ubuntu (20.04, Linux 5.4.0-89-generic) Arch: 64bit, (x86_64) sunpy: 3.1.2 Installation path: /home/cmb/miniconda3/envs/sunpy-latest/lib/python3.8/site-packages
Required Dependencies
packaging: 21.3 numpy: 1.21.4 astropy: 5.0 parfive: 1.5.0
Optional Dependencies
asdf: Missing, need asdf>=2.6.0; extra == “all” beautifulsoup4: 4.10.0 cdflib: Missing, need cdflib>=0.3.19; extra == “all” dask: 2021.11.2 drms: 0.6.2 glymur: 0.9.6 h5netcdf: 0.11.0 h5py: 3.6.0 matplotlib: 3.5.0 mpl-animators: 1.0.0 pandas: 1.3.4 python-dateutil: 2.8.2 reproject: Missing, need reproject; extra == “all” scikit-image: 0.18.3 scipy: 1.7.3 sqlalchemy: 1.4.27 tqdm: 4.62.3 zeep: 4.1.0
macOS
Everything the same as above except:
General
OS: Mac OS 10.16 Arch: 64bit, (i386) Installation path: /Users/conor/miniconda3/envs/sunpy-latest/lib/python3.8/site-packages
Development Environment (works fine)
General
OS: Mac OS 10.16 Arch: 64bit, (i386) sunpy: 4.0.dev81+gf7bdf22e5.d20211127 Installation path: /Users/conor/Documents/sunpy
Required Dependencies
astropy: 5.0 packaging: 20.7 parfive: 1.2.0 numpy: 1.19.4
Optional Dependencies
asdf: 2.7.1 beautifulsoup4: 4.9.3 cdflib: Missing, need cdflib>=0.3.19 dask: 2.30.0 drms: Missing, need drms>=0.6.1 glymur: 0.9.2 h5netcdf: 0.8.1 h5py: 3.1.0 matplotlib: 3.3.3 mpl-animators: 1.0.0 pandas: 1.1.5 python-dateutil: 2.8.1 reproject: 0.7.1 scikit-image: 0.17.2 scipy: 1.5.4 sqlalchemy: 1.3.20 tqdm: 4.54.1 zeep: 4.0.0