astroquery: 504 Server Error in X-Match query

I have been using astroquery’s wrapper around X-Match to run crossmatches against big surveys, and strangely it fails for certain surveys which is new behaviour. I see the following error:

d = XMatch.query(cat1=open(masterlist),cat2='vizier:%s' % code, max_distance=angular_sep * u.arcsec,colRA1='ra',colDec1='dec') File "/Users/moss/anaconda/lib/python3.6/site-packages/astroquery/xmatch/core.py", line 61, in query get_query_payload=get_query_payload) File "/Users/moss/anaconda/lib/python3.6/site-packages/astroquery/xmatch/core.py", line 95, in query_async response.raise_for_status() File "/Users/moss/anaconda/lib/python3.6/site-packages/requests/models.py", line 939, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 504 Server Error: Gateway Time-out for url: http://cdsxmatch.u-strasbg.fr/xmatch/api/v1/sync

I checked the self.TIMEOUT parameter passed to the requests call, which was 300, but I tried setting it to much higher and that made no difference to the error, it still timed out and refused to continue. This was specifically for querying against the ALLWISE catalogue, whereas for other catalogues (e.g. SUMSS, XMMDR8, etc) it finishes within about a minute. It’s possible this is not an astroquery issue but traces back to X-Match, but I found when I did a crossmatch on the X-Match website (against 2MASS, an equally big catalogue) it took a long time but actually did seem to work okay. Wondering if there is any kind of fix possible on the astroquery side? Thanks in advance!

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 15 (6 by maintainers)

Most upvoted comments

Two small updates.

  1. To give an idea of HEALPix conversion runtime (presumably it could be optimised way better): Reading catalogues... 706296 633224 Constructing coordinate list.. Calculating HEALPix using pool... Adding HEALPix info to table... Total time: 14.56 min

  2. Breaking the catalogue into 50k chunks worked, and ALLWISE is now returning a response. Thanks @fxpineau!