syncserver: Potential "audience mismatch" error when running behind TLS-terminating reverse proxy

Hi,

I am running syncserver in docker with user authentication via the default https://account.mozilla.com server. Today I upgraded from an ancient version and Firefox does not longer sync and instead immediately prompts to reconnect to sync.

I tried to identify the issue suspecting db schema changes but eventually tracked it down to the syncserver return 401 Unauthorized responses although my login is clearly valid on https://accounts.firefox.com. Unfortunately the logging output of the syncserver is very sparse, revealing nothing about the failing requests while the about:sync-log is so verbose that I had trouble even finding the 401.

I also tried native serving via make serve instead of docker for the same result. Interestingly make test runs just fine while I am not able to make the tests pass on either my docker or native server which both fail with a traceback ultimately pointing to requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http://localhost:5000/token/1.0/sync/1.5 Is the test suite supposed to fail on real setups as the user is invalid?

To run the tests I ran: /local/bin/python -m syncstorage.tests.functional.test_storage --use-token-server http://localhost:5000/token/1.0/sync/1.5/ as shown in the Makefile.

Is there any way to increase logging for syncserver to investigate this issue? I have no idea where to start to look for the offending code =/

Let me know if there is anything more I can provide to narrow down the issue. If desired I can post an example sync-log but I am not sure if that helps.

EDIT: After changing the log level in syncserver/__init__.py:136 to Info, I found the following:

INFO:mozsvc.metrics:{"code": 401, "request_time": 0.0019838809967041016, "remoteAddressChain": ["172.17.0.1"], "agent": "python-requests/2.13.0", "token.assertion.audience_mismatch_error": 1, "token.assertion.verify_failure": 1, "tokenserver.assertion.verify": 0.00024199485778808594, "path": "http://<redacted>/token/1.0/sync/1.5", "method": "GET"}

which indicates a problem with the audience value? I saw that this value is configurable in the tests but I am not sure what is expected here, any ideas?