xmlschema: `test_normalize_url*` failures with Python 3.10.15, 3.11.10

$ tox -e py310
py310: install_deps> python -I -m pip install 'elementpath<5.0.0,>=4.4.0' jinja2 lxml
.pkg: install_requires> python -I -m pip install setuptools
.pkg: _optional_hooks> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_sdist> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_wheel> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: prepare_metadata_for_build_wheel> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: build_sdist> python /usr/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py310: install_package_deps> python -I -m pip install 'elementpath<5.0.0,>=4.4.0'
py310: install_package> python -I -m pip install --force-reinstall --no-deps /tmp/xmlschema/../.tox/xmlschema/.tmp/package/1/xmlschema-3.4.0.tar.gz
py310: commands[0]> python -m unittest
..................................................................................................................................................................................................................................................................................................................................F...F....ssss........................................................................................................................................................................ssss.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
======================================================================
FAIL: test_normalize_url_slashes (tests.test_locations.TestLocations)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/xmlschema/tests/test_locations.py", line 322, in test_normalize_url_slashes
    self.assertRegex(normalize_url('////root/dir1/schema.xsd'),
AssertionError: Regex didn't match: 'file:////root/dir1/schema.xsd' not found in 'file:///root/dir1/schema.xsd'

======================================================================
FAIL: test_normalize_url_with_base_unc_path (tests.test_locations.TestLocations)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/xmlschema/tests/test_locations.py", line 285, in test_normalize_url_with_base_unc_path
    self.assertEqual(url, 'file:////filer01/MY_HOME/dev/XMLSCHEMA/test.xsd')
AssertionError: 'file:///filer01/MY_HOME/dev/XMLSCHEMA/test.xsd' != 'file:////filer01/MY_HOME/dev/XMLSCHEMA/test.xsd'
- file:///filer01/MY_HOME/dev/XMLSCHEMA/test.xsd
+ file:////filer01/MY_HOME/dev/XMLSCHEMA/test.xsd
?      +


----------------------------------------------------------------------
Ran 1388 tests in 29.941s

FAILED (failures=2, skipped=8)
py310: exit 1 (30.67 seconds) /tmp/xmlschema> python -m unittest pid=230473
  py310: FAIL code 1 (40.38=setup[9.71]+cmd[30.67] seconds)
  evaluation failed :( (40.49 seconds)

I’m getting the same failure with CPython 3.10.15 and 3.11.10, but not PyPy3.10 7.3.17 — so I suspect it’s one of the security backports in the latest round.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 1
  • Comments: 16 (14 by maintainers)

Commits related to this issue

Most upvoted comments

Thanks. If you need any more testing over this in the future, feel free to ping me.

Yeah, I can confirm that on top of 059fd3bfb305809d5ebdede1bd68dff9dbaf3348, both pre-patch and post-patch Python versions pass all tests.

I confirm that https://github.com/sissaschool/xmlschema/commit/059fd3bfb305809d5ebdede1bd68dff9dbaf3348 fixes the failing test on 3.10.15. I didn’t check 3.11.10, but I assume it is fixed as well. Thank you very much!

Python 3.8.20, 3.9.20, 3.10.15 and 3.11.10 all fail on one test:

======================================================================
FAIL: test_get_uri (tests.test_locations.TestLocations.test_get_uri)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/xmlschema/tests/test_locations.py", line 214, in test_get_uri
    self.assertNotEqual(urlsplit(url).geturl(), url)
AssertionError: '////root/dir1/schema.xsd' == '////root/dir1/schema.xsd'

----------------------------------------------------------------------

3.12.6 and 3.13.0rc2 pass.

Ok, so I’m seeing two differences:

  1. In patched versions (3.12.4+, 3.11.10, etc.): urlunsplit(("", "", r"//filer01/MY_HOME/", "", "")) changed from: //filer01/MY_HOME/ to: ////filer01/MY_HOME/
  2. in 3.12+: PureWindowsPath("//foo") changed from: \foo to: \\foo

So both changes combined make PureWindowsPath(urlunsplit(("", "", r"//filer01/MY_HOME/", "", ""))) yield:

< 3.12 unpatched: \\filer01\MY_HOME\ < 3.12 patched: \filer01\MY_HOME 3.12+ unpatched: \\filer01\MY_HOME\ 3.12+ patched: \\\\filer01\MY_HOME

Script used to reproduce this
import os
import os.path

from pathlib import PureWindowsPath
from unittest.mock import patch
from urllib.parse import urlsplit, quote_from_bytes, unquote, urlunsplit

path = r"dev\XMLSCHEMA\test.xsd"
base_url = PureWindowsPath("\\\\filer01\\MY_HOME\\").as_uri()
base_url_host_in_path = base_url.replace('file://', 'file:////')

url_parts = urlsplit(path)
base_url_parts = urlsplit(base_url_host_in_path)
base_path = PureWindowsPath(unquote(urlunsplit(("", base_url_parts.netloc, base_url_parts.path, "", ""))))

path = base_path.joinpath(path)
norm_path = PureWindowsPath(os.path.normpath(str(path)))
norm_uri = "file://" + quote_from_bytes(os.fsencode(norm_path.as_posix()))

print(PureWindowsPath(urlunsplit(("", "", r"//filer01/MY_HOME/", "", ""))))

And I’ve just confirmed that downgrading to 3.10.14 makes the test pass. So yeah, python/cpython@2fa5d706843476cf43ff3b431e3c4a8c34737413 seems like a good candidate.