ArchiveBox: Bug: `UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 110372: surrogates not allowed` when trying to render unprintable filesystem path in view
Describe the bug
Access impossible due to unicode issue
Steps to reproduce
I dont know what exactly changed/happened.
Screenshots or log output
Mar 06 19:18:07 archivebox archivebox[864]: "GET /public/ HTTP/1.1" 500 145
Mar 06 19:29:04 archivebox archivebox[864]: Internal Server Error: /public/
Mar 06 19:29:04 archivebox archivebox[864]: Traceback (most recent call last):
Mar 06 19:29:04 archivebox archivebox[864]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:29:04 archivebox archivebox[864]: response = get_response(request)
Mar 06 19:29:04 archivebox archivebox[864]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:29:04 archivebox archivebox[864]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/base.py", line 204, in _get_resp>
Mar 06 19:29:04 archivebox archivebox[864]: response = response.render()
Mar 06 19:29:04 archivebox archivebox[864]: ^^^^^^^^^^^^^^^^^
Mar 06 19:29:04 archivebox archivebox[864]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/template/response.py", line 105, in render
Mar 06 19:29:04 archivebox archivebox[864]: self.content = self.rendered_content
Mar 06 19:29:04 archivebox archivebox[864]: ^^^^^^^^^^^^
Mar 06 19:29:04 archivebox archivebox[864]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/template/response.py", line 134, in content
Mar 06 19:29:04 archivebox archivebox[864]: HttpResponse.content.fset(self, value)
Mar 06 19:29:04 archivebox archivebox[864]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:29:04 archivebox archivebox[864]: content = self.make_bytes(value)
Mar 06 19:29:04 archivebox archivebox[864]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:29:04 archivebox archivebox[864]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:29:04 archivebox archivebox[864]: return bytes(value.encode(self.charset))
Mar 06 19:29:04 archivebox archivebox[864]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:29:04 archivebox archivebox[864]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 110372: surrogates not allowed
Mar 06 19:29:04 archivebox archivebox[864]: "GET /public/ HTTP/1.1" 500 145
ArchiveBox version
0.7.2
ArchiveBox v0.7.2 BUILD_TIME=2024-03-06 19:08:32 1709752112
IN_DOCKER=False IN_QEMU=False ARCH=x86_64 OS=Linux PLATFORM=Linux-6.7.8-arch1-1-x86_64-with-glibc2.39 PYTHON=Cpython
FS_ATOMIC=True FS_REMOTE=True FS_USER=1000:1000 FS_PERMS=644
DEBUG=False IS_TTY=True TZ=UTC SEARCH_BACKEND=ripgrep LDAP=False
[i] Dependency versions:
√ PYTHON_BINARY v3.11.8 valid /usr/bin/python3.11
√ SQLITE_BINARY v2.6.0 valid /usr/lib/python3.11/sqlite3/dbapi2.py
√ DJANGO_BINARY v3.1.14 valid /home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/__init__.py
√ ARCHIVEBOX_BINARY v0.7.2 valid /home/ol/.local/pipx/venvs/archivebox/bin/archivebox
√ CURL_BINARY v8.6.0 valid /usr/bin/curl
√ WGET_BINARY v1.21.4 valid /usr/bin/wget
√ NODE_BINARY v21.6.2 valid /usr/bin/node
√ SINGLEFILE_BINARY v1.1.49 valid ./node_modules/single-file-cli/single-file
√ READABILITY_BINARY v0.0.11 valid ./node_modules/readability-extractor/readability-extractor
√ MERCURY_BINARY v1.0.0 valid ./node_modules/@postlight/parser/cli.js
√ GIT_BINARY v2.44.0 valid /usr/bin/git
√ YOUTUBEDL_BINARY v2023.12.30 valid /usr/bin/yt-dlp
√ CHROME_BINARY v122.0.6261.111 valid /usr/bin/chromium
√ RIPGREP_BINARY v14.1.0 valid /usr/bin/rg
[i] Source-code locations:
√ PACKAGE_DIR 23 files valid /home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/archivebox
√ TEMPLATES_DIR 3 files valid /home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/archivebox/templates
- CUSTOM_TEMPLATES_DIR - disabled None
[i] Secrets locations:
- CHROME_USER_DATA_DIR - disabled None
- COOKIES_FILE - disabled None
[i] Data locations:
√ OUTPUT_DIR 8 files @ valid /home/ol/data
√ SOURCES_DIR 48 files valid ./sources
√ LOGS_DIR 1 files valid ./logs
√ ARCHIVE_DIR 1683 files valid ./archive
√ CONFIG_FILE 149.0 Bytes valid ./ArchiveBox.conf
√ SQL_INDEX 27.1 MB valid ./index.sqlite3
logs with DEBUG=True
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/base.py", line 204, in _get_response
Mar 06 19:39:25 archivebox archivebox[2001]: response = response.render()
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/template/response.py", line 105, in render
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = self.rendered_content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/template/response.py", line 134, in content
Mar 06 19:39:25 archivebox archivebox[2001]: HttpResponse.content.fset(self, value)
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 110372: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/archivebox/core/middleware.py", line 32, in middleware
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6662: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/utils/deprecation.py", line 114, in __call__
Mar 06 19:39:25 archivebox archivebox[2001]: response = response or self.get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6658: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/utils/deprecation.py", line 114, in __call__
Mar 06 19:39:25 archivebox archivebox[2001]: response = response or self.get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6658: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/utils/deprecation.py", line 114, in __call__
Mar 06 19:39:25 archivebox archivebox[2001]: response = response or self.get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6658: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/utils/deprecation.py", line 114, in __call__
Mar 06 19:39:25 archivebox archivebox[2001]: response = response or self.get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6658: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/utils/deprecation.py", line 114, in __call__
Mar 06 19:39:25 archivebox archivebox[2001]: response = response or self.get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6658: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/utils/deprecation.py", line 114, in __call__
Mar 06 19:39:25 archivebox archivebox[2001]: response = response or self.get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6658: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/utils/deprecation.py", line 114, in __call__
Mar 06 19:39:25 archivebox archivebox[2001]: response = response or self.get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6658: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/archivebox/core/middleware.py", line 25, in middleware
Mar 06 19:39:25 archivebox archivebox[2001]: return get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6658: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: During handling of the above exception, another exception occurred:
Mar 06 19:39:25 archivebox archivebox[2001]: Traceback (most recent call last):
Mar 06 19:39:25 archivebox archivebox[2001]: File "/usr/lib/python3.11/wsgiref/handlers.py", line 137, in run
Mar 06 19:39:25 archivebox archivebox[2001]: self.result = application(self.environ, self.start_response)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/contrib/staticfiles/handlers.py", line 76, in __call__
Mar 06 19:39:25 archivebox archivebox[2001]: return self.application(environ, start_response)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/wsgi.py", line 133, in __call__
Mar 06 19:39:25 archivebox archivebox[2001]: response = self.get_response(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/base.py", line 130, in get_response
Mar 06 19:39:25 archivebox archivebox[2001]: response = self._middleware_chain(request)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 49, in inner
Mar 06 19:39:25 archivebox archivebox[2001]: response = response_for_exception(request, exc)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception
Mar 06 19:39:25 archivebox archivebox[2001]: response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception
Mar 06 19:39:25 archivebox archivebox[2001]: return debug.technical_500_response(request, *exc_info)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/views/debug.py", line 53, in technical_500_response
Mar 06 19:39:25 archivebox archivebox[2001]: return HttpResponse(html, status=status_code, content_type='text/html')
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 298, in __init__
Mar 06 19:39:25 archivebox archivebox[2001]: self.content = content
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 328, in content
Mar 06 19:39:25 archivebox archivebox[2001]: content = self.make_bytes(value)
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: File "/home/ol/.local/pipx/venvs/archivebox/lib/python3.11/site-packages/django/http/response.py", line 241, in make_bytes
Mar 06 19:39:25 archivebox archivebox[2001]: return bytes(value.encode(self.charset))
Mar 06 19:39:25 archivebox archivebox[2001]: ^^^^^^^^^^^^^^^^^^^^^^^^^^
Mar 06 19:39:25 archivebox archivebox[2001]: UnicodeEncodeError: 'utf-8' codec can't encode character '\udcf6' in position 6658: surrogates not allowed
Mar 06 19:39:25 archivebox archivebox[2001]: "GET /public/ HTTP/1.1" 500 59
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 16 (5 by maintainers)
Should be tentatively fixed in https://github.com/ArchiveBox/ArchiveBox/pull/1424
I added workaround logic in
wget_output_path()to fallback to the parent dir if the html path contains unprintable unicode.I’m now running this after moving all directories from
archivetobroken:This is one folder I identified:
So a grep for literally
udcf6would have shown the issue from the beginning…I’ll leave the script running and update again if I come across another issue.
I pulled the whole debug block in the settings.py to the bottom of the file and added
ERROR_LOG="/tmp/err.log"now the server starts and throws the same 500 error as before w/o any debug toolbar 😄Interstingly i did a
sqlite3 database.db '.dump' > foo.sql(besides some strace) which lead to not having the issue anymore. I wonder what that did and if something went wrong insside the sqlite file before.I´d still be interested in getting to know how to debug this 😃
edit: I also moved all archive data back which i suspected to cause issues and it still works.
Looks like you archived a URL that contains unprintable UTF-8 bytes (possibly from a broken emoji/accented character/crylic/chinese/arabic/etc.) and it ended up in a filesystem path, so it’s failing when trying to render the path in the public view.
In the short term you can find/strip all special UTF-8 characters in filenames using this script I wrote:
strip_bad_filename_characters.shor a program likedetox(apt install detox; man detox).In the long term ArchiveBox should fix this by force-normalizing all filenames to UTF-8 form-D on creation so this doesn’t happen in the future.