PyFunceble: BUG: Suttenly just stops....
Description
The log
Process pyfunceble_tester_worker_5: Process pyfunceble_producer_worker_1: Process pyfunceble_tester_worker_22: Traceback (most recent call last): File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/multiprocessing/process.py”, line 315, in _bootstrap self.run() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/cli/processes/workers/base.py”, line 313, in run worker_name, destination_worker, consumed = self.input_queue.get() File “<string>”, line 2, in get File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/multiprocessing/managers.py”, line 809, in _callmethod kind, result = conn.recv() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/multiprocessing/connection.py”, line 255, in recv buf = self._recv_bytes() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/multiprocessing/connection.py”, line 419, in _recv_bytes buf = self._recv(4) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/multiprocessing/connection.py”, line 384, in _recv chunk = read(handle, remaining) KeyboardInterrupt Exception during reset or similar Traceback (most recent call last): File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/pymysql/connections.py”, line 756, in _write_bytes self._sock.sendall(data) ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 749, in _rollback_impl self.engine.dialect.do_rollback(self.connection) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/base.py”, line 2509, in do_rollback dbapi_connection.rollback() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/pymysql/connections.py”, line 479, in rollback self._execute_command(COMMAND.COM_QUERY, “ROLLBACK”) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/pymysql/connections.py”, line 814, in _execute_command self._write_bytes(packet) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/pymysql/connections.py”, line 759, in _write_bytes raise err.OperationalError( pymysql.err.OperationalError: (2006, “MySQL server has gone away (ConnectionResetError(104, ‘Connection reset by peer’))”)
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/pool/base.py”, line 697, in _finalize_fairy fairy._reset(pool) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/pool/base.py”, line 891, in _reset self._reset_agent.rollback() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1751, in rollback self._do_rollback() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1789, in _do_rollback self.connection._rollback_impl() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 751, in _rollback_impl self.handle_dbapi_exception(e, None, None, None, None) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 1510, in handle_dbapi_exception util.raise( File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/util/compat.py”, line 182, in raise raise exception File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/engine/base.py”, line 749, in _rollback_impl self.engine.dialect.do_rollback(self.connection) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/sqlalchemy/dialects/mysql/base.py”, line 2509, in do_rollback dbapi_connection.rollback() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/pymysql/connections.py”, line 479, in rollback self._execute_command(COMMAND.COM_QUERY, “ROLLBACK”) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/pymysql/connections.py”, line 814, in _execute_command self._write_bytes(packet) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/pymysql/connections.py”, line 759, in _write_bytes raise err.OperationalError( sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2006, “MySQL server has gone away (ConnectionResetError(104, ‘Connection reset by peer’))”) (Background on this error at: http://sqlalche.me/e/13/e3q8) Traceback (most recent call last): File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/multiprocessing/process.py”, line 315, in _bootstrap self.run() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/cli/processes/workers/base.py”, line 366, in run result = self.target(consumed) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/cli/processes/workers/tester.py”, line 275, in target self.testing_object.set_subject(test_dataset[“idna_subject”]) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/base.py”, line 143, in wrapper return func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/base.py”, line 168, in wrapper result = func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/availability/domain_and_ip.py”, line 109, in query_status result = query_object.query_status() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/base.py”, line 143, in wrapper return func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/base.py”, line 168, in wrapper result = func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/availability/domain.py”, line 166, in query_status self.try_to_query_status_from_http_status_code() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/availability/base.py”, line 866, in try_to_query_status_from_http_status_code lookup_result = self.http_status_code_query_tool.get_status_code() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/http_status_code.py”, line 125, in wrapper return func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/http_status_code.py”, line 340, in get_status_code req = PyFunceble.factory.Requester.get( File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/requester.py”, line 157, in wrapper req = getattr(self.session, verb.lower())(*args, **kwargs) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/requests/sessions.py”, line 555, in get return self.request(‘GET’, url, **kwargs) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/requests/sessions.py”, line 542, in request resp = self.send(prep, **send_kwargs) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/requests/sessions.py”, line 655, in send r = adapter.send(request, **kwargs) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/adapter/http.py”, line 92, in send hostname_ip = self.resolve(parsed_url.hostname) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/adapter/base.py”, line 180, in resolve return self.resolve_without_cache(hostname) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/adapter/base.py”, line 152, in resolve_without_cache last_cname = get_last_cname(hostname) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/adapter/base.py”, line 140, in get_last_cname time.sleep(self.dns_query_tool.BREAKOFF) KeyboardInterrupt Traceback (most recent call last): File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/multiprocessing/process.py”, line 315, in _bootstrap self.run() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/cli/processes/workers/base.py”, line 366, in run result = self.target(consumed) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/cli/processes/workers/tester.py”, line 275, in target self.testing_object.set_subject(test_dataset[“idna_subject”]) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/base.py”, line 143, in wrapper return func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/base.py”, line 168, in wrapper result = func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/availability/domain_and_ip.py”, line 109, in query_status result = query_object.query_status() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/base.py”, line 143, in wrapper return func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/base.py”, line 168, in wrapper result = func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/availability/domain.py”, line 166, in query_status self.try_to_query_status_from_http_status_code() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/checker/availability/base.py”, line 866, in try_to_query_status_from_http_status_code lookup_result = self.http_status_code_query_tool.get_status_code() File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/http_status_code.py”, line 125, in wrapper return func(self, *args, **kwargs) # pylint: disable=not-callable File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/http_status_code.py”, line 340, in get_status_code req = PyFunceble.factory.Requester.get( File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/requester.py”, line 157, in wrapper req = getattr(self.session, verb.lower())(*args, **kwargs) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/requests/sessions.py”, line 555, in get return self.request(‘GET’, url, **kwargs) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/requests/sessions.py”, line 542, in request resp = self.send(prep, **send_kwargs) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/requests/sessions.py”, line 655, in send r = adapter.send(request, **kwargs) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/adapter/http.py”, line 92, in send hostname_ip = self.resolve(parsed_url.hostname) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/adapter/base.py”, line 180, in resolve return self.resolve_without_cache(hostname) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/adapter/base.py”, line 152, in resolve_without_cache last_cname = get_last_cname(hostname) File “/home/joakim/miniconda/envs/pyfunceble4/lib/python3.9/site-packages/PyFunceble/query/requests/adapter/base.py”, line 140, in get_last_cname time.sleep(self.dns_query_tool.BREAKOFF) KeyboardInterrupt
Configuration
Reproduction
Execute above script whit the following veriables
Next to this… I had --logger-out warning set, but I got no logs…
tree -f --prune test_results/
test_results
├── test_results/dead.hosts.txt
├── test_results/dead.mobile.txt
├── test_results/dead.snuff.txt
├── test_results/dead.strict_adult.txt
├── test_results/dir_structure.json
├── test_results/hosts.active.txt
├── test_results/mobile.active.txt
├── test_results/snuff.active.txt
└── test_results/strict_adult.active.txt
An observation is, it always stops at this recorc, when the test freez 🥶
www.tnaflix.com ACTIVE DNSLOOKUP Unknown Unknown AVAILABILITY
Expected behavior
Screenshots
Versions
OS: Ubuntu 20.04 + Conda conda 4.10.2
Python Version: 3.9.5
PyFunceble Version: 4.0.0b60.dev
Additional context
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 29 (29 by maintainers)
Well, before b53, we were not manually resolving CNAME through our custom resolver …
New version deployed.
LoL …
@spirillen version already bumped 😃 The last commits were just for CI/CD fixes as they were not working. So the last release has everything …
Ping @spirillen ==> Please test.
I saw your commit 👍 I’ll give it a spin a little later… searching for the topic, where you was asking about documenting win+python+cygwin as I have the docs repo opened 😃