pywps: Issue session rollback after some errors
Description
After there is an error, if the pywps service receives new request, the error messages are shown as follows:
[pid 21389:tid 140700116551424] 2016-09-02 16:53:15,709 INFO sqlalchemy.engine.base.Engine BEGIN (implicit)
[pid 21389:tid 140700116551424] 2016-09-02 16:53:15,710 INFO sqlalchemy.engine.base.Engine INSERT INTO pywps_requests (uuid, pid, operation, version, time_start, time_end, identifier, message, percent_done, status) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
[pid 21389:tid 140700116551424] 2016-09-02 16:53:15,710 INFO sqlalchemy.engine.base.Engine ('4e206528-70e2-11e6-891d-ecb1d783f730', 21389, 'execute', '1.0.0', '2016-09-02 16:53:15.708929', None, 'OpenWPS:CV_VisualizeNonSeries', None, None, None)
[pid 21389:tid 140700116551424] 2016-09-02 16:53:15,710 INFO sqlalchemy.engine.base.Engine COMMIT
[pid 21389:tid 140700116551424] [remote xxx:30540] mod_wsgi (pid=21389): Exception occurred processing WSGI script '/xxx/pywps/wps.wsgi'.
[pid 21389:tid 140700116551424] [remote xxx:30540] Traceback (most recent call last):
[pid 21389:tid 140700116551424] [remote xxx:30540] File "/xxx/python2.7/site-packages/werkzeug/wrappers.py", line 291, in application
[pid 21389:tid 140700116551424] [remote xxx:30540] return f(*args[:-2] + (request,))(*args[-2:])
[pid 21389:tid 140700116551424] [remote xxx:30540] File "/xxx/python2.7/site-packages/pywps-4.0.0rc1-py2.7.egg/pywps/app/Service.py", line 585, in __call__
[pid 21389:tid 140700116551424] [remote xxx:30540] request_uuid
[pid 21389:tid 140700116551424] [remote xxx:30540] File "/xxx/python2.7/site-packages/pywps-4.0.0rc1-py2.7.egg/pywps/app/Service.py", line 297, in execute
[pid 21389:tid 140700116551424] [remote xxx:30540] tempdir = tempfile.mkdtemp(prefix='pywps_process_', dir=workdir)
[pid 21389:tid 140700116551424] [remote xxx:30540] File "/xxx/python2.7/tempfile.py", line 333, in mkdtemp
[pid 21389:tid 140700116551424] [remote xxx:30540] _os.mkdir(file, 0700)
[pid 21389:tid 140700116551424] [remote xxx:30540] OSError: [Errno 13] Permission denied: '/tmp/outputs/pywps_process_OyLFRX'
[pid 21388:tid 140700041606912] [remote xxx:65351] mod_wsgi (pid=21388): Exception occurred processing WSGI script '/xxx/pywps/wps.wsgi'.
[pid 21388:tid 140700041606912] [remote xxx:65351] Traceback (most recent call last):
[pid 21388:tid 140700041606912] [remote xxx:65351] File "/xxx/python2.7/site-packages/werkzeug/wrappers.py", line 291, in application
[pid 21388:tid 140700041606912] [remote xxx:65351] return f(*args[:-2] + (request,))(*args[-2:])
[pid 21388:tid 140700041606912] [remote xxx:65351] File "/xxx/python2.7/site-packages/pywps-4.0.0rc1-py2.7.egg/pywps/app/Service.py", line 573, in __call__
[pid 21388:tid 140700041606912] [remote xxx:65351] log_request(request_uuid, wps_request)
[pid 21388:tid 140700041606912] [remote xxx:65351] File "/xxx/python2.7/site-packages/pywps-4.0.0rc1-py2.7.egg/pywps/dblog.py", line 65, in log_request
[pid 21388:tid 140700041606912] [remote xxx:65351] session.commit()
[pid 21388:tid 140700041606912] [remote xxx:65351] File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 811, in commit
[pid 21388:tid 140700041606912] [remote xxx:65351] self.transaction.commit()
[pid 21388:tid 140700041606912] [remote xxx:65351] File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 396, in commit
[pid 21388:tid 140700041606912] [remote xxx:65351] self._assert_active(prepared_ok=True)
[pid 21388:tid 140700041606912] [remote xxx:65351] File "build/bdist.linux-x86_64/egg/sqlalchemy/orm/session.py", line 213, in _assert_active
[pid 21388:tid 140700041606912] [remote xxx:65351] % self._rollback_exception
[pid 21388:tid 140700041606912] [remote xxx:65351] InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140700041606912 and this is thread id 140700158510848
Environment
- operating system: CentOS 6.5 final
- Python version: 2.7.10
- PyWPS version: 4.0.0rc1 (cloned on 31 Aug 2016 13:46 UTC+09:00)
- source/distribution
- [O] git clone
- Debian
- PyPI
- zip/tar.gz
- other (please specify):
- web server
- [O] Apache/mod_wsgi
- CGI
- other (please specify):
Steps to Reproduce
i) changing permission of output directory ii) calling pywps service, then the first error message is shown as ‘permission denied’ ii) re-calling pywps service, then the second error message is shown as ‘InvalidRequestError’
Additional Information
The first error is no problem because I can solve that.
About this issue
- Original URL
- State: closed
- Created 8 years ago
- Comments: 16 (10 by maintainers)
Just to confirm, I also have this problem. This was not the case in RC1. This commit 5f579d4a353677e9b58375fbda9932dd9f4551e2 seem to have introduced the issue. On top of the InvalidRequestError mentioned above, my apache log is also complaining about:
@geoslegend Jachym introduced SQLAlchemy to be able to switch to other DMBS, such as postgres. I assume postgres should work out of the box. Can you give it a try? You only need to adjust the parameters in the config file, correct @jachym?
@geoslegend @jonas-eberle so, do I understand it correctly, that moving from sqlite3 to postgres should solve the issue?
@geoslegend please, your report was OK, my comment was regarding PyWPS: PyWPS should report error messages in more readable manner
@jachym See the error message in the last line:
Original exception was: (sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140700041606912 and this is thread id 140700158510848Stackoverflow shows this explanation (“You may not be using threads explicitly, but mod_wsgi is, and you’ve defined a global session object”) and solution: http://stackoverflow.com/a/34010159