luigi: [Question] How to store history indefinitely?
According to Section 8.6.2 Enabling Task History in the doc, task history is stored for last 24 hours only. Is there a way to override that to make it indefinite?
There is one parameter in Section 8.9.4 [cors]:
max_age Content of Access-Control-Max-Age. Defaults to 86400 (24 hours)
that seems relevant to the duration. Would changing that help?
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 19 (18 by maintainers)
Worked perfectly. Thank you very much @drowoseque , it seems like you are very knowledgeable about the code base.
No worries, you’re more than welcome 😃 I think it’s this file https://github.com/spotify/luigi/blob/master/doc/central_scheduler.rst
@tashrifbillah I’m not sure, but maybe that’s because of a typo in the url it looks like it should be localhost:8082/history/by_name/TaskB instead of localhost:8082/history/by_name/:TaskB you have an odd colon
@tashrifbillah btw if you’re interested in some particular task you can use /history/by_name/ endpoint which doesn’t have any restrictions on dates
unfortunately there is no way to do it right now as limitation of 24 hours is hardcoded