bitcoin: Travis timeouts
Travis generally times out when ./depends
changes or when a commonly included header file is changed. It currently needs human attention to run the job again and continue compiling from the last time the cache was saved.
Solutions to this could be:
- Run a helper script to poll travis failures and rerun them if the failure was due to a timeout.
- Rerun about-to-time-out jobs from within travis
- Switch to another CI solution. We currently have travis, appveyor and (not enabled) Cirrus CI, so we’d have to get rid of at least one of them before adding a new one.
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 38 (35 by maintainers)
Commits related to this issue
- Merge #16214: travis: Fix caching issues 88884c6f75 travis: Use absolute paths for cache dirs (MarcoFalke) fae9d54abc travis: Fix caching issues (MarcoFalke) Pull request description: It appears ... — committed to bitcoin/bitcoin by deleted user 5 years ago
- Merge #16595: travis: use extended 90 minute timeout when available fa1e9c68a72fe8a17b6cb1ce473fd79388b2b61e travis: Bump timeout from 50 minutes to 90 (MarcoFalke) Pull request description: The ... — committed to bitcoin/bitcoin by laanwj 5 years ago
- Merge #17040: ci: Make apt-get more verbose, to debug travis timeouts fa95503d64e1a5ad8aa67d78695ee9222372a16e ci: Make apt-get more verbose, to debug travis timeouts (MarcoFalke) Pull request descr... — committed to bitcoin/bitcoin by deleted user 5 years ago
- Merge #16214: travis: Fix caching issues 88884c6f75 travis: Use absolute paths for cache dirs (MarcoFalke) fae9d54abc travis: Fix caching issues (MarcoFalke) Pull request description: It appears ... — committed to vijaydasmp/dash by deleted user 5 years ago
I don’t but if it would be helpful to the project I can try to establish a connection. If I get a few thumbs up on this comment I’ll prioritize it. I’d want to know what the project’s history is with Travis.
Added a thumbs up, I would encourage others to do the same. We need to transition from Travis but during that transition period we need to get by as best we can.
Would the Travis timeouts be less frequent if we paid them more?
I did get a reply from travis with ideas to explore:
Hey Marco,
Thank you for your reply and for sending us these links to newer instance of the issue.
I have some suggestions for you to try. Ideally you should try them independently.
Right now this file contains the following: {“registry-mirrors”: [“https://mirror.gcr.io”], “mtu”: 1460} and I’m wondering if the mtu setting could be at play.
Please ensure to call this command before calling apt-get update.
I hope this helps and please let us know the results you will get.
GitHub Actions is lacking support for caching last time I looked, so the build take longer than on travis. I’d prefer to wait until caching is supported.
Semaphore just made us a nice offer, basically 16 parallel jobs of 2 vCPU 3.4GHZ 4GB Ram machines for free.
Since travis is back running… I don’t know what to do.
Travis just confirmed to me it was an issue on their side:
@MarcoFalke how difficult would it be to “Run a helper script to poll travis failures and rerun them if the failure was due to a timeout” or “Rerun about-to-time-out jobs from within travis” as suggested in the issue description? I understand doing these things wouldn’t solve every problem, but it seems like they would solve most of the current problems with no downsides and not be too difficult to implement. The second option “Rerun about-to-time-out jobs from within travis” seems especially appealing and it looks like there’s an existing implementation of it with curl here: https://github.com/plume-lib/trigger-travis#use-in-travisyml
The only potential downside I see is that logs from “Error! Initial build successful, but not enough time remains…” builds would be discarded, but since successful build logs probably aren’t very interesting, this doesn’t seem so important.
I guess in the longer term, it does seem like we are pushing against travis limits and might need to find other solutions.
Supports all of those listed points.
To put in a nutshell, I wouldn’t see any blocker that would keep you away from doing the same things that are being done on TravisCI. And there even more pretty neat feature to support easy maintenance for all those tests. But to stay realistic, the only downside I could see is that you have everything inside a
yaml
file which is (in my opinion) not that easily readable and become quite huge after some time : ^(