harbor: Support for CockroachDB (don't use pg_advisory_lock)
Is your feature request related to a problem? Please describe. We would like to use CockroachDB for a replicated / clustered database backend, possibily even for a multi-region setup of Harbor. Since CockroachDB speaks a dialect of PostgreSQL, this should be possible with some modifications.
The main issue with CockroachDB seems to be the lack of support for pg_advisory_lock() - CockroachDB will probably not support this function in the near future.
Describe the solution you’d like
Implement CockroachDB support, e.g. by solving #8511 and #6942 and drop pg_advisory_lock() calls (maybe via a flag / configuration option). I’m honestly not sure if that’s possible, I just wanted to document the main issue with CockroachDB support that we encountered.
Describe the main design/architecture of your solution
---
Describe the development plan you’ve considered
---
Additional context
This is what Harbor Core logs when you configure CockroachDB as target:
2019-08-13T10:11:44Z [ERROR] [/common/dao/pgsql.go:112]: Failed to upgrade schema, error: "try lock failed in line 0: SELECT pg_advisory_lock($1) (details: pq: unknown function: pg_advisory_lock())"
About this issue
- Original URL
- State: open
- Created 5 years ago
- Reactions: 21
- Comments: 28 (7 by maintainers)
@embik @KarstenSiemer Thanks a lot for raising this.
The error is thrown by the migrator, it seems it supports cockroach DB: https://github.com/golang-migrate/migrate/tree/master/database/cockroachdb
We’ll do some investigation to see if we can support that with minimal change in configuration.
Hi, we want also to use CockroachDB for Harbor as we have Harbor clusters in 5 DCs across the world, and CockroachDb DC will be a perfect solution to consolidate our data from all DCs in 1 table
I’ve been working on this; I think the code changes could be minor, but there’s coordination happening with MySQL work, and #16420 is a blocker for me.
Thank you for the clarification. i looked at the code and wondered where the dsn should take effect.
They removed the vendor folder in one of the latest commits. If we could pass the schema from the config to golang-migrate, we would have full cockroachdb support.
Not sure if it’s a matter to document this here, but I’ve switched the protocol on my connection string from
postgresql://user:password@...tocockroachdb://user:password@...and the golang-migrate worked.We’ve been relying on the image storage from our cloud provider.
I’m utterly at a loss as to how to proceed, @wwentland:
The ORM Harbor uses supports Cockroach natively, but the way they manage dependencies is idiosyncratic: code files appear to be manually added to the
vendor/directory, and in this case Cockroach support wasn’t included.The project contribution guides indicate that all dependency updates have to be made by a core contributor. #16420 was the request to import the whole ORM, which was then labelled “help-wanted” and left to go stale.
I’ve tried to join the working group related to DB compatibility, but that was frustrating. Their focus was on MySQL, and building import tooling. Compounded with challenges with time zones and language difficulties, I gave up.
My takeaway from the experience, regarding development practices and project governance has been that I can’t recommend Harbor in my organization or elsewhere. I wish I could say differently.