monstache: Sync MongoDB to Elasticsearch via docker way not work.
I run monstache in my k8s cluster to sync mongo(inside k8s) to elasticsearch(outside k8s), it’s not work without any error, nothing synced to ES.
monstache log:
INFO 2018/05/10 09:04:55 Successfully connected to MongoDB version 3.6.4
INFO 2018/05/10 09:04:55 Successfully connected to Elasticsearch version 5.5.0
INFO 2018/05/10 09:04:55 Entering event loop
...
...
STATS 2018/05/10 09:25:48 {"Flushed":181,"Committed":20,"Indexed":185,"Created":0,"Updated":0,"Deleted":27,"Succeeded":51,"Failed":161,"Workers":[{"Queued":0,"LastDuration":4000000},{"Queued":0,"LastDuration":40000000},{"Queued":0,"LastDuration":1000000},{"Queued":0,"LastDuration":3000000}]}
STATS 2018/05/10 09:26:18 {"Flushed":211,"Committed":20,"Indexed":185,"Created":0,"Updated":0,"Deleted":27,"Succeeded":51,"Failed":161,"Workers":[{"Queued":0,"LastDuration":4000000},{"Queued":0,"LastDuration":40000000},{"Queued":0,"LastDuration":1000000},{"Queued":0,"LastDuration":3000000}]}
My env:
Mongo: replicaset (v3.6) in k8s cluster. Elasticsearch: v5.5 in isolate machine. Monstache: docker image: rwynn/monstache:3.13.3.cgo
My config.toml:
mongo-url = "mongodb://{user}:{password}@mongo-0.mongo.db,mongo-1.mongo.db,mongo-2.mongo.db"
elasticsearch-urls = ["http://{es.ip}:9200"]
gzip = true
stats = true
index-stats = false
fail-fast = true
resume = true
resume-write-unsafe = true
replay = true
elasticsearch-retry = true
dropped-collections = true
dropped-databases = true
mongo-oplog-collection-name = "oplog.rs"
namespace-regex = '^web-db\.(articles|\$cmd)$'
# direct-read-namespaces = ["web-db.articles"]
exit-after-direct-reads = false
enable-http-server = true
Data in monstache.monstache collection
{
"_id" : "default",
"ts" : Timestamp(1525943995, 1)
}
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 18 (11 by maintainers)
I will create a separate issue to figure out what to do with those error messages since they are not truly errors but the versioning system at work.