orchestrator: Maybe a bug during recovery.
Hi @shlomi-noach , As mentioned in https://github.com/github/orchestrator/issues/419: I’ve used RecoveryIgnoreHostnameFilters and still it gets promoted. The topology looks like this:
[root@mysql-sredb02.xh ~]# orchestrator-client -c topology -i mysql-sredb03.yp
mysql-sredb03.yp:3306 [0s,ok,5.6.24-72.2-log,rw,ROW,>>,GTID]
+ mysql-sredb05.yp:3306 [0s,ok,5.6.24-72.2-log,rw,ROW,>>,GTID]
+ mysql-sredb06.yp:3306 [0s,ok,5.6.24-72.2-log,rw,ROW,>>,GTID]
+ mysql-sredb07.yp:3306 [0s,ok,5.6.24-72.2-log,rw,ROW,>>,GTID]

I’ve configure like this:
"RecoveryIgnoreHostnameFilters": [
"mysql-sredb06.yp",
"mysql-sredb05.yp"
],
When I stop the MySQL server on mysql-sredb03.yp, the mysql-sredb06.yp has been promoted.
2018-03-05 14:33:48 Detected DeadMaster on mysql-sredb03.yp:3306. Affected replicas: 3
2018-03-05 14:33:48 Will recover from DeadMaster on mysql-sredb03.yp:3306
2018-03-05 14:33:50 Detected DeadMaster on mysql-sredb03.yp:3306. Affected replicas: 1
2018-03-05 14:33:50 Recovered from DeadMaster on mysql-sredb03.yp:3306. Failed: mysql-sredb03.yp:3306; Promoted: mysql-sredb06.yp:3306
2018-03-05 14:33:50 (for all types) Recovered from DeadMaster on mysql-sredb03.yp:3306. Failed: mysql-sredb03.yp:3306; Successor: mysql-sredb06.yp:3306
Are there something wrong?
About this issue
- Original URL
- State: open
- Created 6 years ago
- Comments: 19
I do not reproduce the same problem now… When it happens again, I will tell you. Thanks.
Yes, I have restarted orchestrator after making configuration changes.
I will test more times to confirm this. I have reproduced several times, but I don’t know whether its happens every 2nd times. Need I acknowledge the recoveries every time? I think its not necessary because the value of “RecoveryPeriodBlockSeconds” is 10.