plugins: rebalance plugin crashes

2021-01-23T23:44:33.003Z INFO    plugin-rebalance.py: Plugin rebalance initialized with 1000msat base / 10 ppm fee, cltv_final: 18
2021-01-23T23:45:35.890Z UNUSUAL plugin-rebalance.py: Exception in thread Thread-1:
2021-01-23T23:45:35.890Z UNUSUAL plugin-rebalance.py: Traceback (most recent call last):
2021-01-23T23:45:35.891Z UNUSUAL plugin-rebalance.py:   File \"/usr/lib/python3.8/threading.py\", line 932, in _bootstrap_inner
2021-01-23T23:45:35.897Z UNUSUAL plugin-rebalance.py:     self.run()
2021-01-23T23:45:35.898Z UNUSUAL plugin-rebalance.py:   File \"/usr/lib/python3.8/threading.py\", line 870, in run
2021-01-23T23:45:35.898Z UNUSUAL plugin-rebalance.py:     self._target(*self._args, **self._kwargs)
2021-01-23T23:45:35.898Z UNUSUAL plugin-rebalance.py:   File \"/home/user/plugins/rebalance/rebalance.py\", line 433, in rebalanceall_thread
2021-01-23T23:45:35.899Z UNUSUAL plugin-rebalance.py:     plugin.ideal_ratio = get_ideal_ratio(channels, plugin.enough_liquidity)
2021-01-23T23:45:35.899Z UNUSUAL plugin-rebalance.py:   File \"/home/user/plugins/rebalance/rebalance.py\", line 304, in get_ideal_ratio
2021-01-23T23:45:35.900Z UNUSUAL plugin-rebalance.py:     if smallest_channel[\"total_msat\"] * min(ratio, 1 - ratio) > enough_liquidity:
2021-01-23T23:45:35.900Z UNUSUAL plugin-rebalance.py:   File \"/usr/local/lib/python3.8/dist-packages/pyln/client/lightning.py\", line 180, in __mul__
2021-01-23T23:45:35.901Z UNUSUAL plugin-rebalance.py:     return Millisatoshi(self.millisatoshis * other)
2021-01-23T23:45:35.901Z UNUSUAL plugin-rebalance.py:   File \"/usr/local/lib/python3.8/dist-packages/pyln/client/lightning.py\", line 72, in __init__
2021-01-23T23:45:35.901Z UNUSUAL plugin-rebalance.py:     raise TypeError(
2021-01-23T23:45:35.901Z UNUSUAL plugin-rebalance.py: TypeError: Millisatoshi must be string with msat/sat/btc suffix or int

pinging @m-schmoock

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 19 (18 by maintainers)

Most upvoted comments

@aido I just merged https://github.com/lightningd/plugins/pull/211 . Can you retry with this changeset. If you still have remaining issues, can you try to construct testcases that cover this? Because the our == total is now covered by test_rebalance_all and its not causing the issues you describe (which is why I did not take all of your suggestions).

m-schmoock on Feb 3, 2021

Hi @m-schmoock

There is an infinite loop that removes objects from a list until there are zero objects left resulting in a division by zero error… By changing:

https://github.com/lightningd/plugins/blob/abbdeabec6504b1f1771473296b7656f53293723/rebalance/rebalance.py#L301

   while len(chs) > 0:

ensures the loop stops before the list is empty.

This fixes the divide by zero error but now the following result needs investigation:

2021-02-01T23:10:55.893Z INFO    plugin-rebalance.py: Automatic rebalance is running with enough liquidity threshold: 0msat, ideal liquidity ratio: 100.00%, min rebalancable amount: 50000000msat, feeratio: 0.5
2021-02-01T23:10:56.130Z INFO    plugin-feeadjuster.py: 0 channels adjusted
2021-02-01T23:10:56.132Z INFO    plugin-rebalance.py: Feeadjust succeeded: 0 channels adjusted
2021-02-01T23:10:56.149Z INFO    plugin-rebalance.py: Automatic rebalance finished: 0 successful rebalance, 0msat fee spent, it took 0:00:00.346

aido on Feb 1, 2021

Hi,

Seeing issues with rebalance plugin also:

2021-02-01T03:05:50.590Z UNUSUAL plugin-rebalance.py: Exception in thread Thread-1:
2021-02-01T03:05:50.591Z UNUSUAL plugin-rebalance.py: Traceback (most recent call last):
2021-02-01T03:05:50.591Z UNUSUAL plugin-rebalance.py:   File \"/usr/lib/python3.9/threading.py\", line 954, in _bootstrap_inner
2021-02-01T03:05:50.611Z UNUSUAL plugin-rebalance.py:     self.run()
2021-02-01T03:05:50.611Z UNUSUAL plugin-rebalance.py:   File \"/usr/lib/python3.9/threading.py\", line 892, in run
2021-02-01T03:05:50.613Z UNUSUAL plugin-rebalance.py:     self._target(*self._args, **self._kwargs)
2021-02-01T03:05:50.614Z UNUSUAL plugin-rebalance.py:   File \"/usr/libexec/c-lightning/plugins/rebalance/rebalance.py\", line 433, in rebalanceall_thread
2021-02-01T03:05:50.615Z UNUSUAL plugin-rebalance.py:     plugin.ideal_ratio = get_ideal_ratio(channels, plugin.enough_liquidity)
2021-02-01T03:05:50.615Z UNUSUAL plugin-rebalance.py:   File \"/usr/libexec/c-lightning/plugins/rebalance/rebalance.py\", line 302, in get_ideal_ratio
2021-02-01T03:05:50.617Z UNUSUAL plugin-rebalance.py:     ratio = int(our) / int(total)
2021-02-01T03:05:50.617Z UNUSUAL plugin-rebalance.py: ZeroDivisionError: division by zero

aido@lightning:~# /usr/bin/lightning-cli --version
v0.9.3rc2-4-g015ac37-modded
aido@lightning:~# pip3 freeze | grep pyln
pyln-client==0.9.3
pyln-proto==0.9.3

Should https://github.com/lightningd/plugins/blob/abbdeabec6504b1f1771473296b7656f53293723/rebalance/rebalance.py#L298-L299 be:

    our = sum(ch["msatoshi_to_us"] for ch in channels)
    total = sum(ch["msatoshi_total"] for ch in channels)

aido on Feb 1, 2021

Nope, the fix was included in 0.9.3 which I haven’t published to pypi yet 🤦 Will do that asap and then we can bump the requirement to pyln-client==0.9.3

cdecker on Jan 25, 2021