go-livepeer: 'Failed: insufficient funds for gas * price + value' - Cannot process transactions after Nitro upgrade

Describe the bug After upgrading to Nitro, some orchestrators including myself cannot execute transactions like calling reward or redeeming a winning ticket.

Initially, the error looked like this:

HTTP Response Error 500: Error calling reward: gas required exceeds allowance (152271)

Upon node restart, the error evolved into this:

 ******************************Eth Transaction******************************
Aug 31 17:21:42 lpr-Not-Specified livepeer[554821]: Invoking transaction: "rewardWithHint". Inputs: "_newPosPrev: 0xE3a5793d7c1D2a04A903FA1695b3E3555d6084CA  _newPosNext: 0x4a43B1D7e62                   27C8b0512e413F406555647ff7bdB"
Aug 31 17:21:42 lpr-Not-Specified livepeer[554821]: Transaction Failed: insufficient funds for gas * price + value: address 0x9D61ae5875E89036FBf6059f3116d01a22ACe3C8 have 604543848649                   53520 want 266627000000000000

Or, on the node trying to redeem a winning ticket, this:

Invoking transaction: "redeemWinningTicket". Inputs: "_ticket: { Recipient: 0x9D61ae5875E89036FBf6059f3116d01a22ACe3C8  Sender: 0xc3c7c4C8f7061B7d6A72766Eee5359fE4F36e61E  FaceValue: 12000000000000000  WinProb: 9649340769776349618630915417390658987772498722136713669954799000000000000  SenderNonce: 1  RecipientRandHash: 0x83fe58b22811435b18d0c2b90eba17206f612f1072f2805fa792c808d06a0170  AuxData: 0x0000000000000000000000000000000000000000000000000000000000000a7a613ebf57b80834b99a2ec150ea7b2b0cb97419d0c2d31dfeca4659fc6a92688e }  _sig: 0x1f956188526c89394e467aecd529669c65b5c5a5913079721585486dee654ccc1dd79c631f18385e1aeb079634202328ce3d85345ccc9e22e0b51d8f7694da5c1b  _recipientRand: 55259894694139197534321694537779125253224093903070031163731600211776028401548"
Transaction Failed: insufficient funds for gas * price + value: address 0xcdF93B13EFCf63f84f2afFB01BeC48A5c096C69E have 25795810339399299 want 328761600000000000

***************************************************************************
E0831 22:08:41.746572       1 queue.go:149] Error redeeming err="insufficient funds for gas * price + value: address 0xcdF93B13EFCf63f84f2afFB01BeC48A5c096C69E have 25795810339399299 want 328761600000000000"

It looks like it’s requesting an absurd amount to redeem a winning ticket or call reward, and the cost seems to get higher over time. We’ve tried multiple RPC providers (Alchemy, offchain, full arb node) and restarting our nodes but have not had any success.

We CAN, however, call reward normally via https://arbiscan.io/address/0x35Bcf3c30594191d53231E4FF333E8A770453e40#writeProxyContract

My node setup: Independent local Ubuntu node for calling reward 7 Dockerized Geolocated O/T’s

Upon talking to other O’s using similar setups, this does not seem to be an issue that affects them. We also had no issues prior to the upgrade.

To Reproduce Steps to reproduce the behavior:

1. Call reward or redeem a winning ticket
2. Observe logs
3. If the transaction completes normally, you’re not affected by this
4. If you see the error listed above, you’re in the same boat as me and some others… sorry

Expected behavior Transactions should process normally after the Nitro upgrade.

Desktop (please complete the following information):

• OS: Ubuntu
• Version: 22.04.1

Additional context It’s hard to see how this could be a configuration error on the orchestrators side, as this started immediately after the Nitro upgrade and several RPC providers have been tested, none of them resolving the issue, while other orchestrators don’t seem to have a problem.