ArchiSteamFarm: Bot won't start and chat becomes unresponsive

Update from Archi:

Notice for everybody facing this issue: enable ASF Debug mode in order to record debug log from the freeze. After you notice freeze, let ASF work for at least ~10 more minutes in such state. Once recorded, pack whole debug directory together with log.txt to zip and send it on my e-mail: JustArchi[at]JustArchi[dot]net, as it can contain sensitive information.

Okay so I’m having this weird issue with only one bot. I’m sure if I restart ASF it will work as expected, but maybe this can help fix some potential bug. Ok, so all my bots started normally (even the one that I’m having issues with), and all of them are redeeming keys too, except one bot, which is returning Timeout:

[8:02:57 PM] master: !redeem buggy_bot XXXXX-XXXXX-XXXXX
<slave_1> Key: XXXXX-XXXXX-XXXXX| Status: Timeout!

The ASF output looks like this:

2016-07-30 20:02:59.9729|ERROR|ArchiSteamFarm.Logging|buggy_bot|RedeemKey()

I’ve had this issue a couple of times and generally I just have to restart the bot to make it work again. But this time, I tried to !stop it and I got no response (I spammed some commands three times and I still didn’t got any response):

[8:03:07 PM] master: !stop buggy_bot
[8:03:11 PM] master: !start buggy_bot
[8:04:05 PM] master: !stop buggy_bot

The ASF output after the first !stop:

2016-07-30 20:03:11.1083|INFO|ArchiSteamFarm.Logging|buggy_bot|Stop() Stopping...

Since bot slave_1’s chat window became unresponsive, I tried to issue the commands to slave_2, but doing that, made slave_2 freeze too:

[8:06:29 PM] master: !statusall
[8:06:31 PM] slave_2: 
<a reaaaally long response that isn't important>
[8:06:54 PM] master: !stop buggy_bot
[8:06:56 PM] slave_2: That bot instance is already inactive!
[8:06:59 PM] master: !start buggy_bot
[8:07:25 PM] master: !statusall

ASF didn’t show any relevant information to above commands. I’m using the latest stable version: V2.1.3.1

About this issue

Original URL
State: closed
Created 8 years ago
Comments: 24 (14 by maintainers)

Commits related to this issue

Implement enhanced HeartBeat The objective of this feature is to detect network malfunctions as well as SK2 connection issues early and initiate a reconnect as soon as possible, instead of relying on ... — committed to JustArchiNET/ArchiSteamFarm by JustArchi 8 years ago

Most upvoted comments

Interesting that you had 16 failed heartbeat attempts, then 16 disconnected callbacks all at the same time.

Are you sure you’re calling the CallbackManager appropriately?

Netshroud on Aug 23, 2016

Can you map those line logs to SteamKit2 / callbacks / function calls? I’m not going to read through all of ASF to find all log output lines.

Also, are you using TCP or UDP?

Netshroud on Jul 31, 2016

Remove NLog.config if you use default settings, there is no need to override them if you don’t have a reason.

Judging from the fact that it was in ArchiWebHandler, most likely it was some kind of networking problem - steam didn’t announce end of connection, so SteamClient probably is trying to end it gracefully while not receiving any response.

This would make sense as SteamClient.Connect() should always disconnect the old session and initiate a connect, instead, it’s stuck in your case, and SK2 still reports that SteamClient.IsConnected.

I’m not sure if there is anything that ASF can do in this case, it should be handled in SK2 library, preferably implementing a timeout and force reconnect instead of something like this. I’ve already experienced this problem - it’s very rare, but it looks like SteamClient simply freezes and it’s not possible to make it reconnect even with SteamClient.Connect() - that’s why your keys redeeming requests are failing, because ASF properly tries to send the request, but Steam network never responds back, as SteamClient is frozen. It became frozen the moment it tried to log in to ISteamUserAuth the last time (and failed) - that initiated a reconnect request that was never fullified.

It’s probably best to ask SK2 devs if they have any clue why SteamClient can reach such state of being unable to operate, and if perhaps it’s possible to do anything to overcome this problem apart from throwing away old object and creating new SteamClient(). Maybe @Netshroud could say something about that.

JustArchi on Jul 31, 2016

This can be true as I didn’t spot any exception back when I had this issue for the first time in ArchiBoT as well, thanks for checking.

JustArchi on Jul 31, 2016

If SteamClient is frozen then no request can reach ASF, even if you do make it reach ASF, through other bot instance (like you did), then redeem request will timeout anyway as SteamClient is non-operative. Thing is, during normal and even faulty operation SteamClient should NEVER reach such state, as SteamClient.Connect() should always force a reconnect, and ASF has routines to initiate reconnect if it’s needed - it did initiate reconnect in your case.

This is somehow problematic as the workaround for this issue would require throwing away currently coded nice way of SteamClient.Disconnect() and SteamClient.Connect() and instead initialize new object on each reconnect attempt. I don’t want to do this as this doesn’t make any sense - SteamClient should be reusable and there should be never a need to throw it away, unless we’re done with it for good.

So for now I don’t have any real workaround for this, I might need to look into SK2 code and check how the whole connection is being handled to spot the deadlock/issue we’re seeing, but hopefully @Netshroud or other dev from @SteamRE/SteamKit2 can at least confirm if my assumptions are right.

Because this is the only possible reason why SteamClient.Connect() freezes and doesn’t return any callback - neither OnConnected() nor OnDisconnected(). If it did, then ASF would react accordingly, but the callback never arrived, so ASF is still waiting for SteamClient.Connect() to do something.

JustArchi on Jul 31, 2016

Yup. Here you go:

http://pastebin.com/raw/H5LWHj5G

guihkx on Jul 30, 2016