Overmind: [BUG] [MAJOR] CPU Reset Routine doesn't work, gets stuck.

Issue summary

Overmind gets stuck looping WARNING 3139967 CPU bucket is critically low (13)! Starting CPU reset routine.

Description of issue:

When Overmind is running out of CPU, it tries to go into CPU reset routine. It never actually goes into it (?) and just tries again every tick, draining CPU the whole time. Occasionally I see that it used the max CPU time limit several ticks in a row, and that’s how it gets to a low number quickly.

After manually pausing the script until enough CPU is saved up, I get: INFO 3140278 Operation suspended until bucket recovery. Bucket: 5520/10000 which seems to be normal behavior

This is on Shard 3

Steps to reproduce:

This has occurred twice:

I was tired of a colony that was in “terminal evacuate” mode for some reason, so I unclaimed it. Overmind immediately spent a lot of CPU doing who knows what (saw the room builder flash for a tick) and got stuck at low CPU (~200, decreasing slowly.) Saved by flushing memory and errant flags, and bypassing execution until bucket was at a safe level.
Now that I had a new room slot, Overmind colonized another room while I was away. Spawn and some extenders were built and RCL was 2, so it likely didn’t happen immediately. Saved by bypassing execution until bucket was at a safe level.

Error message:

// Console output:
WARNING 3139967 CPU bucket is critically low (13)! Starting CPU reset routine.
WARNING 3139968 CPU bucket is critically low (11)! Starting CPU reset routine.
...etc
After manually pausing the script until enough CPU is saved up:
INFO    3140278 Operation suspended until bucket recovery. Bucket: 5520/10000

// Offending line:

Suggested fix (optional):

Other information:

Overmind version: v0.5.1
- Commit hash:

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 16 (7 by maintainers)

Commits related to this issue

Fix (hopefully) CPU reset routine on shard3 (#65) — committed to bencbartlett/Overmind by bencbartlett 5 years ago

Most upvoted comments

Hi,

I’ve made a small change to my script and it now behaves a lot better

Inside:

static shouldRun() {

I have added:

    if (Game.cpu.bucket < 5000) {
        // Additional failsafe for bucket depletion
        log.warning(`CPU bucket is getting low (${Game.cpu.bucket})! Starting CPU reset routine.`);
        Memory.resetBucket = true;
    }

Above:

    if (Memory.resetBucket) {
        if (Game.cpu.bucket < MAX_BUCKET - Game.cpu.limit) {
            log.info(`Operation suspended until bucket recovery. Bucket: ${Game.cpu.bucket}/${MAX_BUCKET}`);
            shouldRun = false;
        }
        else {
            delete Memory.resetBucket;
        }
    }

The script now halts all processing when the bucket reaches 5,000 and waits for it to replenish to 10,000.

Will look at the your fix now too!!

Thanks for looking into this.

mlinky on Feb 22, 2019

@mlinky Glad to hear the bug is fixed now. The bucket recovery strategy was designed for use in shard2 and probably isn’t the best idea for shard3. I plan to improve this later, once I establish a foothold in shard3, but feel free to make a separate issue if you find the current bucket recovery system problematic.

bencbartlett on Feb 27, 2019

Sorry this has taken so long to get around to; I’ve been pretty busy the last two weeks. Here is how I understand this bug: I added Game.cpu.halt() to the CPU reset routine to fix some weird heap issues that I have been experiencing on shard2. However, in low-CPU environments like shard3, the amount of CPU you accumulate per tick is less than the cost of parsing the ~1MB main.js file on the tick following Game.cpu.halt(). This can cause the bucket to dip back below the CPU-reset threshold, causing the endless cycle of CPU resets. I have pushed a short fix in the latest commit which should fix this problem, but I’ll leave this issue open until @Evanito and/or @mlinky can confirm that the problem is no longer an issue.

bencbartlett on Feb 22, 2019

Ah, that is unintended behavior. I’ll look into it again.

bencbartlett on Jan 30, 2019