edge-runtime: Local Edge functions: event loop error after a few minutes

Describe the bug

It seems that after my edge function has been running locally for some time (perhaps a few minutes) I start to get these messages in the logs:

wall clock duration warning. isolate: df3c6dc8-dcc9-4348-a278-e0e1ff102fc3
wall clock duration warning. isolate: 93b745a9-cdb0-49a0-ae88-514a87391cd6
wall clock duration warning. isolate: ce00ba14-d7f7-4e27-8b6e-92191f485a81
wall clock duration warning. isolate: 39ebddfd-8b3e-4af6-a966-178f821877b6

And then eventually I see this:

event loop error: TypeError: Cannot convert undefined or null to object
    at slice (<anonymous>)
    at new DOMException (ext:deno_web/01_dom_exception.js:126:14)
    at [[[signalAbort]]] (ext:deno_web/03_abort_signal.js:83:14)
    at AbortController.abort (ext:deno_web/03_abort_signal.js:172:30)
    at file:///home/deno/main/index.ts:147:31
    at Object.action (ext:deno_web/02_timers.js:151:11)
    at handleTimerMacrotask (ext:deno_web/02_timers.js:65:10)
    at eventLoopTick (ext:core/01_core.js:189:21)

After that error happens, all my requests fail with this:

Error in worker connection: connection closed before message completed
request failed (uri: "/my-function" reason: hyper::Error(Canceled, "connection was not ready"))

The function is as simple as this:

Deno.serve(async (req) => {
  return new Response(
    JSON.stringify({ publicToken: "123" }),
    {
      headers: { "Content-Type": "application/json" },
    }
  );
});

To Reproduce Steps to reproduce the behavior:

Create a new function with the above code
run supabase functions serve
Call the function at least once (not sure if this is actually necessary)
Wait a few minutes… You should eventually see the above event loop error
Try to call the function again. You will get "connection was not ready" error

Expected behavior Local functions should not randomly crash after only a few minutes 😃

Desktop (please complete the following information):

OS: OSX
Version of CLI: 1.110.1

Additional context

At least one other person has reported experiencing this problem in the supabase discord (The above report is using their description).

About this issue

Original URL
State: open
Created 8 months ago
Reactions: 4
Comments: 18

Most upvoted comments

I had this is issue too on my M2 mac until i upgraded Docker Desktop (it was very much out of date) as well as supabase. Now running without a single crash on supabase 1.112.2 and Docker Desktop 4.25.1 (128006). Still seeing the wall clock duration warning though, but no crashes.

hyldmo on Nov 16, 2023

I as well am getting related issues.

I get wall clock log msgs like these:

wall clock duration reached. isolate: d66d7171-0472-48db-a589-a52454d89a26
isolate memory usage sender dropped
wall clock duration warning. isolate: 476ff220-04b9-41ee-a5dc-7db65464616a
wall clock duration reached. isolate: 476ff220-04b9-41ee-a5dc-7db65464616a
isolate memory usage sender dropped

I also am having an issue where I cannot use the existsSync() and Deno.readDirSync() functions. When running these, I get the following error:

runtime has escaped from the event loop unexpectedly: event loop error: TypeError: Deno.statSync is not a function
    at existsSync (https://deno.land/std@0.220.0/fs/exists.ts:121:23)
    at EnvService.dirDoesNotExist (file:///home/deno/functions/instructor-ui/services/env-service.ts:100:13)
    at EnvService.validateInitEnvFile (file:///home/deno/functions/instructor-ui/services/env-service.ts:65:14)
    at EnvService.loadEnv (file:///home/deno/functions/instructor-ui/services/env-service.ts:22:48)
    at file:///home/deno/functions/instructor-ui/index.ts:5:12
failed to send request to user worker: connection error: Connection reset by peer (os error 104)
user worker failed to respond: connection error: Connection reset by peer (os error 104)
InvalidWorkerResponse: user worker failed to respond
    at async Promise.allSettled (index 1)
    at async UserWorker.fetch (ext:sb_user_workers/user_workers.js:79:21)
    at async Server.<anonymous> (file:///home/deno/main/index.ts:146:12)
    at async #respond (https://deno.land/std@0.182.0/http/server.ts:220:18) {
  name: "InvalidWorkerResponse"
}

I have tried to search for any open or closed issues on this to see if I can get some more info but have come up dry.

CalvinWilkinson on Mar 15, 2024

Any comments on whether the wall clock duration reached. isolate: 11455258470253939329 logs are a problem? It kind of seems like the connection isn’t being closed out or something, but the response it getting returned from the endpoint. Is there something else we should be doing to tell supabase that the request is complete?

rothnic on Nov 29, 2023

Thanks @hyldmo. Updating Docker Desktop and my supabase CLI resolved the crashing issue.

scottweinert on Nov 17, 2023

I’m also experiencing this issue with CLI version 1.110.1

d-e-h-i-o on Nov 15, 2023

yep. same

rbkayz on Nov 15, 2023

I’m also seeing this issue, can’t update supabase beyond 1.106.1 or edge fns break every few minutes.

evelant on Nov 10, 2023

Transferring because it sounds like an issue with edge runtime container.

sweatybridge on Nov 9, 2023