workerd: ๐Ÿ› BUG: Worker <-> Worker request over `custom_domain` returns instant 522 timeout response

Which Cloudflare product(s) does this pertain to?

Workers/Other

What version of Wrangler are you using?

2.9.0

What operating system are you using?

Mac

Describe the Bug

Same zone worker <-> worker requests through custom domains returns an immediate 522 timeout http response.

Custom domains were introduced (partly?) to solve same zone worker <-> worker requests: https://blog.cloudflare.com/custom-domains-for-workers/

However, at least on some POPs the request immediately returns with a 522 response. To reproduce create the workers

// worker1.js
export default {
  fetch(req) {
    return fetch("https://worker2.example.com");
  }
};
// worker2.js
export default {
  fetch(req) {
    return new Response("<html>Hello from worker2</html>", {
      headers: { "content-type": "text/html" }
    });
  }
};

Deploy them (wrangle 2.9.0) with the following configs:

# worker1.toml
name = "example-worker1"
main = "worker1.js"
compatibility_date = "2023-01-31"

[route]
pattern = "worker1.example.com"
custom_domain = true
# worker2.toml
name = "example-worker2"
main = "worker2.js"
compatibility_date = "2023-01-31"

[route]
pattern = "worker2.example.com"
custom_domain = true

One needs to replace example.com with a domain the CF account controls. Then

wrangler publish -c worker2.toml
wrangler publish -c worker1.toml
# wait
curl https://worker2.example.com    # Fine!
curl https://worker1.example.com -v # Status 522

For a deployed example Account: 8f0c2f2271ff857947d9a5b2c38595a0 Zone: 97fd67b98d0cc2080e9d13be10b3bca0

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Reactions: 6
  • Comments: 17 (3 by maintainers)

Most upvoted comments

Weโ€™ve looked into this a bit more, and it looks like itโ€™s an internal bug. Weโ€™re working on finding the root cause, but in the meantime, a workaround is to set your compatibility date to before 2022-04-05.

Last update Jul, 13th. Where is the proper place to track progress on this important issue?

Cross-referencing community thread: https://community.cloudflare.com/t/522-when-worker-proxies-to-another-worker/569561

Hey any update on this, I deployed my workers via terraform and also getting 522 when calling from a.domain.com -> b.domain.com

@KimlikDAO-bot as an update, we originally thought this might be an internal issue but have found that the problem doesnโ€™t surface when everything is done through the dashboard, so this does appear to be a bug in wrangler. weโ€™re continuing to investigate ๐Ÿ‘