status-go: Rapid log file growth and repeated Waku errors when DNS encryption is not set

Issue Description

When running the Status app without setting encrypted DNS (e.g., https://9.9.9.9/dns-query), the geth.log file grows rapidly within seconds of opening the app. The log is filled with repetitive error messages, primarily related to Waku and peer selection.

Observed Behavior

  1. Without encrypted DNS, opening the Status app causes geth.log to generate a large volume of repeated error logs within seconds.
  2. The log file size increases rapidly, potentially overwriting initial logs (e.g., ‘status-go version details’).
  3. The majority of error messages are related to peer selection and Waku.

Log Analysis

The log shows multiple recurring errors:

  • “peer selection returned err”
  • “no suitable peers found”
  • “dns discovery error”
  • “could not obtain dns discovery peers”

Example log snippet:

ERROR[08-08|08:12:56.847][github.com/status-im/status-go/vendor/github.com/waku-org/go-waku/waku/v2/protocol/filter/client.go:350] peer selection returned err
error="no suitable peers found"

Potential Causes

  1. GFW (Great Firewall) interference: The issue may be related to network restrictions in certain regions.
  2. Waku-related code issues in the Status app.
  3. DNS resolution problems: Default DNS may fail to resolve Ethereum network node addresses correctly.
  4. Network connectivity issues: Certain network configurations may be blocking the Status app from connecting to the Ethereum network.

Workaround

Setting encrypted DNS (e.g., https://9.9.9.9/dns-query) appears to resolve or mitigate the issue. This suggests that the problem might be related to DNS resolution or network restrictions.

Additional Context

  • The issue seems to create a loop-like behavior in Waku-related processes.
  • The problem may be more pronounced in regions with strict internet controls.

Questions for Investigation

  1. Is there a potential infinite loop in the Waku-related code when peer discovery fails?
  2. How does the encrypted DNS setting affect the peer discovery process?
  3. Are there any network-level checks that can be implemented to gracefully handle situations where peer discovery consistently fails?

Environment

  • Status app version: on branch develop, commit id: 3b4cbfa4bca1b76cf3e9675e2e6b99d14c09b9e1
  • ios simulator: iphone 15 pro
  • surge v5.7.x

Steps to Reproduce

  1. Ensure encrypted DNS is not set in surge.
  2. Open the Status app.
  3. Monitor the geth.log file for rapid growth and repeated error messages.

relate screenshots and comments:

This might be a serious issue. I just opened the Status app for a few seconds, and the log file size grows rapidly. The content seems to repeatedly relate to Waku, as if there is a dead loop. I can’t even see the initial log like status-go version details because it might be getting overwritten.

image

after changing log level to INFO , geth.log get fullfilled with this content, so now I’m sure there’s something relate to waku

image update encrypted DNS

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 18 (18 by maintainers)

Most upvoted comments

maybe we could just search the usages on handleFilterSubscribeOptions, and we can find out all the possibles.

would it be possible to share debug logs?

geth-debug.log.zip

also what is the peer-count do you see in the UI?

5

ah, from the logs it looks like somehow connection to storenodes is successfull eventhough they are configured with DNS name. But for some reason node is unable to connect to other peers.

i think we need to add add some backoff to filter subscribe in case of subsequent failures especially for specific errors such as “no suitable peers found” . This would prevent such situation from occuring. Will start working on a PR for this.

#5909 is being dogfooded in status-mobile which includes a simple backoff in case of certain errors.

mobile PR is status-im/status-mobile#21386

@qfrank please test this in the same env and confirm the resolution of the issue.

thank you for your effort, it works ❤️ @chaitanyaprem

https://github.com/status-im/status-go/pull/5909 is being dogfooded in status-mobile which includes a simple backoff in case of certain errors.

mobile PR is https://github.com/status-im/status-mobile/pull/21386

@qfrank please test this in the same env and confirm the resolution of the issue.

Reopen this as this problem is encountered again on latest branch develop. cc @chaitanyaprem

would help if i can get logs to look into it.

cc @fryorcraken would you mind having somebody look at it?