status-go: Rapid log file growth and repeated Waku errors when DNS encryption is not set
Issue Description
When running the Status app without setting encrypted DNS (e.g., https://9.9.9.9/dns-query), the geth.log file grows rapidly within seconds of opening the app. The log is filled with repetitive error messages, primarily related to Waku and peer selection.
Observed Behavior
- Without encrypted DNS, opening the Status app causes geth.log to generate a large volume of repeated error logs within seconds.
- The log file size increases rapidly, potentially overwriting initial logs (e.g., ‘status-go version details’).
- The majority of error messages are related to peer selection and Waku.
Log Analysis
The log shows multiple recurring errors:
- “peer selection returned err”
- “no suitable peers found”
- “dns discovery error”
- “could not obtain dns discovery peers”
Example log snippet:
ERROR[08-08|08:12:56.847][github.com/status-im/status-go/vendor/github.com/waku-org/go-waku/waku/v2/protocol/filter/client.go:350] peer selection returned err
error="no suitable peers found"
Potential Causes
- GFW (Great Firewall) interference: The issue may be related to network restrictions in certain regions.
- Waku-related code issues in the Status app.
- DNS resolution problems: Default DNS may fail to resolve Ethereum network node addresses correctly.
- Network connectivity issues: Certain network configurations may be blocking the Status app from connecting to the Ethereum network.
Workaround
Setting encrypted DNS (e.g., https://9.9.9.9/dns-query) appears to resolve or mitigate the issue. This suggests that the problem might be related to DNS resolution or network restrictions.
Additional Context
- The issue seems to create a loop-like behavior in Waku-related processes.
- The problem may be more pronounced in regions with strict internet controls.
Questions for Investigation
- Is there a potential infinite loop in the Waku-related code when peer discovery fails?
- How does the encrypted DNS setting affect the peer discovery process?
- Are there any network-level checks that can be implemented to gracefully handle situations where peer discovery consistently fails?
Environment
- Status app version: on branch
develop, commit id:3b4cbfa4bca1b76cf3e9675e2e6b99d14c09b9e1 - ios simulator: iphone 15 pro
- surge v5.7.x
Steps to Reproduce
- Ensure encrypted DNS is not set in surge.
- Open the Status app.
- Monitor the geth.log file for rapid growth and repeated error messages.
relate screenshots and comments:
This might be a serious issue. I just opened the Status app for a few seconds, and the log file size grows rapidly. The content seems to repeatedly relate to Waku, as if there is a dead loop. I can’t even see the initial log like status-go version details because it might be getting overwritten.
after changing log level to INFO , geth.log get fullfilled with this content, so now I’m sure there’s something relate to waku
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 18 (18 by maintainers)
ah, from the logs it looks like somehow connection to storenodes is successfull eventhough they are configured with DNS name. But for some reason node is unable to connect to other peers.
i think we need to add add some backoff to filter subscribe in case of subsequent failures especially for specific errors such as “no suitable peers found” . This would prevent such situation from occuring. Will start working on a PR for this.
thank you for your effort, it works ❤️ @chaitanyaprem
https://github.com/status-im/status-go/pull/5909 is being dogfooded in status-mobile which includes a simple backoff in case of certain errors.
mobile PR is https://github.com/status-im/status-mobile/pull/21386
@qfrank please test this in the same env and confirm the resolution of the issue.
would help if i can get logs to look into it.
cc @fryorcraken would you mind having somebody look at it?