foundry: Failed to deserialize response: expected value at line 1 column 1

Component

Forge

Have you ensured that all of these are up to date?

Foundry
Foundryup

What version of Foundry are you on?

forge 0.2.0 (8307d6d 2023-04-20T00:05:51.425008000Z)

What command(s) is the bug in?

forge verify-contract

Operating System

Linux

Describe the bug

I am running forge in an AWS EC2 instance and on verification keep getting a 403 error from Etherscan which results in the Failed to deserialize response: expected value at line 1 column 1 error as it returns html with a captcha rather than json.

I’ve reached out to Etherscan and this was their response:

We are aware of our CloudFlare protections being particularly sensitive to unknown automated scripts, and returning 403 responses as a result. We’re looking into whitelisting this at the moment, however a quick workaround for this would be to make your request more identifiable by attaching a User Agent string

My understanding is that reqwest doesn’t set a default User-Agent header and so CloudFlare decides to block my request.

Does anyone know of a way to add a User-Agent header to outgoing requests on forge?

This seems like it currently affects just a small subset of users, although it would future proof everyone if a default user agent was set by forge/foundry. For example, curl automatically applies a default user agent so it would make sense for forge/foundry to do the same.

About this issue

Original URL
State: closed
Created a year ago
Reactions: 1
Comments: 30 (6 by maintainers)

Commits related to this issue

fix `forge verify-contract` etherscan cloudflare bug Etherscan verification for non-mainnet chains requires a question mark at the end of the verifier url in order to prevent a forward slash from bei... — committed to tsite/foundry by tsite 8 months ago
fix(forge): fix `verify-contract` etherscan cloudflare bug (#6079) * fix `forge verify-contract` etherscan cloudflare bug Etherscan verification for non-mainnet chains requires a question mark at... — committed to foundry-rs/foundry by tsite 8 months ago

Most upvoted comments

@Evalir yeah totally understood. It seems like the user agent string has not worked and I suspect this has to do with Cloudflare blocking the AWS IP range or something to that extent. The issue still persists with the latest build.

Definitely hard to test locally 😂

Is the foundry cloud build on ubuntu working with verification? Can’t remember if contract verification to etherscan is tested in the github action off the top of my head.

mpeyfuss on Jun 20, 2023

Hey @ratankaliani — we have a fix for contract verification on ethers-rs, we just need to upstream. It’ll be done in the upcoming days

Evalir on Jun 13, 2023

Yeah I agree, would be good to add to these tests or a similar nightly suite to detect when there are Cloudflare errors.

ratankaliani on Oct 10, 2023

I can confirm this issue does not exist for mainnet eth, only testnets it seems. Not sure there is much we can do but we should keep this open until the Etherscan team gives an update.

mpeyfuss on Jun 24, 2023

Issue has resolved for me - I was able to deploy & verify contracts on Optimism Goerli earlier today.

ratankaliani on Jun 20, 2023

Looks like it’s broken on Gnosis as well - @mpeyfuss did you find a fix for Arbitrum Goerli?

ratankaliani on Jun 13, 2023

Latest comms from the Etherscan API team made it seem like this is an issue they are aware of and their suggested User Agent fix doesn’t quite work from my testing. No response back about that. Seems to also be affecting Aribtrum Goerli (which uses etherscan).

I have yet to try on mainnet fully but my initial testing went a lot better on mainnet so it seems restricted to goerli at least.

mpeyfuss on Jun 9, 2023

Ah, unfortunately still getting the same response even with the latest build.

mpeyfuss on May 2, 2023

ah thanks for pointing me to that. I was on an older build but just now built again from source on the EC2. about to test

Not 100% sure the kind of error returned as it doesn’t seem like a regular 403 error. But here is the html body that foundry couldn’t deserialize.

<!DOCTYPE html>
<html lang="en-US">
<head>
    <title>Just a moment...</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=Edge">
    <meta name="robots" content="noindex,nofollow">
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <link href="/cdn-cgi/styles/challenges.css" rel="stylesheet">
    

</head>
<body class="no-js">
    <div class="main-wrapper" role="main">
    <div class="main-content">
        <noscript>
            <div id="challenge-error-title">
                <div class="h2">
                    <span class="icon-wrapper">
                        <div class="heading-icon warning-icon"></div>
                    </span>
                    <span id="challenge-error-text">
                        Enable JavaScript and cookies to continue
                    </span>
                </div>
            </div>
        </noscript>
        <div id="trk_jschal_js" style="display:none;background-image:url('/cdn-cgi/images/trace/captcha/nojs/transparent.gif?ray=7c08c85288277fb7')"></div>
        <form id="challenge-form" action="/api/?__cf_chl_f_tk=Q7F3hkX.VOk8KSj37RUZQAEw34ohIhGvc9K2pEp9d2c-1682952056-0-gaNycGzNMDs" method="POST" enctype="application/x-www-form-urlencoded">
            <input type="hidden" name="md" value="REAALLYY LONG VALUE OMITTED HERE">
        </form>
    </div>
</div>
<script>
    (function(){
        window._cf_chl_opt={
            cvId: '2',
            cZone: 'api-goerli.etherscan.io',
            cType: 'interactive',
            cNounce: '11195',
            cRay: '7c08c85288277fb7',
            cHash: '98433f73c9870d8',
            cUPMDTk: "\\/api\\/?__cf_chl_tk=Q7F3hkX.VOk8KSj37RUZQAEw34ohIhGvc9K2pEp9d2c-1682952056-0-gaNycGzNMDs",
            cFPWv: 'g',
            cTTimeMs: '1000',
            cMTimeMs: '0',
            cTplV: 5,
            cTplB: 'cf',
            cK: "",
            cRq: {
                ru: 'aHR0cHM6Ly9hcGktZ29lcmxpLmV0aGVyc2Nhbi5pby9hcGkv',
                ra: 'Tk9fVUE=',
                rm: 'UE9TVA==',
                d: 'HyOwClHdzKb0QWhXxeflQnZhLi9iZ7fHD9C+JBVClbvYfmbSMr0eTICiMY/XSloNj3BuVgnWI+iu9KVE7ISpl/t5XVS9v+jUUDQib5iMmaaTHcfWA/+cod0Jr95xHjnrv35LHfPkoZPgjKL2zuAlSahaeIRqYat5vw+1SQ9hoApJoTqZ77JRRqrrFodhQdFYRQPT2pl5PrbMb6YrksC26y6IDqUF/3pkVm+A0Qfs0u4Qaxltqm2RvbO7Ibx0MoYHxZyeZhTEa52lwtRNoDvN3vqJWYHlA3A0x8cOQ2YPE1YiuGtF/fjrqTSw9GOXasLgU0UTZXp2xY674bry0b/Cirzb3ElR612wVlOvyo7cfwsljp1lZf5I6OmhQXw/3lTG8lfcsyTYRk7Cvs0vigejlt6VsQXhYhgpXf0OwpPNIsEyjNg4X+7F6AA6zqPwfcSpHIdvmkOc6G+7rH49Ag2L0g9wWGpVGTmWvW9s+Zb/eDogqy3pJrkIVnHBi73V6r7BKMl/LgJoIqolCUlg0DoqdZRIju9ijDlPQYYaYKycLVT033HyzJdDWrJDGKwokRkGIwbZ7+P/eqDpXoUBrr3YzcKfUL5P6aMKrPg66gYrr99etaJ+xnoZ/GmS2/NjqwFZ',
                t: 'MTY4Mjk1MjA1Ni43MjkwMDA=',
                m: '43Q+P85nrusAlMAenGBEOgWlw5AZs3ny6hjIRkI0ew0=',
                i1: 'LNiT0NzLFeZ2oBCfHihVjQ==',
                i2: 'asxnmYxvJS85O7q/1ITnUQ==',
                zh: 'qb4aFuGlbJn/rUOkKXjUqElKDKE10jDqu5PE014OTwk=',
                uh: 'DV4j3Tmrbi5Rs1q3ahwVS6SgbPbI7np5884QO1u1Cgg=',
                hh: 'Ax949TKiHbaXasTISC7ryL1/i3VsF1So3LziNEbpSQM=',
            }
        };
        var trkjs = document.createElement('img');
        trkjs.setAttribute('src', '/cdn-cgi/images/trace/captcha/js/transparent.gif?ray=7c08c85288277fb7');
        trkjs.setAttribute('alt', '');
        trkjs.setAttribute('style', 'display: none');
        document.body.appendChild(trkjs);
        var cpo = document.createElement('script');
        cpo.src = '/cdn-cgi/challenge-platform/h/g/orchestrate/captcha/v1?ray=7c08c85288277fb7';
        window._cf_chl_opt.cOgUHash = location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;
        window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;
        if (window.history && window.history.replaceState) {
            var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;
            history.replaceState(null, null, "\\/api\\/?__cf_chl_rt_tk=Q7F3hkX.VOk8KSj37RUZQAEw34ohIhGvc9K2pEp9d2c-1682952056-0-gaNycGzNMDs" + window._cf_chl_opt.cOgUHash);
            cpo.onload = function() {
                history.replaceState(null, null, ogU);
            };
        }
        document.getElementsByTagName('head')[0].appendChild(cpo);
    }());
</script>


</body>

</html>

mpeyfuss on May 2, 2023