interactsh: Errors in TXT propagation during SSL certificate update/installation

The certificate update randomly fails due to delays/issues in the TXT record propagation containing the challenge:

# ./interactsh-server -domain abcd.efg -ip xxx.xxx.xxx.xxx -listen-ip xxx.xxx.xxx.xxx
2021/12/15 14:24:28 Creating new order for domains: [*.abcd.efg abcd.efg]
2021/12/15 14:24:28 Order created: https://acme-v02.api.letsencrypt.org/acme/order/123456789/123456789
2021/12/15 14:24:28 Fetching authorization: https://acme-v02.api.letsencrypt.org/acme/authz-v3/123456789
2021/12/15 14:24:28 Fetched authorization: abcd.efg
2021/12/15 14:28:28 Updating challenge for authorization abcd.efg: https://acme-v02.api.letsencrypt.org/acme/chall-v3/123456789/123456789
2021/12/15 14:28:59 An error occurred while applying for an certificate, error: could not generate new certs: error updating authorization abcd.efg challenge: acme: error code 400 "urn:ietf:params:acme:error:dns": During secondary validation: DNS problem: query timed out looking up TXT for _abcd.efg
2021/12/15 14:28:59 Could not generate certs for auto TLS, https will be disabled
2021/12/15 14:28:59 Listening on DNS, SMTP and HTTP ports


$ dig abcd.efg txt # from another box

; <<>> DiG 9.16.1-Ubuntu <<>> hackwithautomation.com txt
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15605
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;abcd.efg.		IN	TXT

;; ANSWER SECTION:
abcd.efg.	0	IN	TXT	""

;; Query time: 8 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Wed Dec 15 13:32:01 UTC 2021
;; MSG SIZE  rcvd: 107

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 4
  • Comments: 15 (6 by maintainers)

Most upvoted comments

@mbrownnycnyc Thanks for looking into this. I confirm the server will pick up both the certificate and private key in those locations but will attempt to renew them starting 30 days before the expiration, as recommended by let’s encrypt. Unfortunately, I couldn’t obtain a new certificate yet due to secondary validation failure. Anyway, the workaround that you provided should generally work.

I’m able to workaround this issue by placing the public cert chain at /root/.config/interactsh/cert.crt and private key at /root/.config/interactsh/cert.key. This doesn’t appear to be documented, but I located it with strace.

I generated a cert for fqdn.com via certbot and following the procedure (create a TXT record at the NS, verified it’s visibility to https://unboundtest.com):

certbot certonly --manual -d *.fqdn.com  --agree-tos \
--no-bootstrap --manual-public-ip-logging-ok --preferred-challenges dns-01 \
-m abuse@fqdn.com  \
--server https://acme-v02.api.letsencrypt.org/directory

Then I created two links:

ln -s /etc/letsencrypt/live/fqdn.com/fullchain.pem /root/.config/interactsh/cert.crt
ln -s /etc/letsencrypt/live/fqdn.com/privkey.pem /root/.config/interactsh/cert.key

I invoked interactsh with:

../go/bin/interactsh-server -domain fqdn.com -auth -debug -listen-ip $(hostname -i)

@OctavianGuzu Thanks for pointing this out, I modified the comment to make it more explicit what I meant (the change introduced is wrong and needs to be modified to return the question’s name as it’s received, hence the correlation logic should be moved to interact-client)

I suspect that the error could be caused by a potential blacklisting of some letsencrypt secondary resolvers by GoDaddy, making them unable to access the exposed acme-challenge txt records. letsencrypt follows a two-step verification, the primary one using https://unboundtest.com/ that works just fine against interactsh.com (https://unboundtest.com/m/TXT/_acme-challenge.interactsh.com/ST2HYD5H)

Query results for TXT _acme-challenge.interactsh.com

Response:
;; opcode: QUERY, status: NOERROR, id: 58706
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;_acme-challenge.interactsh.com. IN  TXT

;; ANSWER SECTION:
_acme-challenge.interactsh.com.  0  IN TXT   "jT9jeUx8fpNnmLupv0zfpb7mioTuFWRUGu-UcTVUJEs"
_acme-challenge.interactsh.com.  0  IN TXT   "kKxS1eHBMJSnq3Bmbqt-z4eApULPtUM3kwSkUkjNVrg"

----- Unbound logs -----
Dec 17 19:39:20 unbound[445800:0] notice: init module 0: validator
Dec 17 19:39:20 unbound[445800:0] notice: init module 1: iterator
Dec 17 19:39:20 unbound[445800:0] info: start of service (unbound 1.12.0).
Dec 17 19:39:21 unbound[445800:0] info: 127.0.0.1 _acme-challenge.interactsh.com. TXT IN
Dec 17 19:39:21 unbound[445800:0] info: resolving _acme-challenge.interactsh.com. TXT IN
Dec 17 19:39:21 unbound[445800:0] info: priming . IN NS
Dec 17 19:39:21 unbound[445800:0] info: response for . NS IN
Dec 17 19:39:21 unbound[445800:0] info: reply from <.> 2001:dc3::35#53
Dec 17 19:39:21 unbound[445800:0] info: query response was ANSWER
Dec 17 19:39:21 unbound[445800:0] info: priming successful for . NS IN
Dec 17 19:39:21 unbound[445800:0] info: response for _acme-challenge.interactsh.com. TXT IN
Dec 17 19:39:21 unbound[445800:0] info: reply from <.> 198.97.190.53#53
Dec 17 19:39:21 unbound[445800:0] info: query response was REFERRAL
Dec 17 19:39:21 unbound[445800:0] info: response for _acme-challenge.interactsh.com. TXT IN
Dec 17 19:39:21 unbound[445800:0] info: reply from <com.> 192.12.94.30#53
Dec 17 19:39:21 unbound[445800:0] info: query response was REFERRAL
Dec 17 19:39:21 unbound[445800:0] info: resolving ns2.interactsh.com. AAAA IN
Dec 17 19:39:21 unbound[445800:0] info: resolving ns1.interactsh.com. AAAA IN
Dec 17 19:39:21 unbound[445800:0] info: response for _acme-challenge.interactsh.com. TXT IN
Dec 17 19:39:21 unbound[445800:0] info: reply from <interactsh.com.> 104.248.51.21#53
Dec 17 19:39:21 unbound[445800:0] info: query response was ANSWER
Dec 17 19:39:21 unbound[445800:0] info: response for _acme-challenge.interactsh.com. TXT IN
Dec 17 19:39:21 unbound[445800:0] info: reply from <interactsh.com.> 104.248.51.21#53
Dec 17 19:39:21 unbound[445800:0] info: query response was ANSWER
Dec 17 19:39:21 unbound[445800:0] info: prime trust anchor
Dec 17 19:39:21 unbound[445800:0] info: generate keytag query _ta-4f66. NULL IN
Dec 17 19:39:21 unbound[445800:0] info: resolving . DNSKEY IN
Dec 17 19:39:21 unbound[445800:0] info: resolving _ta-4f66. NULL IN
Dec 17 19:39:21 unbound[445800:0] info: response for _ta-4f66. NULL IN
Dec 17 19:39:21 unbound[445800:0] info: reply from <.> 2001:dc3::35#53
Dec 17 19:39:21 unbound[445800:0] info: query response was NXDOMAIN ANSWER
Dec 17 19:39:21 unbound[445800:0] info: response for . DNSKEY IN
Dec 17 19:39:21 unbound[445800:0] info: reply from <.> 2001:503:ba3e::2:30#53
Dec 17 19:39:21 unbound[445800:0] info: query response was ANSWER
Dec 17 19:39:21 unbound[445800:0] info: validate keys with anchor(DS): sec_status_secure
Dec 17 19:39:21 unbound[445800:0] info: Successfully primed trust anchor . DNSKEY IN
Dec 17 19:39:21 unbound[445800:0] info: validated DS com. DS IN
Dec 17 19:39:21 unbound[445800:0] info: resolving com. DNSKEY IN
Dec 17 19:39:21 unbound[445800:0] info: response for com. DNSKEY IN
Dec 17 19:39:21 unbound[445800:0] info: reply from <com.> 192.43.172.30#53
Dec 17 19:39:21 unbound[445800:0] info: query response was ANSWER
Dec 17 19:39:21 unbound[445800:0] info: validated DNSKEY com. DNSKEY IN
Dec 17 19:39:21 unbound[445800:0] info: NSEC3s for the referral proved no DS.
Dec 17 19:39:21 unbound[445800:0] info: Verified that unsigned response is INSECURE

The check involves an echo mixed case. Hence the change introduced in https://github.com/projectdiscovery/interactsh/pull/121 should be modified to return in the response the same name record from the request question field (with same uppercase/lowercase letters). interactsh-client should handle the case-insensitive correlation. See https://community.letsencrypt.org/t/certificate-verification-failure-certbot-doesnt-reach-our-dns-server-txt-records/151166 for more details. The txt validation may fail for two reasons:

  • Primary validation: Too short wait time between the txt record creation and challenge verification (default to 10 seconds) => switching to a 10-minute delay seems to allow the propagation to happen and pass the first verification step
  • Secondary validation: the letsencrypt resolvers can’t reach the interactsh DNS server (GoDaddy blacklisting? More info at https://community.letsencrypt.org/t/verify-error-during-secondary-validation-fetching-timeout/134905/3, https://community.letsencrypt.org/t/failure-during-secondary-validation-again/135194/8 and https://community.letsencrypt.org/t/repeated-dns-error-during-secondary-validation-dns-problem-networking-error-looking-up-a/135243/2). The second step also fails with cert-bot with manual step-by-step verification and custom DNS server. Therefore it seems not related to interactsh.

Hi @Mzack9999 ,

Sure, no worries.

I was wondering if you got it to work. “May fail randomly” implies it should work occasionally, which I couldn’t reproduce :p.

All the best!