kubernetes: UDP based DNS queries failing for large record-sets.

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:“1”, Minor:“4”, GitVersion:“v1.4.1”, GitCommit:“33cf7b9acbb2cb7c9c72a10d6636321fb180b159”, GitTreeState:“clean”, BuildDate:“2016-10-10T18:19:49Z”, GoVersion:“go1.7.1”, Compiler:“gc”, Platform:“darwin/amd64”} Server Version: version.Info{Major:“1”, Minor:“4”, GitVersion:“v1.4.4”, GitCommit:“3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56”, GitTreeState:“clean”, BuildDate:“2016-10-21T02:42:39Z”, GoVersion:“go1.6.3”, Compiler:“gc”, Platform:“linux/amd64”}

Environment:

  • Cloud provider or hardware configuration: GKE

What happened:

UDP based host queries are failing for large records.

# host api.mixpanel.com
;; connection timed out; no servers could be reached

The same over TCP works

# host -T api.mixpanel.com
api.mixpanel.com has address 169.54.33.186
api.mixpanel.com has address 169.54.33.132
api.mixpanel.com has address 169.54.33.139
api.mixpanel.com has address 169.54.33.207
api.mixpanel.com has address 169.54.33.200
api.mixpanel.com has address 169.54.33.150
api.mixpanel.com has address 169.54.33.136
api.mixpanel.com has address 169.54.33.194
api.mixpanel.com has address 169.54.33.167
api.mixpanel.com has address 169.54.33.155
api.mixpanel.com has address 169.54.33.168
api.mixpanel.com has address 169.54.33.131
api.mixpanel.com has address 169.54.33.164
api.mixpanel.com has address 169.54.33.174
api.mixpanel.com has address 169.54.33.148
api.mixpanel.com has address 169.54.33.152
api.mixpanel.com has address 169.54.33.166
api.mixpanel.com has address 169.54.33.159
api.mixpanel.com has address 169.54.33.181
api.mixpanel.com has address 169.54.33.149
api.mixpanel.com has address 169.54.33.135
api.mixpanel.com has address 169.54.33.163
api.mixpanel.com has address 169.54.33.143
api.mixpanel.com has address 169.54.33.202
api.mixpanel.com has address 169.54.33.173
api.mixpanel.com has address 169.54.33.198
api.mixpanel.com has address 169.54.33.175
api.mixpanel.com has address 169.54.33.183
api.mixpanel.com has address 169.54.33.189
api.mixpanel.com has address 169.54.33.197
api.mixpanel.com has address 169.54.33.144
api.mixpanel.com has address 169.54.33.196
api.mixpanel.com has address 169.54.33.199
api.mixpanel.com has address 169.54.33.151
api.mixpanel.com has address 169.54.33.142
api.mixpanel.com has address 169.54.33.157
api.mixpanel.com has address 169.54.33.182
api.mixpanel.com has address 169.54.33.138
api.mixpanel.com has address 169.54.33.147
api.mixpanel.com has address 169.54.33.169
api.mixpanel.com has address 169.54.33.140
api.mixpanel.com has address 169.54.33.191
api.mixpanel.com has address 169.54.33.172
api.mixpanel.com has address 169.54.33.154
api.mixpanel.com has address 169.54.33.177
api.mixpanel.com has address 169.54.33.156
api.mixpanel.com has address 169.54.33.176
api.mixpanel.com has address 169.54.33.145
api.mixpanel.com has address 169.54.33.178
api.mixpanel.com has address 169.54.33.179
api.mixpanel.com has address 169.54.33.201
api.mixpanel.com has address 169.54.33.160
api.mixpanel.com has address 169.54.33.146
api.mixpanel.com has address 169.54.33.165
api.mixpanel.com has address 169.54.33.184
api.mixpanel.com has address 169.54.33.185
api.mixpanel.com has address 169.54.33.130
api.mixpanel.com has address 169.54.33.187
api.mixpanel.com has address 169.54.33.188
api.mixpanel.com has address 169.54.33.158
api.mixpanel.com has address 169.54.33.133
api.mixpanel.com has address 169.54.33.171
api.mixpanel.com has address 169.54.33.192
api.mixpanel.com has address 169.54.33.193
api.mixpanel.com has address 169.54.33.205
api.mixpanel.com has address 169.54.33.195
api.mixpanel.com has address 169.54.33.161
api.mixpanel.com has address 169.54.33.170
api.mixpanel.com has address 169.54.33.141
api.mixpanel.com has address 169.54.33.162
api.mixpanel.com has address 169.54.33.134
api.mixpanel.com has address 169.54.33.180
api.mixpanel.com has address 169.54.33.153
api.mixpanel.com has address 169.54.33.203
api.mixpanel.com has address 169.54.33.204
api.mixpanel.com has address 169.54.33.137
api.mixpanel.com has address 169.54.33.206
api.mixpanel.com has address 169.54.33.190
api.mixpanel.com has address 169.54.33.208
api.mixpanel.com has address 169.54.33.209

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Comments: 16 (9 by maintainers)

Most upvoted comments

This looks to be an issue in the skydns not retrying TCP for truncated forwarded entries. I’m getting a patch together to fix the issue.

From kube-dns logs:

I1103 20:33:44.153546       1 logs.go:41] skydns: failure to forward request "dns: failed to unpack truncated message"