Kubectl генерирует тайм-аут установления связи TLS с частным кластером EKS
Я вижу следующую ошибку при запуске любой команды kubectl, и данные не возвращаются. Эта ошибка возникает при доступе к частному экземпляру AWS EKS через VPN-соединение.
$ kubectl get pods -A -v=9
...
5800 helpers.go:116] Unable to connect to the server: net/http: TLS handshake timeout
Странность этой ошибки заключается в том, что kubectl генерирует ошибку, но выводит все данные модуля при первом запуске kubectl , если нет кэша обнаружения. Но после этого kubectl выйдет из строя и не вернет данных. Если я удалю каталог кеша (rm -rf ~/.kube/cache), kubectl заработает один раз, а затем снова начнет давать сбой, поскольку ~/.kube/cache будет воссоздан.
Например, когда я впервые запускаю kubectl:
$ kubectl get pods -A -v=9
I0718 14:52:58.797861 15292 loader.go:372] Config loaded from file: U:\.kube\config
I0718 14:52:58.806839 15292 round_trippers.go:435] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl.exe/v1.22.0 (wi
ndows/amd64) kubernetes/c2b5237" 'https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=32s'
I0718 14:53:13.037830 15292 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=
32s in 14230 milliseconds
I0718 14:53:13.038981 15292 round_trippers.go:460] Response Headers:
I0718 14:53:13.044027 15292 cached_discovery.go:121] skipped caching discovery info due to Get "https://C21D1C150B2FC9F1252A79875E11C4BC.gr7
.us-east-2.eks.amazonaws.com/api?timeout=32s": net/http: TLS handshake timeout
I0718 14:53:13.051169 15292 round_trippers.go:435] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl.exe/v1.22.0 (wi
ndows/amd64) kubernetes/c2b5237" 'https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=32s'
I0718 14:53:23.063199 15292 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=
32s in 10010 milliseconds
I0718 14:53:23.065975 15292 round_trippers.go:460] Response Headers:
I0718 14:53:23.065975 15292 cached_discovery.go:121] skipped caching discovery info due to Get "https://C21D1C150B2FC9F1252A79875E11C4BC.gr7
.us-east-2.eks.amazonaws.com/api?timeout=32s": net/http: TLS handshake timeout
I0718 14:53:23.114872 15292 shortcut.go:89] Error loading discovery information: Get "https://C21D1C150B2FC9F1252A79875E11C4BC.gr7.us-east-2
.eks.amazonaws.com/api?timeout=32s": net/http: TLS handshake timeout
I0718 14:53:23.114872 15292 round_trippers.go:435] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl.exe/v1.22.0 (wi
ndows/amd64) kubernetes/c2b5237" 'https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=32s'
I0718 14:53:23.266940 15292 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api?timeout=
32s 200 OK in 152 milliseconds
I0718 14:53:23.267518 15292 round_trippers.go:460] Response Headers:
I0718 14:53:23.268082 15292 round_trippers.go:463] Content-Type: application/json
I0718 14:53:23.268082 15292 round_trippers.go:463] Content-Length: 166
I0718 14:53:23.268082 15292 round_trippers.go:463] Date: Mon, 18 Jul 2022 19:53:23 GMT
I0718 14:53:23.268649 15292 round_trippers.go:463] Audit-Id: dfc5cfe6-08d5-46a8-a61c-632dc3a21613
I0718 14:53:23.268649 15292 round_trippers.go:463] Cache-Control: no-cache, private
I0718 14:53:23.307493 15292 request.go:1181] Response Body: {"kind":"APIVersions","versions":["v1"],"serverAddressByClientCIDRs":[{"clientCI
DR":"0.0.0.0/0","serverAddress":"ip-10-10-1-1.us-east-2.compute.internal:443"}]}
I0718 14:53:23.336044 15292 round_trippers.go:435] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubectl.exe/v1.22.0 (wi
ndows/amd64) kubernetes/c2b5237" 'https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/apis?timeout=32s'
I0718 14:53:23.368489 15292 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/apis?timeout
=32s 200 OK in 32 milliseconds
I0718 14:53:23.369867 15292 round_trippers.go:460] Response Headers:
I0718 14:53:23.369867 15292 round_trippers.go:463] Cache-Control: no-cache, private
I0718 14:53:23.369867 15292 round_trippers.go:463] Content-Type: application/json
I0718 14:53:23.369867 15292 round_trippers.go:463] Date: Mon, 18 Jul 2022 19:53:23 GMT
I0718 14:53:23.369867 15292 round_trippers.go:463] Audit-Id: ba3c50bf-66a3-411e-8763-ec302cc78d03
...
И команда возвращает данные модуля. Я заметил, что требуется 3 попытки завивки, прежде чем http вернет 200 OK, и с этого момента все команды завивки кажутся успешными.
После этого, если я запущу другую команду kubectl, я получу следующий вывод ошибки и никаких данных модуля:
$ kubectl get pods -A -v=9 --insecure-skip-tls-verify=true
I0718 14:51:33.249188 1640 loader.go:372] Config loaded from file: U:\.kube\config
I0718 14:51:33.427333 1640 round_trippers.go:435] curl -v -XGET -H "Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json;
as=Table;v=v1beta1;g=meta.k8s.io,application/json" -H "User-Agent: kubectl.exe/v1.22.0 (windows/amd64) kubernetes/c2b5237" 'https://C21D1C150B
2FC9F1252A79875E11C4BC.gr7.us-east-2.eks.amazonaws.com/api/v1/pods?limit=500'
I0718 14:51:47.439207 1640 round_trippers.go:454] GET https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/api/v1/pods?
limit=500 in 14011 milliseconds
I0718 14:51:47.440457 1640 round_trippers.go:460] Response Headers:
I0718 14:51:47.453797 1640 helpers.go:235] Connection error: Get https://ABCDEFG12345.AB1.us-east-2.eks.amazonaws.com/a
pi/v1/pods?limit=500: net/http: TLS handshake timeout
F0718 14:51:47.453797 1640 helpers.go:116] Unable to connect to the server: net/http: TLS handshake timeout
goroutine 1 [running]:
k8s.io/kubernetes/vendor/k8s.io/klog/v2.stacks(0xc0000d4001, 0xc000804000, 0x6f, 0xf9)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1026 +0xbf
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).output(0x300ff60, 0xc000000003, 0x0, 0x0, 0xc00012c0e0, 0x2, 0x271bb69, 0xa, 0x74, 0x2bef0
0)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:975 +0x1fb
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).printDepth(0x300ff60, 0xc000000003, 0x0, 0x0, 0x0, 0x0, 0x2, 0xc000788270, 0x1, 0x1)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:735 +0x190
k8s.io/kubernetes/vendor/k8s.io/klog/v2.FatalDepth(...)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1500
k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util.fatal(0xc0000af450, 0x41, 0x1)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util/helpers.go:94 +0x296
k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util.checkErr(0x218bc20, 0xc000004198, 0x2003930)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util/helpers.go:178 +0x8b5
k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util.CheckErr(...)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/util/helpers.go:116
k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/get.NewCmdGet.func2(0xc000376280, 0xc0000dc880, 0x1, 0x4)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/cmd/get/get.go:180 +0x15d
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute(0xc000376280, 0xc0000dc840, 0x4, 0x4, 0xc000376280, 0xc0000dc840)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:856 +0x2c2
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc0003bcc80, 0xc0000e0000, 0xc0000de000, 0x6)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:960 +0x375
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute(...)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:897
main.main()
_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/kubectl.go:49 +0x234
goroutine 19 [chan receive]:
k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).flushDaemon(0x300ff60)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1169 +0x92
created by k8s.io/kubernetes/vendor/k8s.io/klog/v2.init.0
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:420 +0xe5
goroutine 21 [select]:
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x2003838, 0x2189500, 0xc000574000, 0x6c612079786f7201, 0xc000082ba0)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:167 +0x1
19
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x2003838, 0x12a05f200, 0x0, 0x6c74636562756b01, 0xc000082ba0)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133 +0x9
f
k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait.Until(0x2003838, 0x12a05f200, 0xc000082ba0)
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:90 +0x54
created by k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/util/logs.InitLogs
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/kubectl/pkg/util/logs/logs.go:51 +0x9e
Я попробовал установить переменную env NO_PROXY, но это не помогло.
Есть предположения?