rancher: Malformed RKE2 ETCD snapshot name causes Rancher memory leak

Rancher Server Setup

Rancher version: 2.6.5 installed via Helm chart on RKE2.

Information about the Cluster

Kubernetes version: v1.22.9+rke2r2
Local cluster encountered the error, but is caused from a snapshot of a downstream cluster. Both clusters are the same rancher and K8s version.

User Information

What is the role of the user logged in? Admin

Describe the bug

While setting up an RKE2 cluster a discovered snapshot in S3 had a malformed name, something similar to something.something.-.s3-.com even though all other snapshots were fine. All snapshots were created via RKE2 automated snapshots set to take one every hour and retain 72 of them.

Rancher was attempting to cleanup old snapshots as they were discovered via the s3 bucket, and when it crossed this snapshot it would panic and stop the process. This would happen until the Rancher container consumed enough memory to start OOM killing processes on the node and crash the RKE2 service.

Once the service crashed the pod would be rescheduled and a different Rancher pod would take up the task and the process would continue.

Removing the malformed name from the snapshot configmap and removing the snapshot from S3 fixed this issue.

To Reproduce

Rename an etcd RKE2 snapshot to an unacceptable name that Rancher can’t parse.

Result

Rancher panics when cleaning up old snapshots and causes a memory leak until Rancher pod reaches it’s resource limits or crashes the host node.

Expected Result

Rancher handles not being able to parse a bad name (as they happen) and instead just skips it and moves on with a log or way to notify operators.

About this issue

Original URL
State: open
Created 2 years ago
Comments: 23 (12 by maintainers)

Most upvoted comments

Just got a report of this today via @Brianalmeida :

Jun 03 20:29:23 ip-10-242-63-234.us-east-2.compute.internal rke2[13846]: time="2022-06-03T20:29:23Z" level=info msg="Reconciliation of snapshot data in rke2-etcd-snapshots ConfigMap complete" Jun 03 20:29:23 ip-10-242-63-234.us-east-2.compute.internal rke2[13846]: time="2022-06-03T20:29:23Z" level=error msg="Failed to record snapshots for cluster: ConfigMap \"rke2-etcd-snapshots\" is invalid: [data[s3-c-9r7sc-rs-w2dks_2022-05-24T17:14:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-w2dks_2022-05-24T17:14:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-djzbw_2022-05-25T11:29:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-djzbw_2022-05-25T11:29:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-vpxkq_2022-05-26T23:59:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-vpxkq_2022-05-26T23:59:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-29422-rs-2b8r8_2022-05-12T11:20:15Z.zip]: Invalid value: \"s3-c-29422-rs-2b8r8_2022-05-12T11:20:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-zbjln_2022-05-09T01:15:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-zbjln_2022-05-09T01:15:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-wtg5x_2022-05-06T00:15:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-wtg5x_2022-05-06T00:15:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-7xh47_2022-05-24T11:09:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-7xh47_2022-05-24T11:09:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-29422-rs-b9s4g_2022-05-12T05:15:15Z.zip]: Invalid value: \"s3-c-29422-rs-b9s4g_2022-05-12T05:15:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-9gmrm_2022-05-21T10:09:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-9gmrm_2022-05-21T10:09:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-hcrnv_2022-05-21T04:04:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-hcrnv_2022-05-21T04:04:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-4vb72_2022-05-21T16:14:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-4vb72_2022-05-21T16:14:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-fqxvf_2022-05-22T10:29:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-fqxvf_2022-05-22T10:29:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-vnzq8_2022-05-23T04:44:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-vnzq8_2022-05-23T04:44:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-hvf99_2022-05-10T19:50:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-hvf99_2022-05-10T19:50:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-xxlkp_2022-05-27T18:14:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-xxlkp_2022-05-27T18:14:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-bdwwh_2022-05-20T21:59:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-bdwwh_2022-05-20T21:59:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-h45dc_2022-05-27T06:04:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-h45dc_2022-05-27T06:04:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-gqb94_2022-05-23T22:59:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-gqb94_2022-05-23T22:59:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-82kxn_2022-05-26T17:54:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-82kxn_2022-05-26T17:54:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-6pkh4_2022-05-06T06:20:17Z.zip]: Invalid value: \"s3-c-zrjnb-rs-6pkh4_2022-05-06T06:20:17Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-9b6qd_2022-05-21T22:19:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-9b6qd_2022-05-21T22:19:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-tdrcd_2022-05-11T14:05:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-tdrcd_2022-05-11T14:05:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-lxscc_2022-05-08T13:05:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-lxscc_2022-05-08T13:05:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-qtxx2_2022-05-11T08:00:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-qtxx2_2022-05-11T08:00:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-mvvgq_2022-05-05T06:00:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-mvvgq_2022-05-05T06:00:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-xndlb_2022-05-04T23:55:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-xndlb_2022-05-04T23:55:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-tjjz2_2022-05-10T13:45:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-tjjz2_2022-05-10T13:45:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-29422-rs-m6nlb_2022-05-11T17:05:15Z.zip]: Invalid value: \"s3-c-29422-rs-m6nlb_2022-05-11T17:05:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-zzgtf_2022-05-23T10:49:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-zzgtf_2022-05-23T10:49:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-w8bsq_2022-05-10T01:35:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-w8bsq_2022-05-10T01:35:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-lm544_2022-05-24T05:04:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-lm544_2022-05-24T05:04:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-29422-rs-tvdh5_2022-05-12T17:25:15Z.zip]: Invalid value: \"s3-c-29422-rs-tvdh5_2022-05-12T17:25:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-wm9bp_2022-05-09T19:30:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-wm9bp_2022-05-09T19:30:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-p57tf_2022-05-09T13:25:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-p57tf_2022-05-09T13:25:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-hs4cn_2022-05-07T18:50:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-hs4cn_2022-05-07T18:50:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-dkwmc_2022-05-20T15:54:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-dkwmc_2022-05-20T15:54:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-6hxpk_2022-05-05T12:05:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-6hxpk_2022-05-05T12:05:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-vgffw_2022-05-07T06:40:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-vgffw_2022-05-07T06:40:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-b9x6p_2022-05-04T11:45:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-b9x6p_2022-05-04T11:45:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-zcl99_2022-05-24T23:19:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-zcl99_2022-05-24T23:19:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-9hdkw_2022-05-25T17:34:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-9hdkw_2022-05-25T17:34:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-n6glm_2022-05-06T18:30:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-n6glm_2022-05-06T18:30:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-rpvsv_2022-05-06T12:25:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-rpvsv_2022-05-06T12:25:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-rhsjh_2022-05-07T00:35:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-rhsjh_2022-05-07T00:35:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-8ncrn_2022-05-28T00:19:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-8ncrn_2022-05-28T00:19:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-5tkkc_2022-05-26T05:44:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-5tkkc_2022-05-26T05:44:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-jnj2v_2022-05-05T18:10:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-jnj2v_2022-05-05T18:10:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-jdgbj_2022-05-23T16:54:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-jdgbj_2022-05-23T16:54:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-29422-rs-gvwzz_2022-05-11T23:10:15Z.zip]: Invalid value: \"s3-c-29422-rs-gvwzz_2022-05-11T23:10:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-vzzn7_2022-05-22T22:39:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-vzzn7_2022-05-22T22:39:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-9t9vq_2022-05-26T11:49:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-9t9vq_2022-05-26T11:49:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-7phln_2022-05-22T16:34:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-7phln_2022-05-22T16:34:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-9hjwq_2022-05-08T07:00:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-9hjwq_2022-05-08T07:00:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-pqvfz_2022-05-09T07:20:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-pqvfz_2022-05-09T07:20:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-p4q5t_2022-05-08T19:10:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-p4q5t_2022-05-08T19:10:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-v2gl8_2022-05-10T07:40:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-v2gl8_2022-05-10T07:40:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-57fzg_2022-05-11T01:55:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-57fzg_2022-05-11T01:55:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-22h5c_2022-05-08T00:55:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-22h5c_2022-05-08T00:55:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-vcw6f_2022-05-04T17:50:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-vcw6f_2022-05-04T17:50:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-wg476_2022-05-25T23:39:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-wg476_2022-05-25T23:39:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-ttpk8_2022-05-04T05:40:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-ttpk8_2022-05-04T05:40:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-nbnsz_2022-05-25T05:24:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-nbnsz_2022-05-25T05:24:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-zrjnb-rs-49m2w_2022-05-07T12:45:15Z.zip]: Invalid value: \"s3-c-zrjnb-rs-49m2w_2022-05-07T12:45:15Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-ltg9p_2022-05-22T04:24:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-ltg9p_2022-05-22T04:24:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+'), data[s3-c-9r7sc-rs-4wvrm_2022-05-27T12:09:18Z.zip]: Invalid value: \"s3-c-9r7sc-rs-4wvrm_2022-05-27T12:09:18Z.zip\": a valid config key must consist of alphanumeric characters, '-', '_' or '.' (e.g. 'key.name', or 'KEY_NAME', or 'key-name', regex used for validation is '[-._a-zA-Z0-9]+')]" Jun 03 20:29:25 ip-10-242-63-234.us-east-2.compute.internal rke2[13846]: time="2022-06-03T20:29:25Z" level=info msg="Stopped tunnel to 127.0.0.1:9345" Jun 03 20:29:25 ip-10-242-63-234.us-east-2.compute.internal rke2[13846]: time="2022-06-03T20:29:25Z" level=info msg="Connecting to proxy" url="wss://10.242.63.234:9345/v1-rke2/connect" Jun 03 20:29:25 ip-10-242-63-234.us-east-2.compute.internal rke2[13846]: time="2022-06-03T20:29:25Z" level=info msg="Proxy done" err="context canceled" url="wss://127.0.0.1:9345/v1-rke2/connect" Jun 03 20:29:25 ip-10-242-63-234.us-east-2.compute.internal rke2[13846]: time="2022-06-03T20:29:25Z" level=info msg="error in remotedialer server [400]: websocket: close 1006 (abnormal closure): unexpected EOF"

brandond on Jun 14, 2022