terraform-provider-confluent: provisioning connector fails

While provisioning Postgres Debezium connector using terraform provider version 0.8.1 connector fails after provisioning. In the confluent control centre you see this message.

Unexpected error occurred with connector. Confluent connect team is looking at your failure. We will reach out to you via support if we need more details. Please check back here for an update. An in the connector logs you see this entry.

{
  "datacontenttype": "application/json",
  "data": {
    "level": "ERROR",
    "context": {
      "connectorId": "lcc-yo6mvk"
    },
    "summary": {
      "connectorErrorSummary": {
        "message": "Non tolerated exception in error handler",
        "rootCause": "Failed to access Avro data from topic name.of.topic : Unauthorized; error code: 401"
      }
    }
  },
  "subject": "lcc-yo6mvk-lcc-yo6mvk-0",
  "specversion": "1.0",
  "id": "23710d52-5689-4599-9002-e2168e0f7231",
  "source": "crn://confluent.cloud/connector=lcc-yo6mvk",
  "time": "2022-05-20T11:04:39.677Z",
  "type": "io.confluent.logevents.connect.TaskFailed"
}

After some time the connector goes from failed to running state. but because of the first failure terraform aborts and the state file doesn’t get updated.

About this issue

Original URL
State: open
Created 2 years ago
Comments: 41

Commits related to this issue

Increase waiting time for provisioning connector resource to avoid provisioning errors (#43) — committed to confluentinc/terraform-provider-confluent by linouk23 2 years ago

Most upvoted comments

@linouk23 the 1.10.0 release fixed our issue.

nmaves on Oct 27, 2022

👋 update: we released a new 1.10.0 version of TF Provider that should mitigate this issue by adding an initial wait when creating a confluent_connector resource.

cc @bjaggi

linouk23 on Oct 27, 2022

@linouk23 @nmaves we are seeing the same issue again…

bjaggi on Oct 24, 2022

Is there any update on the internal ticket? I wonder if it even makes sense to keep the status of the connector as part of terraform state. Imagine a situation where the terraform apply is done as part of an automated CD pipeline.

A developer makes some modification in the configuration, runs a terraform plan and finds the new plan matches with what he/she needs.
Developer opens a PR and merges it to master.
CD pipeline runs terraform apply.

Now if an existing connector fails between steps 2 and 3, for any number of reasons, terraform apply will delete and recreate that connector. This is behaviour in my opinion developer would not be expecting and could cause cascading failures in downstream.

abintrd on Oct 4, 2022

For the record, we received an email and created an internal ticket to track the issue.

linouk23 on Sep 14, 2022

I think that the idea makes perfect sense for temporary workaround.

iamjekyun on Jun 8, 2022

Based on the UI, it seemed like it took 3~5 minutes. Sad part is that we are using atlantis + terraform remote, so it’s pretty hard to import the resource. We’ll look forward to the further release for now.

iamjekyun on Jun 3, 2022

@abintrd we’re happy to let you know we’ve just released 0.9.0 version of TF Provider that fixes the issue.

linouk23 on May 25, 2022