terraform-provider-confluent: Error customizing diff Schema: 422 Unprocessable Entity

I started seeing this error customizing diff Schema: 422 Unprocessable Entity after upgrading to https://github.com/confluentinc/terraform-provider-confluent/releases/tag/v1.29.0

I a proto file that looks like this:

syntax = "proto3";

package examples.v1alpha;

import "proto_options/kafka/v1alpha/kafka_options.proto";

// Some proto message.
// v=3
message Person {
  string name = 1;

  option (proto_options.kafka.v1alpha.kafka) = {};
}

And a custom options that looks like this:

syntax = "proto3";

import "google/protobuf/descriptor.proto";

package proto_options.kafka.v1alpha;

message KafkaOptions {
  map<string, string> config = 2;
}

extend google.protobuf.MessageOptions {
  repeated KafkaOptions kafka = 80000;
}

After applying a change to a comment in the schema file:

  // Some proto message.
-// v=3
+// v=4
 message Person {
   string name = 1;

I get this error:

╷
│ Error: error customizing diff Schema: 422 Unprocessable Entity
│ 
│   with module.kafka.confluent_schema.schemas["examples.v1alpha.person-value"],
│   on .terraform/modules/kafka/modules/kafka-topic/main.tf line 69, in resource "confluent_schema" "schemas":
│   69: resource "confluent_schema" "schemas" {
│

debug logs logs

[DEBUG] provider.terraform-provider-confluent_1.29.0: Finished reading Schema "xxx-xxxxx/examples.v1alpha.person-value/latest": schema_id=xxx-xxxxx/examples.v1alpha.person-value/latest tf_provider_addr=provider @caller=src/github.com/confluentinc/terraform-provider-confluent/internal/provider/resource_schema.go:373 @module=provider tf_resource_type=confluent_schema tf_rpc=ReadResource tf_req_id=b9f5d916-0a5f-d670-9522-91adc1d755be timestamp=2023-02-09T15:47:27.976-0500
[DEBUG] provider.terraform-provider-confluent_1.29.0: Customizing diff new Schema: {"references":[],"schema":"syntax = \"proto3\";\n\npackage examples.v1alpha;\n\nimport \"proto_options/kafka/v1alpha/kafka_options.proto\";\n\n// Some proto message.\n// v=4\nmessage Person {\n  string name = 1;\n\n  option (proto_options.kafka.v1alpha.kafka) = {};\n}","schemaType":"PROTOBUF"}: @module=provider tf_provider_addr=provider @caller=src/github.com/confluentinc/terraform-provider-confluent/internal/provider/resource_schema.go:209 tf_req_id=b6f0b108-c337-6f90-5d75-6a71a46d9fb6 tf_resource_type=confluent_schema tf_rpc=PlanResourceChange timestamp=2023-02-09T15:47:27.980-0500
[DEBUG] provider.terraform-provider-confluent_1.29.0: 2023/02/09 15:47:27 [DEBUG] POST https://xxx-xxxxx.us-central1.gcp.confluent.cloud/subjects/examples.v1alpha.person-value?normalize=true
[ERROR] vertex "module.kafka.confluent_schema.schemas[\"examples.v1alpha.person-value\"]" error: error customizing diff Schema: 422 Unprocessable Entity
[ERROR] vertex "module.kafka.confluent_schema.schemas (expand)" error: error customizing diff Schema: 422 Unprocessable Entity
module.kafka.data.confluent_kafka_cluster.cluster: Read complete after 1s [id=xxx-xxxxx]

About this issue

Original URL
State: closed
Created a year ago
Reactions: 1
Comments: 21

Most upvoted comments

@linouk23 Awesome…

I can confirm it works…

Thanks 👍

FabioBatSilva on Feb 16, 2023

No prob…

I think this did it …

Before:

resource "confluent_schema" "person" {
  #... 

  rest_endpoint      = data.confluent_schema_registry_cluster.registry.rest_endpoint
  subject_name       = "terraform.examples.v1alpha.person"
  format             = "PROTOBUF"
  recreate_on_update = false

  schema             = <<-EOT
    syntax = "proto3";

    package examples.v1alpha;

    message Person {
      string name = 1;
    }
  EOT
}

After:

resource "confluent_schema" "kafka_options" {

  #... 


  subject_name       = "terraform.proto_options.kafka.v1alpha.kafka_options"
  format             = "PROTOBUF"
  recreate_on_update = false
  schema             = <<-EOT
    syntax = "proto3";

    import "google/protobuf/descriptor.proto";

    package proto_options.kafka.v1alpha;

    message KafkaOptions {
      map<string, string> config = 2;
    }

    extend google.protobuf.MessageOptions {
      repeated KafkaOptions kafka = 80000;
    }
  EOT
}

resource "confluent_schema" "person" {
  
  #... 


  subject_name       = "terraform.examples.v1alpha.person"
  format             = "PROTOBUF"
  recreate_on_update = false
  schema             = <<-EOT
    syntax = "proto3";

    package examples.v1alpha;

    import "proto_options/kafka/v1alpha/kafka_options.proto";

    message Person {
      string name = 1;

      option (proto_options.kafka.v1alpha.kafka) = {};
    }
  EOT

  schema_reference {
    name         = "proto_options/kafka/v1alpha/kafka_options.proto"
    subject_name = confluent_schema.kafka_options.subject_name
    version      = confluent_schema.kafka_options.version
  }
}

It seems to happens only when the state changes from the Before to the After version.

Running a fresh install on the After version seems to work fine… 🤔

FabioBatSilva on Feb 15, 2023

To clarify…

It looks like it happens to any change where I have the option in place…

For instance:

 message Person {
   string name = 1;
+  string name2 = 2;
 
   option (proto_options.kafka.v1alpha.kafka) = {};
 }

│ Error: error customizing diff Schema: 422 Unprocessable Entity
│ 
│   with module.kafka.confluent_schema.schemas["examples.v1alpha.person-value"],
│   on .terraform/modules/kafka/modules/kafka-topic/main.tf line 69, in resource "confluent_schema" "schemas":
│   69: resource "confluent_schema" "schemas" {

If i also remove the option it plan works.

 message Person {
   string name = 1;
-
-  option (proto_options.kafka.v1alpha.kafka) = {};
+  string name2 = 2;
 }

  ~ resource "confluent_schema" "schemas" {
        id                 = "xxxx-xxx/examples.v1alpha.person-value/latest"
      ~ schema             = <<-EOT
            syntax = "proto3";
            
            package examples.v1alpha;
            
            import "proto_options/kafka/v1alpha/kafka_options.proto";
            
            // Some proto message.
            // v=3
            message Person {
              string name = 1;
          - 
          -   option (proto_options.kafka.v1alpha.kafka) = {};
          +   string name2 = 2;
            }
        EOT
    }

But when using the UI i’m able to evolve the schema with the option present…

FabioBatSilva on Feb 9, 2023