cloud-foundation-toolkit: cloud_sql.py returns 400

With some very minor naming changes to fit this into our own codebase but this doesn’t work as it is returning this error on the first run after the creation preview.

Waiting for update [operation-1578326201106-59b7ab48e8013-4ab7a06c-f57787da]...failed.                                                                                                                            
ERROR: (gcloud.deployment-manager.deployments.update) Error in Operation [operation-1578326201106-59b7ab48e8013-4ab7a06c-f57787da]: errors:
- code: RESOURCE_ERROR
  location: /deployments/test-cloud-sql-deploy/resources/cloud-sql-user-1
  message: '{"ResourceType":"gcp-types/sqladmin-v1beta4:users","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"Request
    contains an invalid argument.","status":"INVALID_ARGUMENT","statusMessage":"Bad
    Request","requestPath":"https://www.googleapis.com/sql/v1beta4/projects/<proj id>/instances/cloud-sql-postgres-instance/users","httpMethod":"POST"}}'

Using the cloud_sql.py, cloud_sql.py.schema and cloud_sql_postgres.yaml and deploying using deployment-manager.

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 20 (17 by maintainers)

Most upvoted comments

Ah, that makes sense. Although, is there a way to get the secrets out of secret manager rather than hardcoding them into the yaml file. 😕

We have our own deployment manager configurations in a git repo which we need to move over to cft at some point (we started our work on our own DM templates some 9 months before work even began on cft) and even though it’s a private repo we don’t want to be storing this stuff(passwords) in plain text.

In puppet for example, we store passwords in eyaml(encrypted yaml) so anyone with access to our puppet-control repo doesn’t automatically have access to the passwords/secret keys etc.

@rick-pri The deployment manager team released an update, from now on every value which is called password or secret will be redacted and will not show up in logs or DM portal.

@matthias-baetens Would you mind to give me your yaml so I can reproduce your error?

ocsig on Jun 10, 2020

Thank you for your deep investigation. Let try to reproduce your errors. I have to admit I am not a Postgres expert but let me share my early findings:

The current example fails for postgres

jsonPayload: {
  "absoluteUri": "https://www.googleapis.com/sql/v1beta4/projects/cft-test-workspace-221111/instances/cloud-sql-postgres-instance/users" ,
  "body": {
   "host": "10.1.1.1" ,
   "name": "user1"    
  }
}

For a currently unknown reason if I call the api manually I am getting a better error message, which helped me to find the issue:

{
  "error": {
    "code": 400,
    "message": "Invalid request: Missing native user password for PostgreSQL instance.",
    "errors": [
      {
        "message": "Invalid request: Missing native user password for PostgreSQL instance.",
        "domain": "global",
        "reason": "invalid"
      }
    ]
  }
}

This means the following example is working:

# Example of the Cloud SQL template usage.
# Note that Postgres instances use different instance types then MySQL instances

imports:
  - path: templates/cloud_sql/cloud_sql.py
    name: cloud_sql.py

resources:
  - name: cloud-sql-postgres-instance
    type: cloud_sql.py
    properties:
      region: us-central1
      databaseVersion: POSTGRES_9_6
      instanceType: CLOUD_SQL_INSTANCE
      settings:
        tier: db-f1-micro
        backupConfiguration:
          startTime: '02:00'
          enabled: true
        locationPreference:
          zone: us-central1-c
      users:
        - name: user-1
          host: 10.1.1.1
          password: dummy2
        - name: user-2
          host: 10.1.1.2
          password: dummy3
      databases:
        - name: db-1
        - name: db-2

I will try to find time to create a test for postgres as. Please let me know if this solved your issue or you still having problems with regions/zones?

ocsig on Feb 20, 2020