dbt-databricks: Too many redirects - Response Error when running "dbt seed"

Describe the bug

Running a dbt job on databricks workflows with a custom profile. The job exits with exit code 2 with an error message shown below (too many redirects) when running dbt seed. I’m running couple of other dbt commands before running dbt seed and they run just fine - dbt debug gives a proper out with all tests passing but still fails with database error when running dbt seed.

Sreenshots provided below

Steps To Reproduce

Regular dbt workflow setup with custom profile as mentioned in the docs.

Expected behavior

If I issue the same command (dbt seed) locally with my personal access token - it works but fails when running as a part of a workflow. I expect it to just run normally without failing.

Screenshots and log output

dbt debug output log: Screenshot 2023-09-22 at 11 13 12 am

dbt seed output log: Screenshot 2023-09-22 at 11 15 38 am

System information

dbt version = 1.6.2 dbt-databricks adapter version = 1.6.4 operating system = Linux Python version = 3.9

Additional context

If I issue the same command (dbt seed) locally with my personal access token - it works but fails when running as a part of a workflow. Requesting guidance on how I can avoid hitting this error. Any help is appreciated sinc the error message doesn’t really tell us a lot.

I have tried looking for this error within the adapter but looks like any database error message is just relayed by the adapter.

About this issue

  • Original URL
  • State: closed
  • Created 9 months ago
  • Comments: 15

Most upvoted comments

I’ve brought your issue to the attention of the Jobs team; will update here when we figure something out.