great_expectations: Lowercase table and column names in Snowflake are unreachable using BatchRequest API
Description Lowercase table and column names in Snowflake are unreachable using the BatchRequest API, as the identifiers are not wrapped in quotation marks.
any queries trying to access these columns or tables will fail with the following error
(snowflake.connector.errors.ProgrammingError) 000904 (42000): SQL compilation error: error invalid identifier '<some column>'
To Reproduce Steps to reproduce the behavior:
- Connect to a snowflake database with lowercase column or table names
- Run anything that will query the db (profiler, checkpoint etc…)
- look at output of data docs
Expected behavior
There should be a way to specify that identifiers should be wrapped in quotes. similar to the use_quoted_name kwarg in the v2 BatchKwargs API.
There is an optional argument on the great_expectations.execution_engine.sqlalchemy_batch_data.SqlAlchemyBatchData initializer to use quoted_name but it defaults to false, and there is no way to pass True up to it from a batch request as it is called without **kwargs when fetching a Batch.
Environment:
- Windows
- Great Expectations version 0.15.3
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 15 (9 by maintainers)
Solution is provided in Pull Request #6450 – thank you! /cc @minhuipath @cKennnedy @talagluck
Thanks @alexsherstinsky , I reach out to you in slack to share a bit more details on our situation and will get back to this thread with the finding
@rdodev Many thanks indeed 😃.