amundsen: Recieve error "Something went wrong..." when upload data from PostgreSQL database
I have uploaded information about my postgresql tables. When I try to see Information about tables I see error "Something went wrong… "
Expected Behavior
I want to see metadata information about tables.
Current Behavior
I can see that upload was succesfull because information about tables availible in neo4j store.
I have used docker-amundsen.yml script for deloyment. In amundsenfrontend container logs recieve this error
2021-03-14T13:11:33+0000.463 [ERROR] v0._get_table_metadata:149 (11:MainThread) - Encountered exception: {'columns': {0: {'col_type': ['Field may not be null.']}, 1: {'col_type': ['Field may not be null.']}}} Traceback (most recent call last): File "/app/amundsen_application/api/metadata/v0.py", line 143, in _get_table_metadata results_dict['tableData'] = marshall_table_full(table_data_raw) File "/app/amundsen_application/api/utils/metadata_utils.py", line 109, in marshall_table_full table: Table = schema.load(table_dict).data File "/usr/local/lib/python3.7/site-packages/marshmallow/schema.py", line 588, in load result, errors = self._do_load(data, many, partial=partial, postprocess=True) File "/usr/local/lib/python3.7/site-packages/marshmallow/schema.py", line 711, in _do_load raise exc marshmallow.exceptions.ValidationError: {'columns': {0: {'col_type': ['Field may not be null.']}, 1: {'col_type': ['Field may not be null.']}}} 2021-03-14T13:11:33+0000.463 [DEBUG] action_log_callback.on_post_execution:70 (11:MainThread) - Calling callbacks: [<function logging_action_log at 0x7f26506bf8c0>] 2021-03-14T13:11:33+0000.464 [DEBUG] action_log_callback.logging_action_log:85 (11:MainThread) - logging_action_log: ActionLogParams(command='_get_table_metadata', start_epoch_ms=1615727493322, end_epoch_ms=1615727493463, user='test@email.com', host_name='ee9f995f6b9d', pos_args_json='[]', keyword_args_json='{"table_key": "postgres://ioekgftt.public/demo", "index": "0", "source": "search_results"}', output='{"tableData": {}, "msg": "Encountered exception: {\'columns\': {0: {\'col_type\': [\'Field may not be null.\']}, 1: {\'col_type\': [\'Field may not be null.\']}}}", "status_code": 500}', error=None)
Steps to Reproduce
- Deploy amundsen on docker
- Use sample_postgres_loader.py to upload data
Your Environment
- Amunsen version used: amundsen-frontend:3.1.0, amundsen-search:2.4.1, amundsen-metadata:3.3.0
- Data warehouse stores: postgresql
- Deployment (k8s or native): native(docker)
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 2
- Comments: 21 (17 by maintainers)
Commits related to this issue
- Make application docker image configurable (#955) This patch enables docker image users to add their custom config files easily. Previous `setup.py install` had installed the application to system s... — committed to dorianj/amundsen by shuichiro-makigaki 3 years ago
- Make application docker image configurable (#955) This patch enables docker image users to add their custom config files easily. Previous `setup.py install` had installed the application to system s... — committed to amundsen-io/amundsen by shuichiro-makigaki 3 years ago
- Make application docker image configurable (#955) This patch enables docker image users to add their custom config files easily. Previous `setup.py install` had installed the application to system s... — committed to SaltIO/amundsen by shuichiro-makigaki 3 years ago
- Make application docker image configurable (#955) This patch enables docker image users to add their custom config files easily. Previous `setup.py install` had installed the application to system s... — committed to DataChefHQ/amundsen by shuichiro-makigaki 3 years ago
@alldoami I was confirmed with a different deployment that
amundsen-databuilder==4.2.2
should fix your initial issue. I am not sure what else could cause your current error.@alldoami could you use https://pypi.org/project/amundsen-databuilder/4.2.2/ which should fix the issue?
I think we add the deps in CI but miss in setup.py (https://github.com/amundsen-io/amundsendatabuilder/pull/462#pullrequestreview-621620935) which will fix it. The new release will be release once pr is merged.