athena-express: The response json returns the empty columns

When there are some columns in a row that are empty / null, then the response json that is returned from the query is having empty indexes.

I think you should create an array with all falsey values removed. So that only the columns that actually exist in a specific row will be returned in the response.

{ resource: 'dynamo',
       name: 'customers",,,,"mpla",,,"table',
       bucket_name: 'fetchCustomer',
       directory_path: 'production' }

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 37 (18 by maintainers)

Most upvoted comments

Just pushed v4.0.0 that parses using csvtojson.

ghdna on Jan 29, 2019

@ghdna so I debugged the code now and like I said the csv parser is not robust enough.

line = line.substring(1, line.length - 1).split('","');

This line messes up. Replace the whole process with csvtojson and it will work.

wasserholz on Jan 29, 2019

OK so the problem is the fact that the GetQueryExecution of AWS Athena, does not respond by default with a vaiue at StatementType. So If it does not return with DML value then the json trasnformation fails.

So you should consider the following:

			const s3Output =
				queryStatus.QueryExecution.ResultConfiguration
				.OutputLocation,
				statementType = queryStatus.QueryExecution.StatementType || "DML";

which solves the whole opened issue with the data etc.

deliverymanager on Jan 28, 2019