local-data-api: JVM Crashes

@koxudaxi happy to help - thanks so much for making this in the first place!

Hi @koxudaxi

I also have this issue on a Mac.

@MarkHerhold Thank you for showing me the way to reproduce the error.

I understand why the error happens.

I fixed a bug around auto-commit on JDBC. But, I forgot to handle a case that runs execute-statement without transactionId. https://github.com/koxudaxi/local-data-api/pull/67

I have fixed the problem and released it as 0.4.15.

Would you please test it?

I think this project must have unit tests with real MySQL and PostgreSQL. I will check your PR for it 🤓

@koxudaxi I encounter different errors with that PR, even locally.

Query:

[AWS rdsdataservice 500 1.217s 3 retries] executeStatement({
  secretArn: 'arn:aws:secretsmanager:us-east-1:123456789012:secret:dummy',
  resourceArn: 'arn:aws:rds:us-east-1:123456789012:cluster:dummy',
  database: 'test',
  includeResultMetadata: true,
  parameters: [ [length]: 0 ],
  sql: 'SELECT 1 as id'
})

Logs:

[8] [INFO] Waiting for application startup.
[8] [INFO] Application startup complete.
172.19.0.1:38468 - "POST /Execute HTTP/1.1" 500
[8] [ERROR] Exception in ASGI application

Reproduce:

export RDS_DATA_API_CLIENT_RESOURCE_ARN=arn:aws:rds:us-east-1:123456789012:cluster:dummy
export RDS_DATA_API_CLIENT_SECRETARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:dummy
aws --endpoint-url 'http://localhost:8080' rds-data execute-statement --database 'test' --resource-arn $RDS_DATA_API_CLIENT_RESOURCE_ARN --secret-arn $RDS_DATA_API_CLIENT_SECRETARN --sql 'SELECT 1 AS hi'

Side note: I also saw jaydebeapi.DatabaseError: org.postgresql.util.PSQLException: Cannot commit when autoCommit is enabled but that could be my fault. Not sure yet.

As a potentially helpful tip (how I solved this for my case), when you are NOT using Docker Compose, and working with a locally running database, at least on macOS, you can set the DB HOST (e.g. POSTGRES_HOST or MYSQL_HOST ) to host.docker.internal. For example, I’m using this with Postgres that is already running via Brew on my machine:

docker run --rm -it --name my-data-api -p 8080:80 \
  -e ENGINE=PostgreSQLJDBC \
  -e POSTGRES_HOST=host.docker.internal \
  -e POSTGRES_PORT=5432 \
  -e POSTGRES_USER=postgres \
  -e RESOURCE_ARN=arn:aws:rds:us-east-2:123456789012:cluster:dummy \
  -e SECRET_ARN=arn:aws:secretsmanager:us-east-2:123456789012:secret:dummy \
  koxudaxi/local-data-api

arn:aws:rds:us-east-1:123456789012:cluster:dummy

Thank you, I’ve got this working now!

Facing the same problem in OS X:

#
#  SIGSEGV (0xb) at pc=0x00007fd84c48f0f9, pid=12, tid=0x00007fd8541cbb88
#
# JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build 1.8.0_222-b10)
# Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-amd64 compressed oops)
# Derivative: IcedTea 3.13.0
# Distribution: Custom build (Tue Oct  1 11:00:28 UTC 2019)
# Problematic frame:
# C  [_jpype.cpython-37m-x86_64-linux-gnu.so+0x430f9]  JPJavaEnv::NewLocalRef(_jobject*)+0x9

when running command in cli: aws --endpoint-url 'http://127.0.0.1:8080' rds-data execute-statement --database 'test_db' --resource-arn 'test_resource_arn' --secret-arn 'test_secrect_arn' --sql 'CREATE TABLE TEST(id Int)'

using docker-compose:

version: '3.1'

services:
  database:
    image: mysql:5.6
    restart: always
    command: --default-authentication-plugin=mysql_native_password
    environment:
      MYSQL_DATABASE: test_db
      MYSQL_ROOT_PASSWORD: test_psw
    volumes:
    - ../aws/rds/sql:/docker-entrypoint-initdb.d
    ports:
      - "3360:3306"
  local-data-api:
    image: koxudaxi/local-data-api
    restart: always
    environment:
      MYSQL_HOST: database
      MYSQL_PORT: 3306
      MYSQL_USER: root
      MYSQL_PASSWORD: test_psw
      RESOURCE_ARN: test_resource_arn
      SECRET_ARN: test_secrect_arn
    ports:
      - "8080:80"
    depends_on:
      - database

However, a few days ago it worked, the same setup. Tried to prune the docker system, additionally deleted the images for mysql and local-data-api and pulled it again - it didn’t resolve the issue above.