beam: [Failing Test]: Python PostCommit failing due to duplicate AvroSchemaIO autoservice
What happened?
Python PostCommit jdbcio_xlang_it_test failing. Jenkins log does not show much message, but running locally the actual error shows:
ERROR:apache_beam.utils.subprocess_server:Starting job service with ['java', '-jar', '/Users/yathu/.apache_beam/cache/jars/composite-jars/9b6e2cb01bd723cbd87a5e71462f25664f165993697afc98e1c64c34bf814f98.jar', '50709', '--filesToStage=/Users/yathu/dev/virtualenv/py38beam/lib/sdks/java/extensions/schemaio-expansion-service/build/libs/beam-sdks-java-extensions-schemaio-expansion-service-2.47.0-SNAPSHOT.jar,/Users/yathu/.apache_beam/cache/jars/postgresql-42.2.16.jar']
ERROR:apache_beam.utils.subprocess_server:Error bringing up service
Traceback (most recent call last):
File "/Users/yathu/dev/virtualenv/py38beam/lib/python3.8/site-packages/apache_beam/utils/subprocess_server.py", line 88, in start
raise RuntimeError(
RuntimeError: Service failed to start up with error 1
Traceback (most recent call last):
This is because there are two @AutoService AvroSchemaIOProvider classes so the schemaio expansion service fails to start.
The fix could either be similar to the workaround here: https://github.com/apache/beam/blob/011296c14659f80c8ecbeefda79ecc3f1113bd95/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/io/Providers.java#L45
Or simply remove AvroSchemaIOProvider in core.
Issue Failure
Failure: Test is continually failing
Issue Priority
Priority: 1 (unhealthy code / failing or flaky postcommit so we cannot be sure the product is healthy)
Issue Components
- Component: Python SDK
- Component: Java SDK
- Component: Go SDK
- Component: Typescript SDK
- Component: IO connector
- Component: Beam examples
- Component: Beam playground
- Component: Beam katas
- Component: Website
- Component: Spark Runner
- Component: Flink Runner
- Component: Samza Runner
- Component: Twister2 Runner
- Component: Hazelcast Jet Runner
- Component: Google Cloud Dataflow Runner
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 21 (21 by maintainers)
Ok, so, let’s wait for tests passed and if it’s ok, I’ll merge it. Then, we can cherry-pick it to release branch.
@aromanenko-dev I think there’s two more follow ups required:
AvroSchemaIOProvideris a good example where users might never notice such a deprecation as there’s no direct dependency on the class.AvroSchemaIOProvideris removed fromcoreany x-lang pipeline using it would break.@damccorm @Abacn Before any reverts or deletes, I’d like to try a fix proposed by @mosche if it will work. Let me create a PR for that and check.