beam: [Bug]: Python x-lang jobs seems to be getting stuck when run from Mac OS Sonoma

What happened?

I recently upgraded to Mac OS Sonoma and seems like x-lang jobs are getting stuck during job submission when run with this OS.

The job seems to be getting stuck when connecting to Java expansion service using gRPC (see below for the stack-trace). I tried following scenarios:

• Running job with Beam HEAD on Mac OS Sonoma - job gets stuck
• Running job with Beam 2.51.0 and dependencies constrained to https://github.com/apache/beam/blob/release-2.51.0/sdks/python/container/py38/base_image_requirements.txt on Mac OS Sonoma - job gets stuck
• Running job with Beam HEAD on Linux (Debian) - job runs fine

INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:transform:org.apache.beam:kafka_read_with_metadata:v1, beam:transform:org.apache.beam:kafka_read_without_metadata:v1, beam:transform:org.apache.beam:kafka_write:v1, beam:external:java:generate_sequence:v1]
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Registered transforms:
INFO:apache_beam.utils.subprocess_server:	beam:transform:org.apache.beam:kafka_read_with_metadata:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@71075444
INFO:apache_beam.utils.subprocess_server:	beam:transform:org.apache.beam:kafka_read_without_metadata:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@4f32a3ad
INFO:apache_beam.utils.subprocess_server:	beam:transform:org.apache.beam:kafka_write:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@6b695b06
INFO:apache_beam.utils.subprocess_server:	beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@4d1bf319
INFO:apache_beam.utils.subprocess_server:
INFO:apache_beam.utils.subprocess_server:Registered SchemaTransformProviders:
INFO:apache_beam.utils.subprocess_server:	beam:schematransform:org.apache.beam:kafka_read:v1
INFO:apache_beam.utils.subprocess_server:	beam:schematransform:org.apache.beam:kafka_write:v1
INFO:apache_beam.utils.subprocess_server:Nov 13, 2023 4:23:36 PM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'WriteToKafka(beam:transform:org.apache.beam:kafka_write:v1)' with URN 'beam:transform:org.apache.beam:kafka_write:v1'
INFO:apache_beam.utils.subprocess_server:Dependencies list: {}
INFO:apache_beam.utils.subprocess_server:Nov 13, 2023 4:23:36 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.io.kafka.KafkaIO$Write$External$Configuration' has no schema registered. Attempting to construct with setter approach.
INFO:apache_beam.utils.subprocess_server:Nov 13, 2023 4:23:36 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig
INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.io.kafka.KafkaIO$Write$External$Configuration' has no schema registered. Attempting to construct with setter approach.
/Users/chamikara/testing/xlang_job_stuck/env_2_51_0_with_constraints/lib/python3.8/site-packages/apache_beam/transforms/external.py:725: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
  self._expansion_service, pipeline.options)

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner