arrow: [Python] Unable to turn-off signal blocking for pyarrow flight RPC server

Greetings,

The python-binding of Apache Arrow Flight RPC server blocks SIGTERM and SIGINT signals during long-running operations. And I have been unable to unblock these signals.

It seems like the enable_signal_handlers() method from pyarrow.lib is there for the exact purpose of turning on/off signal blocking, but it does not work, at least not for the following sample code,

  • server.py
import signal
import threading
import time
from datetime import datetime

import pyarrow as pa
import pyarrow.flight as pf


class TickServer(pf.FlightServerBase):

    SCHEMA = pa.schema([
        ('time', pa.timestamp("us")),
    ])

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.stopped = threading.Event()

    def _generate_ticks(self, context: pf.ServerCallContext):
        while not self.stopped.is_set() and not context.is_cancelled():
            yield pa.record_batch([
                pa.array([datetime.now(), ]),
            ], schema=TickServer.SCHEMA)
            time.sleep(1)

    def do_get(self, context: pf.ServerCallContext, ticket: pf.Ticket):
        key = ticket.ticket.decode('utf-8')
        if key == "tick":
            return pf.GeneratorStream(TickServer.SCHEMA, self._generate_ticks(context))
        raise pa.ArrowKeyError("Unknown ticket: {}".format(ticket))

    def shutdown(self):
        self.stopped.set()
        return super().shutdown()


if __name__ == "__main__":
    # NOTE: this doesn't work
    pa.enable_signal_handlers(False)

    with TickServer(location="grpc://127.0.0.1:8000") as server:

        def signal_handler(signum, frame):
            server.shutdown()
            print("interrupted")

        signal.signal(signal.SIGINT, signal_handler)
        server.serve()

  • client.py
import pyarrow as pa
import pyarrow.flight as pf

if __name__ == "__main__":
    try:
        client: pf.FlightClient = pf.connect(location="grpc://127.0.0.1:8000")
        client.wait_for_available()
        for batch in client.do_get(pf.Ticket("tick")):
            table = pa.Table.from_batches([batch.data])
            for record in table:
                print(record[0])
    except KeyboardInterrupt:
        print("interrupted")

The current behavior of server.py is that it can only be interrupted by SIGINT (i.e. Ctrl+C) when no client is active. I’d like to be able to interrupt server.py regardless of whether there are active clients not not.

My environment is Ubuntu 20.04 + python 3.8 + pyarrow 6.0.1.

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 15 (12 by maintainers)

Most upvoted comments

Overall there are three things to handle here:

  • Allow disabling of the default signal handler for consistency, with the caveats I’ve given above;
  • Documentation that serve() is not actually necessary to start the server,
  • Update the default shutdown() to actually signal cancellation ongoing RPCs

Does this sound reasonable?