grpc: gRPC Python 1.6.0 Library Incompatibility with `fork()`

We intend to support fork() going forward for the 1.7 release.

The 1.6 release introduced more background threads at the c-level, which are not carried over to child processes after the fork() call.

In the interim, pinning grpcio==1.4.0 is the best workaround.

I seem to be encountering the same bug using grpcio==1.8.2. Here’s a minimal repro using datastore as the example, but you can do it with nearly anything:

import multiprocessing

from google.cloud import datastore


def causeTrouble(where: str):
    client = datastore.Client(project='dev-storage-humu', namespace='aquarium')
    client.get(client.key('c', 'aquarium'))
    # The call to get() hangs forever; this line is never reached.
    print('OK')


if __name__ == '__main__':
    # Create a datastore client and do an RPC on it.
    client = datastore.Client(project='dev-storage-humu', namespace='aquarium')
    client.get(client.key('c', 'aquarium'))

    # Kick off a child process while the first client is still in scope.
    process = multiprocessing.Process(target=causeTrouble,
                                      args=['child process'])
    process.start()

If the first client goes out of scope before you call Process() (e.g., if you call causeTrouble() from main instead of doing it right there) then this works correctly. (This was tested using Py3.6 on OSX, but I’ve seen similar issues running on GCP machines as well)

This is a pretty serious obstacle in my case, because I’ve got a server which uses all sorts of GCP systems, and it needs to fork subprocesses to do expensive asynchronous tasks, which also require those. Short of rearchitecting to pre-fork a bunch of processes before any of grpc is invoked, is there any way to deal with this? (And if @katbusch’s result that simply importing various libraries before fork is enough to trigger the bug, that’s going to be particularly difficult)

Saying ‘grpc is incompatible with fork()’ feels like a pretty unsatisfying answer, especially given the lack of non-fork() based parallelism in Python.

Another update: I can only reproduce this on mac, not on linux

Will take me a moment to test an example minimal enough to post but the general idea is

pool = multiprocessing.Pool(initializer={initializer calls grpc.insecure_channel}).
pool.map(fn that sends messages over channel)

I just upgraded to 1.7 and am still seeing an issue with multiprocessing. It works on 1.4

Note: I am updating all libraries to pin to < 1.6dev for now.