bleak: Connecting to device which is already connected fails (Device with address ... was not found)

Hi @newAM, I am using this to list already connected devices (for BlueZ):

def bluez_connected():
    import dbus
    bus=dbus.SystemBus()
    manager=dbus.Interface(bus.get_object('org.bluez','/'),'org.freedesktop.DBus.ObjectManager')
    objs=manager.GetManagedObjects()
    return [str(props['org.bluez.Device1']['Address']) for path,props in objs.items() if 'org.bluez.Device1' in props and props['org.bluez.Device1']['Connected']]

If the device I want to connect to is connected already, then I disconnect like this (could be also done over DBus, this is a quick hack):

try:
    subprocess.call(['bluetoothctl','disconnect',addr,timeout=2)
except (subprocess.CalledProcessError,subprocess,TimeoutExpired):
    # do what you need to do

Afterwards, Bleak will connect happily.

What might the connected-devices API look like?

Yes, something like this.

There’s some subtlety around the effect of client.connect() and client.disconnect()

Indeed. For Mac/Windows, I think the existing code will just “do the right thing”. For BlueZ, we will probably just have to pick a behavior that works for most users until it gets fixed upstream, i.e. if the device is already connected, then return from the Bleak connect method successfully without calling the D-Bus connect method (which would fail with “already connected”) - and it should set a flag so that the disconnect method doesn’t actually call the D-Bus disconnect method since another app may be using the device.

One thing that I’m not sure about though is how Mac/Windows would handle the BleakClient disconnect method when another app is still using the device. In all backends, Bleak currently waits for feedback that the device has actually been disconnected. Do Mac/Windows send artificial disconnected signals to say that our specific OS handle has been released? Or should Bleak not actually be waiting for this feedback? We will have to do some testing to find out.

Hello friends and much gratitude 🙏 for Bleak which seems to be far and away the most thoughtfully designed BT/LE interface layer for Python!

There is quite a bit of discussion about the “disconnected device issue” around:

• https://github.com/hbldh/bleak/issues/315
• https://github.com/hbldh/bleak/issues/412
• https://github.com/hbldh/bleak/issues/613

And in other projects:

• https://github.com/newAM/idasen/issues/29
• https://github.com/gopro/OpenGoPro/issues/14
• https://github.com/gopro/OpenGoPro/issues/39

This seems to be the most promising thread (including some great positive suggestions by @dlech above). I’m happy to help if there’s a good agreeable direction! I apologize profusely if I’m just stirring the pot uselessly. Please check my understanding of the situation and the possible solutions, corrections much appreciated…

My understanding: there is an issue (quirk?) with the bluez (Linux) implementation (and possibly other platforms?), where if an app is connected to a device and then terminates without disconnection, the device remains connected and the connection is kept alive by bluez. That means the device no longer shows up in discovery, and becomes an “orphan” unavailable for use until the device is restarted, the computer is restarted, bluetoothd is restarted, or some bluetoothctl incantations are made.

(OPEN QUESTION: What does happen on other platforms currently?)

SOAPBOX RANT 📦 (skip this blockquote you don’t want to argue the subject)

I believe that a system component which requires programs to perform “clean” shutdown to avoid broken system state (that cannot be recovered by an app restart) is extremely undesirable and seems like important to fix. “You can force-kill and restart apps without leaving the system broken” is a core design principle of all modern operating systems. When you kill an X11 (or Wayland) app, its window goes away; when you kill a socket using app, the socket is closed; when you kill an app talking to a USB device, the device is released; this is a widely observed principle.

I believe that careful clean shutdown with signal handlers and the like is not a complete solution, because program death is a fact of life and a design feature of the operating system. Programs seg fault, they run out of memory, they get killed by ornery users, they are force-killed by window managers, they lock up and need to be terminated, and so on. This is not a wildly exceptional circumstance, program termination happens all the time on every modern OS to every computer user, expert and novice alike. This is especially true of user-facing apps which are likely to be using a framework like Bleak. Asking every user to understand bluetoothctl incantations to “liberate” the device seems unreasonable, and more of a burden than almost any other device subsystem requires. (Serial port locks used to be like this, and it was a nightmare, and eventually got mostly fixed.)

At some level this is the application’s responsibility, but a framework/library such as bleak needs to make it possible to make resilient and robust apps, otherwise it is a much less useful framework/library.

END SOAPBOX 🧼

Even if people agree with that rant, the question is how to deal with it. There seem to be a few options:

Cleanup hygiene: Try to ensure bleak-using apps all cleanly disconnect when exiting for any reason. While probably good practice, I believe this is insufficient (see rant above).

Enumerate connected devices: Add some cross-platform facility to enumerate connected Bluetooth devices, and also allow them to be force-disconnected. This is a little problematic because how exactly is an app supposed to know which devices are “orphaned” and up for “adoption”, and which are legit in use by some other app? As a messy solution, bleak could stash a datum saying which devices it opened and the PID it had, so future processes know which devices are candidates for “reclaiming”… but that sounds complex and subject to its own race conditions and failure modes.

Arrange for auto-disconnect: In an ideal world, bluez (and similar system layers) would notice that the requesting app has gone off the bus, and disconnect any devices that app had requested connections to (at least for apps which request this service). However, that requires cooperation on bluez’s part. I’m not well enough read into the semantics of dbus to know how big a deal this would be. (OPEN QUESTION: Is there any hope in this direction?)

Other: ???

Again, I really, really like Bleak and am enjoying using it and have a ton of appreciation for the people giving their free time to make a not-so-shiny area of the system shinier! I’m just trying to figure out how to untangle this particular knot so that I can make robust apps and services.

I’ve done a bit of digging and it does look like it is possible to enumerate already connected devices on all OSes.

• BlueZ already caches all devices, so we can just use the Connected property as mentioned above.
• CoreBluetooth can retrieve a list of connected peripherals
• Windows can enumerate connected devices.

So perhaps we could add an async static method to BleakScanner that gets connected devices. This would return a list of BleDevice just like the discover() method. find_device_by_address() could then internally first check already connected devices, then star scanning if the device is not found.

If the device is already connected, then it will not advertise and can therefore not be found by Bleak. You will never be able to connect to an already connected device.

If you e.g. press Ctrl-C in your program, you might be left with a connected device. Try to ensure a clean exit.

This is BLE device listing under Windows with winrt https://stackoverflow.com/a/72045486 (we sponsored that code, and it is actually used in production, with some non-essential adaptations).

A device connected before bleak’s lifecycle is a normal scenario, eg., a BLE keyboard with an extra configuration service, it will always be connected by the system automatically, and a configuration program that is only executed when needed. Without the function to retrieve paired/connected devices, user will have to forget the device in this case.

My understanding: there is an issue (quirk?) with the bluez (Linux) implementation (and possibly other platforms?),

There is an open issue at BlueZ for this: https://github.com/bluez/bluez/issues/89. It sounds like there is agreement that it needs to be fixed but no one has stepped up to do the work.

On other OSes, if multiple apps or the OS itself was also using a BLE device, then it would be possible to have a device that didn’t actually disconnect when the Bleak program exits (even if it exited “cleanly”). But I don’t recall any issues raised with this behavior, so I’m guessing it is not common.

I don’t have specific knowledge of the internals of Windows/Mac but I assume that when you connect a device, it creates an OS handle (like a file handle) for the connection. If the program crashes, the handle is automatically released by the OS. If this was the only handle to the Blueooth device, then the device is disconnected.

This also means that BlueZ is the only backend that could “force disconnect” a device as suggested. So since this isn’t available cross-platform, probably it isn’t the best choice for Bleak. Instead, I think it makes sense for Bleak to be able to enumerate and “connect” to already connected devices. This would allow working around the BlueZ issue (users could create their own force-disconnect if they think that is the best solution). But at the same time, the feature stands on its own (e.g. connecting to a BLE mouse that is already connected to the OS).

The problem is seems to be exactly that: that your application leaves stale connections after it exits. But if you use the async with bleak.BleakClient(…) solution, you should not experience that.

The async context manager does not work for me as I inherit from BleakClient and need to keep the instance connected while other parts of the code run. I define destructor like def __del__(self): asyncio.run(self.disconnect()), which eliminated most stale connections, but not all. Like if the app is killed or someone connects e.g. through the Bluetooth applet.

Anyway, I see what you mean. I thought Bleak would ask BlueZ over DBus whether the connection exists already but I am okay to do it myself.

Thanks for the advice and helpful hints, let me close this now. And thanks for Bleak 😃