dapr: [Actor Reminders] Potential memory leak

cc @yaron2 @cgillum

As per our chats on Discord, I believe there is another memory leak in dapr workflows or the underlying subsystems (reminders subsystem has been mentioned as a possible root cause)

For example, on start-up, the idling sidecar is taking up 26mb, however after running the below repro, the mem sits around 104mb.

After 5 hours of observation, this has not dropped, which leads me to believe there is another memory leak.

---

my local environment

• Mac OS - M1
• Docker Desktop 4.19.0 (106363)

Can reproduce this on the following dapr sidecar versions

1.11.2

client libraries

<PackageReference Include="Dapr.Client" Version="1.11.0" />
<PackageReference Include="Dapr.Workflow" Version="1.11.0" />`

Repro steps

1. pull repo https://github.com/olitomlinson/dapr-workflow-examples
2. docker compose build
3. docker compose up
4. Use insomnia/postman/whatever : POST http://localhost:5112/start?count=2000&async=true&runId={runId}<- make this request just once to start 2000 workflows. Replace {runId} with a unique string for each time you run this request, i.e. 001, 002,, 003 etc
5. Takes approx 10-20 minutes to start and complete all workflows.
6. Observe the dapr sidecar consume memory, most of which doesn’t get released.