MediatR: Error when using MediatR with NetCore's new Worker service

I am currently using MediatR successfully with my .net core web project. But unfortunately when using a Worker service the DI breaks and can not find my Handlers. Worker Services require an Environment Var of "DOTNET_ENVIRONMENT" : "Development" and my web project uses "ASPNETCORE_ENVIRONMENT" : "Development" when i switch my worker service to use the later of the two EV vars, there are no problems with my worker services finding the Handlers, but the worker services picks up the wrong appsettings.[ENV].json (Production instead of Development).

Thanks