azure-pipelines-agent: "The SSL connection could not be established" when configuring agent behind enterprise root cert

Agent Version and Platform

2.194.0, MacOS

Azure DevOps Type and Version

Azure DevOps Services (dev.azure.com)

What’s not working?

I’m attempting to install the Azure DevOps agent behind an enterprise proxy which injects a root certificate. I’ve added that root certificate to the OS X keychain and marked it as trusted, and when I go to https://dev.azure.com in my browser, I see it successfully being served up as trusted using my organization’s root cert. When I try to run config.sh to configure the agent, I get this message, though:

The SSL connection could not be established, see inner exception.

In the generated log, here’s the error I see:

[2021-11-08 22:07:33Z ERR VisualStudioServices] GET request to https://dev.azure.com/progcloud/_apis/connectionData?connectOptions=1&lastChangeId=-1&lastChangeId64=-1 failed. System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. —> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure. at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception) (… more boilerplate stacktrace)

The closest thing to guidance I can find is the article on how to Run the agent with a self-signed certificate, which suggests using curl -v to validate that your SSL is working. When I run a curl on the URL in question, the response includes:

Server certificate:

subject: CN=dev.azure.com

start date: Nov 18 13:11:54 2020 GMT

expire date: Nov 18 13:11:54 2021 GMT

subjectAltName: host “dev.azure.com” matched cert’s “dev.azure.com”

issuer: // my organization’s issuer information

SSL certificate verify ok.

Is there something I’m missing? If my SSL trust chain is validating in the browser and via CURL, is there something else I need to do to cause the agent config utility to validate my cert?

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 26 (2 by maintainers)

Most upvoted comments

I just discovered that if I run export AZP_AGENT_USE_LEGACY_HTTP=true prior to running config.sh, then my SSL validation works as expected. @anatolybolshakov, if you or anyone more familiar with this codebase can help provide info on what may have changed about SSL validation between the legacy HTTP handler and the new default handler, it would be immensely helpful.

sphanley on Dec 2, 2021

Running on Ubuntu 22.04.1 LTS I had to do the following to install the agent. Any one of the solutions above wouldn’t work, I had to have a combination of the solutions:

mkdir ~/Downloads
cd ~/Downloads
wget https://vstsagentpackage.azureedge.net/agent/2.213.2/vsts-agent-linux-x64-2.213.2.tar.gz
mkdir ~/myagent && cd ~/myagent
tar zxvf ~/Downloads/vsts-agent-linux-x64-2.213.2.tar.gz
cd ~/Downloads
wget http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2_amd64.deb
sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2_amd64.deb
cd ../myagent
export AZP_AGENT_USE_LEGACY_HTTP=true
./config.sh
sudo ./svc.sh install
sudo ./svc.sh start

Edit: I also had to add export AZP_AGENT_USE_LEGACY_HTTP=true to the top of the runsvc.sh file

bbrougher-cerium on Dec 9, 2022

Also having this issue when I upgraded to Ubuntu 22.04 in preparation for that release on 4/21/2022. I am running agents self hosted in docker, managed by kubernetes: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops

When i added the environment variable

AZP_AGENT_USE_LEGACY_HTTP=true

It works now.

johnynfulleffect on Mar 30, 2022

Worked for me on Ubuntu 22.04 - export AZP_AGENT_USE_LEGACY_HTTP=true

ruaan on Sep 29, 2022

I have the same problem…

[2022-01-31 16:37:45Z INFO LocationServer] Unable to connect to https://dev.azure.com/xxxxx/. [2022-01-31 16:37:45Z ERR LocationServer] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host. --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request) at System.Net.Security.SslStream.ThrowIfExceptional() at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result) at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult) at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) --- End of inner exception stack trace --- at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection) [2022-01-31 16:37:45Z INFO CommandSettings] Flag ‘unattended’: ‘False’ [2022-01-31 16:37:45Z ERR Terminal] WRITE ERROR (exception): [2022-01-31 16:37:45Z ERR Terminal] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. —> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host… —> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host. — End of inner exception stack trace — at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request) at System.Net.Security.SslStream.ThrowIfExceptional() at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result) at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult) at System.Net.Security.SslStream.<>c.b__65_1(IAsyncResult iar) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) — End of stack trace from previous location where exception was thrown — at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) — End of inner exception stack trace — at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection) at Microsoft.VisualStudio.Services.Agent.Util.ServerUtil.GetConnectionData(String serverUrl, VssCredentials credentials, ILocationServer locationServer) at Microsoft.VisualStudio.Services.Agent.Util.ServerUtil.DetermineDeploymentType(String serverUrl, VssCredentials credentials, ILocationServer locationServer) at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.ConfigurationManager.ConfigureAsync(CommandSettings command) [2022-01-31 16:37:45Z ERR Terminal] WRITE ERROR: Failed to connect. Try again or ctrl-c to quit [2022-01-31 16:37:45Z INFO CommandSettings] Arg 'url': '' [2022-01-31 16:37:45Z INFO CommandSettings] Flag 'unattended': 'False' [2022-01-31 16:37:45Z INFO PromptManager] ReadValue [2022-01-31 16:37:45Z INFO Terminal] WRITE: Enter server URL > [2022-01-31 16:37:45Z INFO Terminal] READ LINE

I’m having the same issue as well, started this morning…

dhlee90 on Jan 31, 2022

If you’re getting an error with --skipsslcertvalidation, then it sounds like you’re running into something different than what I’m experiencing. In my case, --skipsslcertvalidation causes the agent registration to work successfully, but then I continue to get other SSL errors when trying to run certain pipeline tasks. But in my case, my certificate is valid and trusted by the MacOS machine, but for some reason won’t pass the pipeline agent’s cert validation.

sphanley on Dec 25, 2021