Invoke-WebRequest Always Fails For HTTPS. The remote certificate is invalid according to the validation procedure

I’m trying to use Invoke-WebRequest inside of pwsh (on Linux), but it’s always failing. Here’s an example:

    Invoke-WebRequest https://www.google.com

The site/URL doesn’t matter, if it’s using HTTPS I’ll get the error. The exact same script on a Windows machine works without issue.

$PSVersionTable

> Name                           Value
> ----                           -----
> PSVersion                      7.0.0
> PSEdition                      Core
> GitCommitId                    7.0.0
> OS                             Linux 3.10.0-514.21.1.el7.x86_64 #1 SMP Thu May 25 17:04:51 UTC 2017
> Platform                       Unix
> PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
> PSRemotingProtocolVersion      2.3
> SerializationVersion           1.1.0.1
> WSManStackVersion              3.0

dotnet –version

3.1.403

openssl version

OpenSSL 1.0.1e-fips 11 Feb 2013

I know it’s possible to disable SSL validation on the Invoke-WebRequest method but I’d really rather avoid it and, also, I’m seeing the same issue when doing things like dotnet restore

If I use curl https://www.google.com it seemingly works fine. My understanding was that pwsh uses dotnet to actually make this webrequest, and that dotnet uses openssl for https…that might all be wrong, but I’ve been trying to debug this using openssl.

When I do openssl s_client -connect www.google.com:443 it seems to show that everything is fine. I can see the certificate chain:

> Certificate chain  
>  0 s:/C=US/ST=California/L=Mountain View/O=Google LLC/CN=www.google.com    
>    i:/C=US/O=Google Trust Services/CN=GTS CA 1O1
>  1 s:/C=US/O=Google Trust Services/CN=GTS CA 1O1    
>    i:/OU=GlobalSign Root CA - R2/O=GlobalSign/CN=GlobalSign

And the return code of 0 (ok)

Start Time: 1612285043
Timeout   : 300 (sec)
Verify return code: 0 (ok)

If I catch the exception in my .ps1 script I can see a little more information:

The remote certificate is invalid according to the validation procedure.
System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
   at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Security.SslStream.ThrowIfExceptional()
   at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)
   at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
   at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.GetResponse(HttpClient client, HttpRequestMessage request, Boolean keepAuthorization)
   at Microsoft.PowerShell.Commands.WebRequestPSCmdlet.ProcessRecord()

But I’m still at loss for where to go next.

Finally – while I can’t pinpoint what has changed, I know this used to work on this machine. I was tempted to upgrade everything to the latest versions but it’s hard to justify that sort of change when it was working with the versions I have now.

Can anyone help point me in the right direction?

EDIT

Additional information – when I run curl -verbose https://www.google.com I see which cert is being used:

* Connected to www.google.com (74.125.195.105) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256

I didn’t see an equivalent for openssl’s s_client, but running strace -e open openssl s_client -connect www.google.com:443 |& grep open shows:

open("/etc/pki/tls/openssl.cnf", O_RDONLY) = 3
open("/etc/pki/tls/cert.pem", O_RDONLY) = 3

But I haven’t been able to see pwsh attempting to access any of the root ca’s I have installed. When I search online, I just see a lot of people saying that dotnet core on Linux uses OpenSSL and that I should look at the openssl.cnf file – but when I use openssl from the command line, it seems to work?!?

Again – any help would be greatly appreciated it. It seems like pwsh just isn’t using the certs I have?

Answer

Since you’re asking for a pointer and not a solution: it’s almost certainly that powershell doesn’t know where to find the trust store of certificate authorities to trust. Figuring out how to tell it where the trust store is, or figuring out how to tell invoke-webrequest with a specific certificate is where I would start.

Advertisement

Answer