PSResourceGet: Register-PSRepository raises errors about "invalid Web Uri", despite underlying httpclient recognizing the Uri and [Uri] parsing
I have a very simple problem I want to solve: Due to reasons beyond my control, the company I work for uses a “.local” domain name internally, for everything. My repository url is “http://packages.mycompany.local/nuget/Mycompany-Development/api/v2” - which according to Register-PSRepository is not a “Web Uri” (whatever the heck that means).
It appears this is a bogus error message, largely due to the sophisticated and overly complicated sanity checks Register-PSRepository tries to perform. I’ve extracted all the required sub-functions to replicate the behavior, modulo the final error trap that surfaces the wrong error.
function Get-LocationString
{
[CmdletBinding(PositionalBinding=$false)]
Param
(
[Parameter()]
[Uri]
$LocationUri
)
$LocationString = $null
if($LocationUri)
{
if($LocationUri.Scheme -eq 'file')
{
$LocationString = $LocationUri.OriginalString
}
elseif($LocationUri.AbsoluteUri)
{
$LocationString = $LocationUri.AbsoluteUri
}
else
{
$LocationString = $LocationUri.ToString()
}
}
return $LocationString
}
function Resolve-Location
{
[CmdletBinding()]
[OutputType([string])]
Param
(
[Parameter(Mandatory=$true)]
[string]
$Location,
[Parameter(Mandatory=$true)]
[string]
$LocationParameterName,
[Parameter()]
$Credential,
[Parameter()]
$Proxy,
[Parameter()]
$ProxyCredential,
[Parameter()]
[System.Management.Automation.PSCmdlet]
$CallerPSCmdlet
)
# Ping and resolve the specified location
if(-not (Test-WebUri -uri $Location))
{
if(Microsoft.PowerShell.Management\Test-Path -Path $Location)
{
return $Location
}
elseif($CallerPSCmdlet)
{
$message = $LocalizedData.PathNotFound -f ($Location)
ThrowError -ExceptionName "System.ArgumentException" `
-ExceptionMessage $message `
-ErrorId "PathNotFound" `
-CallerPSCmdlet $CallerPSCmdlet `
-ErrorCategory InvalidArgument `
-ExceptionObject $Location
}
}
else
{
$pingResult = Ping-Endpoint -Endpoint $Location -Credential $Credential -Proxy $Proxy -ProxyCredential $ProxyCredential
$statusCode = $null
$exception = $null
$resolvedLocation = $null
if($pingResult -and $pingResult.ContainsKey($Script:ResponseUri))
{
$resolvedLocation = $pingResult[$Script:ResponseUri]
}
if($pingResult -and $pingResult.ContainsKey($Script:StatusCode))
{
$statusCode = $pingResult[$Script:StatusCode]
}
Write-Debug -Message "Ping-Endpoint: location=$Location, statuscode=$statusCode, resolvedLocation=$resolvedLocation"
if((($statusCode -eq 200) -or ($statusCode -eq 401)) -and $resolvedLocation)
{
return $resolvedLocation
}
elseif($CallerPSCmdlet)
{
$message = $LocalizedData.InvalidWebUri -f ($Location, $LocationParameterName)
ThrowError -ExceptionName "System.ArgumentException" `
-ExceptionMessage $message `
-ErrorId "InvalidWebUri" `
-CallerPSCmdlet $CallerPSCmdlet `
-ErrorCategory InvalidArgument `
-ExceptionObject $Location
}
}
}
function Test-WebUri
{
[CmdletBinding()]
[OutputType([bool])]
Param
(
[Parameter(Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[Uri]
$uri
)
return ($uri.AbsoluteURI -ne $null) -and ($uri.Scheme -match '[http|https]')
}
function Ping-Endpoint
{
[CmdletBinding()]
param
(
[Parameter()]
[ValidateNotNullOrEmpty()]
[string]
$Endpoint,
[Parameter()]
$Credential,
[Parameter()]
$Proxy,
[Parameter()]
$ProxyCredential,
[Parameter()]
[switch]
$AllowAutoRedirect = $true
)
$results = @{}
$WebProxy = $null
if($Proxy -and ('Microsoft.PowerShell.Commands.PowerShellGet.InternalWebProxy' -as [Type]))
{
$ProxyNetworkCredential = $null
if($ProxyCredential)
{
$ProxyNetworkCredential = $ProxyCredential.GetNetworkCredential()
}
$WebProxy = New-Object Microsoft.PowerShell.Commands.PowerShellGet.InternalWebProxy -ArgumentList $Proxy,$ProxyNetworkCredential
}
if(HttpClientApisAvailable)
{
$response = $null
try
{
$handler = New-Object System.Net.Http.HttpClientHandler
if($Credential)
{
$handler.Credentials = $Credential.GetNetworkCredential()
}
else
{
$handler.UseDefaultCredentials = $true
}
if($WebProxy)
{
$handler.Proxy = $WebProxy
}
$httpClient = New-Object System.Net.Http.HttpClient -ArgumentList $handler
$response = $httpclient.GetAsync($endpoint)
}
catch
{
}
if ($response -ne $null -and $response.result -ne $null)
{
Write-Host $response.Result
$results.Add($Script:ResponseUri,$response.Result.RequestMessage.RequestUri.AbsoluteUri.ToString())
$results.Add($Script:StatusCode,$response.result.StatusCode.value__)
}
}
else
{
$iss = [System.Management.Automation.Runspaces.InitialSessionState]::Create()
$iss.types.clear()
$iss.formats.clear()
$iss.LanguageMode = "FullLanguage"
$WebRequestcmd = @'
param($Credential, $WebProxy)
try
{{
$request = [System.Net.WebRequest]::Create("{0}")
$request.Method = 'GET'
$request.Timeout = 30000
if($Credential)
{{
$request.Credentials = $Credential.GetNetworkCredential()
}}
else
{{
$request.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
}}
$request.AllowAutoRedirect = ${1}
if($WebProxy)
{{
$request.Proxy = $WebProxy
}}
$response = [System.Net.HttpWebResponse]$request.GetResponse()
if($response.StatusCode.value__ -eq 302)
{{
$response.Headers["Location"].ToString()
}}
else
{{
$response
}}
$response.Close()
}}
catch [System.Net.WebException]
{{
"Error:System.Net.WebException"
}}
'@ -f $EndPoint, $AllowAutoRedirect
$ps = [powershell]::Create($iss).AddScript($WebRequestcmd)
if($WebProxy)
{
$null = $ps.AddParameter('WebProxy', $WebProxy)
}
if($Credential)
{
$null = $ps.AddParameter('Credential', $Credential)
}
$response = $ps.Invoke()
$ps.dispose()
if ($response -ne "Error:System.Net.WebException")
{
if($AllowAutoRedirect)
{
$results.Add($Script:ResponseUri,$response.ResponseUri.ToString())
$results.Add($Script:StatusCode,$response.StatusCode.value__)
}
else
{
$results.Add($Script:ResponseUri,[String]$response)
}
}
}
return $results
}
function HttpClientApisAvailable
{
$HttpClientApisAvailable = $false
try
{
[System.Net.Http.HttpClient]
$HttpClientApisAvailable = $true
}
catch
{
}
return $HttpClientApisAvailable
}
$SourceLocation = [Uri] "http://packages.mycompany.local/nuget/Mycompany-Development/api/v2"
# Ping and resolve the specified location
$SourceLocation = Resolve-Location -Location (Get-LocationString -LocationUri $SourceLocation) `
-LocationParameterName 'SourceLocation' `
-Credential $Credential `
-Proxy $Proxy `
-ProxyCredential $ProxyCredential `
-CallerPSCmdlet $PSCmdlet
I’m not 100% sure, but I think the issue is that Resolve-Location treats any failures as “Web Uri” failures, even though the error appears to be due to $Script:$ResponseUri not existing/being a null key. (I’m not a PowerShell expert and while I generally understand the concept of scope-modifiers, I generally program in C# where you can’t dynamically change scopes. To a C# programmer, this looks like bad code or non-sensical code.)
This appears to be an issue since PowerShellGet was called OneGet: https://github.com/OneGet/TestBuild/issues/2
In that issue, it appears that Ping-Endpoint was to blame. Oddly, this was 3 years ago and still not fixed as OneGet got renamed PowerShellGet.
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Reactions: 2
- Comments: 24
Commits related to this issue
- Merge pull request #23 from anamnavi/update-release-build Updates to CI: Change image and set useMinimatch for ESRP sign task to true — committed to anamnavi/PowerShellGet by JamesWTruher a year ago
still facing same issue. even with powershell 7 core preview
Anyone found some kind of workaround for this for the current powershellget version(2.2.5)? powershellget 3.0 has been in eternal beta(and we’re not even sure we’ll be able to upgrade to it, we haven’t looked at the prerequisites) so it’s not a solution for the moment and we simply can’t register our internal repo with any new servers we build.
I have the same issues with Artifactory with some spice. We do not use https internally, and auth only needed for publishing with a nugetapikey. We have only one location, so no separate publish/source for modules and scripts. http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo
We have several domains, artifactory resides on the INT one. The repository works as intended from our %MYCOMPANY% domain, and our DEV domain, where there are no whitelisting policies on network level. Below the Invoke-WebRequest block: ` Invoke-WebRequest -Uri ‘http://artifactory-pro.mycompany.com/artifactory/api/nuget/AppMgmtRepo/’ -Verbose -Credential $(Get-Credential)
cmdlet Get-Credential at command pipeline position 1 Supply values for the following parameters: Credential VERBOSE: GET http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/ with 0-byte payload VERBOSE: received -1-byte response of content type application/xml;charset=utf-8
StatusCode : 200 StatusDescription : OK Content : <?xml version="1.0" encoding="utf-8"?>
RawContent : HTTP/1.1 200 OK X-Artifactory-Id: $$$$$$$$$$$$$$:$$$$$$$$:$$$$$$$$:-8000 DataServiceVersion: 1.0 Transfer-Encoding: chunked Content-Type: application/xml;charset=utf-8 Date: Wed, 13 Nov 2019… Forms : {} Headers : {[X-Artifactory-Id, $$$$$$$$$$$$$$:$$$$$$$$:$$$$$$$$:-8000], [DataServiceVersion, 1.0], [Transfer-Encoding, chunked], [Content-Type, application/xml;charset=utf-8]…} Images : {} InputFields : {} Links : {} ParsedHtml : mshtml.HTMLDocumentClass RawContentLength : 570 `
Tough we do have a PROD and UAT domain, where you need to allow it through the firewall. We have let both http and https comm through, I can confirm that telnet works on port 80 and 443 from the correspondant servers, but getting this nasty invalid Web Uri error.
I have also tried the workaround mentioned at stackoverflow, it successfully registers the repo with SourceLocation set as $env:TEMP, but when I try to use Set-PsRepository cmdlet to fix it fails with same Web Uri error.
Futhermore I have tried to manually edit the “C:\Users%USER%\AppData\Local\Microsoft\Windows\PowerShell\PowerShellGet\PSRepositories.xml” file with correct path, after restarting WinRM it picked up the endpoint without any error, altough when I try to list the modules it throws the below error:
Find-Module -Repository MyRepo -Verbose VERBOSE: Repository details, Name = 'MyRepo', Location = 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/'; IsTrusted = 'True'; IsRegistered = 'True'. VERBOSE: Using the provider 'PowerShellGet' for searching packages. VERBOSE: Using the specified source names : 'MyRepo'. VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'. VERBOSE: The specified Location is 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/' and PackageManagementProvider is 'NuGet'. VERBOSE: Total package yield:'0' for the specified package ''. VERBOSE: An error occurred while sending the request. VERBOSE: Retry downloading 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/' for '2' more times VERBOSE: An error occurred while sending the request. VERBOSE: Retry downloading 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/' for '1' more times VERBOSE: An error occurred while sending the request. VERBOSE: Retry downloading 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/' for '0' more times WARNING: Unable to resolve package source 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/'. VERBOSE: Total package yield:'0' for the specified package ''.I’ve tried to resolve the Artifactory as backend server FQDN and IP as well, all resulted with the same errors.
These errors appears on Server 2016 Hosts with PowerShell 5.1, but only on our domains where we need to allow these connections through firewall. Everywhere else I do have upgraded WMF to 5.1, but the underlying servers spans from Server 2008 to Server 2016, and all works as it should.
All help much appreciated.
@jzabroski this error exactly. Artifactory doesn’t register any calls at all; as far as I can tell , PowerShellGet is simply angry that the provided URI doesn’t look like a URI that it expects.
This is an issue for us using Artifactory, is there any chance we could get it addressed?
@edyoung Shouldn’t registering the repository be responsible for telling the PSGet the endpoint to query? If not in the URL then in metadata. Otherwise I’ll just go ahead and install Office 2017 on my Windows 95
I get the same error. On both PowerShell 5.1 and PowerShell Core (6.1) when trying to Register-PSRepository for our local ProGet repo. It’s funny, though, that
Register-PSRepositoryfails if specified the following URL:https://nuget.our.domain/feeds/feed-name/, but succeeds, if specified the following URL:https://nuget.our.domain/api/v2/feed-name/. Although, entering 2nd URL in Web browser automatically switches us to the 1st URL.That’s kind of confusing.
I have a similar problem. I try to check if the PSGallery is present on a system.
Although PSGallery is present on the system, $ev1 is still set and therefor my code tries to install it . If I output the content of $ev1 it shows:
PowerShellGet version is 1.0.0.1 on Windows Server 2016 (14393.2551)