runtime: [Uri] System.Uri rejects otherwise valid strings with length >= 65520

The System.Uri class seems to reject URI strings greater than a strangely short length (experimentally, 65520 or higher, so probably this constant). I’m not able to find any documentation of this fact at https://docs.microsoft.com/en-us/dotnet/api/system.uri?view=netcore-3.1, nor does RFC 3986 discuss length restrictions (though I went through those two sources very quickly, so this could be my error).

Most URIs I’ve seen are short enough, but data URIs can easily be imagined to exceed this limit.

CA1056:UriPropertiesShouldNotBeStrings and related rules have been guiding me towards using this class to represent URI strings for a while now; their documentation does not list “you want to support URI strings that exceed 65519 characters in length” as good reasons to suppress warnings.

In my perfect world, I would appreciate it if the System.Uri class could represent all URI values that are legal in RFC 3986 (or at least the subset that can be represented by System.String).

I totally understand why an artificial restriction might appear as a result of engineering trade-offs, so perhaps a decent first step might be to document it better?

Repro program

The following C# program writes out the longest URI I could manage.

Incrementing the commented line causes it to throw an exception:

Unhandled exception. System.UriFormatException: Invalid URI: The Uri string is too long.
   at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind)
   at System.Uri..ctor(String uriString)
   at Program.Main() in <<AIRBREATHER_REDACTED>>\Program.cs:line 10

Program.cs

using System;
using static System.Console;

static class Program
{
    static void Main()
    {
        const int TextLength = 65489; // any higher will throw an exception
        string txt = new string('a', TextLength);
        WriteLine(new Uri($"data:text/plain;charset=utf-8,{txt}"));
    }
}

ConsoleApp0.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.1</TargetFramework>
  </PropertyGroup>

</Project>

System Information

>dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.1.101
 Commit:    b377529961

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.101\

Host (useful for support):
  Version: 3.1.1
  Commit:  a1388f194c

.NET Core SDKs installed:
  2.2.402 [C:\Program Files\dotnet\sdk]
  3.1.100 [C:\Program Files\dotnet\sdk]
  3.1.101 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 20 (13 by maintainers)

Commits related to this issue

Allow overlong data URIs in GpxWebLink. Workaround for dotnet/runtime#1857 Resolves NetTopologySuite.IO.GPX#39 — committed to NetTopologySuite/NetTopologySuite.IO.GPX by airbreather 4 years ago

Most upvoted comments

The System.Uri class seems to reject URI strings greater than a strangely short length (experimentally, 65520 or higher

I’m really not sure you can consider that short.

Wraith2 on Jan 17, 2020

In my original comment, I wrote this:

I totally understand why an artificial restriction might appear as a result of engineering trade-offs, so perhaps a decent first step might be to document it better?

Should open an issue on dotnet/dotnet-api-docs for that “decent first step” part of it?

As of right now, the “Filing Issues” section of this repo doesn’t link over there (I only found out about it when scrolling through and noticing the cross-reference from dotnet/dotnet-api-docs#3906).

airbreather on Feb 29, 2020

MoreInfo looks like it’s implementing a pattern that I’ve used before when storing property values that aren’t needed in performance-sensitive cases; is that right?

Correct

In case my intuition is wrong, another possible way of doing this without storing extra data per instance would be to continue to store offsets in ushort values the way we do when the values are short enough, but for offsets >65519, store a sentinel value of 65535 indicating that the “real” value lies somewhere beyond. When we need to access such an offset, we would need to re-scan the original string to figure it out.

65535 is also a valid length that could be stored if the input is shorter than 65520 and has characters that would expand it to 65535.

If we were to implement it, it would most likely be done by changing the minimum amount of fields to int (we could likely keep Scheme, User, Host, PortValue and Path as ushort). That would result in a 4-8 byte size increase of Uri, which might be acceptable considering other inefficiencies in Uri.

Considering that data Uris are the only realistic scenario for Uris longer than 65k, I don’t think it is worth making such a change.

I understand that in your case you may not be interacting with the created Uri other than for validation / satisfying the object model. But otherwise the utility of Uri for data uris is practically equivalent to

bool IsValid(string uriString) =>
    uriString.StartsWith("data:", StringComparison.OrdinalIgnoreCase);

It provides no utility for accesing the media type/data - those have to be manually parsed out. It also provides no validation for it except for the scheme name.

There are other downsides to removing the length restriction.

Further complicate the already messy Uri code
There are likely some sub-optimal algorithmical approaches in Uri that could ‘explode’ should the length limit be relaxed (in which case they should be fixed ofc).
There are also a lot of cases where Uri will allocate temporary buffers/strings. While obviously not ideal in any case, if the 65k length restriction wasn’t in place, these could end up going to the LOH instead of being quickly dealt with by the GC.

So if for some reason we decided to remove the length restriction in the future, a lot of work would have to be done to improve Uri overall first.

Given that this issue is the only ask for relaxing the length restriction that I know of and you already implemented a workaround for your use case, I don’t see a real benefit for moving forward here.

Consider as won’t-fix until then.

MihaZupan on Feb 28, 2020

This is a documented length limit for Uri

See https://docs.microsoft.com/en-us/dotnet/api/system.uri.-ctor?view=netcore-3.1#System_Uri__ctor_System_String_

One of the conditions listed for throwing UriFormatException:

The length of uriString exceeds 65519 characters.

I agree that we can add a comment about this limit to the main Uri page as rn it is only documented on constructors.

MihaZupan on Jan 17, 2020