runtime: [API Proposal]: Support error codes for HttpRequestException

Edited by @antonfirsov Update 1: changes based on feedback - https://github.com/dotnet/runtime/issues/76644#issuecomment-1525754867

Background and motivation

HttpRequestException is thrown under various conditions but it does not provide an error code to differentiate those conditions. Currently, it provides ‘string Message’ property inherited from Exception type and it can contain various error messages:

  • The HTTP response headers length exceeded the set limit
  • Cannot write more bytes to the buffer
  • The response ended prematurely
  • Received chunk header length could not be parsed
  • Received an invalid status
  • Received an invalid header line
  • Received an invalid header name
  • The server returned an invalid or unrecognized response
  • etc.

The above strings are defined in https://github.com/dotnet/runtime/blob/main/src/libraries/System.Net.Http/src/Resources/Strings.resx

If we want to pragmatically handle exceptions based on the cause, it would be better dealing with error codes rather than dealing with string values. For example, when we create a dashboard for different exceptions encountered, aggregating by error codes would make more sense that doing by string values. And we could categorize some errors into a group for a special handling.

Many other exceptions like SocketException and Win32Exception have an error code field already. Would it make sense to provide a new property for HttpRequestException to hold error codes?

API Proposal


public class HttpRequestException : Exception
{
    // -- New properties --
    public HttpRequestError HttpRequestError { get; }

    // -- New constructors --
    public HttpRequestException(HttpRequestError httpRequestError, string? message) {}
    public HttpRequestException(HttpRequestError httpRequestError, string? message, Exception? inner) {}
    public HttpRequestException(HttpRequestError httpRequestError, string? message, Exception? inner, HttpStatusCode? statusCode) {}
    // ALTERNATIVE:
    public HttpRequestException(HttpRequestError httpRequestError, string? message, Exception? inner = null, HttpStatusCode? statusCode = null) {}
}

// IOException subtype to throw from response read streams
public class HttpIOException : IOException  // ALTERNATIVE name: HttpResponseReadException
{
    public HttpRequestError HttpRequestError { get; }
    public HttpResponseReadException(HttpRequestError httpRequestError, string? message = null, Exception? innerException = null) {}
} 

// The EXISTING HttpProtocolException should be the subtype of this new exception type
public class HttpProtocolException : HttpResponseReadException // WAS: IOException
{
}

// Defines disjoint error categories with high granularity.
public enum HttpRequestError
{
    Undefined = 0,                          // Uncategorized/generic error -OR-
                                            // the underlying HttpMessageHandler does not implement HttpRequestError

    NameResolutionError,                    // DNS request failed
    ConnectionError,                        // Transport-level error during connection
    TransportError,                         // Transport-level error after connection
    SecureConnectionError,                  // SSL/TLS error
    HttpProtocolError,                      // HTTP 2.0/3.0 protocol error occurred
    UnsupportedExtendedConnect,             // Extended CONNECT for WebSockets over HTTP/2 is not supported.
                                            // (SETTINGS_ENABLE_CONNECT_PROTOCOL has not been sent).
    VersionNegotiationError,                // Cannot negotiate the HTTP Version requested
    UserAuthenticationError,                // Authentication failed with the provided credentials
    ProxyTunnelError,

    StatusNotSuccess,                       // Set by EnsureSuccessStatusCode() in case of non-success status code.
    InvalidResponse,                        // General error in response/malformed response
    
    // ALTERNATIVE name: InvalidResponseHeaders
    InvalidResponseHeader,                  // Error with response headers
    
    // OPTIONAL TO NOT INCLUDE: The one's below are specific cases of errors with the response.
    // We may consider merging them into "InvalidResponse" and "InvalidResponseHeader"
    ResponseEnded,                          // Received EOF
    // These depend on SocketsHttpHandler configuration
    ContentBufferSizeExceeded,              // Response Content size exceeded MaxResponseContentBufferSize
    ResponseHeaderExceededLengthLimit,      // Response Header length exceeded MaxResponseHeadersLength
}

API Usage

With HttpClient methods:

try
{
    using var response = await httpClient.GetStringAsync(url);
}
catch (HttpRequestException ex) when (ex.HttpRequestError is HttpRequestError.ConnectionError or
                                                             HttpRequestError.SecureConnectionError or
                                                             HttpRequestError.NameResolutionError)
{
    // Retry
}

With response read Stream:

using var response = await httpClient.GetAsync(url, HttpCompletionOption.ResponseHeadersRead);
using var responseStream = await response.Content.ReadAsStreamAsync();
try
{
    await responseStream.ReadAsync(buffer);
}
catch (HttpResponseReadException ex)
{
    switch (ex.HttpRequestError)
    {
        // ....
    }
}

Risks

Any change to the error categorization is a breaking change, meaning that if we introduce new error codes for some users, we would break others.

Mobile handlers would need to implement their error mapping separately, meaning that we would lack platform consistency until that point. Whenever it’s impossible to detect some error categories on those platforms, they would need to report a more generic error code eg. InvalidResponse instead of ResponseEnded. This is a challenge for users who need to include mobile targets in their cross-platform HttpClient code.

Since requests and connections are decoupled, connection-establishment errors (NameResolutionError, ConnectionError and SecureConnectionError) will be only reported if the originating request is still in the queue, otherwise we will swallow them. Observing these error codes is not a reliable way to observe ALL connection-level errors happening under the hood. Some users may find this surprising.

Alternative Designs

  • Move errors that can occur during response content read to a separate HttpResponseReadError enum. HttpResponseReadException would hold values from that separate enum.
    • Upside: This might make it easier for users to implement unified error handling for the two API scenarios (HttpClient and Stream).
    • Downsides: might be harder to follow. Some error cases like TransportError or ProtocolError can occur in bothe cases, meaning that enum values would be shared.
  • Reduce the granularity of the error codes, eg. avoid defining cases like ContentBufferSizeExceeded, and threat them as InvalidResponse.
  • Use exception subclasses instead of error codes for better extensibility. The most practical approach would be to keep HttpRequestException sealed, and define a class hierarchy under IOException (which then should be aways InnerException of HttpRequestException).
    • Downside: too many subclasses, such hierarchy is unprecedented in the BCL

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 18 (15 by maintainers)

Commits related to this issue

Most upvoted comments

Video

  • Changed the HttpRequestException constructor to move the HttpRequestError to the end, and default it.
  • Changed the HttpRequestError property on HttpRequestException to be nullable, to distinguish “the provider didn’t give one” from “the provider had an error it couldn’t map”.
  • We decided it’s correct that HttpIOException.HttpRequestError is non-nullable.
  • UnsupportedExtendedConnect => ExtendedConnectNotSupported
  • We removed InvalidResponseHeader, recommending it just be bucketed to InvalidResponse for long-term compatibility and provider variance reasons.
  • ContentBufferSizeExceeded and ResponseHeaderExceededLengthLimit were merged into ConfigurationLimitExceeded
  • We renamed Undefined to Unknown
public class HttpRequestException : Exception
{
    public HttpRequestError? HttpRequestError { get; }

    public HttpRequestException(string? message, Exception? inner = null, HttpStatusCode? statusCode = null, HttpRequestError? httpRequestError = null) {}
}

// IOException subtype to throw from response read streams
public class HttpIOException : IOException  // ALTERNATIVE name: HttpResponseReadException
{
    public HttpRequestError HttpRequestError { get; }
    public HttpIOException(HttpRequestError httpRequestError, string? message = null, Exception? innerException = null) {}
} 

// The EXISTING HttpProtocolException should be the subtype of this new exception type
public class HttpProtocolException : HttpIOException
{
}

// Defines disjoint error categories with high granularity.
public enum HttpRequestError
{
    Unknown = 0,                          // Uncategorized/generic error

    NameResolutionError,                    // DNS request failed
    ConnectionError,                        // Transport-level error during connection
    TransportError,                         // Transport-level error after connection
    SecureConnectionError,                  // SSL/TLS error
    HttpProtocolError,                      // HTTP 2.0/3.0 protocol error occurred
    ExtendedConnectNotSupported,             // Extended CONNECT for WebSockets over HTTP/2 is not supported.
                                            // (SETTINGS_ENABLE_CONNECT_PROTOCOL has not been sent).
    VersionNegotiationError,                // Cannot negotiate the HTTP Version requested
    UserAuthenticationError,                // Authentication failed with the provided credentials
    ProxyTunnelError,

    InvalidResponse,                        // General error in response/malformed response
    
    // OPTIONAL TO NOT INCLUDE: The one's below are specific cases of errors with the response.
    // We may consider merging them into "InvalidResponse" and "InvalidResponseHeader"
    ResponseEnded,                          // Received EOF
    // These depend on SocketsHttpHandler configuration
    ConfigurationLimitExceeded,             // Response Content size exceeded MaxResponseContentBufferSize or
                                            // Response Header length exceeded MaxResponseHeadersLength or
                                            // any future limits are exceeded
}

At least the cases found by Norm were specifically EOF, though I’m not sure if that’s an exhaustive set of cases we would want to catch or just a point-in-time response. Since this particular handler is an attempt to detect and give the user a way to workaround a specific error scenario, I think any of the other response type you mention would be ok for us to catch as well. Any sort of hard-disconnect error from this particular API would require the same user mitigation - go create the missing resource through some other means.