runtime: NET5.0 vs NETCore3.1 - Strings comparison between space and non-breaking-space characters

Description

I am encountering different behavior comparing SPACE (ALT+032) and NBSP (ALT+255) characters using NET 5.0 or NETCore 3.1. It seems that the case insensitive comparison in NET 5.0 treats a space and a non-breaking-space as equal.

Configuration

• Which version of .NET is the code running on? Microsoft.NETCore.App 5.0.6 and Microsoft.NETCore.App 3.1.15
• What is the architecture (x64, x86, ARM, ARM64)? x64
• Do you know whether it is specific to that configuration? No

Steps to Reproduce

Here’s a small example code to reproduce the issue

var space = "Equality Test";
var nbsp = "Equality\x00A0Test";

Console.WriteLine($"Test 1: { space == nbsp }");
Console.WriteLine($"Test 2: { space.Equals(nbsp, StringComparison.CurrentCultureIgnoreCase) }");

Actual Output

Target Framework NET 5.0

Test 1: False
Test 2: True

Target Framework NETCore 3.1

Test 1: False
Test 2: False

Expected Output

This is the expected output for Target Framework NET 5.0

Test 1: False
Test 2: False