StringComparer not working as expected on Linux

[ChristophB125]

The following test works on Windows but not on Linux (RHEL 7.2 or Ubuntu 16.04):

Assert.AreEqual(0, StringComparer.CurrentCultureIgnoreCase.Compare("ss", "ß"));

Our Microsoft consultant Ben Gimblett has confirmed it and requested that we raise it as an issue here for follow-up.

We understand that .Net Core uses Windows's own comparison service, which is not available on Linux operating systems. How would one perform these culture aware operations on .Net core - irrespective of operating system?

[tarekgh]

this can be by design as we are using the OS for such features. I'll need to look first if there is anything else missing in our wrappers.

[migajek]

I'm afraid the problem is wider.
I was struggling to find out why my app doesn't work on .net core on Ubuntu.
I've been digging through librariers, but was able to narrow it down to just this

ladies, gentlemen ... behold:
apparently on en-US-POSIX culture on my Ubuntu, "Id" != "id"

me@system:/tmp/netcoredemo$ more Program.cs
using System;

namespace netcoredemo
{
    class Program
    {
        static void Main(string[] args)
        {
            var s = "Id";
            Console.WriteLine($"current / ignore {s.Equals("id", StringComparison.CurrentCultureIgnoreCase)}");
            Console.WriteLine($"invariant / ignore {s.Equals("id", StringComparison.InvariantCultureIgnoreCase)}");
            Console.WriteLine($"culture: {System.Globalization.CultureInfo.CurrentCulture.Name}");
        }
    }
}
me@system:/tmp/netcoredemo$ dotnet run
current / ignore False <--- !!!!!!!!!!
invariant / ignore True
culture: en-US-POSIX

env details

me@system:/tmp/netcoredemo$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial

me@system:/tmp/netcoredemo$ dotnet --version
2.1.4

[migajek]

setting the culture at the beginning of the program fixes the issue

System.Globalization.CultureInfo.CurrentCulture = new System.Globalization.CultureInfo("en-US");

[tarekgh]

@ChristophB125 you can use the CompareOptions.IgnoreNonSpace to get comparison of "ß" evaluated as equal to "ss". CompareOptions can be passed to string APIs, CompareInfo APIs and also you can create StringComparer in 2.1 version using the following code:

        string sb = "ß";
        string ss = "ss";

        StringComparer sc1 = StringComparer.Create(CultureInfo.CurrentCulture, CompareOptions.IgnoreNonSpace);
        Console.WriteLine(sc1.Compare(sb, ss));

@migajek en_US_POSIX is very special locale and its string sorting behavior is special too. This is why you got this surprising result when you used it. switching to en-US as you did will give you the logical results.

[tarekgh]

I am closing this issue, feel free to send any more questions or issue. thanks all.

[migajek]

@tarekgh so the right question is .. why does .net core detect current culture as en-US-POSIX?
I cannot seem to find any information on where does current culture come from on Linux.

LANG env variable is en_US.UTF-8 ...

root@system:~/netcoredemo# dotnet run
current / ignore False
invariant / ignore True
culture: en-US-POSIX
root@system:~/netcoredemo# echo $LANG
en_US.UTF-8

[tarekgh]

@migajek

You may look at https://github.com/dotnet/coreclr/blob/master/src/corefx/System.Globalization.Native/locale.cpp#L123 for how we get the default user locale. in short, we call ICU function uloc_getDefault() to get the default locale.

[tarekgh]

you may check the values of the environment variables LC_MESSAGES and LC_ALL in your system.

[migajek]

@tarekgh thank you, this seems to work as expected then as uloc_getDefault returns en_US_POSIX for me.
I'll look into system configuration