collation: add pinyin collation for chinese charset support

[xiongjiwei]

What problem does this PR solve?

Issue Number: This issue provides an alternative for issue #19747 and #10192

Problem Summary:

What is changed and how it works?

many users in China would like to sort Chinese characters with pinyin order.
Proposal: #19984

How it Works:

we use a big table to store the character's weight. To compare two strings, just find the character's weight one by one.

For all Chinese characters, we sort them and give them the weight from 1 to n according to zh.xml file in cldr24 and then plus 0xFFA00000

For non-Chinese characters, we convert them to gb18030 codepoint, and if

1. codepoint is not greater than 0xFFFF, use the codepoint to be weight directly.
2. codepoint greater than 0xFFFF, use 0xFF000000 + codepoint + 0x1E248 to be weight.

benchmark

goos: linux
goarch: amd64
pkg: github.com/pingcap/tidb/util/collate
BenchmarkUtf8mb4Bin_CompareLong-8                   2907            417144 ns/op
BenchmarkUtf8mb4GeneralCI_CompareLong-8               58          20108234 ns/op
BenchmarkUtf8mb4UnicodeCI_CompareLong-8               55          22296453 ns/op
BenchmarkUtf8mb4Pinyin_CompareLong-8                  68          17060721 ns/op
BenchmarkUtf8mb4Bin_KeyLong-8                       1724            692919 ns/op
BenchmarkUtf8mb4GeneralCI_KeyLong-8                   88          12352256 ns/op
BenchmarkUtf8mb4UnicodeCI_KeyLong-8                   82          14197344 ns/op
BenchmarkUtf8mb4Pinyin_KeyLong-8                      90          12048163 ns/op

The test string length is 2<<20

Check List

Tests

• Unit test
• Integration test

Release note

• add pinyin order support

[bb7133]

LGTM

[ti-chi-bot]

@xiongjiwei: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.