Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Console encoding breaking change #34365

Merged
merged 1 commit into from
Mar 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/core/compatibility/7.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ If you're migrating an app to .NET 7, the breaking changes listed here might aff
| - | :-: | :-: | - |
| [Automatic RuntimeIdentifier for certain projects](sdk/7.0/automatic-runtimeidentifier.md) | ✔️ | ❌ | 7.0.100 |
| [Automatic RuntimeIdentifier for publish only](sdk/7.0/automatic-rid-publish-only.md) | ❌ | ❌ | 7.0.200 |
| [CLI console output uses UTF-8](sdk/8.0/console-encoding.md) | ❌ | ❌ | 7.0.3xx |
| [MSBuild serialization of custom types in .NET 7](sdk/7.0/custom-serialization.md) | ❌ | ❌ | 7.0.100 |
| [Side-by-side SDK installations](sdk/7.0/side-by-side-install.md) | ❌ | ❌ | 7.0.100 |
| [Version requirements for .NET 7 SDK](sdk/7.0/vs-msbuild-version.md) | ✔️ | ✔️ | 7.0.100 |
Expand Down
9 changes: 5 additions & 4 deletions docs/core/compatibility/8.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,11 @@ If you're migrating an app to .NET 8, the breaking changes listed here might aff

## SDK

| Title | Type of change | Introduced |
| ------------------------------------------------------------------------------- | ------------------------------------- | ---------- |
| ['dotnet pack' uses Release configuration](sdk/8.0/dotnet-pack-config.md) | Behavioral change/Source incompatible | Preview 1 |
| ['dotnet publish' uses Release configuration](sdk/8.0/dotnet-publish-config.md) | Behavioral change/Source incompatible | Preview 1 |
| Title | Type of change | Introduced |
| ------------------------------------------------------------------------------- | ------------------------------------------------ | ---------- |
| [CLI console output uses UTF-8](sdk/8.0/console-encoding.md) | Behavioral change/Source and binary incompatible | Preview 1 |
| ['dotnet pack' uses Release configuration](sdk/8.0/dotnet-pack-config.md) | Behavioral change/Source incompatible | Preview 1 |
| ['dotnet publish' uses Release configuration](sdk/8.0/dotnet-publish-config.md) | Behavioral change/Source incompatible | Preview 1 |

## Windows Forms

Expand Down
56 changes: 56 additions & 0 deletions docs/core/compatibility/sdk/8.0/console-encoding.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
---
title: "Breaking change: CLI console output uses UTF-8"
description: Learn about a breaking change in the .NET 8 SDK where the encoding for console input and output is now UTF-8.
ms.date: 03/03/2023
---
# CLI console output uses UTF-8

If the [`DOTNET_CLI_UI_LANGUAGE`](../../../tools/dotnet-environment-variables.md#dotnet_cli_ui_language) or `VSLANG` environment variable is set, the .NET CLI console output and input encoding changes to UTF-8, so that the code page can change to UTF-8 as well. This new behavior allows characters from languages set by those environment variables to be rendered correctly.

This change only affects Windows operating systems (the encoding was okay on other platforms). Moreover, it only applies to Windows 10 and later versions where the UI culture set by the user is non-English.

## Previous behavior

Characters in certain languages, including Chinese, German, Japanese, and Russian, would sometimes display as garbled characters or as `?` in the console. For example:

```console
C:\>dotnet build
MSBuild version 17.3.0-preview[...] for .NET
???????????????...
```

## New behavior

Starting in .NET 7 (version 7.0.3xx) and .NET 8, characters render correctly. Both the encoding and the code page change. For example:

```console
C:\>dotnet build
MSBuild version 17.3.0-preview[...] for .NET
正在确定要还原的项目…
```

Versions of Windows older than Windows 10 1909 don't fully support UTF-8 and may experience issues after this change.

In addition, there's [an existing bug](https://github.com/dotnet/sdk/issues/30170) where the SDK can affect the encoding of other commands and programs called in the same command prompt after the SDK has finished execution. Now that the SDK more frequently changes the encoding, the impact of this bug may increase.

## Version introduced

7.0.3xx
.NET 8 Preview 1

## Type of breaking change

This change can affect [source compatibility](../../categories.md#source-compatibility) and [binary compatibility](../../categories.md#binary-compatibility). It's also a [behavioral change](../../categories.md#behavioral-change).

## Reason for change

Using the .NET CLI in non-English languages provided a poor experience.

Developers that weren't already using the `VSLANG` and `DOTNET_CLI_UI_LANGUAGE` variables aren't impacted. The impact should be minimal, as this language setting wouldn't have worked well in the first place due to garbled characters. Also, only developers using Windows 10 or later might be impacted, most of which are likely using version 1909 or later.

The legacy scenarios are already less likely to support the broken languages, so it's unlikely you'd want to use another language that might expose this break anyway.

## Recommended action

- If you're using an older version of windows 10, upgrade to version 1909 or later.
- If you want to use a legacy console or are facing build issues or others due to the encoding change, unset `VSLANG` and `DOTNET_CLI_UI_LANGUAGE` to disable this change.
8 changes: 8 additions & 0 deletions docs/core/compatibility/toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ items:
href: globalization/8.0/twodigityearmax-default.md
- name: SDK
items:
- name: CLI console output uses UTF-8
href: sdk/8.0/console-encoding.md
- name: "'dotnet pack' uses Release configuration"
href: sdk/8.0/dotnet-pack-config.md
- name: "'dotnet publish' uses Release configuration"
Expand Down Expand Up @@ -192,6 +194,8 @@ items:
href: sdk/7.0/automatic-runtimeidentifier.md
- name: Automatic RuntimeIdentifier for publish only
href: sdk/7.0/automatic-rid-publish-only.md
- name: CLI console output uses UTF-8
href: sdk/8.0/console-encoding.md
- name: Version requirements for .NET 7 SDK
href: sdk/7.0/vs-msbuild-version.md
- name: Serialization of custom types in .NET 7
Expand Down Expand Up @@ -1256,6 +1260,8 @@ items:
items:
- name: .NET 8
items:
- name: CLI console output uses UTF-8
href: sdk/8.0/console-encoding.md
- name: "'dotnet pack' uses Release configuration"
href: sdk/8.0/dotnet-pack-config.md
- name: "'dotnet publish' uses Release configuration"
Expand All @@ -1266,6 +1272,8 @@ items:
href: sdk/7.0/automatic-runtimeidentifier.md
- name: Automatic RuntimeIdentifier for publish only
href: sdk/7.0/automatic-rid-publish-only.md
- name: CLI console output uses UTF-8
href: sdk/8.0/console-encoding.md
- name: Version requirements for .NET 7 SDK
href: sdk/7.0/vs-msbuild-version.md
- name: Serialization of custom types in .NET 7
Expand Down