-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
System.Runtime.Serialization.Schema.Tests failures #74370
Comments
Seen in a 7.0 backport PR as well: #74447 Queue: Libraries Test Run release coreclr windows x64 Debug Job: https://dev.azure.com/dnceng/public/_build/results?buildId=1961997&view=results
|
This is blocking CI - regression on 8/15 with 1+ failures per day, 3 rolling runs impacted (as of 8/24). @mconnew @StephenMolloy @HongGit can you please take a look? |
@dotnet/dncenghot it seems some runs are not recorded in Kusto -- when I click on the top post Runfo links, I can see e.g. runs 1944867 on 8/15 and 1950291 on 8/17, while Kusto shows first failures on 8/20 -- is that a known issue? let failedTests = (testNameSubstring : string, methodName : string, messageSubstr: string, includePR : bool, includePassedOnRerun : bool) {
cluster('engsrvprod.kusto.windows.net').database('engineeringdata').AzureDevOpsTests
| where TestName contains testNameSubstring
| where includePassedOnRerun or (Outcome == 'Failed')
| extend startOfTestName = indexof_regex(TestName, @"[^.]+$")
| extend Method = substring(TestName, startOfTestName)
| extend Type = substring(TestName, 0, startOfTestName - 1)
| project-away startOfTestName
| where (methodName == '') or (Method == methodName)
| where Message contains messageSubstr
| distinct JobId, WorkItemId, Message, StackTrace, Method, Type, Arguments, Outcome
| join kind=inner (cluster('engsrvprod.kusto.windows.net').database('engineeringdata').Jobs
| where ((Branch == 'refs/heads/main') or (Branch == 'refs/heads/master') or (Branch startswith 'refs/heads/release/7.0') or (includePR and (Source startswith "pr/")))
| where Type startswith "test/functional/cli/"
and not(Properties contains "runtime-staging")
| where Branch <> 'refs/pull/73374/merge'
| summarize arg_max(Finished, Properties, Type, Branch, Source, Started, QueueName) by JobId
| project-rename JobType = Type) on JobId
| extend PropertiesJson = parse_json(Properties)
| extend OS = replace_regex(tostring(PropertiesJson.operatingSystem), @'\((.*)\).*|([^\(].*)', @'\1\2')
| extend Runtime = iif(PropertiesJson.runtimeFlavor == "mono", "Mono", iif(PropertiesJson.DefinitionName contains "coreclr", "CoreCLR", ""))
| extend TargetBranch = extractjson("$.['System.PullRequest.TargetBranch']", Properties)
| extend Architecture = PropertiesJson.architecture
| extend Scenario = iif(isempty(PropertiesJson.scenario), "--", PropertiesJson.scenario)
//| extend DefinitionName = PropertiesJson.DefinitionName
| project-away PropertiesJson
};
failedTests(
'System.Runtime.Serialization.Schema', //testNameSubstring
'', //methodName
'Compilation on the XmlSchemaSet failed', //messageSubstr
true, //includePR
true); //includePassedOnRerun |
Hello, we're aware that there are missing runs, there was a discussion recently, I'll bring this up again |
There was an AzDo Ingestion bug, it appears to have been fixed on the 19th August, so it's highly likely that the runs are missing because of it |
I think the bot was a little over-zealous here. I believe this needs to be backported to actually unblock anybody stuck on this issue. @karelz, do we need this in 7.0-rc1 AND 7.0? |
@StephenMolloy release/7.0 branch is sufficient. RC1 is closed at this point AFAIK - cc @carlossanlop Thanks for fast turnaround! |
Looks like a regression introduced on 8/15.
Affected tests with status as of 8/24:
System.Runtime.Serialization.Schema.Tests.ImporterTests.GetKnownTypeReferences
| last 120 days - 26x failures reported (each failure is reported twice) -- 13x runs (incl. 3x Rolling runs) - first on 8/15 in run 1944867System.Runtime.Serialization.Schema.Tests.ImporterTests.CanImport
- last 120 days - 56x reported failures (each failure is reported 4x, in one run it is reported 8x) -- 13x runs (incl. 3x Rolling runs) - first on 8/15 in run 1944867System.Runtime.Serialization.Schema.Tests.ImporterTests.Import
- last 120 days - 56x failures reportedSystem.Runtime.Serialization.Schema.Tests.ImporterTests.GetCodeTypeReference
- last 120 days - 56x failures reportedSystem.Runtime.Serialization.Schema.Tests.ImportOptionsTests.ReferencedTypes
- last 120 days - 56x failures reportedSystem.Runtime.Serialization.Schema.Tests.ImportOptionsTests.CodeProvider
- last 120 days - 14x failures reported -- 13x runs (one run reported twice) - first on 8/15 in run 1944867System.Runtime.Serialization.Schema.Tests.ImportOptionsTests.Namespaces
- last 120 days - 14x failures reportedSystem.Runtime.Serialization.Schema.Tests.ImportOptionsTests.ImportXmlType
- last 120 days - 14x failures reportedSystem.Runtime.Serialization.Schema.Tests.ImportOptionsTests.EnableDataBinding
- last 120 days - 13x failures reportedSystem.Runtime.Serialization.Schema.Tests.ImportOptionsTests.GenerateInternal
- last 120 days - 14x failures reportedSystem.Runtime.Serialization.Schema.Tests.ImportOptionsTests.GenerateSerializable
- last 120 days - 14x failures reportedSystem.Runtime.Serialization.Schema.Tests.ImporterTests and System.Runtime.Serialization.Schema.Tests.ImportOptionsTests failes on
CI leg: https://github.com/dotnet/runtime/pull/72772/checks?check_run_id=7932703826
Logs: https://helix.dot.net/api/2019-06-17/jobs/6db75ebd-6f45-4213-a444-987288186b07/workitems/System.Runtime.Serialization.Schema.Tests/console
The text was updated successfully, but these errors were encountered: