Repository navigation
[ci] Fix macOS MSBuild test discovery; remove overlapping tests and one lane per OS - #13550
Draft
jonathanpeppers wants to merge 2 commits into
Draft
jonathanpeppers wants to merge 2 commits into
jonathanpeppers wants to merge 2 commits into
Conversation
Since #13376 moved Xamarin.Android.Build.Tests to net11.0, every macOS MSBuild test lane has found 0 tests and passed. The slicer apphost used .NET 10 on macOS even with DOTNET_ROOT set, and .NET 10 cannot load the net11.0 test assembly. Run the slicer dll with the repo-local dotnet so it uses the SDK installed during test setup. Fail the slice step when it finds 0 tests, so this cannot pass silently again. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Remove NativeAOT and other cases that repeat what CoreCLR cases or other NativeAOT cases already test. Keep at least one NativeAOT case where the test can find a runtime-specific problem. Reduce the Windows MSBuild lanes from 8 to 7 and the macOS MSBuild lanes from 10 to 9. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
macOS MSBuild lanes run no tests
Since #13376 moved
Xamarin.Android.Build.Teststonet11.0, all 10macOS > Tests > MSBuildlanes reportFound 0 matching tests.and pass. Builds before #13376 found about 1,957 tests. Every build after it found 0.On macOS, the
dotnet-test-slicerapphost uses .NET 10 even withDOTNET_ROOTset. .NET 10 cannot load thenet11.0assembly. On Windows, the slicer uses .NET 12 and finds the tests.The slicer now runs as
bin/$(Configuration)/dotnet/dotnet <dotnet-test-slicer.dll>, so it uses the SDK installed during test setup. The slice step also fails when it finds 0 tests, so this cannot pass silently again.Overlapping tests and lane count
Many MSBuild tests repeat each CoreCLR case on NativeAOT, or test many values of one input. This removes cases that repeat what other cases already test, and keeps at least one NativeAOT case where a runtime-specific problem is possible. In 5 recent builds, the removed Windows cases took about 174 test-minutes per build.
The 7-lane Windows time is not verified. The removed time is below an earlier estimate of 210–230 test-minutes for one full lane, because the slicer splits tests by count, not by duration.
Testing
dotnet-test-slicer0.1.0-alpha7 with the new commands against thenet11.0test assembly:dotnet exec --fx-version 10.0.15): found 0 tests, and the new check fails the step.dotnet-local.cmd test bin\TestDebug\net11.0\Xamarin.Android.Build.Tests.dll --filter "<18 changed test methods>" -- NUnit.NumberOfTestWorkers=2on Windows: 141 passed, 0 failed, 6 skipped. The 6 skipped cases are theBuildApkWithManyAssetsNativeAOT cases, which were already skipped because that test runs only in Debug.