镜像站点 · 本页由第三方 GitHub 只读镜像提供，非 GitHub 官方站点，不接受任何登录或凭据输入。前往 github.com
Skip to content

Regressions in System.Tests.Perf_Int16 #89790

New issue
New issue
Closed
Closed
Regressions in System.Tests.Perf_Int16#89790
Assignees
EgorBo
Labels
arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windowsruntime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark
Milestone
9.0.0

Description

@performanceautofiler
performanceautofiler
bot
opened on Aug 1, 2023

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.86 ns 14.14 ns 1.30 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'
Details

Payloads

Baseline
Compare

System.Tests.Perf_Int16.ToString(value: 32767)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.14247358346259 > 11.413447568727415.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/5/2023 10:43:22 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -42.619443874450205 (T) = (0 -14.079840274334877) / Math.Sqrt((0.06128570950382653 / (13)) + (0.011907101932591717 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.2757309147088261 = (11.036685018759204 - 14.079840274334877) / 11.036685018759204 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
13.59 ns 15.23 ns 1.12 0.14 False
10.41 ns 11.53 ns 1.11 0.15 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
Details

Payloads

Baseline
Compare

System.Tests.Perf_Enum.GetValuesAsUnderlyingType_Generic

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.228957156617469 > 14.578445360547185.
IsChangePoint: Marked as a change because one of 5/11/2023 5:02:01 PM, 5/16/2023 12:30:38 AM, 6/5/2023 10:43:22 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -16.426516857898907 (T) = (0 -15.439862373772359) / Math.Sqrt((0.021366311189630637 / (11)) + (0.2035868483271795 / (28))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (28) - 2, .025) and -0.11373459119181577 = (13.863143423829591 - 15.439862373772359) / 13.863143423829591 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Format_NonFlags(value: Monday, format: "g")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.534257647238416 > 10.754153777821605.
IsChangePoint: Marked as a change because one of 5/20/2023 9:58:12 PM, 7/6/2023 1:53:22 PM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -15.018087565613055 (T) = (0 -11.629641045432903) / Math.Sqrt((0.01300566568906393 / (13)) + (0.20937345959886758 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.1283069562251724 = (10.30716063680105 - 11.629641045432903) / 10.30716063680105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
23.85 ns 25.58 ns 1.07 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
Details

Payloads

Baseline
Compare

System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.582589146418908 > 25.165638299842794.
IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 6/5/2023 10:43:22 AM, 6/8/2023 1:48:44 PM, 6/26/2023 2:16:43 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -14.510187082352958 (T) = (0 -25.91503108447964) / Math.Sqrt((0.11624662641135099 / (13)) + (0.1152229131646731 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.06723238839727558 = (24.28246309446974 - 25.91503108447964) / 24.28246309446974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
278.39 μs 306.28 μs 1.10 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'
Details

Payloads

Baseline
Compare

PerfLabTests.CastingPerf.FooObjIsFoo2

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.27955419580417 > 292.5684479895105.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 10:17:01 AM, 5/16/2023 5:40:18 PM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -95.27514369482915 (T) = (0 -306391.5660666573) / Math.Sqrt((1071991.1854392563 / (13)) + (10840.445168492337 / (29))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (29) - 2, .025) and -0.09829411896706958 = (278970.41491473553 - 306391.5660666573) / 278970.41491473553 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.30 μs 2.72 μs 1.18 0.39 False
1.58 μs 1.72 μs 1.09 0.19 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'
Details

Payloads

Baseline
Compare

System.Collections.AddGivenSize<String>.ICollection(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7208333333333337 > 2.4159216238472174.
IsChangePoint: Marked as a change because one of 5/19/2023 12:51:11 AM, 6/13/2023 11:57:38 AM, 6/21/2023 9:22:33 AM, 6/30/2023 8:34:57 AM, 7/17/2023 6:26:12 PM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -5.706558490671429 (T) = (0 -2627.705125534047) / Math.Sqrt((20559.38823998931 / (13)) + (4331.381039749639 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.0990194172486573 = (2390.9542309201247 - 2627.705125534047) / 2390.9542309201247 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.AddGivenSize<String>.Stack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7246590387167013 > 1.659051067819718.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -142.20801682505194 (T) = (0 -1722.2693178100615) / Math.Sqrt((10.911187904247434 / (13)) + (3.3413744868835784 / (29))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (29) - 2, .025) and -0.08775054797356352 = (1583.3311424375574 - 1722.2693178100615) / 1583.3311424375574 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
178.38 μs 193.49 μs 1.08 0.09 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
Details

Payloads

Baseline
Compare

PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 193.4878549382716 > 187.8497904600643.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 5/16/2023 12:30:38 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -22.357211550009108 (T) = (0 -193194.4501824932) / Math.Sqrt((3699006.544652325 / (11)) + (515376.7375766015 / (27))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (27) - 2, .025) and -0.07409717661999342 = (179866.82619392432 - 193194.4501824932) / 179866.82619392432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
167.07 μs 250.66 μs 1.50 0.07 False
111.41 μs 167.18 μs 1.50 0.06 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
Details

Payloads

Baseline
Compare

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.66410856009068 > 175.43872091090424.
IsChangePoint: Marked as a change because one of 6/13/2023 11:57:38 AM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -4509.901492912683 (T) = (0 -250663.42461286503) / Math.Sqrt((2209.877932823912 / (13)) + (5197.437352880435 / (30))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (30) - 2, .025) and -0.4999919758119941 = (167109.84368911228 - 250663.42461286503) / 167109.84368911228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf2.CastingPerf.IntObj

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.1805661094225 > 116.96357302748227.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 6/13/2023 11:57:38 AM, 6/19/2023 1:02:34 PM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -4600.348055702014 (T) = (0 -167117.99375520102) / Math.Sqrt((737.0768144361135 / (13)) + (2789.7329636364825 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.5001526893009134 = (111400.65604460552 - 167117.99375520102) / 111400.65604460552 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.19042
Queue OwlWindows
Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.68 μs 6.04 μs 1.06 0.09 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

System.Collections.IterateFor<Int32>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.0377685519733175 > 5.823231628327663.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 6/30/2023 8:34:57 AM, 7/17/2023 6:26:12 PM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -24.502572131288655 (T) = (0 -6143.388434163453) / Math.Sqrt((5448.8297240419115 / (13)) + (5559.161622279784 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.10812141262244855 = (5543.966901266428 - 6143.388434163453) / 5543.966901266428 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. performanceautofiler
    added
    arch-x64
    os-windows
    runtime-coreclrspecific to the CoreCLR runtime
    untriagedNew issue has not been triaged by the area owner
    on Aug 1, 2023

  2. performanceautofiler commented on Aug 1, 2023

    @performanceautofiler
    performanceautofilerBot
    on Aug 1, 2023
    Author

    Run Information

    Name Value
    Architecture x64
    OS Windows 10.0.19042
    Queue OwlWindows
    Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
    Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in System.MathBenchmarks.Single

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    19.50 μs 20.90 μs 1.07 0.04 False
    19.51 μs 20.90 μs 1.07 0.08 False

    graph
    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'
    Details

    Payloads

    Baseline
    Compare

    System.MathBenchmarks.Single.Exp

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.89602829768271 > 20.477927369077314.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 6/13/2023 11:57:38 AM, 6/19/2023 1:02:34 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -655.335949203705 (T) = (0 -20896.364181416317) / Math.Sqrt((40.17758702066901 / (13)) + (41.61583359222963 / (29))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (29) - 2, .025) and -0.07148554918995902 = (19502.23612181604 - 20896.364181416317) / 19502.23612181604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    System.MathBenchmarks.Single.ExpM1

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.900895979020977 > 20.479824496722852.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 6/13/2023 11:57:38 AM, 6/19/2023 1:02:34 PM, 6/30/2023 8:34:57 AM, 7/12/2023 9:48:15 PM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -810.1514609931146 (T) = (0 -20897.121012100346) / Math.Sqrt((17.55173933672352 / (12)) + (41.685999592766514 / (28))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (28) - 2, .025) and -0.07135588775065031 = (19505.30281396464 - 20897.121012100346) / 19505.30281396464 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS Windows 10.0.19042
    Queue OwlWindows
    Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
    Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in System.Collections.CtorDefaultSizeNonGeneric

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    9.35 ns 10.93 ns 1.17 0.07 False
    13.39 ns 14.74 ns 1.10 0.10 False
    17.76 ns 20.41 ns 1.15 0.08 False

    graph
    graph
    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSizeNonGeneric*'
    Details

    Payloads

    Baseline
    Compare

    System.Collections.CtorDefaultSizeNonGeneric.Stack

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.932039866066837 > 10.107874469115403.
IsChangePoint: Marked as a change because one of 5/22/2023 2:06:57 AM, 5/26/2023 6:22:05 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -17.25314313105284 (T) = (0 -10.819700698650307) / Math.Sqrt((0.042079729243106885 / (13)) + (0.10216832823724074 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.147952281853175 = (9.425218164280947 - 10.819700698650307) / 9.425218164280947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    System.Collections.CtorDefaultSizeNonGeneric.Hashtable

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.74443046272499 > 14.301528270190383.
IsChangePoint: Marked as a change because one of 5/27/2023 2:31:56 PM, 6/8/2023 8:02:22 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -21.918501081695513 (T) = (0 -15.012252165113182) / Math.Sqrt((0.023297441997376777 / (13)) + (0.0988705509575178 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.11488827186229812 = (13.465252567449532 - 15.012252165113182) / 13.465252567449532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    System.Collections.CtorDefaultSizeNonGeneric.SortedList

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.407228293697592 > 17.987433026991145.
IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 5/19/2023 10:30:38 AM, 5/22/2023 7:40:07 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -26.775160969513717 (T) = (0 -20.745933817174663) / Math.Sqrt((0.05984172213284456 / (13)) + (0.3775851660306739 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.20076987508981725 = (17.277193780051213 - 20.745933817174663) / 17.277193780051213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS Windows 10.0.19042
    Queue OwlWindows
    Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
    Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in Benchstone.BenchI.TreeInsert

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    44.03 μs 48.67 μs 1.11 0.04 False

    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.TreeInsert*'
    Details

    Payloads

    Baseline
    Compare

    Benchstone.BenchI.TreeInsert.Test

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.67096411934339 > 46.31346658981389.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/1/2023 3:42:23 PM, 6/13/2023 11:57:38 AM, 6/22/2023 7:55:14 AM, 7/1/2023 6:36:20 PM, 7/5/2023 7:07:19 PM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -27.710762499062483 (T) = (0 -48676.78591567696) / Math.Sqrt((246798.7737337005 / (13)) + (223279.72069055095 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.10147134778847257 = (44192.51214605801 - 48676.78591567696) / 44192.51214605801 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS Windows 10.0.19042
    Queue OwlWindows
    Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
    Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in System.Tests.Perf_Char

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    20.54 ns 28.06 ns 1.37 0.18 False

    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'
    Details

    Payloads

    Baseline
    Compare

    System.Tests.Perf_Char.Char_IsLower(input: "Good afternoon, Constable!")

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.05809342245437 > 21.89258896306033.
IsChangePoint: Marked as a change because one of 4/25/2023 3:46:39 AM, 5/1/2023 3:42:23 PM, 5/9/2023 1:22:51 PM, 6/19/2023 6:37:24 PM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -30.633100228490065 (T) = (0 -28.628841383596065) / Math.Sqrt((0.13990449192789264 / (13)) + (1.7243185582103528 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.3806259553951021 = (20.736131514638355 - 28.628841383596065) / 20.736131514638355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS Windows 10.0.19042
    Queue OwlWindows
    Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
    Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in System.IO.Tests.BinaryWriterTests

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    7.96 ns 9.07 ns 1.14 0.08 False

    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterTests*'
    Details

    Payloads

    Baseline
    Compare

    System.IO.Tests.BinaryWriterTests.DefaultCtor

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.072859109776312 > 8.354720929930942.
IsChangePoint: Marked as a change because one of 5/12/2023 4:02:48 AM, 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -26.61547016679733 (T) = (0 -9.078944219146809) / Math.Sqrt((0.02585258533960576 / (13)) + (0.010688748446311147 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.1649738171195371 = (7.79326031686704 - 9.078944219146809) / 7.79326031686704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS Windows 10.0.19042
    Queue OwlWindows
    Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
    Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    7.59 ns 9.14 ns 1.20 0.20 False

    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
    Details

    Payloads

    Baseline
    Compare

    Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimStart

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.13891100297057 > 7.378869670302504.
IsChangePoint: Marked as a change because one of 6/5/2023 10:43:22 AM, 6/9/2023 2:21:24 AM, 6/30/2023 8:34:57 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -13.154116555259943 (T) = (0 -8.881237739973487) / Math.Sqrt((0.18699310089062662 / (13)) + (0.06992436773695317 / (31))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (31) - 2, .025) and -0.23617976736482144 = (7.184422504265479 - 8.881237739973487) / 7.184422504265479 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS Windows 10.0.19042
    Queue OwlWindows
    Baseline d40ca00ca2fb5411ff159b2e84b800e8dfbd681e
    Compare 65536eb2f01ebb85a54e792a2b974fe76af7e230
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in System.Collections.ContainsFalse<Int32>

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    16.63 μs 20.22 μs 1.22 0.12 False

    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
    Details

    Payloads

    Baseline
    Compare

    System.Collections.ContainsFalse<Int32>.Stack(Size: 512)

    ETL Files

    Histogram

    Description of detection logic

    IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.22386281326959 > 17.249698305660147.
IsChangePoint: Marked as a change because one of 5/26/2023 6:22:05 AM, 7/5/2023 7:07:19 PM, 7/12/2023 2:12:47 AM, 7/23/2023 4:13:29 AM, 7/31/2023 7:57:50 PM falls between 7/23/2023 4:13:29 AM and 7/31/2023 7:57:50 PM.
IsRegressionStdDev: Marked as regression because -12.797091281790742 (T) = (0 -19577.57278907806) / Math.Sqrt((15408.291694399582 / (12)) + (1613603.2924137847 / (28))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (28) - 2, .025) and -0.18857469446258693 = (16471.47030833434 - 19577.57278907806) / 16471.47030833434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

  3. cincuranet
    changed the title [-][Perf] Windows/x64: 21 Regressions on 7/23/2023 2:51:47 PM[/-] [+]Regressions in System.Tests.Perf_Int16[/+] on Aug 1, 2023

  4. cincuranet commented on Aug 1, 2023

    @cincuranet
    cincuranet
    on Aug 1, 2023
    Contributor

    The commit range points only to one commit: #85562. Not super sure if it's related.

  5. cincuranet
    removed
    untriagedNew issue has not been triaged by the area owner
    on Aug 1, 2023
  6. cincuranet
    transferred this issue fromdotnet/perf-autofiling-issueson Aug 1, 2023
  7. ghost added
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Aug 1, 2023
  8. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Aug 1, 2023
  9. cincuranet
    added
    tenet-performancePerformance related issue
    tenet-performance-benchmarksIssue from performance benchmark
    on Aug 1, 2023

  10. AndyAyersMS commented on Aug 1, 2023

    @AndyAyersMS
    AndyAyersMS
    on Aug 1, 2023
    Member

    Also possibly related: dotnet/perf-autofiling-issues#20151

  11. jeffschwMSFT
    added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    on Aug 2, 2023

  13. JulieLeeMSFT commented on Aug 2, 2023

    @JulieLeeMSFT
    JulieLeeMSFT
    on Aug 2, 2023
    Member

    CC @MichalPetryka @EgorBo PTAL the perf regression.

  14. JulieLeeMSFT
    added this to the 8.0.0 milestone on Aug 2, 2023
  15. ghost removed
    untriagedNew issue has not been triaged by the area owner
    on Aug 2, 2023
  16. JulieLeeMSFT
    assigned
    EgorBo
    on Aug 2, 2023
  17. vcsjones
    removed
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Aug 3, 2023

  18. EgorBo commented on Aug 5, 2023

    @EgorBo
    EgorBo
    on Aug 5, 2023
    Member

    Only a few benchmarks in this list regressed, presumably, because of different inlining decisions - will investigate in 9.0 (or in 8.0 if I have time)

  19. EgorBo
    modified the milestones: 8.0.0, 9.0.0 on Aug 5, 2023
  20. EgorBo
    closed this as completedon May 3, 2024
  21. github-actions
    locked and limited conversation to collaborators on Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-windowsruntime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions