镜像站点 · 本页由第三方 GitHub 只读镜像提供，非 GitHub 官方站点，不接受任何登录或凭据输入。前往 github.com
Skip to content

[Perf] Linux/x64: 22 Regressions on 2/13/2025 12:40:20 AM +00:00 #112657

New issue
New issue
Closed
Closed
[Perf] Linux/x64: 22 Regressions on 2/13/2025 12:40:20 AM +00:00#112657
Assignees
EgorBo
Labels
arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark
Milestone
10.0.0

Description

@performanceautofiler
performanceautofiler
bot
opened on Feb 18, 2025

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in IfStatements.IfStatements

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
27.13 μs 29.84 μs 1.10 0.02 False
24.42 μs 29.84 μs 1.22 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'IfStatements.IfStatements*'
Details

IfStatements.IfStatements.And

ETL Files

Histogram

JIT Disasms

IfStatements.IfStatements.Single

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Perf_LengthBucketsFrozenDictionary

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.01 μs 1.07 μs 1.06 0.02 False
1.16 μs 1.27 μs 1.10 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Perf_LengthBucketsFrozenDictionary*'
Details

System.Collections.Perf_LengthBucketsFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 1000, ItemsPerBucket: 5)

ETL Files

Histogram

JIT Disasms

System.Collections.Perf_LengthBucketsFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 1000, ItemsPerBucket: 1)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
40.68 μs 54.26 μs 1.33 0.03 False
271.18 μs 298.27 μs 1.10 0.02 False
135.64 μs 162.74 μs 1.20 0.03 False
135.56 μs 162.74 μs 1.20 0.02 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
Details

PerfLabTests.LowLevelPerf.ObjectStringIsString

ETL Files

Histogram

JIT Disasms

PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodLongHierarchy

ETL Files

Histogram

JIT Disasms

PerfLabTests.LowLevelPerf.ClassVirtualMethod

ETL Files

Histogram

JIT Disasms

PerfLabTests.LowLevelPerf.StructWithInterfaceInterfaceMethod

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
17.74 μs 25.16 μs 1.42 0.19 False
2.86 μs 3.03 μs 1.06 0.22 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
Details

System.Collections.IterateForEach<String>.ImmutableDictionary(Size: 512)

ETL Files

Histogram

JIT Disasms

System.Collections.IterateForEach<String>.ConcurrentQueue(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<SmallClass, SmallClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
16.33 μs 17.56 μs 1.08 0.13 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;SmallClass, SmallClass&gt;*'
Details

System.Collections.TryGetValueFalse<SmallClass, SmallClass>.SortedList(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
287.73 ns 353.15 ns 1.23 0.11 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForNonGeneric&lt;Int32&gt;*'
Details

System.Collections.IterateForNonGeneric<Int32>.ArrayList(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
21.71 μs 23.08 μs 1.06 0.06 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'
Details

System.MathBenchmarks.Double.Exp

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
54.26 μs 108.54 μs 2.00 0.03 False
81.38 μs 162.79 μs 2.00 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'
Details

PerfLabTests.CastingPerf2.CastingPerf.IntObj

ETL Files

Histogram

JIT Disasms

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.BenchE

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
226.41 ms 290.33 ms 1.28 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BenchE*'
Details

Benchstone.BenchI.BenchE.Test

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
96.58 ns 115.85 ns 1.20 0.02 False
20.09 ns 38.79 ns 1.93 0.03 False
19.56 ns 38.79 ns 1.98 0.03 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'
Details

System.Tests.Perf_String.IndexerCheckPathLength

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_String.IndexerCheckLengthHoisting

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_String.IndexerCheckBoundCheckHoist

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
410.99 ns 546.24 ns 1.33 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'
Details

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.MeasureReverseUsingNtoH

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
57.96 ns 84.50 ns 1.46 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
Details

System.Linq.Tests.Perf_Enumerable.SelectToArray(input: Range)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue ViperUbuntu
Baseline 109c9469a7f76bdc98300bd9c7bb4eaf014be971
Compare 505e3171007810999a7f56672b9f70332609273f
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEachNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.13 μs 1.25 μs 1.10 0.13 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;String&gt;*'
Details

System.Collections.IterateForEachNonGeneric<String>.Stack(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. performanceautofiler
    added
    arch-x64
    os-linuxLinux OS (any supported distro)
    PGO
    runtime-coreclrspecific to the CoreCLR runtime
    untriagedNew issue has not been triaged by the area owner
    on Feb 18, 2025
  2. LoopedBard3
    removed
    PGO
    on Feb 18, 2025
  3. LoopedBard3
    changed the title [-][Perf] Linux/x64: 22 Regressions on 2/13/2025 12:40:20 AM +00:00[/-] [+][Perf] Linux/x64: System.Collections.IterateForEach<String> Regressions on 2/13/2025 12:40:20 AM +00:00[/+] on Feb 18, 2025
  4. LoopedBard3
    transferred this issue fromdotnet/perf-autofiling-issueson Feb 18, 2025
  5. ghost added
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Feb 18, 2025
  6. LoopedBard3
    changed the title [-][Perf] Linux/x64: System.Collections.IterateForEach<String> Regressions on 2/13/2025 12:40:20 AM +00:00[/-] [+][Perf] Linux/x64: 22 Regressions on 2/13/2025 12:40:20 AM +00:00[/+] on Feb 18, 2025

  7. LoopedBard3 commented on Feb 18, 2025

    @LoopedBard3
    LoopedBard3
    on Feb 18, 2025
    Member

    Likely due to #112060. Most of the regressions appear to be noise/bimodal while the System.Collections.IterateForEach test regressions seem potentially real. FYI @EgorBo

    Related Regressions

  8. LoopedBard3
    added
    tenet-performancePerformance related issue
    tenet-performance-benchmarksIssue from performance benchmark
    on Feb 18, 2025

  9. EgorBo commented on Feb 19, 2025

    @EgorBo
    EgorBo
    on Feb 19, 2025 · Hidden as resolved

  10. EgorBo commented on Feb 19, 2025

    @EgorBo
    EgorBo
    on Feb 19, 2025 · Hidden as resolved

  11. EgorBo commented on Feb 19, 2025

    @EgorBo
    EgorBo
    on Feb 19, 2025
    Member

    @EgorBot -amd -intel -commit b1ab309 vs previous --filter System.Collections.IterateForEach*

  12. EgorBo commented on Feb 19, 2025

    @EgorBo
    EgorBo
    on Feb 19, 2025
    Member

    Looks like there are only a few benchmarks regressed from #112060, the rest are just bi-modal noise

    Overall, it was expected that #112060 would have regressed some things, it's listed here: #112060 (comment)

    Glad that there are more improvements than regressions

  13. jeffschwMSFT
    added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    on Feb 19, 2025

  14. dotnet-policy-service commented on Feb 19, 2025

    @dotnet-policy-service
    dotnet-policy-serviceBot
    on Feb 19, 2025 – with Dotnet Policy Service
    Contributor

    Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
    See info in area-owners.md if you want to be subscribed.

  15. JulieLeeMSFT commented on Feb 25, 2025

    @JulieLeeMSFT
    JulieLeeMSFT
    on Feb 25, 2025
    Member

    @EgorBo, is it ok to close this issue because it is expected to have some regressions?

    Looks like there are only a few benchmarks regressed from #112060, the rest are just bi-modal noise

    Overall, it was expected that #112060 would have regressed some things, it's listed here: #112060 (comment)

    Glad that there are more improvements than regressions

  16. vcsjones
    removed
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Mar 3, 2025
  17. JulieLeeMSFT
    assigned
    EgorBo
    on Mar 3, 2025
  18. JulieLeeMSFT
    removed
    untriagedNew issue has not been triaged by the area owner
    on Mar 3, 2025
  19. JulieLeeMSFT
    added this to the 10.0.0 milestone on Mar 3, 2025
  20. EgorBo
    closed this as completedon Mar 21, 2025
  21. github-actions
    locked and limited conversation to collaborators on Apr 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions