[lewing]

Stores through an unknown address (STOREIND not to a field, array element or local) are value-numbered as an opaque memory mutation, so a reload from the address just written always gets a fresh VN. This implements the VNF_ByrefExposedStore idea from the existing comment in fgValueNumberStore. A non-volatile int/long/float/double/ref/byref store records a memory state carrying its address and value; long is excluded where it is decomposed. This also applies when the byref-exposed and GC heap memory states differ. A following same-type load from the same liberal address while that state is current takes the stored value.

• The forwarded load records its loop memory dependence, so LICM can't hoist it above the store.
• When the conservative address also matches, the store already faulted on a null address, so the load's VN omits that NullPtrExc. VN does not change IR flags.
• Reinterpreting loads (e.g. long store, double load) still read memory.
• There is no xarch special case for read-modify-write sources; a forwarded update source can now use a register instead of op [mem], x. Screening such loads in CSE is split into a separate PR.

Motivation: Utf8JsonReader.SkipWhiteSpace on Wasm R2R reloads _consumed after each store. This removes the loop-latch reload. In an isolated-method d8 measurement, the 64-space case went from 77 to 45 ns.

SuperPMI shrinks code on every platform (arm64 −34 KB/−33 KB, arm −17 KB, x64/x86 about −7 KB each, wasm −1 KB). PerfScore improves in every collection, and throughput cost is +0.01–0.07%. The remaining x64/x86 size growth is mostly a read after an update now coming from a register. Details and examples are in the SuperPMI comment; those numbers predate the ref/byref changes and the removal of the xarch read-modify-write check.

Tests: ARM64/X64 disasm checks in ByrefStoreForwarding, and correctness and exception-preservation cases in ExceptionSets. These include a hoisting regression that fails if the loop-dependence record is removed.

Note

This PR description and the change were prepared with GitHub Copilot.

[azure-pipelines]

Azure Pipelines:
Successfully started running 5 pipeline(s).
11 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[lewing]

SuperPMI results from the CI run:

Platform	Net size	Methods smaller / larger	PerfScore better / worse	Throughput
linux arm64	−17,572 B	1,709 / 264	2,007 / 48	+0.00–0.01%
windows arm64	−13,376 B	1,797 / 49	1,950 / 13	+0.01–0.02%
linux arm	−13,196 B	2,590 / 211	2,746 / 386	+0.01–0.03%
linux x64	−3,128 B	1,760 / 1,995	4,119 / 53	+0.03–0.07%
windows x64	−2,299 B	956 / 908	2,245 / 42	+0.03–0.06%
windows x86	−2,292 B	806 / 744	2,028 / 43	+0.01–0.03%
browser wasm	−1,108 B	227 / 5	115 / 11	+0.01%

Few methods change: whole-collection PerfScore moves by −0.002% to −0.04%. Browser wasm has a single 89K-method crossgen2 library collection, so its coverage is thin.

arm64: the reload is gone. Repeated x &= mask through a byref (AllowedBmpCodePointsBitmap.ForbidHtmlCharacters, 112 → 92 bytes):

             and     w2, w2, #0xD1FFAB1E
             str     w2, [x1]
             mov     x1, x0
-            ldr     w2, [x1]
             and     w2, w2, #0xD1FFAB1E
             str     w2, [x1]

x64/x86: the main source of size regressions. In the same method, x64 loses its memory-operand read-modify-write forms because the forwarded value is CSE'd into a register (52 → 66 bytes). PerfScore goes from 22.5 to 14.0, but that model overstates the cost of RMW instructions, so I'd treat this as roughly neutral at best:

-       and      dword ptr [rax], 0xD1FFAB1E
+       mov      edx, dword ptr [rax]
+       and      edx, 0xD1FFAB1E
+       mov      dword ptr [rax], edx
        mov      rax, rcx
-       and      dword ptr [rax], 0xD1FFAB1E
+       and      edx, 0xD1FFAB1E
+       mov      dword ptr [rax], edx

I could stop forwarding when the reload only feeds a store back to the same address.

arm32: Thumb-2 encoding growth and register pressure. Most arm32 regressions use the same or fewer instructions but grow by 2–10 bytes: the extra live value moves operands into high registers, which need 32-bit encodings. A few are real loop costs, because the forwarded value has to stay live across the loop. In SpanReader.ReadVarUInt64 (PerfScore 318 → 328), a header reload is replaced by a register kept live through the loop body:

-            ldr     r1, [r4+0x0C]
-            cmp     r1, 0
+            cmp     r10, 0
             beq     SHORT G_M52454_IG08
 ...
-            subs    r1, r2, 1
+            add     r10, r2, -1
+            mov     r1, r10
             str     r0, [r4+0x08]
             str     r1, [r4+0x0C]

This also adds a callee-saved register and 12 bytes of frame.

wasm. In BlobReader.TryReadCompressedSignedInteger, a store to an out param followed by value |= mask no longer reloads the value (345 → 329 bytes). Each of the three |= branches changes like this:

             local.get 0
             i32.load 0 20      ;; V02
-            local.get 0
-            i32.load 0 20      ;; V02
-            i32.load 0 0
+            local.get 9
             i32.const -64
             i32.or
             i32.store 0 0

The byref itself (V02) still comes from the frame.

No replay failures or asserts. A handful of contexts per platform (≤12) are missing with the diff JIT because the new JIT asks for data the old collection didn't record. I haven't measured end-to-end performance; the only timing I have is the isolated Utf8JsonReader.SkipWhiteSpace measurement from the description.

Note

This comment was prepared with GitHub Copilot from the CI SuperPMI summaries.

[lewing]

/azp run runtime-coreclr jitstress, runtime-coreclr libraries-jitstress, runtime-coreclr outerloop

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).

[lewing]

/azp run runtime-coreclr superpmi-diffs

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[lewing]

Updated SuperPMI results after the second commit, which stops forwarding into xarch read-modify-write updates (CI run, full collection sets for the current JIT-EE GUID):

Platform	Net size	Methods smaller / larger	Throughput
linux arm64	−33,968 B	3,861 / 87	+0.01–0.04%
windows arm64	−32,984 B	3,580 / 84	+0.01–0.04%
linux arm	−16,966 B	3,454 / 249	+0.01–0.03%
osx arm64	−9,008 B	1,157 / 23	+0.02–0.03%
linux x64	−7,342 B	2,459 / 2,033	+0.03–0.07%
windows x64	−7,230 B	1,920 / 1,275	+0.03–0.07%
windows x86	−7,111 B	1,692 / 1,298	+0.01–0.03%
browser wasm	−1,107 B	226 / 5	+0.01%

PerfScore improves overall in every collection. There are no replay failures. A few contexts per platform are missing with the diff JIT (≤3, 17 on windows x86) because the new JIT asks for data the collections didn't record.

xarch read-modify-write. On x64/x86, a load that is the direct source of op [mem], x stored back to the same address is no longer forwarded, so lowering keeps the memory-operand form. Chains like v &= a; v &= b (ForbidHtmlCharacters, PInvokeFlags.set_BestFitMapping) compile as before and no longer appear in the x64 diffs. On the collections common to both CI runs:

Platform	Net	Methods larger
windows x64	−2,299 → −3,913 B	908 → 725
windows x86	−2,292 → −3,816 B	744 → 617
linux x64	−3,128 → −7,067 B	1,995 → 1,671

This isn't a clean A/B because main and the collections moved between runs. A local A/B on the three windows x64 collections with base, before and after JITs built from the same main commit shows the gate alone: 173 changed methods, 166 smaller and 5 larger, −801 B. arm64 and wasm codegen is unchanged by the gate.

The remaining x64/x86 growth is mostly a read after an update now coming from a register. For example:

-       add      dword ptr [rcx], edx
-       cmp      dword ptr [rcx], edx
+       mov      eax, edx
+       add      eax, dword ptr [rcx]
+       mov      dword ptr [rcx], eax
+       cmp      eax, edx

That's one fewer memory read for about 4 bytes. Also skipping these updates would save a further ~1 KB on windows x64 but gives back about half of the PerfScore gain on the library collections, so I left it.

Stress runs on the first commit: jitstress and outerloop passed. libraries-jitstress failed only on TensorValueReductionsPreserveScalarOrder, which also fails on main (#136647, fix in #136650).

Note

This comment was prepared with GitHub Copilot from the CI SuperPMI summaries and local SuperPMI runs.

[lewing]

@AndyAyersMS any thoughts on this one?

[AndyAyersMS]

@AndyAyersMS any thoughts on this one?

Let's settle #137032 first, and then take another look at this.