镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Bench: add a duration-qualified arithmetic publication tier #843

Closed

Description

@chrisbbreuer

Problem

The frozen required-bytecode/no-JIT arithmetic profile uses 100 jobs for smoke validation and 10,000 jobs for its full structural row. On the reference Apple Silicon host, the stable-Number full row completes in roughly 7.5 ms. proc_pid_rusage(RUSAGE_INFO_V6).ri_energy_nj is quantized at that boundary: a clean seven-pair AC run reported exact/stable instructions and cycles but process-energy deltas alternating between zero and coarse nonzero values. The exact-parent collector correctly blocked publication because energy RSD exceeded the frozen 5% limit.

This is a benchmark-contract defect, not permission to discard energy, weaken the efficiency gate, repeat until favorable, or publish a diagnostic row. The existing quick/full rows are useful and must remain byte-for-byte comparable for smoke, checksum, attribution, and allocation validation, but the scored publication boundary needs enough identical logical work to make every required efficiency counter measurable.

Scope

  • Version the representative matrix and preserve the existing 100-job quick and 10,000-job full contracts unchanged.
  • Add an explicit duration-qualified publication tier to the four required-bytecode/no-JIT arithmetic workloads.
  • Freeze 500,000-job checksums and require the exact-parent collector to use that tier for public efficiency evidence. Seven-pair sizing probes rejected 100,000 jobs because the coercion control remained at 6.2-6.8% process-energy RSD and rejected 200,000 jobs because its parent remained at 6.7%; all four parent/candidate rows are below 2.1% at 500,000 jobs.
  • Document that the publication tier changes only measurement duration, not the workload source, Context options, required-bytecode/no-native boundary, warmup policy, attribution semantics, or control coverage.
  • Extend matrix validation and rejection tests so missing, shortened, or checksum-drifting publication tiers fail closed.

Acceptance

  • Matrix version advances without rewriting historical manifests.
  • 100-job quick and 10,000-job full jobs/checksums remain unchanged.
  • All four 500,000-job publication checksums match exact parent and candidate binaries.
  • Matrix validator and structural mutation suite reject publication-tier drift.
  • Docs state why sub-counter-resolution rows cannot support public energy/efficiency claims.
  • The VM: quicken monomorphic Number arithmetic sites #734 exact-parent batch passes all stable-Number and control rows without weakening the 5% instruction/cycle/energy or nominal-thermal gates.

Related: #734, #503, #842.

Activity

  1. chrisbbreuer commented on Sep 1, 2026

    @chrisbbreuer
    MemberAuthor

    Completed in:

    V32 preserves the existing 100-job smoke and 10,000-job structural jobs/checksums. It adds only a 500,000-job publication tier for the four required-bytecode/no-JIT arithmetic witnesses, with exact parent/candidate checksums and the existing 5% instruction/cycle/process-energy, nominal-thermal, AC-power, no-discard, and provenance gates unchanged.

    Sizing was fail-closed: 100,000 and 200,000 jobs were rejected because the shortest control still exceeded the process-energy RSD limit; seven-pair 500,000-job probes put every parent/candidate energy series below 2.1%. The final four artifacts all have stable efficiency evidence and no publication block. Matrix validation and 70/70 structural mutations, the 18/18 benchmark harness, 313-file link audit, dependency audit, and checked 50-page docs build pass.

    Remote main is verified at 4d347739524d44fd1f4ae64096f2e9a786a20c22; all #843 acceptance items are complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions