镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Small optimizations that improve search speed by 20% and reduce memory usage with no behavior differences. - #1008

Open
Dragon-Hatcher wants to merge 15 commits into
vprover:masterfrom
Dragon-Hatcher:faster-search
Open

Dragon-Hatcher wants to merge 15 commits into
vprover:masterfrom
Dragon-Hatcher:faster-search

Conversation

@Dragon-Hatcher

Copy link
Copy Markdown
Contributor

A set of independent performance changes, mostly in code tree matching, flat terms, substitution application and forward demodulation. The only behavior difference comes from the polarity flipping change in the last commit which only effects term ids.

Results

Plain -DCMAKE_BUILD_TYPE=Release builds of master (af03e15) and this branch, -sa discount -p off -t 30, on 72 TPTP problems from 23 domains used for benchmarking to find the optimizations and on 133 further problems sampled at random from TPTP as a holdout:

instructions, median instructions, total wall time, median memory, median
72 problems −27% −30% −23% −5%
133 random problems −21% −23% −3.5%

No problem got slower by more than measurement noise.

Commits

(The commits are logical so it is probably easier to review them one at a time rather than looking at the whole diff).

  • Scratch: a per-nesting-depth alternative to Recycled for working stacks needed on every call of small reentrant functions; used by SubstHelper::applyImpl and BottomUpEvaluation::apply.
  • EqHelper::replace skips subterms that cannot contain the replaced term.
  • CodeTreeTIS answers queries whose top symbol has no key (and there are no variable keys) without walking the tree.
  • Term, literal and clause matchers reuse their previous query's flat terms; copying a flat term copies only initialized entries; Entry::expand's already-expanded check is inline.
  • ClauseMatcher keeps one literal matcher per depth.
  • The code tree matcher pushes a backtracking point only after an operation succeeds, and goes to the alternative directly on failure.
  • Recycled uses [[no_unique_address]] for its policies, so it is the size of a pointer.
  • The variant index hash counts variables in an array and walks subterms with an explicit stack (same hash values).
  • Substitution tree insertion/removal avoid per-call allocations; Map::reset returns early when empty.
  • Forward demodulation remembers, per term, the last time no demodulator applied, and skips the term until a demodulator that could rewrite it (same top symbol, or a variable left-hand side) is inserted. Failures that depended on the clause (colours, the premise redundancy check) or on randomization are not recorded. The stamp uses existing padding in Term so doesn't increase the size of Term.
  • The passive queues store the age/weight key in their skip list nodes and read the clause only on ties.
  • Opposite and complementary query literals flip the polarity in the flat term header instead of building the complementary literal.

Dragon-Hatcher and others added 15 commits October 3, 2026 17:15
…plication

SubstHelper::applyImpl and BottomUpEvaluation::apply took their stacks from
Recycled on every call, which on small terms costs more than the work itself.
Scratch keeps one object per nesting depth instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A subterm no heavier than the term being replaced (or a ground one, when it
is a variable) cannot contain it, so it need not be traversed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CodeTreeTIS counts its keys by top symbol, so a query term whose top symbol
has no key, while there are no variable keys, has no generalizations. Most
forward demodulation queries are of this kind.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Term, literal and clause matchers are recycled between queries, but each
query allocated new flat terms. They now keep the flat terms of the previous
query and build the next ones in them, up to a size limit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The argument entries of an unexpanded function are uninitialized until it is
expanded, and most of a fresh flat term is such entries, so copying (for the
reversed and opposite literals of subsumption queries) skips them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The matcher calls expand() on every function entry it checks, and most are
already expanded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Literal matchers are entered and left many times in each subsumption query,
so the clause matcher keeps one for each depth instead of taking them from
Recycled each time.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When an operation fails, its alternative was pushed onto the backtracking
stack only to be popped again immediately. Now the matcher moves to the
alternative directly and pushes it only after a successful operation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Its default reset and keep policies are empty, so [[no_unique_address]]
keeps them from taking space. Index query result iterators then fit the
small object allocator.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Count variable occurrences in an array indexed by variable instead of a
hash map, and walk subterms with an explicit stack instead of
SubtermIterator. The hash values are unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…trees

The binding heap of an insertion is kept between calls, as unresolvedSplits
already is, and the subterm stacks are recycled.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ast tried

A term that had no applicable demodulator stays so until a demodulator whose
left-hand side has the same top symbol, or is a variable, is inserted
(removals cannot make one applicable). DemodulationLHSIndex records when such
demodulators were last inserted, and terms record when they were last found
irreducible, in what was padding in Term. Failures that depended on the
clause (colours, the premise redundancy check) or on randomization are not
recorded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Inserting into or removing from the age and weight queues compared against
the clause of every node passed, through a virtual call. Nodes now store the
age and weight packed into one word, and the clause is only read on ties.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ary literal

Matching reads a literal's polarity from the header entry only, so opposite
literals of subsumption resolution queries and complementary literal code
tree queries no longer look up or create the complementary literal.

Not creating these literals changes the ids of literals made later, so where
the search breaks ties by id it can differ on some problems.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@quickbeam123

Copy link
Copy Markdown
Collaborator

Looking cool... I am curious to see how much each of these contributes!

Can take care of giving it a thorough performance check :)

@Dragon-Hatcher

Copy link
Copy Markdown
Contributor Author

Would appreciate a double check of my measurements. The changes are basically all 1-2% each. Just a lot of small improvements.

@mezpusz

mezpusz commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

It indeed looks really cool, thanks! I was also wondering how much each of these contribute.. we could decide on a case-by-case basis which ones are reasonable to merge without sacrificing code quality too much.

@easychair

easychair commented Oct 5, 2026 via email

Copy link
Copy Markdown
Contributor

@Dragon-Hatcher

Dragon-Hatcher commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor Author

I do not have access to a lot of compute so it is hard for me to do large scale tests. Obviously bigger tests will be much more decisive about the true improvement. My hope is that most of these changes are simple enough that they are obvious wins even without testing. None are huge improvements on their own so if any individual one seems too messy to maintain, its fine to drop.

@easychair

easychair commented Oct 5, 2026 via email

Copy link
Copy Markdown
Contributor

@quickbeam123

Copy link
Copy Markdown
Collaborator

Here are my first results for a two short instruction-limited (-i 5000) full TPTP (v9.3.1) sweeps (one -sa otter and one -sa discount -awr 1:10).

There are only two sequential diffs with no gain (or a tiny loss) in the number of problems solved and I will next have these tested uder a time limit (cache/memory related optimization sometimes only show up under wall clock).

40 5a66bca31c Flip the polarity of query flat terms without building the complementary literal
-- otter: 10781
-- dis10: 10689

39 413e087f4b Keep passive queue sort keys in the skip list nodes
-- otter: 10775
-- dis10: 10674

38 6b8a8907c5 Skip forward demodulation of terms that could not be rewritten when last tried
-- otter: 10772
-- dis10: 10667

37 1fc5c61dde Do not walk an empty Map on reset
-- otter: 10759
-- dis10: 10639

36 049f35dcf1 Avoid allocations when inserting into and removing from substitution trees
-- otter: 10746 (no gain)?
-- dis10: 10626 (no gain)?

35 fa1ea897f6 Speed up the variant index hash
-- otter: 10746
-- dis10: 10626

34 fba8bbb3e7 Make Recycled no larger than the pointer it holds
-- otter: 10744
-- dis10: 10623

33 8446ac357a Push code tree backtracking points only for operations that succeed
-- otter: 10737
-- dis10: 10619

32 1f566aa3ad Keep a literal matcher per depth in ClauseMatcher
-- otter: 10711
-- dis10: 10609

31 002136e41e Inline the already-expanded check of FlatTerm::Entry::expand
-- otter: 10704
-- dis10: 10605

30 2ea81dc66b Copy only the initialized entries of flat terms
-- otter: 10700 (not generally useful)?
-- dis10: 10604 (does nothing here)?
A candidate to check also just under a time limit!

29 de7bc3a858 Reuse flat terms across code tree queries
-- otter: 10701
-- dis10: 10604

28 96f59c5603 Answer generalization queries with no candidate key without a tree walk
-- otter: 10684
-- dis10: 10581

27 675fad8617 Skip subterms that cannot contain the replaced term in EqHelper::replace
-- otter: 10677
-- dis10: 10575

26 c93461d6ea Add Scratch for per-call working stacks and use it in substitution application
-- otter: 10673
-- dis10: 10571

25 af03e1547d (origin/master, origin/HEAD, master)
-- otter: 10670
-- dis10: 10565

@quickbeam123

Copy link
Copy Markdown
Collaborator

Overall, I think, we want to pick the really unproblematic ones immediately, but maybe not everything related to code trees, which are going to be an optimization target of the Vampire team in the near future anyway and abrupt changes there could complicate the work.

If I do this by cherry-pick, would you be able to rebase your PR on top of the new master when done, so that we only see what remains? (Or tell me if you can think of an alternative way that wouldn't confuse Github...)

hash = termFunctorHash(sub,hash);
toDo->push(ts->next());
ts = sub->args();
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this basically inlining the SubtermIterator iteration?

Can this really be making a difference?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By my benchmarks this and the other half of the commit contribute equally to the gain.

@quickbeam123 quickbeam123 Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I see. It's not just inlining the SubtermIterator.

It's also using it as an opportunity to use Scratch.

However, I would argue using Scratch here is not well-justified. As we never call computeHashAndCountVariables(TermList*,...) recursively (it it will cost use 8 entries on the Scratch's stack). And SubtermIterator already uses a Recycled<Stack<const TermList*>>.

Shouldn't this be an opportunity to make SubtermIterator itself more efficient (better inline-able, perhaps, maybe moving hasNext() to the header), rather than bloating the iteration code here?

if (counts[v]++ == 0) seen.push(v);
}
unsigned size() const { return seen.size(); }
};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This other half seems reasonable if not major.

BinaryHeap<Binding, BindingComparator<LeafData_>> remainingBindings;
for (auto [var, term] : iterTraits(svBindings.items())) {
remainingBindings.insert(Binding(var, term));
static BinaryHeap<Binding, BindingComparator<LeafData_>> remainingBindings;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are trying to get rid off the static as a design pattern.

Would it make sense to make it a member instead?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I measured and a member is fine.

remainingBindings.insert(Binding(var, term));
static BinaryHeap<Binding, BindingComparator<LeafData_>> remainingBindings;
remainingBindings.reset();
typename BindingMap::Iterator bit(svBindings);

@quickbeam123 quickbeam123 Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, iterTraits is the modern way and while (bit.hasNext()) slowly under removal.

Is the change of the loop essential for the improvement?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll change it. Performance is the same without the while.

@quickbeam123

Copy link
Copy Markdown
Collaborator

Under -t 5 instead of the instruction limit we get:

29 -> 30
-- otter: 11536 -> 11547
-- disc: 11382 -> 11383

35 -> 36
-- otter: 11574 -> 11569 (a decline)
-- disc: 11410 -> 11407 (also here)

38 -> 39
-- otter: 11590 -> 11590 (no gain)
-- disc: 11443 -> 11449 

So I would drop 049f35dcf1 (the substitution tree allocations) on the basis of insufficient evidence of a benefit.

I was expecting to see more benefit of 413e087f4b (Keep passive queue sort keys in the skip list nodes) under a time-limit; but I was still using a lot of parallel processes so it's a bit hard to argue about caches. But could suspect instruction counting only "paying" one instruction for the whole 64bit age+weight/weight+age compare, so just hiding the cost instead of really improving. I still like the change more than not - I just wished it improved more ;)

@Dragon-Hatcher

Copy link
Copy Markdown
Contributor Author

Dropping the substitution tree change is fine with me. I wonder if the easiest way would be to open separate PRs for the changes we actually want to merge and treating this as a draft? I would have opened separate PRs for each change but I didn't want to open 15 separate PRs.

Comment thread Kernel/FlatTerm.hpp

void swapCommutativePredicateArguments();
/** Like changeLiteralPolarity(), but leaves the term entry pointing to the original literal. */
void flipPolarity()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we simply rewrite changeLiteralPolarity instead of adding a new function?

quickbeam123 added a commit that referenced this pull request Oct 6, 2026
A selected subset of improvements from #1008
@quickbeam123

Copy link
Copy Markdown
Collaborator

So, I went ahead and merged:

26 c93461d6ea Add Scratch for per-call working stacks and use it in substitution application
27 675fad8617 Skip subterms that cannot contain the replaced term in EqHelper::replace
34 fba8bbb3e7 Make Recycled no larger than the pointer it holds
37 1fc5c61dde Do not walk an empty Map on reset
39 413e087f4b Keep passive queue sort keys in the skip list nodes

Could you, @Dragon-Hatcher , please pull the new master, rebase your faster-search (interactively, dropping
36 049f35dcf1 Avoid allocations when inserting into and removing from substitution trees) and then push -f again? When we do this with local branches, the PR updates correspondingly.

In general, the code tree updates will be considered, but probably later as part of a larger effort.

In "Speed up the variant index hash", I left a comment. I would love to save the SubtermIterator there, if we could make that one itself more efficient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants