Inventory Flow tab: backdate edits so every column adds up (builds on #5313) by awwaiid · Pull Request #5730 · rubyforgood/human-essentials

awwaiid · 2026-09-27T18:22:20Z

Builds on #5313 (Resolves #5249). This branch contains #5313's commits plus one more. #5313 comes from a fork, so this PR is based on main.

This implements (and re-works previous implementation) a better inventory-flow tab for a storage location and date range. It uses the event log from the "now" perspective -- meaning if there were retroactive edits then those edits show up when we view an earlier window of time.

I did actual benchmarks using prod data to validate that this performs good enough as an in-memory generation (thereby simplifying away some complex SQL from a previous attempt).

Also:

The date range now goes through the shared DateRangeHelper#selected_range
The query returns rows and totals separately instead of copying the totals into every row.

Verification on a production snapshot

0 of 455 quarterly windows break start + change = end (65 storage locations across the 15 busiest orgs).
0 of 65 locations differ between end quantity (for a window ending today) and the current inventory.
Timing on the busiest location of each of the top 6 orgs, last 30 days: 306–680ms (the app itself loading/rendering takes more time than that)

Notes for reviewers

The date range filters by when things were recorded (event time), not by the donation or distribution's own date. This matches the rest of inventory history.
The page now does two replays: the Inventory tab and this one. If that's too slow for the largest orgs, a follow-up could load the Flow tab only when someone opens it.

Example from prod-clone data:

…ing and display

… code

…curate item creation timestamps

…ccuracy

…d accuracy and clarity

…e related specs for accurate quantity calculations

Validating against a production snapshot showed the flow tab's numbers diverging from inventory by millions of units. Three causes: 1. Editing a donation/purchase/distribution/transfer/adjustment republishes the full event for the same eventable, and inventory replay diffs against the previous version - but the query counted every version at full value. Dedupe to the latest version per eventable (destroy events publish zeroed items, so destroyed records net to nothing and drop out). 2. Kit allocation/deallocation events were not counted at all. On the heaviest production location that is 1.6M items consumed into kits that the tab silently omitted. They now flow like any other event (each row applies in full, matching InventoryAggregate). 3. Audits record the absolute counted quantity, not a delta, so the tab showed e.g. 'audited at 5000' as 5000 in. Audit contributions are now computed as deltas via a single replay pass through the event log. Also collapse the per-type to/from conditions (which had and/or precedence bugs) into generic to/from matching over an explicit list of flow event types, filter by event_time rather than created_at to match replay semantics, drop all-zero rows, and move totals to Ruby. With these fixes, snapshot + in - out reconciles with current inventory to within a few hundred units over multi-year windows moving 7M+ units on production data (residuals trace to UpdateExistingEvent group handling quirks), and the page renders in under a second on the largest production org. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNy2e1J81K8xbvXraCeN8e

The flow tab now shows, per item: Quantity at Start, Quantity In, Quantity Out, Quantity Adjustment/Audit, Quantity Change, and Quantity at End, so each row self-validates (start + in - out + adjustment = end) and users can see stock levels bracketing the flows. - Start/end quantities come from raw InventoryAggregate state at the window boundaries (View::Inventory prunes inactive items, so the raw aggregate is used instead) - Adjustments move out of in/out into the combined adjustment/audit column, netted with their signed quantities; audit deltas land there too. The audit replay pass now doubles as the window-end state computation, so the whole page needs only two replays - The effective window start is clamped to just after the organization's most recent usable snapshot: inventory state cannot be derived from events before it, and unclamped ranges silently double-counted pre-snapshot events against the snapshot baseline - Rows appear when they have flow activity or a start/end difference Validated against a production snapshot: row identity holds for all but 1-7 rows per organization (rare event-correction corner cases in aggregate diff ordering, off by tens of units against multi-million unit flows), totals reconcile start to end, and the page renders in under a second on the largest organizations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNy2e1J81K8xbvXraCeN8e

ItemsFlowQuery now replays the inventory once, records the change each event made at the storage location, and attributes it to when the record was first created. Edits and deletions are therefore reported "as of now" on the record's original date, and start/end quantities are computed from the same changes, so start + in - out + adjustment always equals end and a window ending today matches the Inventory tab. Previously the in/out columns counted the latest version of a record at the time of its last edit while start/end came from separate replays, so any edited or deleted record made the columns disagree (38% of rows on a production snapshot). Also: use the shared DateRangeHelper range (handles bad input and matches what the picker displays), return rows and totals separately, and remove the unused line_item_row partial. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CSmvTcnvvTEKz5CmX9kNm8

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CSmvTcnvvTEKz5CmX9kNm8

dorner · 2026-10-01T15:16:58Z

+  ADJUSTMENT_TYPES = %w[AdjustmentEvent AuditEvent].freeze
+  # These events stand alone rather than being a version of their record, so
+  # their changes stay at their own event time.
+  STANDALONE_TYPES = %w[KitAllocateEvent KitDeallocateEvent AuditEvent UpdateExistingEvent].freeze


Wonder if we should have a method on the event indicating editable? or something similar. I think this crops up in at least one more place.

ah, and then like if it is before a snapshot or depending on the type we'd get false?

dorner · 2026-10-01T15:18:22Z

+      end_qty = end_quantities[item_id]
+      next if flow.values.all?(&:zero?) && start_qty.zero? && end_qty.zero?
+
+      {


Can we make this a Struct / Data class?

dorner · 2026-10-01T15:18:39Z

+# end quantities are then the current quantity minus the changes attributed
+# after those times, so start + change always equals end.
+class ItemsFlowQuery
+  Result = Struct.new(:rows, :totals)


Data instead of Struct?

/me refreshes data-vs-struct

sure!

dorner · 2026-10-01T15:21:42Z

+
+  # Replays the inventory, recording the change each event made to each item
+  # at this location.
+  # @return [Array(Hash<Integer, Integer>, Array<Change>)] current quantities and changes


I feel like the logic is complex enough to extract some actual data classes here... I'd probably consider almost any data structure that uses bare integers/strings as a code smell for this use case. E.g. lines 93-94 seem like they're just different ways of getting data into / out of a collection, maybe we introduce one that automatically merges the data when you insert it? Not a desperate need, but anything we can do to make the flow simpler / easier to follow would help here.

dorner · 2026-10-01T15:24:02Z

+  # Nets each record's changes (so an edited donation counts once, at its
+  # latest quantity) and sorts them into in, out and adjustment.
+  # @return [Hash<Integer, Hash>] item_id => {in:, out:, adjustment:}
+  def window_flows(changes)


Usually window implies that it's moving or that there are multiple windows... maybe rename this to flows_in_range?

yeah I like that, window was annoying me a little too. "view" almost comes to mind but that is taken :)

dorner · 2026-10-01T15:24:18Z

+      elsif net.positive?
+        flow[:in] += net
+      else
+        flow[:out] -= net


more data classes here 😂

dorner · 2026-10-01T15:25:20Z

+  # Current quantities with the changes attributed to the selected times undone.
+  # @yieldparam time [Time] when a change is attributed to
+  # @return [Hash<Integer, Integer>] item_id => quantity
+  def quantities_without(current, changes)


Is the reason this isn't a filter because we don't know how to associate which event to which time yet (it depends on when it was edited)?

Seems like ideally we do that calculation first and then we can filter this out proactively instead of after the fact.

correct -- the idea is to show the post-edited data from a given time-range. I'll see how I can simplify this.

Olexii Kasianenko and others added 17 commits September 22, 2025 18:14

Add Inventory Flow tab with quantity tracking and date filtering

53eaa68

Refactor inventory flow to use ItemsFlowQuery for improved data handl…

bb56879

…ing and display

Add date range filtering to ItemsFlowQuery and update Inventory Flow tab

6e923a6

Remove unused inventory storage methods and related tests for cleaner…

2439b14

… code

Fix date filtering in ItemsFlowQuery to use item creation timestamps

5233723

Fix date range filtering in storage location system spec to ensure ac…

0eda035

…curate item creation timestamps

Refactor item creation in flow query specs for improved clarity and a…

216dc2c

…ccuracy

Refactor inventory flow query to utilize event-based data for improve…

0d50f31

…d accuracy and clarity

Enhance inventory flow query to include AuditEvent handling and updat…

bfd1db6

…e related specs for accurate quantity calculations

Merge branch 'main' into 5249-inventory-flow-tab

233dd10

Merge branch 'main' into 5249-inventory-flow-tab

2c1a417

Merge remote-tracking branch 'origin/main' into 5249-inventory-flow-tab

146129b

Merge remote-tracking branch 'origin/main' into inventory-flow-backdated

f3c8552

Drop redundant keyword_init on Structs

a740f90

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CSmvTcnvvTEKz5CmX9kNm8

awwaiid requested a review from dorner September 27, 2026 19:16

dorner requested changes Oct 1, 2026

View reviewed changes

Navigation Menu

Uh oh!

Inventory Flow tab: backdate edits so every column adds up (builds on #5313) - #5730

Inventory Flow tab: backdate edits so every column adds up (builds on #5313)#5730
awwaiid wants to merge 17 commits into
mainfrom
inventory-flow-backdated

awwaiid commented Sep 27, 2026 •

edited

Loading

Uh oh!

dorner Oct 1, 2026

Uh oh!

awwaiid Oct 1, 2026

Uh oh!

dorner Oct 1, 2026

Uh oh!

dorner Oct 1, 2026

Uh oh!

awwaiid Oct 1, 2026

Uh oh!

dorner Oct 1, 2026

Uh oh!

dorner Oct 1, 2026

Uh oh!

awwaiid Oct 1, 2026

Uh oh!

dorner Oct 1, 2026

Uh oh!

dorner Oct 1, 2026

Uh oh!

awwaiid Oct 1, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

awwaiid commented Sep 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Verification on a production snapshot

Notes for reviewers

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

awwaiid commented Sep 27, 2026 •

edited

Loading