Targeted match query for the matcher create-path (avoid full-collection scans) #44

New issue

Closed

opened 2026-07-04 12:11:37 +00:00 by momsse · 0 comments

momsse commented

2026-07-04 12:11:37 +00:00

Owner

Migré depuis viziertronic/octant#101 — ouvert le 2026-06-26 par @momsse.

Follow-up from the #100 review (epic #42).

Problem

The matcher create-path scans the full counterpart collection on every event:

TransactionMatchingSaga.matchTransaction → documentsQuery.findAllDocuments() + in-memory amount/date filter
DocumentMatchingSaga.matchDocument → bankTransactionsQuery.findAllBankTransactions() + in-memory amount/date filter

This is O(N) per BankTransactionCreated / DocumentAnalyzed event. It was a deliberate simplification in #38 (#100) — DocumentsQueryShape and BankTransactionsQueryShape only expose findAll*, and exact-match needs amount + a date window, which isn't a single equality lookup. The drop-path was already moved to targeted queries (findByBankTransactionId / findByDocumentId) in that PR; only the create-path remains a scan.

Proposed work

Add a filtered query to both shapes, e.g. findByAmountWithinDateWindow({ amount, from, to }), with in-memory + Postgres adapters.
Add the companion index for the heuristic: read.documents (amount_minor, currency, issued_at) and read.bank_transactions (amount_minor, currency, value_date).
Rewire both sagas' create-path off findAll*.

Not urgent

Fine at current data volumes; needed before the matcher runs against production-scale document/transaction sets.

> _Migré depuis [viziertronic/octant#101](https://github.com/viziertronic/octant/issues/101) — ouvert le 2026-06-26 par @momsse._ Follow-up from the #100 review (epic #42). ## Problem The matcher create-path scans the full counterpart collection on every event: - `TransactionMatchingSaga.matchTransaction` → `documentsQuery.findAllDocuments()` + in-memory amount/date filter - `DocumentMatchingSaga.matchDocument` → `bankTransactionsQuery.findAllBankTransactions()` + in-memory amount/date filter This is O(N) per `BankTransactionCreated` / `DocumentAnalyzed` event. It was a deliberate simplification in #38 (#100) — `DocumentsQueryShape` and `BankTransactionsQueryShape` only expose `findAll*`, and exact-match needs amount + a date *window*, which isn't a single equality lookup. The drop-path was already moved to targeted queries (`findByBankTransactionId` / `findByDocumentId`) in that PR; only the create-path remains a scan. ## Proposed work - Add a filtered query to both shapes, e.g. `findByAmountWithinDateWindow({ amount, from, to })`, with in-memory + Postgres adapters. - Add the companion index for the heuristic: `read.documents (amount_minor, currency, issued_at)` and `read.bank_transactions (amount_minor, currency, value_date)`. - Rewire both sagas' create-path off `findAll*`. ## Not urgent Fine at current data volumes; needed before the matcher runs against production-scale document/transaction sets.

momsse closed this issue

2026-07-04 12:11:37 +00:00

momsse referenced this issue

2026-07-04 12:18:56 +00:00

bug: bank-account sync re-emits a detected event for every transaction every run (no dedup/cursor → unbounded stream) #54