Memory Recall: Ranking and Relevance

PROJECT CONTEXT:
My agent's memory recall returns the wrong things: exact names miss while vague semantic matches flood in. Retrieval research in agent memory systems converges on hybrid search - lexical plus vector, fused - with explainability for debugging.

GOAL:
Build hybrid recall: BM25-style lexical search plus embedding similarity, fused rankings, recency/confidence boosts, and explainable results.

REQUIREMENTS:
- Lexical search (exact and keyword matching) over memory entries
- Semantic search via embeddings over the same entries
- Rank fusion (reciprocal rank fusion or equivalent) combining both
- Boost signals: recency, confidence, and access frequency
- Explainability: each result shows why it surfaced (matched terms, similarity, boosts)

CONSTRAINTS:
- Exact identifiers (file names, error strings) must be findable - lexical is not optional
- Recall returns a bounded top-K with scores, never the whole store

DEFINITION OF DONE:
- An exact-name query finds its entry even when embeddings rank it poorly
- A paraphrased query finds semantically related entries lexical search misses
- Fused results beat either method alone on a small labeled test set
- Every returned memory carries a human-readable retrieval explanation

COMMON FAILURES TO AVOID:
- Vector-only recall that cannot find an exact error string
- Lexical-only recall that misses every paraphrase
- Unbounded result lists stuffing recall noise into context
- Opaque retrieval nobody can debug when the wrong memory surfaces