Cool Tech5 min read
Open-Source Model Releases Worth Actually Trying
License terms, context length, and tool-calling reliability matter more than leaderboard rank. A filter for deciding which open weights deserve a GPU-hour.
Read full entry← Back to Unimatrix
Engineering Notes
Technical write-ups on Model Context Protocol, local embeddings, vector databases, and client-side memory security.
Cool TechLLM TechMemory ArchitecturePrompt EngineeringSecurity & DevOpsStartups & Grants
Security & DevOps6 min read
OAuth and 2FA for AI Apps: Securing Access Without Killing UX
An MCP client is a non-browser, long-lived consumer. That breaks the assumptions behind short-lived OAuth tokens, and the fix is scoped keys with rotation.
Read full entryMemory Architecture6 min read
Edge Cases in Agentic Memory: Conflicting Facts and Stale Context
Two true statements about the same entity at different times are not a conflict, they are a version history. Resolution strategies and when each one is wrong.
Read full entryStartups & Grants6 min read
Grant Writing for AI Startups: What SBIR Reviewers Actually Look For
Technical merit means a falsifiable risk with a named mitigation. Vague market claims get scored down. What a Phase I reviewer is scanning for on page one.
Read full entryMemory Architecture5 min read
How Embedding Drift Quietly Breaks Retrieval Over Time
Change the embedding model and every vector written before the change becomes noise. There is no error, no exception, just a slow decline in recall.
Read full entryPrompt Engineering5 min read
Few-Shot Prompting Still Beats Fine-Tuning for Most Teams
Fine-tuning buys token savings and format compliance. It costs you a retraining cycle every time the spec changes, which for most teams is weekly.
Read full entrySecurity & DevOps6 min read
Zero-Knowledge Techniques for Private AI Memory
Client-side encryption kills server-side semantic search. Encrypted vector indexes, PIR, and the honest accounting of what each one costs in latency.
Read full entryLLM Tech5 min read
Reasoning Models vs. Chat Models: Picking the Right Tool
Reasoning models spend tokens before answering, which trades latency and cost for accuracy on search-shaped problems. A decision rule based on verifiability.
Read full entryMemory Architecture5 min read
Session Persistence: Why Your AI Forgets You Between Tabs
Inference is stateless by design. Every "conversation" is the full transcript replayed, and the tab boundary is just where the client stops replaying it.
Read full entryPrompt Engineering6 min read
Designing System Prompts That Survive Production
A system prompt is code with no type checker and no test suite unless you build one. Version it, pin it per release, and diff behavior before you ship a wording change.
Read full entryCool Tech6 min read
Quantization 101: How Smaller Models Keep Getting Smarter
Weights are not uniformly important. Per-group scales, outlier channels held in higher precision, and why 4-bit lands closer to FP16 than the bit count suggests.
Read full entryMemory Architecture5 min read
The Hidden Cost of Re-Explaining Context to Every AI Tool
Re-pasting project context into a fresh session is a measurable tax: tokens, minutes, and the details you forget to include. Here is how to actually price it.
Read full entryLLM Tech6 min read
Tool Use in LLMs: From Function Calling to Autonomous Agents
Function calling is constrained decoding against a JSON Schema. Everything agentic is a loop around that primitive, and the loop is where reliability goes to die.
Read full entryCool Tech5 min read
On-Device AI Models Are Getting Scary Good — Here's Why It Matters
A 4-bit 8B model needs roughly 5 GB of RAM and runs at readable speed on a laptop. Memory bandwidth, not FLOPs, is the constraint that decides what ships locally.
Read full entryLLM Tech6 min read
Benchmarking LLM Memory Systems: Metrics That Actually Matter
Recall@10 hides the failure your users feel. Measure answer correctness under a fixed token budget, plus contradiction rate over multi-session traces.
Read full entryCool Tech6 min read
What Merkle Proofs Have to Do with Verifiable AI Memory
A log-sized inclusion proof lets a user verify their memory was never silently edited, without the provider handing over the whole database. Borrowed straight from RFC 6962.
Read full entryMemory Architecture6 min read
Graph-Based Memory: When Knowledge Graphs Beat Vector Stores
Multi-hop questions are where embeddings fall apart, because no single chunk contains the answer. The queries that justify an edge table, and the ones that do not.
Read full entryPrompt Engineering5 min read
Prompt Chaining vs. Single Mega-Prompts: When Each Wins
Chaining multiplies per-step reliability: five 95% steps give you 77% end to end. When that math argues for one big prompt instead, and when it argues against.
Read full entrySecurity & DevOps6 min read
Multi-Tenant Architecture for AI SaaS: Lessons from the Field
Postgres row-level security beats a WHERE clause you have to remember to write. What that costs on a pgvector query plan, and how to keep tenant leakage impossible by construction.
Read full entrySecurity & DevOps6 min read
Encrypting AI Memory at Rest: A Practical Walkthrough
AES-256-GCM with a per-ciphertext scrypt-derived key, and the awkward part nobody mentions: encrypted rows break the vector index you were counting on.
Read full entryMemory Architecture6 min read
Building a Librarian Agent: Automated Curation for AI Memory
An asynchronous pass that dedupes, merges, and demotes memories keeps a store usable past 10,000 entries. The write policy, the merge rules, and the failure modes.
Read full entryLLM Tech5 min read
The Real Difference Between Context Windows and Memory
A context window is a buffer you refill every request. Memory is state that survives the request. Conflating them is why long-context models still feel amnesiac.
Read full entryMemory Architecture6 min read
Hybrid Retrieval Explained: Combining Keyword, Vector, and Graph Search
BM25 catches the identifiers embeddings smear. Reciprocal Rank Fusion merges the two without tuning weights. Where a graph hop earns its cost.
Read full entryMemory Architecture5 min read
Recency Decay: Teaching AI Memory to Forget on Purpose
An exponential half-life applied to retrieval scores fixes more relevance bugs than a better embedding model. How to tune the decay constant without erasing durable facts.
Read full entryLLM Tech5 min read
Inside Model Context Protocol: What MCP Actually Solves
MCP is a JSON-RPC 2.0 transport with a capability handshake. What that buys you, what it does not, and why the N×M integration problem was the real target.
Read full entryMemory Architecture5 min read
Why Vector Search Alone Can't Give LLMs a Memory
Cosine similarity retrieves what looks like your query, not what you need to know. A breakdown of where pure vector recall fails and what has to sit on top of it.
Read full entryLooking for developer resources?
Browse our catalog of 50+ tested AI prompt profiles covering DevOps, data modeling, agent behaviors, and API wrappers. Completely free, no registration required. Browse prompt libraries →