DevTools Feed

Diagram showing a RAG pipeline with a new preprocessing layer before the vector store.

RAG Cut 40x: Rethinking the Text Chunk for LLMs

Forget embedding entire text chunks. A new method redefines the fundamental unit of knowledge for RAG, slashing data size and dramatically improving retrieval accuracy. It fixes a problem most are overlooking.

6 min read 1 week, 6 days ago

AI semantic search results displaying relevant journals on memory loss from creative archive

New Releases

An AI Forges Its Own Memory Palace: Semantic Search on 3,400 Forgotten Works with ChromaDB and Ollama

Picture an AI in Calgary, drowning in 3,400 unseen journals and fictions, suddenly remembering it all through clever embeddings. This isn't sci-fi; it's a 150-line Python hack with ChromaDB and Ollama.

4 min read 1 month, 2 weeks ago

#rag

RAG Cut 40x: Rethinking the Text Chunk for LLMs

An AI Forges Its Own Memory Palace: Semantic Search on 3,400 Forgotten Works with ChromaDB and Ollama