Recursive Text Chunker (RAG)

Split long documents into overlapping chunks for RAG (Retrieval-Augmented Generation). Implements LangChain-style recursive character splitting: try paragraph break → sentence → word → character, preferring natural boundaries. Configurable chunk size, overlap, and separator list.

How to use the Recursive Text Chunker (RAG)

Paste a document. Pick a chunk size (500-1500 chars is typical for RAG; smaller = more precise retrieval but more chunks to manage) and overlap (50-100 chars helps preserve context across boundaries). The recursive splitter prefers paragraph breaks over sentence breaks over word breaks — chunks end naturally where possible.

Chunk size strategy

Optimal chunk size depends on your retrieval pattern. For embedding-based retrieval where you'll feed top-3 chunks to a small context window: 300-500 chars keeps each chunk semantically focused. For passing top-1 chunk to a long-context model: 1500-2500 chars gives more context per match.

Overlap is insurance against information getting cut at a chunk boundary. 10-20% of chunk size is typical. Too much overlap inflates index size and increases the chance of retrieving redundant chunks.

Frequently asked questions

What chunk size and overlap should I use?
A common starting point is 500 to 1000 tokens with 10 to 20% overlap. Smaller chunks improve retrieval precision; larger chunks keep more context together. Tune for your data.

Why use overlap between chunks?
Overlap keeps sentences that straddle a boundary intact in at least one chunk, so retrieval does not lose meaning split across the cut.

How does recursive splitting work?
It tries to break on the largest natural separator first (paragraph, then sentence, then word) before falling back to a hard character cut, which keeps chunks readable.
Embed this tool on your site

Free to embed, no attribution required (but appreciated). Paste this where you want the tool to appear: