SPLIT LONG TEXT WITHOUT LOSING ITS STRUCTURE

Text Chunking Workbench

Prepare bounded text chunks for review, translation or retrieval workflows. Prefer paragraph, sentence or word boundaries, add controlled overlap, and verify exact reconstruction.

1. Source & chunk rules

Source textOpen UTF-8 text

Boundary preferenceMaximum code points per chunkOverlap code pointsSentence language tag

Restore a saved chunk manifest

250,000 input UTF-16 units · 100–20,000 code points per chunk · 2,000 chunks maximum · overlap must be smaller than chunk size · 60-second job limit. Manifest imports up to 5 MB.

2. Preview & export

Try the example or open your text.

Chunk	UTF-16 range (end excluded)	Code points	Text

Selected chunk

These are character-based chunks, not model tokens. Code points can split combining sequences or emoji families; this is not grapheme-based segmentation. Preferred boundaries may fall back to a hard split for long paragraphs or sentences.

How to chunk and restore text

Paste or import text, choose a preferred boundary and set a maximum size and overlap.
Run and review the chunk ranges. A reconstruction check ensures every source UTF-16 unit is retained in order.
Download individual .txt chunks in a ZIP, plus a full manifest containing offsets and chunk text.
Restore the manifest to check overlapping text consistency and its optional UTF-8 source hash, then download the reconstructed text.

What stays unchanged

The chunking algorithm never trims whitespace or changes line endings. Pasted text may already have been normalized by the browser textarea; UTF-8 imports use the shared text reader, which removes an initial BOM. Hashes refer to the text as imported into the workspace, not necessarily the original file bytes.

Overlaps intentionally duplicate characters. Simply concatenating all chunks will duplicate those regions; use manifest offsets for reconstruction. Sentence boundaries depend on browser Unicode data and language. The full manifest contains the entire text; treat it as sensitive as the source.

Using Text Chunking Workbench

Split long text into bounded, overlapping sections and preserve enough information to reconstruct it exactly.

Load text and choose boundary preference, size and overlap.
Run and inspect chunk offsets and previews; exact reconstruction is checked.
Export text chunks and manifest, or restore a saved manifest for verification.

Example

The sample includes paragraph breaks, accents and emoji, with 300-code-point chunks and 40-code-point overlap.

Questions & answers

Are sizes measured in tokens?

No. Sizes and overlap count Unicode code points. Model-specific token counts and grapheme-cluster boundaries are not provided.

Can chunks reconstruct the original?

The manifest retains UTF-16 offsets and exact chunk text. Reconstruction rejects gaps and conflicting overlaps; the workspace also checks the reconstructed text against its source. UTF-8 imports strip an initial BOM.

What are the limits?

250,000 source UTF-16 units, 100–20,000 code points per chunk, fewer overlap points than chunk size and at most 2,000 chunks. Jobs stop after 60 seconds.

What happens to long paragraphs or sentences?

The preferred boundary is used when it fits and allows progress. Otherwise the algorithm falls back to a hard code-point boundary. Sentence mode requires Intl.Segmenter support.

Does the manifest contain my text?

Yes. It includes full chunk text and therefore contains the complete source with overlap. Treat the manifest and ZIP as sensitive as your original text; no automatic storage is used.