How to chunk PDFs, HTML, Markdown, and code differently

The default advice — "split every 512 tokens" — treats all content as an undifferentiated stream of text. But a PDF, an HTML page, a Markdown doc, and a source file have completely different structures, and each has natural boundaries that a blind token-counter runs straight through. Content-type-aware chunking respects those boundaries, which is the difference between chunks that are coherent retrievable units and chunks that are arbitrary fragments. Here's how each major format wants to be chunked, and why.

The principle: chunk along the document's own structure

Every well-structured document already tells you where its natural seams are — headings, sections, paragraphs, tags, function definitions. The core rule of good chunking is: split on the boundaries the format gives you, not on a token count that ignores them. A chunk that aligns with a section is a coherent unit whose embedding represents one idea. A chunk that ends mid-function or mid-table is a fragment whose embedding represents confusion. The token budget should constrain chunk size; the document's structure should decide chunk boundaries.

With that principle, each format becomes a specific application.

Markdown: the easy case

Markdown hands you structure explicitly. Headings (#, ##, ###) define a hierarchy; lists and code blocks are self-contained units.

• Split on heading boundaries, so each chunk is a coherent section.
• Preserve the heading path as metadata — a chunk from "Setup > Configuration > Environment Variables" carries that context, which both improves retrieval and lets you tell the model where the passage came from.
• Keep code blocks and tables intact — never split in the middle of one.

Markdown is the friendliest format precisely because its structure is machine-readable. Respect the headings and you're most of the way there.

HTML: structure buried in noise

HTML has rich structure (headings, sections, lists, tables via tags) but wrapped in a lot of non-content — navigation, scripts, styling, boilerplate.

• Strip the noise first. Remove nav, footers, scripts, and chrome so you're chunking actual content, not menu links. Failing to do this pollutes every chunk with boilerplate.
• Then chunk on semantic tags — <h1>/<h2> sections, <article>, <section> — the same heading-aware approach as Markdown, once the page is cleaned.
• Preserve tables and lists as units. HTML tables especially carry meaning in their structure; flattening them carelessly destroys it.

The extra work in HTML is the cleanup. Chunk raw HTML and half your vectors will be embeddings of navigation menus.

PDF: the hard case

PDFs are the least cooperative because they encode layout, not structure. A PDF knows where characters sit on a page, not that a block is a heading or that a region is a table. This makes PDFs the format where naive chunking fails most often.

• Extract with layout awareness. Recover reading order (multi-column layouts scramble easily), detect headings by font/position, and identify tables and figures rather than letting their text bleed into surrounding prose.
• Handle tables specially — they're a notorious PDF failure mode and deserve dedicated extraction, not token-splitting.
• Watch for scanned PDFs, which need OCR before any of this — there's no text to extract until you recover it from the image.

PDF chunking quality is dominated by extraction quality. Get a clean structural representation out of the PDF first, and only then chunk it like a structured document. Skip that and you're chunking scrambled, table-corrupted text.

Code: split on program structure

Source code has the clearest structure of all — and the worst outcomes when you ignore it. Splitting code every N tokens routinely cuts a function in half, separating a signature from its body or a class from its methods.

• Split on syntactic units — functions, classes, methods — so each chunk is a complete, meaningful piece of code. A chunk should be something a developer would recognize as a unit.
• Keep signature with body. A function's name and parameters are what a query will match; separating them from the implementation breaks retrieval.
• Preserve context — imports, class context, surrounding scope — as metadata or lightweight inclusion, so a retrieved method isn't floating without the type it belongs to.

Syntax-aware splitting (parsing the code structure) beats token-splitting decisively for code, because code's boundaries are formally defined and violating them produces nonsense chunks.

The practical takeaway

The unifying idea is simple: detect the content type and chunk according to its structure, rather than running one token-counter over everything. A system ingesting mixed content — some Markdown docs, some PDFs, some HTML pages, maybe some code — should route each to a strategy that respects its format, not flatten them all into the same blind splitter. The token budget is a constraint on size; the format decides where the cuts go. Get that right and your chunks are coherent units the rest of the pipeline can actually work with. Get it wrong — one strategy for all formats — and you've quietly capped retrieval quality on most of your corpus before embedding even starts.