Third-party toolUpdated 2026.11.14

Trafilatura: Extract HTML Main Text and Handle Empty Results

Test Trafilatura 2.3.1 on synthetic HTML. Verify two records, navigation and footer exclusion, and None for empty HTML. Live downloading is not covered.

Aaron recommends Open official GitHub

Trafilatura extracts main text from HTML, which can help prepare material for an AI model or search index. In this 2.3.1 test, it retained two synthetic dispatch records and omitted navigation and footer controls. Empty HTML returned None. The test did not download live sites or execute browser JavaScript.

For a broader web extraction workflow, compare Crawl4AI's extraction settings. If you also need clicks or page-state checks, see Playwright MCP's operations.

Use an HTML fixture with noise controls

Build an HTML document with title, nav, main/article, and footer elements. Include full paragraphs for NOTE-31 at 08:30 and NOTE-32 at 16:45. Put NOISE_NAVIGATION and NOISE_FOOTER in the surrounding controls. Meaningful paragraphs make the extraction check more useful than a one-word document.

Extract from HTML before adding downloads

Install the pinned version under Python 3.12 and save the HTML as fixture.html. The example passes a string directly to extract with Markdown output and include_tables=True. The fixture has no table: enabling the option does not constitute a table-preservation test.

These commands use a POSIX shell. The test ran in Linux; Windows shell behavior was not tested.

python -m pip install "trafilatura==2.3.1"
python - <<'PY'
from pathlib import Path
from trafilatura import extract
text = extract(Path("fixture.html").read_text(), output_format="markdown", include_tables=True)
if text is None or not text.strip():
    raise ValueError("No main text extracted")
print(text)
PY

Check records and exclusions together

The output retained both NOTE identifiers with their times and contained neither NOISE marker. An empty html/body returned None. Handle None before measuring text length or writing a string. The existence of Markdown alone does not prove full coverage of every record on a page.

Separate extraction from browser rendering

The downloader, command-line crawling, dynamic pages, and tables are described from official v2.3.1 documentation and were not locally tested. This test starts with HTML already in hand. For browser rendering and selector checks, see the existing Crawl4AI guide. Live-site layouts, permissions, and availability require separate checks; this fixture yields no general success-rate estimate.

Official sources and test scope

Codex ran this synthetic-data test in a credential-free, non-root, network-disabled container. It is not a claim that the site owner tested it. Package version: 2.3.1; documentation snapshot: 8f054a17a68d.

Subscribe to AI tool picks and practical advice

Get worthwhile tool picks, test findings and important updates in your inbox.

Get a monthly-updated list of AI tools and Skills right away. We email when there are worthwhile tool picks, test findings or important updates. Unsubscribe anytime. Privacy

Almost done: check your inboxClick the button in the confirmation email and you will get the download link for the tool list. If you do not see it, check your spam or Promotions folder.

Or follow us on: