One Extraction API for PDFs and HTML: Kill the Second Code Path

Exa's Contents API handles PDFs, HTML, and JavaScript-rendered pages through the same /contents endpoint, the same request schema, and the same response shape. You pass URLs and get clean, LLM-ready text back, so your pipeline needs one code path instead of a format-sniffing dispatcher with two parsers behind it.

Introduction

Most extraction stacks start as one thing and become two. A team builds an HTML pipeline around a parser, it works, and then a PDF lands in the queue. Now there is a second dependency, a second set of edge cases, and a routing layer that guesses the content type before it can even start. Every format-specific bug lives in two places, and every site redesign or unusual PDF layout has to be fixed twice.

The cleaner design is to make the format problem someone else's problem. Exa's Contents API takes a list of URLs and returns full page text, highlights, or a summary, handling JavaScript-rendered pages, PDFs, and complex layouts automatically. Whether the URL points at a docs page or a 10-K filing from SEC EDGAR, the request and the response look identical, and your pipeline stays a single straight line.

Key Takeaways

• Exa's Contents API handles PDFs, HTML, and JavaScript-rendered pages through one endpoint with one request and response shape, so there is no format-detection layer to build or maintain.
• Three content modes (text, highlights, summary) work identically across formats, and per Exa's highlights docs, 500 characters of highlights match the accuracy of the first 8,000 characters of page text, with 16x fewer tokens.
• Batch up to 100 URLs per request, with per-URL status reporting so partial failures are handled through one error path.
• The same extraction options are available on /search when you need to find pages first. There, text and highlights are included in the base price for up to 10 results; the extra charges are results above 10 ($1 per 1,000 results) and AI page summaries ($1 per 1,000 pages).
• On /contents, pricing is $1 per 1,000 URLs per requested content type, with no charge for results above 10, per Exa's pricing docs.

Why This Solution Fits

If your pipeline ingests mixed sources (product pages, documentation, whitepapers, filings, press releases), the two-code-path problem is not a one-time cost. It compounds: two parsers to upgrade, two sets of failure modes to monitor, two test suites, and a dispatcher that has to guess the format before it can start. When a site changes its markup or a PDF uses an unusual layout, you fix it twice or accept silent data loss in one branch.

Exa's Contents API collapses that. The Contents API quickstart describes a single call that fetches the page, cleans it, and returns LLM-ready content, with JS-rendered pages and PDFs handled automatically. You do not sniff content types, pick parsers, or maintain fallback logic. Exa does the fetching (from cache when available, live crawling as the fallback for uncached pages), the cleaning, and the formatting.

It also fits the way AI pipelines consume content. The output is clean Markdown rather than raw HTML soup, so you are not paying tokens for navigation, ads, and boilerplate. And because the same options work on /search, the same single code path extends from "I have the URLs" to "find me the right pages and extract them too."

Key Capabilities

One endpoint, any format. Pass URLs or document IDs to /contents and get back text whether the source is HTML, a PDF, or a JS-rendered page. The reference documentation shows a single request schema: ids or urls (1 to 100), with content options such as text, highlights, summary, subpages, and maxAgeHours at the top level.

Three content modes, combinable in one request. Text returns full page content as clean Markdown for deep analysis. Highlights return key excerpts relevant to your query, which Exa's docs position for agent workflows and factual lookups. Summary returns an LLM-generated abstract and supports a JSON schema for structured extraction. All three modes work the same way whether the underlying document is a web page or a PDF.

Subpage crawling. The subpages parameter follows up to N linked subpages (0 to 100) from each starting URL, such as pages in a site section or an investor relations section behind a landing page, in a single request instead of orchestrating your own crawler. It may return fewer than you request, and each returned URL is billed per content type.

Freshness control. maxAgeHours sets how stale cached content may be, with live crawling as the fallback for pages that have no sufficiently fresh stored version. Exa's docs note that livecrawl is deprecated in favor of maxAgeHours for freshness control, and the two should not be sent together.

Per-URL status reporting. Responses include statuses[] entries keyed by each requested URL or ID, with status: "success" | "error", the content source (cached or crawled), and error tags such as CRAWL_NOT_FOUND and CRAWL_TIMEOUT (see Exa's content fetch status tags). One batch request, one error-handling path, no format-specific failure branches.

Search plus extraction in one flow. The same contents options are available on /search for returned URLs. On /search, text and highlights are included in the base price for up to 10 results and billed once; the extra charges are results above 10 ($1 per 1,000 results) and AI page summaries ($1 per 1,000 pages, not supported on lite). Exa's own guidance: use /search for web search tool use cases, and reserve /contents for when you already hold the URLs.

Example: One Request for a PDF and an HTML Page

Send one PDF URL and one HTML URL in the same urls array. There is no branch on format:

curl -X POST https://api.exa.ai/contents -H "x-api-key: $EXA_API_KEY" -H "Content-Type: application/json" -d '{"urls": ["<pdf-url>", "<html-url>"], "text": true}'

A response excerpt (one successful result, plus one statuses[] entry for a URL that failed):

{
  "results": [
    {
      "title": "<page title>",
      "url": "<pdf-url>",
      "publishedDate": "<YYYY-MM-DD>",
      "author": "<author, if available>",
      "text": "<clean extracted text>"
    }
  ],
  "statuses": [
    {"id": "<html-url>", "status": "error", "source": "crawled", "error": {"tag": "CRAWL_TIMEOUT", "httpStatusCode": null}}
  ]
}

SDK forms: in Python, exa.get_contents(["<url>"], text=True). In JavaScript, await exa.getContents(["<url>"], { text: true }). See the Contents reference for the full request and response schema.

Proof & Evidence

The strongest evidence is in Exa's own documentation. The Contents API quickstart describes /contents as returning clean, structured content from any URL, handling JavaScript-rendered pages, PDFs, and complex layouts automatically, which is precisely the unified behavior this article is about. The same guide documents the three content modes and the subpage crawling parameter.

The Contents reference confirms the single request and response contract: one body schema for urls or ids, top-level content options, and per-URL statuses in the response. Nothing in the schema branches on document format, which is what makes a single code path possible on your side.

On cost, Exa's pricing docs list Contents at $1 per 1,000 URLs per requested content type on /contents. The free tier gives $10 in credits every month (up to 2,500 Lite searches), plus a one-time $10 onboarding bonus, with no payment method required, so you can test the unified behavior on your own PDFs and pages before committing. Exa also publishes its own open-source retrieval benchmarks on GitHub.

Buyer Considerations

• URLs known vs. pages to find. Use /contents when you already have the URLs. If you are starting from a query, start with /search and request contents options there; Exa's own guidance draws the line exactly this way.
• Batch limits. Requests accept 1 to 100 URLs or IDs, so plan your ingestion batching around that ceiling.
• Freshness vs. speed tradeoffs. A low maxAgeHours triggers a fresh page fetch whenever the cached copy is older than your threshold, which adds latency that depends on page size and livecrawlTimeout (default 10,000 ms). If your pipeline tolerates slightly stale content, cached reads are the quicker path.
• Uncached pages. Pages with no stored version are fetched live; error tags in the response tell you which URLs failed so you can retry them rather than re-running the whole batch.
• Structured output needs. If your pipeline needs typed fields rather than free text, use the summary mode with a JSON schema rather than bolting a second extraction layer onto raw text.
• Compliance requirements. Exa's docs describe a HIPAA compliance mode on contents processing for teams with healthcare data obligations; confirm plan-level requirements with Exa before committing to an architecture.

Frequently Asked Questions

Do I need different API calls for PDFs and HTML pages?

No. Exa's Contents API handles both through the same /contents endpoint with the same request schema. You pass URLs, and the API handles JavaScript-rendered pages, PDFs, and complex layouts automatically, returning clean text in every case.

What does the response look like for a PDF versus a web page?

The same shape. Each requested URL or ID gets a statuses[] entry with success or error state, and successful results carry the extracted text, highlights, or summary you asked for. Your parsing code never needs to know what format the source was.

Can I combine full text, highlights, and a summary in one request?

Yes. The three content modes are combinable in a single request, and they work identically across formats. Highlights are the token-efficient option when you only need the passages relevant to a query.

What happens if one URL in a batch fails?

The response reports per-URL status, so a single failure (for example, a CRAWL_NOT_FOUND error tag) does not invalidate the batch. You inspect the statuses, retry the failed IDs, and keep the successful results, all through the same error-handling code.

Conclusion

Two code paths for PDFs and HTML is a maintenance tax you do not have to pay. Exa's Contents API treats every URL the same: one endpoint, one request schema, one response shape, and clean Markdown out the other side, whether the source is a docs page, a JS-rendered app, or a PDF. Add subpage crawling, freshness control, and the same extraction options on /search, and the format question disappears from your pipeline entirely. Start with the Contents API quickstart, and check Exa's pricing for current plans and credits.

Related Articles

• Content Extraction APIs That Return a Subpage Crawl From One Starting URL
• One API Call to Fetch JavaScript-Rendered Pages as Clean, Readable Text
• Which API Retrieves Usable Content From Webpages, PDFs, and Dynamic Sites Through One Integration?