Skip to content

Install

Core install

bash
npm i zod playwright llm-scraper

LLM Scraper requires three peer packages:

  • zod - schema definition for structured output
  • playwright - headless browser for page loading
  • llm-scraper - the library itself

Install a Playwright browser

After installing, download the Chromium browser binary:

bash
npx playwright install chromium

Add your LLM provider SDK

Install one provider package depending on which model you want to use:

ProviderInstall command
OpenAI (GPT-4o)npm i @ai-sdk/openai
Anthropic (Claude)npm i @ai-sdk/anthropic
Google Gemininpm i @ai-sdk/google
Ollama (local)npm i ollama-ai-provider-v2

Set the corresponding API key as an environment variable before running:

bash
# OpenAI
export OPENAI_API_KEY=sk-...

# Anthropic
export ANTHROPIC_API_KEY=sk-ant-...

# Google
export GOOGLE_GENERATIVE_AI_API_KEY=...

TypeScript setup

LLM Scraper is written in TypeScript. Add a tsconfig.json if you do not have one:

json
{
  "compilerOptions": {
    "target": "ES2022",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true
  }
}

Run with ts-node, tsx, or compile first with tsc.