LLM Scraper SOP

Appearance

Test ​

Status: PASS (Node.js package) ​

Classification: PASS for the npm package. An LLM API key (OpenAI or compatible) and Playwright are required for actual use, but the package installs without an account.

Verified install ​

The llm-scraper package installs successfully from npm.

bash
npm install llm-scraper

Version confirmed via:

bash
node -e "const p = require('./node_modules/llm-scraper/package.json'); console.log(p.name + '@' + p.version)"

Actual output:

llm-scraper@2.0.0

llm-scraper is a TypeScript library that uses LLMs to extract structured data from web pages. It requires playwright and an LLM API key (OpenAI or compatible) for actual extraction, but the package itself installs without any account.

Check the installed version ​

LLM Scraper does not expose a CLI binary, so there is no --version flag. Check the installed version via npm:

bash
npm view llm-scraper version

Expected success output:

2.0.x

To check what is installed locally in a project:

bash
npm ls llm-scraper

Expected success output:

your-project@...
└── llm-scraper@2.0.x

Smoke test: verify peer dependencies resolve ​

After installing all packages, confirm everything resolves without errors:

bash
npm i zod playwright llm-scraper @ai-sdk/openai
node -e "require('llm-scraper'); console.log('llm-scraper ok')"

Expected success output:

llm-scraper ok

If you see Cannot find module 'llm-scraper', the install did not complete or you are in the wrong directory.

Full integration smoke test (requires API key) ​

This is the minimal end-to-end test. It requires a valid OPENAI_API_KEY and a downloaded Playwright browser (npx playwright install chromium).

typescript
// smoke.ts
import { chromium } from 'playwright'
import { z } from 'zod'
import { Output } from 'ai'
import { openai } from '@ai-sdk/openai'
import LLMScraper from 'llm-scraper'

const browser = await chromium.launch()
const llm = openai('gpt-4o-mini')
const scraper = new LLMScraper(llm)
const page = await browser.newPage()
await page.goto('https://example.com')

const schema = z.object({ heading: z.string() })
const { data } = await scraper.run(page, Output.object({ schema }))

console.log('heading:', data.heading)
await browser.close()

Run with:

bash
npx tsx smoke.ts

Expected success output:

heading: Example Domain

The exact text depends on the LLM response, but it should return something like "Example Domain" from the <h1> on example.com.

Notes ​

  • tested=false for the full integration test: it requires an LLM API key and a downloaded Playwright browser, which are not available in the build environment. The package install and version check were verified and passed.
  • If the LLM API call fails, check that your OPENAI_API_KEY (or equivalent) is set and has credits.
  • If Playwright throws browserType.launch: Executable doesn't exist, run npx playwright install chromium.