just-scrape

Made with love by the ScrapeGraphAI team 💜

Command-line interface for ScrapeGraph AI — AI-powered web scraping, data extraction, search, crawling, and page-change monitoring.

v1.0.0 — SDK v2 migration. This release migrates the CLI to the scrapegraph-js v2 SDK. The v1 endpoints (smart-scraper, search-scraper, markdownify, sitemap, agentic-scraper, generate-schema) have been removed. Use scrape --format … for multi-format output, extract for structured data, and the new monitor command for page-change tracking.

Project Structure

just-scrape/
├── src/
│   ├── cli.ts
│   ├── lib/
│   │   ├── env.ts
│   │   ├── folders.ts
│   │   └── log.ts
│   ├── commands/
│   │   ├── scrape.ts
│   │   ├── extract.ts
│   │   ├── search.ts
│   │   ├── crawl.ts
│   │   ├── monitor.ts
│   │   ├── history.ts
│   │   ├── credits.ts
│   │   └── validate.ts
│   └── utils/
│       └── banner.ts
├── dist/
├── tests/
├── package.json
├── tsconfig.json
├── tsup.config.ts
├── biome.json
└── .gitignore

Installation

npm install -g just-scrape
pnpm add -g just-scrape
yarn global add just-scrape
bun add -g just-scrape
npx just-scrape --help
bunx just-scrape --help

Package: just-scrape on npm.

Coding Agent Skill

You can use just-scrape as a skill for AI coding agents via Vercel's skills.sh.

Or you can manually install it:

bunx skills add https://github.com/ScrapeGraphAI/just-scrape

Browse the skill: skills.sh/scrapegraphai/just-scrape/just-scrape

Configuration

The CLI needs a ScrapeGraph API key. Get one at dashboard.scrapegraphai.com.

Four ways to provide it:

1. Environment variable: export SGAI_API_KEY="sgai-..."
2. .env file: SGAI_API_KEY=sgai-...
3. Config file: ~/.scrapegraphai/config.json
4. Interactive prompt

Environment Variables

Variable	Description	Default
SGAI_API_KEY	ScrapeGraph API key	—
SGAI_API_URL	Override API base URL	https://api.scrapegraphai.com/api/v2
SGAI_TIMEOUT	Timeout (seconds)	120
SGAI_DEBUG	Debug logs	0

JSON Mode (--json)

just-scrape credits --json | jq '.remaining'
just-scrape scrape https://example.com --json > result.json
just-scrape history scrape --json | jq '.[].id'

---

Scrape

docs

Extract

docs

Search

docs

Crawl

docs

Monitor

docs

History

docs

Credits

just-scrape credits

Validate

just-scrape validate

---

Contributing

git clone https://github.com/ScrapeGraphAI/just-scrape
cd just-scrape
bun install
bun run dev --help

---

Made with love by the ScrapeGraphAI team 💜