Module adapters 

Adapter implementations - infrastructure concerns

Concrete implementations of port traits:

• HTTP client with anti-bot features
• AI providers (Claude, GPT, Gemini, Ollama)
• Storage backends (file, S3, database)
• Cache backends (memory, Redis, file) Adapter implementations - infrastructure concerns

Concrete implementations of port traits:

• HTTP client with anti-bot features
• AI providers (Claude, GPT, Gemini, Ollama, Copilot)
• Storage backends
• Cache backends

Modules

ai

AI provider adapters AI provider adapters

browser

JavaScript rendering adapter (headless browser via stygian-browser) JavaScript rendering adapter using stygian-browser

cache

Cache adapters (memory, Redis) Cache adapters

cloudflare_crawl

Cloudflare Browser Rendering crawl adapter (feature = “cloudflare-crawl”) Cloudflare Browser Rendering crawl adapter

distributed

Distributed work queue and executor adapters Distributed execution adapters

graphql

GraphQL API adapter — generic ScrapingService for any GraphQL endpoint GraphQL API adapter — a generic ScrapingService for any spec-compliant GraphQL endpoint.

graphql_plugins

GraphQL target plugin implementations (one file per API target) GraphQL target plugin implementations.

graphql_rate_limit

Request-count rate limiter for GraphQL API targets with pluggable algorithms.

graphql_throttle

Proactive cost-throttle management for GraphQL APIs Proactive GraphQL cost-throttle management.

http

HTTP scraping adapter with anti-bot capabilities HTTP scraping adapter with anti-bot features

mock_ai

Mock AI provider for testing Mock AI provider adapter for testing

multimodal

Multi-modal content extraction (CSV, JSON, XML, images, PDFs) Multi-modal content extraction adapter

noop

No-op service for testing No-op service adapter for testing

openapi

OpenAPI 3.x introspection adapter — resolves operations from an OpenAPI spec and delegates to RestApiAdapter OpenAPI 3.x introspection adapter.

output_format

Output format helpers — CSV, JSONL, JSON Output format helpers — CSV, JSONL, JSON.

resilience

Resilience adapters (circuit breaker, retry) Resilience adapters

rest_api

REST API adapter — JSON APIs with auth, pagination, and data extraction REST API scraping adapter with authentication and pagination support.

signing

Request signing adapters — Noop passthrough and HTTP sidecar bridge. Covers Frida RPC, AWS Sig V4, OAuth 1.0a, custom HMAC, and device attestation. Request signing adapters.

storage

Storage adapters (file, S3, database) Storage adapters — persist and retrieve pipeline StorageRecords.

wasm_plugin

WASM plugin adapter (feature = “wasm-plugins”) WASM plugin adapter