agatha/proxy-pool
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
proxy-pool/docs/04-api-reference.md

8.7 KiB
Raw Permalink Blame History

API reference

Authentication

All endpoints except POST /auth/register and GET /health require an API key in the Authorization header:

Authorization: Bearer pp_a1b2c3d4e5f6g7h8i9j0...

API keys use a pp_ prefix (proxy pool) followed by 48 random characters. The prefix aids visual identification and log filtering.

Unauthenticated or invalid requests return 401 Unauthorized. Requests with valid keys but insufficient credits return 402 Payment Required.

Common response patterns

Pagination

List endpoints support cursor-based pagination:

GET /proxies?limit=50&cursor=eyJpZCI6Ii4uLiJ9

Response includes a next_cursor field when more results exist:

{
  "items": [...],
  "next_cursor": "eyJpZCI6Ii4uLiJ9",
  "total_count": 1234
}

Error responses

All errors follow a consistent shape:

{
  "error": {
    "code": "INSUFFICIENT_CREDITS",
    "message": "You need at least 1 credit to acquire a proxy. Current balance: 0.",
    "details": {}
  }
}

Proxy domain endpoints

Sources

GET /sources

List all configured proxy sources.

Query parameters: is_active (bool, optional), limit (int, default 50), cursor (string, optional).

Response 200:

{
  "items": [
    {
      "id": "uuid",
      "url": "https://example.com/proxies.txt",
      "parser_name": "plaintext",
      "cron_schedule": "*/30 * * * *",
      "default_protocol": "http",
      "is_active": true,
      "last_scraped_at": "2025-01-15T10:30:00Z",
      "created_at": "2025-01-01T00:00:00Z"
    }
  ],
  "next_cursor": null,
  "total_count": 5
}

POST /sources

Add a new proxy source.

Request body:

{
  "url": "https://example.com/proxies.txt",
  "parser_name": "plaintext",
  "cron_schedule": "*/30 * * * *",
  "default_protocol": "http"
}

Validation: The parser_name must match a registered plugin. If omitted, the registry attempts auto-detection via supports(). Returns 422 if no matching parser is found.

Response 201: The created source object.

PATCH /sources/{source_id}

Update a source. All fields are optional — only provided fields are changed.

DELETE /sources/{source_id}

Delete a source. Associated proxies are NOT deleted (they may have been discovered by multiple sources).

POST /sources/{source_id}/scrape

Trigger an immediate scrape of the source, bypassing the cron schedule. Returns the scrape result.

Response 200:

{
  "source_id": "uuid",
  "proxies_discovered": 142,
  "proxies_new": 23,
  "proxies_updated": 119,
  "duration_ms": 1540
}

Proxies

GET /proxies

Query the proxy pool with filtering and sorting.

Query parameters:

Parameter Type Description
status string Filter by status: active, dead, unchecked
protocol string Filter by protocol: http, https, socks4, socks5
anonymity string Filter by anonymity: transparent, anonymous, elite
country string ISO 3166-1 alpha-2 country code
min_score float Minimum composite score (0.0–1.0)
max_latency_ms float Maximum average latency
min_uptime_pct float Minimum uptime percentage
verified_within_minutes int Only proxies checked within the last N minutes
sort string Sort field: score, latency, uptime, last_checked
order string Sort order: asc, desc (default: desc for score)
limit int Results per page (default: 50, max: 200)
cursor string Pagination cursor

Response 200:

{
  "items": [
    {
      "id": "uuid",
      "ip": "203.0.113.42",
      "port": 8080,
      "protocol": "http",
      "status": "active",
      "anonymity": "elite",
      "exit_ip": "203.0.113.42",
      "country": "US",
      "score": 0.87,
      "avg_latency_ms": 245.3,
      "uptime_pct": 94.2,
      "last_checked_at": "2025-01-15T10:25:00Z",
      "first_seen_at": "2025-01-10T08:00:00Z",
      "tags": {"provider": "datacenter"}
    }
  ],
  "next_cursor": "...",
  "total_count": 892
}

GET /proxies/{proxy_id}

Get detailed info for a single proxy, including recent check history.

Response 200:

{
  "proxy": { ... },
  "recent_checks": [
    {
      "checker_name": "tcp_connect",
      "stage": 1,
      "passed": true,
      "latency_ms": 120.5,
      "detail": "TCP connect OK",
      "created_at": "2025-01-15T10:25:00Z"
    }
  ]
}

POST /proxies/acquire

Acquire a proxy with an exclusive lease. Costs 1 credit.

Request body:

{
  "protocol": "http",
  "country": "US",
  "anonymity": "elite",
  "min_score": 0.7,
  "lease_duration_seconds": 300
}

All filter fields are optional. lease_duration_seconds defaults to 300 (5 minutes), max 3600 (1 hour).

Response 200:

{
  "lease_id": "uuid",
  "proxy": {
    "ip": "203.0.113.42",
    "port": 8080,
    "protocol": "http",
    "country": "US",
    "anonymity": "elite"
  },
  "expires_at": "2025-01-15T10:30:00Z",
  "credits_remaining": 42
}

Error responses: 402 if insufficient credits, 404 if no proxy matches the filters, 409 if all matching proxies are currently leased.

POST /proxies/acquire/{lease_id}/release

Release a lease early. The proxy becomes available immediately. The credit is NOT refunded (credits are consumed on acquisition).

POST /proxies/test

Test whether good proxies can reach a specific URL.

Request body:

{
  "url": "https://example.com",
  "count": 5,
  "timeout_seconds": 10,
  "protocol": "http",
  "country": "US"
}

Response 200:

{
  "url": "https://example.com",
  "results": [
    {
      "proxy_id": "uuid",
      "ip": "203.0.113.42",
      "port": 8080,
      "reachable": true,
      "status_code": 200,
      "latency_ms": 340,
      "error": null
    },
    {
      "proxy_id": "uuid",
      "ip": "198.51.100.10",
      "port": 3128,
      "reachable": false,
      "status_code": null,
      "latency_ms": null,
      "error": "Connection refused by target"
    }
  ],
  "success_rate": 0.8
}

Stats

GET /stats/pool

Pool health overview.

Response 200:

{
  "total_proxies": 15420,
  "by_status": {"active": 3200, "dead": 11800, "unchecked": 420},
  "by_protocol": {"http": 8000, "https": 4000, "socks4": 1200, "socks5": 2220},
  "by_anonymity": {"transparent": 1500, "anonymous": 1000, "elite": 700},
  "avg_score": 0.62,
  "avg_latency_ms": 380.5,
  "sources_active": 12,
  "sources_total": 15,
  "last_scrape_at": "2025-01-15T10:30:00Z",
  "last_validation_at": "2025-01-15T10:25:00Z"
}

GET /stats/plugins

Plugin registry status.

Response 200:

{
  "parsers": [{"name": "plaintext", "type": "SourceParser"}],
  "checkers": [
    {"name": "tcp_connect", "stage": 1, "priority": 0},
    {"name": "http_anonymity", "stage": 2, "priority": 0}
  ],
  "notifiers": [
    {"name": "smtp", "healthy": true, "subscribes_to": ["proxy.pool_low", "credits.*"]},
    {"name": "webhook", "healthy": false, "subscribes_to": ["*"]}
  ]
}

Accounts domain endpoints

Auth

POST /auth/register

Create a new user account and initial API key. No authentication required.

Request body:

{
  "email": "user@example.com",
  "display_name": "Alice"
}

Response 201:

{
  "user": {"id": "uuid", "email": "user@example.com", "display_name": "Alice"},
  "api_key": {
    "id": "uuid",
    "key": "pp_a1b2c3d4e5f6g7h8i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4",
    "prefix": "pp_a1b2c",
    "label": "default"
  }
}

Important: The key field is the raw API key. It is returned ONLY in this response. Store it securely — it cannot be retrieved again.

POST /auth/keys

Create an additional API key for the authenticated user.

GET /auth/keys

List all API keys for the authenticated user (returns prefix and metadata, never the full key).

DELETE /auth/keys/{key_id}

Revoke an API key.

Account

GET /account

Get the authenticated user's account info.

GET /account/credits

Get current credit balance and recent transaction history.

Response 200:

{
  "balance": 42,
  "recent_transactions": [
    {
      "amount": -1,
      "tx_type": "acquire",
      "description": "Proxy acquired: 203.0.113.42:8080",
      "created_at": "2025-01-15T10:25:00Z"
    },
    {
      "amount": 100,
      "tx_type": "purchase",
      "description": "Credit purchase",
      "created_at": "2025-01-14T00:00:00Z"
    }
  ]
}

GET /account/leases

List the authenticated user's active proxy leases.

System endpoints

GET /health

Basic health check. No authentication required.

Response 200:

{
  "status": "healthy",
  "postgres": "connected",
  "redis": "connected",
  "version": "0.1.0"
}