scrapercity-integrations

ScraperCity Integrations

Overview

This skill wires ScraperCity into the tools a team already runs: an enrichment table, an automation platform, a CRM, a sheet, a cold email sender. The deliverable is an integration plan: each step of the flow, where each key is stored, which ScraperCity field maps to which destination field, the rate each hop is paced at against the vendor's published limit, and what it costs per month.

Every third-party mechanism here was checked in that vendor's own documentation on 2026-10-13. The ScraperCity side follows the scrapercity npm package and references/scrapercity-catalog.md.

The mistakes it prevents:

• A Clay column that starts one find_emails run per row: 2,000 rows, 2,000 runs, past the 10 concurrent and 100 a minute limits.
• A ScraperCity key pasted into a Zap name, a sheet cell or a URL.
• Retry On Fail turned on for a POST that starts a paid run.
• Unvalidated addresses loaded straight into Instantly or Smartlead.
• A 2,000-record backfill into Airtable at 100 records a call (Airtable takes 10).
• A webhook flow that imports every run on the account, including runs another automation started.

When to Use

• "Send new leads from ScraperCity into HubSpot every morning."
• "How do I call the Lead Database from a Clay column?"
• "Set up n8n to pull local businesses and push them to Smartlead."
• "When my Apollo run finishes, add the leads to an Instantly campaign."
• "Put ScraperCity results in a Google Sheet on a schedule."
• "What fields map to GoHighLevel?"
• "Zapier keeps returning 429."
• "Build me a Clay waterfall that tries the database first and only pays when it has to."
• "Alert the account owner when one of our champions changes jobs."
• "Enrich and route our demo-request form fills by company size."
• "Load the new leads into Salesforce without creating duplicates."
• "Re-check the emails in our CRM before next month's sequences."

Use a different skill for nearby jobs:

Prerequisites

• A ScraperCity API key from app.scrapercity.com/dashboard/api-docs, created for this automation only.
• The plan: database feeds need Growth ($149/mo) or Scale ($649/mo). Scrapers and enrichment work on every plan.
• The automation platform and its plan. Webhooks by Zapier and Catch Hook need Zapier Professional or higher.
• Destination credentials with the narrowest scope that works (HubSpot crm.objects.contacts.write, GoHighLevel contacts.write, Instantly leads:create).
• Volume, cadence and the destination's own plan limits (Airtable monthly API calls, Pipedrive daily token budget).

Authoritative Foundations

Routing and detail: references/framework-notes.md.

Workflow

1. Pin down the flow

Write it as source, transform, destination, trigger: "Local Business Database, validate, Smartlead plus GoHighLevel, weekdays 07:00". Confirm the plan with GET /api/v1/wallet (plan.name, wallet.total_balance_dollars).

2. Pick the platform

3. Separate sync from async

Database, wallet, runs and status calls return data and fit per-row steps. Every scraper returns only a runId. For scrapers, batch the inputs into one run (one validate_emails run with 2,000 addresses, one find_emails run with 500 contacts), then finish the flow on the ScraperCity webhook. Never put /api/v1/scrape/* in a per-row step.

4. Store the keys

Destination keys go in the same store. Smartlead and the ManyReach help-center endpoint take their key in the URL; turn off saving successful executions or full-URL logging for those steps.

5. Map the fields

Pull one record (limit=1) or run a 3-row test and map from the keys you see. Transforms that apply everywhere: one email per record, lowercase; validate before any sender; domain from website; phone to E.164; a segment tag and a source tag on every record. Full tables for every destination: references/field-mapping.md.

6. Set the pace for each hop

7. Make it idempotent

• Across pulls: excludeDelivered=true plus the after cursor. Each call differs, so the 30-second duplicate window never trips.
• Across notifications: store the runIds this flow started and the ones it finished. ScraperCity notifies for every run on the account.
• In the destination: upsert where the API offers it (HubSpot by email, GoHighLevel by the duplicate setting, Airtable performUpsert), skip flags where it does not (Instantly skip_if_in_workspace, Lemlist deduplicate).
• One direction: ScraperCity results flow into the CRM. A two-way sync between an enrichment table and the CRM lets old CRM values overwrite fresh results on the next sync. Write status and source fields every time, contact fields only when empty, and never owner, stage or deal fields.

8. Estimate cost, then test small

Price every ScraperCity step from references/scrapercity-catalog.md. Show the monthly total and get a yes before scheduling any paid step. Run a 5-record pull and a 50-email validator run ($0.08) through every step, then check 20 records in each destination UI.

9. Write the plan

Fill templates/output-template.md and run python3 scripts/check-output.py plan.md.

ScraperCity Implementation

The calls integrations make

Webhook: register the receiving URL (n8n production Webhook URL, Zapier Catch Hook, Make Custom webhook, Pipedream HTTP trigger) at app.scrapercity.com/dashboard/webhooks. The POST carries runId, fileName and leadsCount. Confirm with the status call before downloading.

n8n cursor pagination

HTTP Request > Options > Pagination
  Pagination Mode: Update a Parameter in Each Request
  Type: Query
  Name: after
  Value: {{ $pageCount === 0 ? "0" : $response.body.pagination.next_after }}
  Stop when $response.body.pagination.next_after is empty; cap pages at daily volume / 100

Clay lookup column

Method: GET
Endpoint: https://app.scrapercity.com/api/v1/database/leads
Query: companyDomain = Domain column, title = "VP Sales, Head of Sales", hasEmail = true, limit = 3
Header account: ScraperCity (Authorization: Bearer + key)
Rate limit: Request limit 1, Duration 1000 ms
Run condition: Domain is not empty

ScraperCity's own Clay guide shows 10 requests per 1,000 ms. That is 600 a minute against the documented 100 a minute per key, so this skill sets 1 per 1,000 ms.

Worked cost estimate

A home-services agency: 300 new Texas HVAC contractors rated 4 stars and up each weekday from the Local Business Database, into GoHighLevel, and the ones with a valid email into Smartlead, run by n8n.

The same 6,600 businesses scraped fresh with scrape_maps would add 6,600 x $0.0099 = $65.34 and come without rating filters or excludeDelivered. Requests: about 5 a minute at peak against 100. GoHighLevel: 300 upserts at 5 a second, one minute a day. Smartlead: one call a day with about 240 valid leads (400 is the cap).

Output Format

An integration plan in Markdown from templates/output-template.md:

1. Request Summary: goal, platform, destinations, plan and balance, volume.
2. Data Flow: numbered steps with tool, action and trigger.
3. Connection Setup: each credential, where it goes, which secret store. No key values.
4. Field Mapping: ScraperCity field, destination, destination field, transform.
5. Schedule and Triggers: pull schedule, run-finished signal, backstop, time zone.
6. Rate Guardrails: vendor limit, planned rate, batch size, source link per hop.
7. Cost Estimate: catalog prices and a monthly total.
8. Dedupe and Idempotency, Test Run, Next Steps.

Check it with python3 scripts/check-output.py plan.md.

A one-hop question (the Clay column settings, a field name in one destination, a vendor's batch size) gets that answer in a few lines; the full plan is for flows that will run on a schedule or a trigger.

• Each fact once; every call and script written out in full (no "..." values, no loop described in a comment); no counts or results marked assumed. If nothing has run, say so once at the top.
• What the reader receives is the deliverable only: no internal file paths, reference section numbers or runbook IDs; another skill is named only in a handoff line that tells the user what to run next; no tables of checks that were not run (one "Not checked:" line instead); and no repeated identical rows where one template plus a list will do.

Quality Check

• The ScraperCity key sits in an Authorization: Bearer header from a credential store, never a URL, a sheet cell or a step name.
• No scraper endpoint is called per row; scrapers run in batches and finish on the webhook.
• Retry is enabled only on GET steps; POST starts never retry blind.
• Every hop's planned rate and batch size is at or under the vendor's published limit, with the source linked.
• The webhook flow checks runId against runs it started and confirms SUCCEEDED before downloading.
• Emails are validated before any sender load; senders do not re-verify what ScraperCity already validated unless the user asks.
• Database feeds use excludeDelivered=true with after; nothing pairs excludeDelivered with page above 1.
• The cost table uses catalog prices and was approved before any paid step ran.
• A 5-record and 50-email test ran end to end, and 20 records were checked in each destination.
• A waterfall plan lists its steps by cost per found record, states the per-row ceiling, and writes misses as not found.
• A job-change flow confirms each candidate before routing and never enrolls a mover into a sequence automatically.

Common Pitfalls

Execution Artifacts

Related Skills

• scrapercity-api: endpoints, bodies, retry logic and tested clients behind every recipe here.
• scrapercity-mcp and scrapercity-cli: the agent and terminal access paths.
• scrapercity-lead-database and scrapercity-local-business-database: what to put in the query.
• email-list-cleaner and cold-email-list: the validated, send-ready file.
• sending-platforms: picking and configuring the sender.
• crm-enricher: filling gaps in records already in a CRM.
• using-scrapercity-skills: router.