WebDoppler
Monitor any URL for changes, describe what data you want to capture in plain English, and receive structured webhook alerts automatically on every future change. WebDoppler is domain-agnostic — regulatory filings, competitor pricing, medical guidance updates, zoning notices — any public URL, any extraction intent.
WebDoppler is entitled per workspace and is not automatically included with
general API access. A non-entitled workspace calling /api/v1/monitor/register
receives:
{ "ok": false, "code": "forbidden", "error": "WebDoppler is not enabled for this workspace..." }Contact support or request access from the WebDoppler console page to get started.
Use this when
- You need to know the moment a specific page changes — not just that it changed, but what changed and what the new data contains.
- You want structured data extracted from a page automatically on each change, without manual review or re-entry.
- You're monitoring IR pages, regulatory notice boards, or any site that posts time-sensitive documents.
- You want webhook delivery to your own system when a monitored URL changes.
How it works
- Register a URL with optional keywords and a plain-English extraction intent.
- Preview — Gyrence fetches the page and Claude extracts a structured preview based on your intent. You refine until it's right, then approve.
- Monitor — Gyrence checks the URL on your configured schedule. When content changes, it runs the locked extraction automatically.
- Receive — a signed webhook fires to your endpoint with the change signal and extracted fields.
The iterative preview/approve loop happens once at setup. Once you approve an extraction, it runs automatically on every future alert — no further iteration required.
Endpoints
| Method | POST |
| Path | /api/v1/monitor/register |
| Auth | Bearer |
| Credits | 0 (registration) |
| Parameter | Type | Description |
|---|---|---|
urlrequired | string | Absolute http(s) URL to monitor. SSRF guard applies — private and loopback hosts are rejected. |
label | string | Human-readable name for this monitor. Shown in the UI and webhook payload. |
channels | string[]default: ["ir_page"] | Acquisition channels: "ir_page" | "businesswire" | "globenewswire" | "edgar_8k" | "direct_pdf". Controls which discovery paths are active. |
check_interval_hours | integerdefault: 24 | How often to check the URL, in hours. Minimum 1, maximum 168 (1 week). |
discovery_keywords | string[] | Keywords that must appear in changed content for an alert to fire. Acts as a signal filter — nav/footer changes that don't match are suppressed. |
extract_intent | string | Plain-English description of what data to extract. Example: 'Form 8937 data by CUSIP — adjustment factor, return of capital per share, event date.' Triggers an extraction preview after registration. |
accept_selectors | string[] | CSS selectors to click before page capture — for cookie banners or terms modals. Example: ["button:has-text(\"I Agree\")"]. |
Example
curl -X POST https://www.gyrence.com/api/v1/monitor/register \
-H "Authorization: Bearer $GYRENCE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"url": "https://bip.brookfield.com/distribution-history",
"label": "BIP Distribution Matrix",
"channels": ["ir_page"],
"check_interval_hours": 24,
"discovery_keywords": ["distribution matrix", "withholding", "1446"],
"extract_intent": "PTP distribution matrix — extract per unit class: CUSIP, currency, record date, payment date, and per-source withholding amounts for both US and Canadian withholding."
}'{ "ok": true, "data": { "monitor_id": "uuid", "status": "active" } }| Method | GET |
| Path | /api/v1/monitor/:id |
| Auth | Bearer |
| Credits | 0 |
Returns current monitor status.
| Field | Type | Description |
|---|---|---|
monitor_id | string | The monitor UUID. |
url | string | The monitored URL. |
status | string | "active" | "paused" | "error" |
last_checked | string | ISO timestamp of the most recent check. |
last_changed | string | ISO timestamp of the most recent detected change. |
anchor_set | boolean | Whether a fingerprint anchor has been established. False on the first run. |
last_outcome | string | Outcome of the most recent run. See Outcomes table below. |
top_score | number | Relocation similarity score from the most recent run (0–100). |
| Method | GET |
| Path | /api/v1/monitor/:id/runs |
| Auth | Bearer |
| Credits | 0 |
Returns paginated run history. Query params: limit (default 20, max 100), before (ISO timestamp cursor).
| Method | DELETE |
| Path | /api/v1/monitor/:id |
| Auth | Bearer |
| Credits | 0 |
Removes the monitor. Returns { "ok": true, "data": { "deleted": true } }.
| Method | POST |
| Path | /api/v1/monitor/webhook |
| Auth | Bearer |
| Credits | 0 |
Configure webhook delivery for your workspace. Accepts { webhook_url, webhook_secret }. Pass null to clear.
Alert channels
Alerts can be delivered via webhook, email, SMS, or Slack. Configure defaults per workspace or override per monitor.
Slack uses a standard Incoming Webhook URL. Paste the URL into your alert settings — no OAuth flow required.
| Method | POST |
| Path | /api/v1/monitor/:id/preview-extract |
| Auth | Bearer |
| Credits | 1 |
Fetch the monitored URL and run Claude extraction against your intent. Returns a structured preview for you to review. Optionally pass { refinement: "..." } in the body to iterate.
| Method | POST |
| Path | /api/v1/monitor/:id/approve-extract |
| Auth | Bearer |
| Credits | 0 |
Lock the extraction template. Pass { extracted_sample, locked_prompt }. Once approved, every future alert runs the same extraction automatically. Returns { "ok": true, "data": { "approved": true } }.
Webhook payload
Gyrence signs every webhook with HMAC-SHA256 over the raw body using your webhook_secret. Verify the X-Gyrence-Signature header before processing.
{
"monitor_id": "uuid",
"caller_id": "crescentic_subscriber_id",
"url": "https://bip.brookfield.com/distribution-history",
"detected_at": "2026-06-28T14:22:00Z",
"outcome": "changed",
"top_score": 84.2,
"channel": "ir_page",
"change_signal": [
{ "field": "attr:href", "from": null, "to": "/files/distribution-matrix-q2-2026.pdf", "ratio": 0 },
{ "field": "text", "from": "Q1 2026", "to": "Q2 2026", "ratio": 0.67 }
],
"pdf_links": ["https://bip.brookfield.com/files/distribution-matrix-q2-2026.pdf"],
"extracted_fields": {
"found": true,
"data": [
{
"unit_class": "Regular LP Units",
"cusip": "G16252101",
"currency": "USD",
"periods": [
{
"record_date": "2026-05-29",
"payment_date": "2026-06-30",
"quarter_label": "Q2",
"total_distribution": 0.4550,
"us_withholding": { "total_subject_to_us_withholding": 0.0474 },
"canadian_withholding": { "total_subject_to_canadian_withholding": 0.3754 }
}
]
}
]
},
"content_preview": "Distribution Matrix Q2 2026 — Brookfield Infrastructure Partners L.P..."
}extracted_fields is only present when an extraction template has been approved for the monitor. The structure matches whatever schema Claude produced during your preview/approve session — it is bespoke per monitor, not a fixed format.
Outcomes
| Outcome | Meaning |
|---|---|
unchanged | Content hash matched — no change detected. No webhook fired. |
anchored | First run — fingerprint anchor established. Webhook fired (informational). |
changed | Material change detected. Extraction ran if a template is locked. Webhook fired. |
relocation_failed | Page restructured enough that the fingerprint anchor was lost. Always fires webhook — never suppressed. Re-anchor required. |
payload_too_large | Detected document exceeds the 25 MB passthrough limit. pdf_links and size_bytes included in webhook. |
blocked | Page returned a bot-detection challenge or a blocked status code. Escalated to browser tier if available. |
error | Discovery keywords found no matching content on the page, or an unhandled error occurred. |
When a page is redesigned significantly enough that WebDoppler loses its anchor, the alert always fires regardless of Gate B magnitude filtering. A lost anchor is a monitoring gap — it must be surfaced, not silently dropped.
Extraction intent
When you register a monitor with extract_intent, WebDoppler opens an iterative preview loop:
- Gyrence fetches the page and scopes content to your
discovery_keywordsmatches. - Claude reads the scoped content and extracts structured data based on your intent.
- You see the result in the WebDoppler UI — rendered as a labeled tree, not raw JSON.
- Refine if anything is missing or wrong. Claude sees your original intent plus each refinement.
- Approve to lock the template. Every future alert runs the locked extraction automatically.
The approved extraction is bespoke to your monitor. There are no fixed schemas — Claude interprets your plain-English intent and produces whatever structure fits the data it found. Your approval is the quality gate.
"extract_intent": "1099-DIV box percentages by share class for the full tax year.
I need box 1a (ordinary dividend), 1b (qualified), 2a (capital gains),
3 (return of capital), and 5 (Section 199A) as decimals summing to 1.0."Errors
| Code | HTTP | Meaning |
|---|---|---|
bad_request | 400 | Missing or invalid fields. |
unauthorized | 401 | Missing, malformed, or revoked Authorization header. |
not_found | 404 | Monitor ID not found or belongs to a different caller. |
conflict | 409 | URL already registered for this caller. |
unavailable | 503 | Upstream fetch failed, extraction service error, or database error. |
Open WebDoppler from the console at /app/monitor — register a URL, preview an extraction, and approve it without writing any curl commands.
