> ## Documentation Index
> Fetch the complete documentation index at: https://docs.trybluesky.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Monitor AI Crawler Activity with Agent Analytics
> Track which AI agents crawled your site, how requests were verified, and whether the optimized artifact was served or passed through.
Agent Analytics is your window into AI crawler activity across your site. Once the Bluesky edge middleware is installed on your app, every time an AI agent — ChatGPT, Perplexity, Claude, and others — fetches one of your pages, the middleware logs the hit and forwards telemetry to the dashboard. You can see who crawled, how each request was verified (or wasn't), whether the optimized artifact was served, and — the part that matters — how many of the pages you served also appear in this domain's citation records.
The dashboard summarizes the **last 30 days** of activity for the selected domain.
## How to find it
Navigate to [app.trybluesky.com](https://app.trybluesky.com) and select **Agent Analytics** from the left sidebar. If you haven't connected the selected domain yet, you'll see a **Connect** call-to-action; follow the steps below.
## Connecting your domain
In Agent Analytics, pick your domain and click **Connect**. This provisions the edge for that site and displays an **edge token** (format `aen_…`). It is shown **once**, is scoped to that single site, and is revocable. It is not your account API key.
In your host (for example Vercel → your project → **Settings → Environment Variables**), add the edge token as `AEN_EDGE_TOKEN`:
```bash theme={null}
AEN_EDGE_TOKEN=aen_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
```
Add the Bluesky Agent Edge Network middleware (`middleware.ts`) to the root of your Next.js app and deploy. It is self-contained with zero dependencies and runs on the Vercel Edge runtime.
It ships in `shadow` mode by default: it serves your normal page to everyone and only logs what it *would* have served. There is no risk to visitors or SEO while you validate.
Open ChatGPT or Perplexity with browsing enabled and ask a question that makes it fetch one of your pages. The crawler's IP is matched against the vendor's published CIDR ranges, the hit is classified `IP_VERIFIED`, and it appears in Agent Analytics.
Once you see verified hits flowing in, change `SERVE_MODE` in `middleware.ts` from `"shadow"` to `"default"` and redeploy. See [Serve Modes](/guides/serve-modes) for a safe rollout guide.
Install paths available today: [Next.js / Vercel](/guides/nextjs-middleware), [Cloudflare Worker](/guides/cloudflare-worker), [WordPress / PHP](/guides/wordpress), and a [custom integration](/guides/custom-integration) for any other stack. A zero-code CNAME reverse-proxy is on the roadmap but not yet available.
## What the dashboard shows
Once connected, the page surfaces four KPIs, two breakdowns, and a list of your optimized pages.
### KPIs
| KPI | What it means |
| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Total hits** | AI-agent requests logged in the window that carried a purpose class — any trust level, verified or not |
| **Served (variant)** | Requests that received the optimized artifact instead of your normal page |
| **Served URLs** | Distinct URLs that were served a variant at least once |
| **Cited share** | Of your served URLs, the percentage that also appear as a citation for this domain (shown as `—` until at least one URL has been served a variant) |
### By purpose
A breakdown of hits by **purpose class**, split by whether the request was **served** the variant or **passed through**. This tells you whether your AI traffic is real-time user queries or background retrieval, and how much of each you are actually serving.
### By trust
A breakdown of hits by **verification trust level** — the "verified detection" story, showing how each crawler's identity was (or wasn't) confirmed. Unverified `UA_ONLY` hits appear here too.
### Optimized pages
A list of the pre-generated artifacts for the domain. Each row shows the URL, its status (`fresh` / `stale`), a **citability score**, and a **Regenerate** action that flags the artifact stale so the engine regenerates it.
## Verification and trust levels
Bluesky verifies each crawler's identity before deciding whether to serve the optimized artifact. Trust is ascending:
* **`SIGNED`** — The request carried a valid [Web Bot Auth](https://www.ietf.org/rfc/rfc9421.html) signature (RFC 9421 / Ed25519), cryptographically verified against the bot's pinned keys. Highest trust.
* **`IP_VERIFIED`** — The source IP falls inside the vendor's published CIDR range. This is what the drop-in middleware verifies today.
* **`UA_ONLY`** — The user-agent matched a known bot token, but nothing else could be verified. Treated as **unverified**.
Only `SIGNED` and `IP_VERIFIED` (verified) requests are ever served the optimized variant. `UA_ONLY` hits are still logged so you can see them, but they always pass through to your normal page. The drop-in middleware produces `IP_VERIFIED` or `UA_ONLY`; `SIGNED` requires an agent that signs its requests.
## Purpose classes
Each matched bot carries a purpose class:
| Purpose | Description |
| ---------------- | ------------------------------------------------------------------------------------------ |
| `USER_TRIGGERED` | A user explicitly asked an AI assistant to browse your page — high-intent, real-time fetch |
| `RETRIEVAL` | An answer-engine crawler fetching to build or refresh its index |
| `TRAINING` | A corpus / training crawler |
| `UNKNOWN` | Matched a bot but no purpose could be assigned |
By default, only `USER_TRIGGERED` and `RETRIEVAL` are eligible to be served the variant. `SEARCH_INDEX` bots (Googlebot / Bingbot / Applebot) are never served — see the cloaking firewall below.
## Served status
* **`VARIANT`** — The request received Bluesky's pre-generated, AEO-optimized representation of the page (answer-first markdown plus `Article` / `FAQPage` JSON-LD).
* **`PASSTHROUGH`** — The request received your normal page. This happens when the serve mode is `shadow`, the agent is unverified (`UA_ONLY`), the URL has no generated artifact yet, the purpose class isn't enabled, or the bot is a classic search engine protected by the cloaking firewall.
Seeing lots of `PASSTHROUGH` for verified agents? Check whether a generated artifact exists for those URLs in the Optimized pages list, and regenerate if needed.
## The served-to-cited loop
The **Cited share** KPI is the point of the whole product. Bluesky joins the URLs you actually served a variant for (within the window) against this domain's citation records, and reports what percentage of those served URLs also appear as a citation. The join is set-membership against the domain's citation records — it does not assert the citation happened *after* the serve. It is empty (shown as `—`) until you have served at least one URL a variant; once you have, it shows the percentage (which is `0%` if none of the served URLs are cited yet).
## The cloaking firewall
Classic search-engine bots — Googlebot, Bingbot, and Applebot — are **always** passed through to your normal page, before any serve-mode logic runs. This is a hard guard:
```ts theme={null}
// CLOAKING FIREWALL: classic search bots (Googlebot/Bingbot/Applebot) → always the human page.
if (bot.isSearchIndex) return NextResponse.next();
```
Serving different content to the search index is cloaking, which search engines penalize. Because AI Overviews run on Googlebot and the search index, branching them would cloak Search itself — so they never see the variant.
## Optimized pages and regeneration
Artifacts are generated **offline** and cached — there is no LLM call on the request path. Each optimized page carries a **citability score** so you can judge quality at a glance, and can be flagged for regeneration from the Optimized pages list.
The generator restructures your existing page into answer-first markdown plus `Article` / `FAQPage` JSON-LD under a **strict-parity guardrail**: it will not publish a claim your source page didn't make. The artifact carries the same substance as your original page, formatted for machine readability.
Serving a verified crawler never triggers a slow round-trip or an LLM call at request time — the middleware fetches the pre-generated artifact and serves it.
## How telemetry flows to the dashboard
Every request the middleware processes fires a non-blocking, fire-and-forget telemetry event:
```
Edge middleware → POST /api/v1/edge/telemetry → Agent Analytics
```
The payload includes the bot token, path, IP, trust level, purpose class, verification method, and whether the variant was served. It is sent asynchronously and never adds latency to the response.
## Troubleshooting
Check that (1) `middleware.ts` deployed successfully, (2) `AEN_EDGE_TOKEN` is set and matches the token from the app, and (3) you actually triggered an AI crawler to fetch a page — not just your own browser. Also note the dashboard summarizes the last 30 days.
A variant is only served when the agent is verified (`SIGNED` / `IP_VERIFIED`), its purpose is enabled, the path is allowed, and a generated artifact exists. Find the URL in Optimized pages and click **Regenerate** if there is no artifact. Search-index bots are always passthrough by design.
Yes. Googlebot, Bingbot, and Applebot always get your normal page — they are excluded at the middleware level before any serve-mode logic runs. The optimized artifact is only served to verified, non-search AI answer-engine agents.
Yes. There are drop-ins for [Cloudflare](/guides/cloudflare-worker) and [WordPress / PHP](/guides/wordpress), plus a [custom integration](/guides/custom-integration) with copy-paste snippets for Node, PHP, Ruby, nginx and Apache. A zero-code CNAME reverse-proxy is on the roadmap.