PANOPTICON
API
No key · Open to any origin · v2.0.0

Panopticon
API

A read-only HTTP API over the archive of a design and creative community: 3.9 million replies, 568 thousand topics and 41 thousand front-page snapshots, scraped every five minutes since 2001. Every endpoint is a GET. There are no writes.

ENDPOINTS → CONVENTIONS
26
Endpoints
10
Groups
120/min
Rate limit per IP
01

Conventions

Base /api/v1

Base path /api/v1. Every response is JSON. Cross-origin requests are allowed from any origin, GET and OPTIONS only. That governs whether browser JavaScript may read a response; it is not authentication.

SHAPES
Lists{ data, meta }
Singlea bare object
Errors{ error, status }
TimesISO-8601 UTC
LIMITS
Rate120 / minute / IP
Codes400 · 404 · 429 · 500
Windows24h 7d 30d 90d 1y all
Health/healthz
  • meta.has_moreComputed by fetching one extra row, so paging never costs a COUNT(*) over 3.9 million replies. There is no total on unbounded lists, because there honestly cannot be a cheap one.
  • urlEvery entity carries one, pointing back at the original post.
  • Deleted contentNever appears. 250,781 replies and 145,414 topics are tombstones with empty bodies; they are gaps, not content, and are filtered in one place.
  • to is inclusivePeople say “up to the 5th” and mean it. Internally the range is half-open so it stays index-friendly.
THE LIST ENVELOPEGET /api/v1/users?sort=replies&window=7d&limit=2
{
  "data": [
    { "rank": 1, "user": "Gardener", "replies": 25, "topics": 0,
      "notes": 2, "score": 32, "active_days": 6,
      "first_active": "2026-08-14", "last_active": "2026-08-19" },
    { "rank": 2, "user": "NBQ00", "replies": 16, "topics": 0, … }
  ],
  "meta": { "limit": 2, "offset": 0, "count": 2, "has_more": true,
             "sort": "replies", "window": "7d",
             "from": "2026-08-14", "to": "2026-08-21" }
}
02

The index

10 groups
Status · 1 Users · 6 Replies · 2 Topics · 2 Notes · 2 Profiles · 2 NT · 2 Front page · 3 Statistics · 4 Bulk exports · 2
03

Status

1 endpoint

One endpoint, built to be the first request a client makes.

GET /overview

The state of the community in a single round trip: activity for today, the last 7 days and the last 30 days; the most active users this week; the busiest threads this week; and the current front page.

No parameters.

Today is counted live from replies, not from the rollups. The scraper runs every five minutes but the rollup job runs hourly, and “what is happening right now” is exactly the number that must not be an hour stale. The 7d/30d figures and both rankings come from the rollups; rollups_current_to says what they are current to.

Returns { generated_at, as_of, activity: { today, last_7d, last_30d }, most_active_users_7d[], busiest_topics_7d[], front_page, notes_epoch, rollups_current_to }
04

Users

6 endpoints

The ranking is why this API was rebuilt. It used to be impossible to ask.

GET /users try →

Rank users by activity over a window. This answers “what user has been the most active in the past week”. Pass search instead to look somebody up by handle or display name.

sort
replies · topics · notes · score. Default replies.
window
Time window: 24h · 7d · 30d · 90d · 1y · all. Default 30d.
from / to
Explicit range instead of a window.
search
Switches to lookup mode. Matches handle or display name.
limit / offset
Default 20, max 200.

sort=notes is refused for windows starting before the notes epoch. See what this cannot tell you.

Returns { data: [{ rank, user, replies, topics, notes, score, active_days, first_active, last_active }], meta }
GET /users/:name try →

One user: their profile if they have one, lifetime totals, and how active they have been recently. There was no such resource before.

No parameters.
Returns { user, profile, totals: { replies, topics, notes, score, active_days, first_active, last_active }, recent: { replies_7d, replies_30d, replies_365d, active_days_7d } }
GET /users/:name/replies try →

Replies by one user.

window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all
sort
recent (default) · oldest
notes
Attach each reply’s notes from the notes table. Default false.
limit / offset
Default 50, max 200.
Returns The standard list envelope of reply objects.
GET /users/:name/notes

Notes written by one user, each carrying the reply and topic it was left on.

limit / offset
Default 50, max 200.
Returns The standard list envelope of note objects.
GET /users/:name/topics

Threads started by one user. Takes every parameter /topics takes.

sort · window · limit · offset
As /topics.
Returns The standard list envelope of topic objects.
GET /users/:name/timeline try →

One user’s activity as a time series. This is the chart the notebooks could not draw, because it would have meant fetching every year of the spreadsheet.

bucket
day · month (default) · year
window · from · to
Default all.
Returns { user, bucket, window, data: [{ bucket, replies, topics, notes, score }] }
05

Replies

2 endpoints

Reply bodies are HTML, as scraped. Notes are short remarks attached to a reply.

GET /replies try →

Replies across the archive.

user
Filter by author.
topic_id
Filter by thread.
window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all. Default 30d.
sort
recent (default) · oldest
notes
Attach each reply’s notes from the notes table. Default false.
limit / offset
Default 50, max 200.
Returns The standard list envelope of reply objects.
GET /replies/:id

One reply with its full text and all of its notes.

No parameters.
Returns A reply object with notes[] and notes_count.
06

Topics

2 endpoints

Replaces the three by-* ranking endpoints, one of which used to time out after two minutes.

GET /topics try →

Threads, filtered and sorted. Every result carries its lifetime reply, note and participant counts.

sort
recent (default) · oldest · active · replies · notes · users
window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all. Default all.
type
thread · profile · nt
user
Who started the thread.
search
Match against the title. Minimum 2 characters.
limit / offset
Default 20, max 100.

sort=users is exact up to a 90-day window, or all-time. sort=notes is all-time only. Both refusals are explained under what the data cannot tell you.

Returns The standard list envelope of topic objects.
GET /topics/:id

A topic and a page of its replies in posting order, each with its notes attached.

user
Only this user’s replies in the thread.
limit / offset
Default 50, max 200. See replies_meta.has_more.
Returns A topic object with replies[] and replies_meta.
07

Notes

2 endpoints

A note’s only timestamp is when the scraper saw it, so it is called observed_at rather than pretending otherwise.

GET /notes

Notes across the archive, newest first.

user
Filter by author.
reply_id
Notes on one reply.
topic_id
Notes anywhere in one thread.
days
Last N days, 1–365.
limit / offset
Default 50, max 200.
Returns The standard list envelope of note objects.
GET /notes/:id

One note, with the reply and topic it belongs to.

No parameters.
Returns A note object.
08

Profiles

2 endpoints

topics.user is the handle; profiles.name is the display name. They differ: defineipr is “Define IPR”, so search covers both.

GET /profiles

Profiles, newest first. Useful for spotting new accounts and spam waves.

search
Handle or display name. Minimum 2 characters.
limit / offset
Default 20, max 100.

A search returns a single page: the fallback path scans, and paging deep into a scan means doing it repeatedly.

Returns The standard list envelope of profile objects.
GET /profiles/:key try →

One profile. :key may be a handle or a topic id. Both are how people refer to them.

No parameters.
Returns A profile object.
09

NT · newsthread

2 endpoints

Link submissions with a thumbnail and a target URL.

GET /nt

Newsthread posts, newest first.

limit / offset
Default 20, max 100.
Returns The standard list envelope of NT objects.
GET /nt/:topic_id

One newsthread post.

No parameters.
Returns An NT object.
10

Front page

3 endpoints

Snapshotted every five minutes since 2024. 826k rows across 41k snapshots.

GET /homepage try →

The front page now, or as it stood at the end of a past date.

at
The snapshot nearest the end of this YYYY-MM-DD.
Returns { snapshot_id, scraped_at, entries: [{ position, score, notes_count, reply }] }
GET /homepage/history

Snapshots over a window. Always the same shape. The old endpoint returned two entirely different item shapes from one path.

window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all. Default 7d.
limit / offset
Default 50, max 500.
Returns { data: [{ snapshot_id, scraped_at, entry_count }], meta }
GET /homepage/replies/:id

One reply’s full front-page career: when it first appeared, its best position, how long it lasted, and every sample in between.

limit
Default 500, max 5000.
Returns { reply_id, url, first_seen, last_seen, snapshots, best_position, data[] }
11

Statistics

4 endpoints

Precomputed. Ranking the whole archive per request is what used to take minutes.

GET /stats/daily

Site-wide activity by day.

window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all. Default 30d.
Returns { window, from, to, notes_epoch, data: [{ day, replies, active_users, new_topics, new_profiles, new_nt, notes, note_users }] }
GET /stats/monthly try →

One year month by month, or one month day by day.

year
Required in practice. 2001 onward.
month
Omit for a per-month summary of the year.
Returns { year, month?, data[] }
GET /stats/years try →

Per-year totals for the whole archive. This is what the recap notebooks are built on.

No parameters.

active_users is a true distinct count per year, not a sum of daily counts. Someone active on three hundred days is one user.

Returns { notes_epoch, data: [{ year, replies, active_users, new_topics, new_profiles, new_nt, notes, first_day, last_day }] }
GET /stats/freshness

When the derived tables were last rebuilt, so a client can judge how current a rolled-up figure is.

No parameters.
Returns { rolled_up_at, backfilled_at, notes_epoch }
12

Bulk exports

2 endpoints

The whole archive is 3.9M replies and 2 GB of text: a fine download, a terrible pagination exercise.

GET /exports try →

The manifest: every file with its row count, byte size and SHA-256.

No parameters.
Returns { generated_at, format, current_year, files: [{ file, rows, bytes, sha256 }] }
GET /exports/:file try →

One gzipped NDJSON file, one JSON object per line, with each reply’s notes already attached. This is the join the Observable notebooks used to do by hand, and got wrong.

No parameters.

Closed years never change, so they are served immutable with a one-year max-age; only the current year is regenerated, weekly. That means a CDN can absorb the traffic and the origin never sees a repeat request.

Returns application/gzip
13

What this cannot tell you

Limits, stated

Some questions this archive cannot answer. Where that is true, the API refuses the request and says why, rather than returning a number that looks like a measurement and is not.

  • note timesA note’s only timestamp is when the scraper saw it, and 83,949 of 167,171 notes share one bulk-import instant (2024-04-23). Note activity is a real time series only from 2024-04-24, reported as notes_epoch. Daily note figures before it are null, never 0. Windowed note rankings over earlier dates are rejected: they would rank by when the scraper ran.
  • unique usersDistinct-user counts cannot be summed across days; someone active on thirty days is one user. /topics?sort=users is therefore exact up to 90 days, or all-time, and refuses longer windows rather than double-counting.
  • topic timesEvery topic was stamped -0800 regardless of daylight saving until 2024, so a pre-2024 topic time may be up to an hour late. The original string is passed through as date_raw.
  • freshnessScraping runs every five minutes; the rollups refresh hourly. So /overview counts today live, and everything rolled up carries rollups_current_to.
  • scoreOnly a net score is stored. There is no upvote/downvote split, and nothing indexes @user mentions. Both would need a scraper change, not an API change.