PANOPTICON
API
No key · Open to any origin · v2.0.0

Panopticon
API

A read-only HTTP API over the archive of a design and creative community: 3.9 million replies, 568 thousand topics and 41 thousand front-page snapshots, scraped every five minutes since 2001. Every endpoint is a GET. There are no writes.

ENDPOINTS → CONVENTIONS
27
Endpoints
10
Groups
300/min
Rate limit per IP
01

Conventions

Base /api/v1

Base path /api/v1. Every response is JSON. Cross-origin requests are allowed from any origin, GET and OPTIONS only. That governs whether browser JavaScript may read a response; it is not authentication.

SHAPES
Lists{ data, meta }
Singlea bare object
Errors{ error, status }
TimesISO-8601 UTC
LIMITS
Rate300 / minute / IP
Codes400 · 404 · 429 · 500
Windows24h 7d 30d 90d 1y all
Health/healthz
  • meta.has_moreComputed by fetching one extra row, so paging never costs a COUNT(*) over 3.9 million replies. There is no total on unbounded lists, because there honestly cannot be a cheap one.
  • urlEvery entity carries one, pointing back at the original post.
  • Deleted contentNever appears. 250,781 replies and 145,414 topics are tombstones with empty bodies; they are gaps, not content, and are filtered in one place.
  • to is inclusivePeople say “up to the 5th” and mean it. Internally the range is half-open so it stays index-friendly.
THE LIST ENVELOPEGET /api/v1/users?sort=replies&window=7d&limit=2
{
  "data": [
    { "rank": 1, "user": "Gardener", "replies": 25, "topics": 0,
      "notes": 2, "score": 32, "active_days": 6,
      "first_active": "2026-08-14", "last_active": "2026-08-19" },
    { "rank": 2, "user": "NBQ00", "replies": 16, "topics": 0, … }
  ],
  "meta": { "limit": 2, "offset": 0, "count": 2, "has_more": true,
             "sort": "replies", "window": "7d",
             "from": "2026-08-14", "to": "2026-08-21" }
}
02

The index

10 groups
Status · 1 Users · 7 Replies · 2 Topics · 2 Notes · 2 Profiles · 2 NT · 2 Front page · 3 Statistics · 4 Bulk exports · 2
03

Status

1 endpoint

One endpoint, built to be the first request a client makes.

GET /overview try →

The state of the community in a single round trip: activity for today, the last 7 days and the last 30 days; the most active users this week; the busiest threads this week; and the current front page.

No parameters.

Today is counted live from replies, not from the rollups. The scraper runs every five minutes but the rollup job runs hourly, and “what is happening right now” is exactly the number that must not be an hour stale. The 7d/30d figures and both rankings come from the rollups; rollups_current_to says what they are current to.

Returns { generated_at, as_of, activity: { today, last_7d, last_30d }, most_active_users_7d[], busiest_topics_7d[], front_page, notes_epoch, rollups_current_to }
04

Users

7 endpoints

The ranking is why this API was rebuilt. It used to be impossible to ask.

GET /users try →

Rank users by activity over a window. This answers “what user has been the most active in the past week”. Pass search instead to look somebody up by handle or display name.

sort
replies · topics · notes · score. Default replies.
window
Time window: 24h · 7d · 30d · 90d · 1y · all. Default 30d.
from / to
Explicit range instead of a window.
search
Switches to lookup mode. Matches handle or display name.
limit / offset
Default 20, max 200.

sort=notes is refused for windows starting before the notes epoch. See what this cannot tell you.

Returns { data: [{ rank, user, replies, topics, notes, score, active_days, first_active, last_active }], meta }
GET /users/:name try →

One user: their profile if they have one, lifetime totals, and how active they have been recently. There was no such resource before.

No parameters.
Returns { user, profile, totals: { replies, topics, notes, score, active_days, first_active, last_active }, recent: { replies_7d, replies_30d, replies_365d, active_days_7d } }
GET /users/:name/replies try →

Replies by one user.

window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all
sort
recent (default) · oldest
notes
Attach each reply’s notes from the notes table. Default false.
limit / offset
Default 50, max 200.
Returns The standard list envelope of reply objects.
GET /users/:name/notes try →

Notes written by one user, each carrying the reply and topic it was left on.

limit / offset
Default 50, max 200.
Returns The standard list envelope of note objects.
GET /users/:name/topics try →

Threads started by one user. Takes every parameter /topics takes.

sort · window · limit · offset
As /topics.
Returns The standard list envelope of topic objects.
GET /users/:name/replied-topics try →

Threads one user has replied in, most recently replied first. The threads they started are /users/:name/topics.

search
Match the thread title. At least 2 characters.
limit / offset
Default 20, max 100.
Returns The standard list envelope of { id, url, title, type, replies, first_reply_at, last_reply_at }, where replies counts this user’s replies only.
GET /users/:name/timeline try →

One user’s activity as a time series. This is the chart the notebooks could not draw, because it would have meant fetching every year of the spreadsheet.

bucket
day · month (default) · year
window · from · to
Default all.
Returns { user, bucket, window, data: [{ bucket, replies, topics, notes, score }] }
05

Replies

2 endpoints

Reply bodies are HTML, as scraped. Notes are short remarks attached to a reply.

GET /replies try →

Replies across the archive.

user
Filter by author.
topic_id
Filter by thread.
window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all. Default 30d.
sort
recent (default) · oldest
notes
Attach each reply’s notes from the notes table. Default false.
limit / offset
Default 50, max 200.
Returns The standard list envelope of reply objects.
GET /replies/:id try →

One reply with its full text and all of its notes.

No parameters.
Returns A reply object with notes[] and notes_count.
06

Topics

2 endpoints

Replaces the three by-* ranking endpoints, one of which used to time out after two minutes.

GET /topics try →

Threads, filtered and sorted. Every result carries its lifetime reply, note and participant counts.

sort
recent (default) · oldest · active · replies · notes · users
window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all. Default all.
type
thread · profile · nt
user
Who started the thread.
search
Match against the title. Minimum 2 characters.
limit / offset
Default 20, max 100.

sort=users is exact up to a 90-day window, or all-time. sort=notes is all-time only. Both refusals are explained under what the data cannot tell you.

Returns The standard list envelope of topic objects.
GET /topics/:id try →

A topic and a page of its replies in posting order, each with its notes attached.

user
Only this user’s replies in the thread.
limit / offset
Default 50, max 200. See replies_meta.has_more.
Returns A topic object with replies[] and replies_meta.
07

Notes

2 endpoints

A note’s only timestamp is when the scraper saw it, so it is called observed_at rather than pretending otherwise.

GET /notes try →

Notes across the archive, newest first.

user
Filter by author.
reply_id
Notes on one reply.
topic_id
Notes anywhere in one thread.
days
Last N days, 1–365.
limit / offset
Default 50, max 200.
Returns The standard list envelope of note objects.
GET /notes/:id try →

One note, with the reply and topic it belongs to.

No parameters.
Returns A note object.
08

Profiles

2 endpoints

topics.user is the handle; profiles.name is the display name. They differ: defineipr is “Define IPR”, so search covers both.

GET /profiles try →

Profiles, newest first. Useful for spotting new accounts and spam waves.

search
Handle or display name. Minimum 2 characters.
limit / offset
Default 20, max 100.

A search returns a single page: the fallback path scans, and paging deep into a scan means doing it repeatedly.

Returns The standard list envelope of profile objects.
GET /profiles/:key try →

One profile. :key may be a handle or a topic id. Both are how people refer to them.

No parameters.
Returns A profile object.
09

NT · newsthread

2 endpoints

Link submissions with a thumbnail and a target URL.

GET /nt try →

Newsthread posts, newest first.

limit / offset
Default 20, max 100.
Returns The standard list envelope of NT objects.
GET /nt/:topic_id try →

One newsthread post.

No parameters.
Returns An NT object.
10

Front page

3 endpoints

Snapshotted every five minutes since 2024. 826k rows across 41k snapshots.

GET /homepage try →

The front page now, or as it stood at the end of a past date.

at
The snapshot nearest the end of this YYYY-MM-DD.
Returns { snapshot_id, scraped_at, entries: [{ position, score, notes_count, reply }] }
GET /homepage/history try →

Snapshots over a window. Always the same shape. The old endpoint returned two entirely different item shapes from one path.

window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all. Default 7d.
limit / offset
Default 50, max 500.
Returns { data: [{ snapshot_id, scraped_at, entry_count }], meta }
GET /homepage/replies/:id try →

One reply’s full front-page career: when it first appeared, its best position, how long it lasted, and every sample in between.

limit
Default 500, max 5000.
Returns { reply_id, url, first_seen, last_seen, snapshots, best_position, data[] }
11

Statistics

4 endpoints

Precomputed. Ranking the whole archive per request is what used to take minutes.

GET /stats/daily try →

Site-wide activity by day.

window · from · to
Time window: 24h · 7d · 30d · 90d · 1y · all. Default 30d.

Every day in the window is returned, including days nothing happened — those come back as zeros, not as missing rows. A quiet day is a real observation: the scraper polls every five minutes. Days outside the range the archive covers are never invented, so a window running into the future stops at the last rolled-up day. notes and note_users stay null before notes_epoch, where a count would not mean anything.

Returns { window, from, to, notes_epoch, data: [{ day, replies, active_users, new_topics, new_profiles, new_nt, notes, note_users }] }
GET /stats/monthly try →

One year month by month, or one month day by day.

year
Required in practice. 2001 onward.
month
Omit for a per-month summary of the year.

With month, the day-by-day series covers the whole calendar month, quiet days included, on the same terms as /stats/daily.

Returns { year, month?, data[] }
GET /stats/years try →

Per-year totals for the whole archive. This is what the recap notebooks are built on.

No parameters.

active_users is a true distinct count per year, not a sum of daily counts. Someone active on three hundred days is one user.

Returns { notes_epoch, data: [{ year, replies, active_users, new_topics, new_profiles, new_nt, notes, first_day, last_day }] }
GET /stats/freshness try →

When the derived tables were last rebuilt, so a client can judge how current a rolled-up figure is.

No parameters.
Returns { rolled_up_at, backfilled_at, notes_epoch }
12

Bulk exports

2 endpoints

The whole archive is 3.9M replies and 2 GB of text: a fine download, a terrible pagination exercise.

GET /exports try →

The manifest: every file with its row count, byte size and SHA-256.

No parameters.
Returns { generated_at, format, current_year, files: [{ file, rows, bytes, sha256 }] }
GET /exports/:file try →

One gzipped NDJSON file, one JSON object per line, with each reply’s notes already attached. This is the join the Observable notebooks used to do by hand, and got wrong.

No parameters.

A closed year is settled but not fixed. No new replies are posted to 2008, but a note can be attached to a 2008 reply at any time, and a year file carries its replies’ notes inline — replies from 2003, 2008 and 2009 have all gained notes recently. So a year is regenerated whenever it gains notes, not just when it is the current year. Closed years are served with a one-day max-age and revalidation rather than immutable: a CDN still absorbs nearly all of the traffic, because a revalidation costs a 304 rather than the body. Compare the sha256 in /exports rather than assuming a year you already hold is current.

Returns application/gzip
13

What this cannot tell you

Limits, stated

Some questions this archive cannot answer. Where that is true, the API refuses the request and says why, rather than returning a number that looks like a measurement and is not.

  • note timesA note has no timestamp of its own — the live site never exposes when one was written. The only time a note carries is observed_at, when the scraper saw it, and for most of the archive that is null: of ~2.97M notes, ~2.8M were recovered from an old column that recorded the note but not when it was read. source says which — only live notes were watched arriving. So note coverage runs the whole archive back to 2002, but note activity over time does not: it is a real series only from 2024-04-24, reported as notes_epoch, and counts live observations alone. Daily note figures before it are null, never 0. Windowed note rankings over earlier dates are rejected: they would rank by when the scraper ran.
  • note order and markupposition is where a note stood when we first saw it, not where it stands now — notes deleted from the live site are kept, because they exist nowhere else, so a reply’s stored notes can outnumber what the live site shows. html_comment is null for notes recovered from the old column: their markup was never captured, only their text.
  • unique usersDistinct-user counts cannot be summed across days; someone active on thirty days is one user. /topics?sort=users is therefore exact up to 90 days, or all-time, and refuses longer windows rather than double-counting.
  • topic timesEvery topic was stamped -0800 regardless of daylight saving until 2024, so a pre-2024 topic time may be up to an hour late. The original string is passed through as date_raw.
  • freshnessScraping runs every five minutes; the rollups refresh hourly. So /overview counts today live, and everything rolled up carries rollups_current_to.
  • scoreOnly a net score is stored. There is no upvote/downvote split, and nothing indexes @user mentions. Both would need a scraper change, not an API change.
14

Source

Read the code

This API is routing and nothing else. Every SQL statement it runs lives in one shared package, published beside it, so any number returned here can be traced to the query that produced it.