Archivepanel Web Preservation System

Documentation

The application in ten sections · The dashboard lives at dashboard.archivepanel.com

The short version. Paste an address, and a real browser saves that page as one self-contained HTML file in your account — assets embedded, scripts stripped. A preview, a plain-text rendering, a WARC container and a JSON manifest are stored alongside it. Nothing is ever rewritten afterwards, and nothing is readable except through your own signed-in session.

01Getting started

Create an account at dashboard.archivepanel.com/register. Your email address is your account identifier — there are no usernames. Registration finishes by entering a six-digit code sent to that address, so an address that cannot receive mail cannot complete sign-up.

Every later sign-in is the same two steps: password, then a fresh emailed code. A proof-of-work check runs on both forms and solves itself as the page loads; there is nothing to click and no puzzle to read.

Once you are in, the Overview carries the capture box. Paste an address, press capture, and the first snapshot is on its way.

02Capturing a page

The address is checked before anything is fetched: anything resolving to loopback, private, link-local, carrier-grade NAT or unique-local space is refused, and every redirect hop is checked again.

Capture then runs away from your request, one page at a time. A capture that arrives while another is running waits its turn, so a snapshot is queued before it is running. Most pages take a minute or two.

StatusMeans
queuedAccepted and waiting for the capture worker.
runningA browser is on the page now.
completeStored. This is the only status with files behind it.
failedSomething went wrong; the record says what.
canceledYou stopped it. Not a failure, and shown in grey rather than red.

While it runs

The progress card reports the stage in plain words — opening the page, waiting for it to finish loading, saving the text, images and layout, finishing the snapshot — with a bar estimated from elapsed time. The estimate is capped below 100% until the record actually says the capture is stored, because the underlying tool reports nothing until it is done.

You do not have to wait. Close the tab and a notification lands in the masthead tray with an unread count when the capture is stored, fails, or is stopped by a storage limit; if you happen to be looking at the dashboard it also arrives as a toast.

Stopping a capture

A queued or running capture can be stopped from the progress card, from the Actions column of the archive list, or from its record page. Stopping kills the browser immediately and writes nothing to storage, so a capture you abandon costs you no quota. It is recorded in the activity log, raises no failure alert, and sends no email.

Snapshots are immutable. Nothing ever rewrites one. Capture again files a new snapshot beside the old, exactly as a schedule does, and no retry or duplicate submission can replace stored bytes. Deleting is the only way an archive leaves your account.

03What a snapshot is

Capture drives a real Chromium at a 1440×900 viewport, waits for the page to settle, and writes one document with every stylesheet, image and font embedded as a data URI and unused CSS pruned away. Scripting is stripped, and a content policy is embedded in the saved file so an opened archive cannot reach the network wherever it ends up. Video and audio are not captured.

Where a cookie banner is recognised it is dismissed before the page is saved, so the snapshot is the page rather than the overlay — and the record says what was done about it.

Five files are stored per capture:

FileMade fromWhat it is for
.html The live page The archive itself. Self-contained, script-free, opens offline anywhere.
.jpg The saved file Verification and recognition. A screenshot of the archive, never of the live page — so a capture that lost its styling looks wrong on day one rather than in year three.
.txt The saved file Reading, quoting, grepping, diffing. Headings underlined, lists bulleted, tables drawn as an ASCII grid, link targets printed beside their text.
.warc.gz All of the above Interchange. WARC 1.1, for tools that read nothing else.
.json All of the above Audit and inventory. A size, a SHA-256 digest and a content type for every file, plus the addresses, times and tool versions behind the capture.
Four of the five are companions, not the archive. If the preview, the text, the WARC or the manifest cannot be produced, the capture is still kept and the record says so. Nothing is discarded because an accessory did not render.

04The archive record

Each snapshot has a record page. It carries what was asked for and where the browser actually ended up — the two differ whenever a redirect happened — along with the title, the capture time, the stored size, and whether you asked for it or a schedule did.

Two further sections describe the page and the moment it was taken:

  • What the page declared about itself — language, description, the canonical address it names, author and published date, where it states them.
  • How the host resolved at that moment — the addresses it answered on, the CNAME it resolves through (often the only visible sign of a CDN), the authoritative nameservers, and the TLS certificate it served.

From there you can view the archive, download it, read the text rendering, take the WARC or the JSON manifest, capture the address again, or delete the record.

Nothing is public. Every file is streamed out of private storage behind your session, and captured markup is served pinned to an opaque origin with scripts and forms refused. A link to an archive is worthless to a browser that is not signed in to the account that owns it.

05Finding things again

Archives

The archive list searches title, address and host a word at a time: every word you type must appear somewhere, in any order, so nos italie finds the article about Italy on nos.nl. Filter by status, and sort by newest, oldest, largest, title or site.

Sites

Sites groups every snapshot you hold by host, which is the view to use when you want the history of one place rather than one page. Searching inside a site goes further than the list does: it reads the stored text of that site's snapshots and shows you the passage that matched, so you can find a page by something it said rather than by its title.

That scan is bounded — the newest 200 snapshots of the site, within a few seconds — and the page tells you when it did not cover everything rather than implying it did.

Text view

The text rendering opens in the dashboard with a find box that matches as you type, highlights every occurrence and steps between them with Enter and Shift+Enter. It runs entirely in your browser over text the page already holds: no request per keystroke, and what you search an archived document for is never sent anywhere. Very long snapshots are shown up to 512 KB, with the whole file a download away.

Activity log

The activity log records what happened in the account — captures queued, stored, stopped and deleted, schedules changed, sign-ins. It is where a question about a missing snapshot gets answered.

06Scheduled captures

A schedule captures on its own, and comes in two kinds. A page schedule captures its own address, one snapshot per run. A sitemap schedule reads its address as an XML sitemap and captures the pages that sitemap lists — so one schedule covers a whole section of a site, and keeps covering it as the site adds pages.

CadenceFires
HourlyEvery hour, at the minute you choose.
DailyEvery day, at each hour you list — 6,12,18 is three captures a day.
WeeklyOn the weekdays you pick, at each hour listed.
MonthlyOn the days of the month you pick, at each hour listed.

Hours are read in the schedule's own time zone, chosen by name. A day of the month past the end of a short month is clamped to its last day, so a schedule set for the 31st still fires in February. A schedule lands on the same wall-clock time every period rather than drifting, and one that could not run does not owe the runs it missed — it resumes at its next occurrence. Editing a schedule moves its next run to the first time the new rule matches.

Schedules can be paused, run once on demand, edited or deleted at any time.

Sitemap runs

Both sitemaps.org document types are understood: a urlset listing pages, and a sitemapindex pointing at further sitemaps, which is followed one level down. The gzipped form is decompressed transparently, detected by content rather than by file extension. Every address a sitemap names gets the same screening a manual capture gets, before it is fetched or queued.

BoundValueWhy
Pages per run20 default, 100 maxSet per schedule when you create it.
Sitemap files read12An index pointing at hundreds of children costs twelve requests, not hundreds.
Document size12 MBAfter decompression.
Whole traversal45 sA sitemap that will not resolve does not hold a worker.
Outstanding captures5A run is skipped rather than queued behind itself.
Failure is reported once per run, not once per page. A sitemap that cannot be read raises one alert and one email. Dozens of messages for one event is not a notice, it is a flood.

07Getting your data out

A snapshot taken here is not trapped here. Four of the five stored files are yours to take at any time, from the record page.

Document
One .html file. Open it in any browser, on any machine, with no network at all.
Text
The plain-text rendering, for quoting, grepping and diffing one capture against the next.
WARC
ISO 28500, WARC 1.1. Opens in pywb, replays through ReplayWeb.page, and satisfies an institution that accepts nothing else. One download carries the document, the text and the preview together.
Manifest
metadata.json — a digest and a size for every stored file, plus the addresses, times, viewport and tool versions. An inventory or an integrity check can be answered from it without opening the snapshot.

The WARC states plainly what it can and cannot claim: the response headers in it are reconstructed, because capture drives a browser and returns one document with its subresources already inlined — the individual HTTP exchanges that built the page are never observed. The container says so in its own metadata rather than leaving a later reader to assume a crawler produced it.

Older snapshots may hold fewer files. An archive is never rewritten, so a capture taken before a companion file existed does not gain one — its record simply says "none". Capturing the address again produces a snapshot that has it.

08Storage and deletion

Each account has a snapshot storage allowance, 50 MB by default. Usage counts every file a capture stores, since all of them occupy the bucket, and your current usage is on the dashboard rather than buried in a settings page.

  • Queuing is refused up front when the account is already full — a courtesy, since a page's size is unknown until it has been captured.
  • The binding check is immediately before upload. A capture that would cross the line is failed with an explanation and never written.
  • Scheduled captures are skipped while an account is over its limit and resume on their own once space is freed. The schedule is not disabled behind your back.

Deleting an archive is the only way to free space, and the only irreversible action in the product: it takes your password again, is rate limited, and removes every stored file belonging to that snapshot. Deleting one that is still running stops the capture first. Need more room? Allowances are set per account — write to us.

Two larger versions of the same action are in Settings. Erase stored data empties the account — every snapshot, every schedule, every alert — and keeps the account itself, your address, your password and the activity log, so the record of who has been in the account is not quietly erased along with its contents. Close account removes the account outright. Both take your password, closing also takes your address typed out in full, and a capture still running is stopped before either proceeds.

09Account and security

Settings shows your storage usage, lets you change your password, and lists the sessions currently signed in to the account so you can revoke the ones that are not you. It is also where you erase the account's stored data or close the account altogether.

  • Two-step sign-in on every account — password, then a six-digit emailed code. Only the code's digest is stored, five attempts per challenge.
  • argon2id password hashing at 64 MB with a unique salt; the password itself is never written down.
  • Session cookies hold a 256-bit token and only its digest is stored, so a copy of the database yields no usable sessions.
  • Every archive query is scoped to the owning account, and captured markup can neither read a cookie here nor reach the network.

The security section of the feature detail covers the threat model in full — what the capturer will and will not fetch, and what an archived page is allowed to do when you open it. What we store, how long, and the rights you have over it are in the Privacy Policy.

10Limits and defaults

Everything in one table. These are the defaults on the hosted service; the per-account allowance is the one that moves.

LimitValue
Storage per account50 MB
Captures at once1
Time allowed for one capture150 s
Capture viewport1440 × 900
Largest document stored48 MB
Sitemap pages per run20 default, 100 max
Site text searchnewest 200 snapshots
Text shown in the page512 KB
Two-factor attempts5 per code

Anything not covered here: support@archivepanel.com.