How to compare two versions of a web page and show exactly what changed

You have two copies of the same page, taken a few weeks apart, and you think a clause, a price or a claim has changed. Opening them in two windows and reading both is how people miss the one sentence that matters. A comparison tool finds every difference in seconds, but only if you give it the right input and read the output with care.

This article covers three ways to compare versions, how to deal with the changes that do not matter, and how to report what you found.

Compare text, not appearance

Most of what you care about in a page is its words: a term, a price, a promise. Compare those first.

A visual comparison of two screenshots flags every moved banner and every re-flowed paragraph. A comparison of the underlying HTML flags every script, tracking parameter and class name, none of which a reader sees. Plain text sits between the two. It holds what the page said, in order, and little else.

The cost is what text leaves out. A changed image, a price shown only in a graphic, or a layout change that hides a notice will not appear. If the question is about something visual, compare the two copies by eye as well, and keep both.

Three ways to compare

1. The Wayback Machine's Changes view

If both versions exist in the Internet Archive, you do not need to save anything. The Archive's "Changes" feature lets you select two versions of a URL and compare them side by side, with differences in text highlighted in yellow and blue. You reach it from the Calendar View page, and you can share the comparison link with others. The Archive's developer documentation gives the address pattern: https://web.archive.org/web/changes/ followed by the page's URL.

It is the quickest route for a page you did not capture yourself. Its limits are the Archive's: you cannot choose when it captures a page, and a version that lived for two days may never have been saved. We set out what its captures can and cannot show in Wayback Machine captures as evidence.

The same developer page describes a manual route. List the captures of a URL through the CDX API, pick two, and open each at http://web.archive.org/web/<timestamp>/<URL>, where the timestamp is 14 digits (YYYYMMDDhhmmss). Each listing includes a digest, which the Archive defines as the SHA-1 hash of the content, excluding headers. Two captures with the same digest hold identical content, so pick two with different digests.

2. A line-by-line diff of two text files

If you hold your own copies, save the text of each, then run the standard diff tool in unified format:

diff -u old.txt new.txt

In unified format, lines common to both files begin with a space, a + marks a line added to the second file, and a - marks a line removed from the first. By default, three lines of context are shown around each change, which is enough to find the place in the page.

Put the older file first. If you reverse them, every addition reads as a deletion, and the report you write will say the opposite of what happened.

3. A word-by-word diff

A line diff marks a whole paragraph as changed when one number inside it moved. For prose, a word diff shows the number alone. Git can do this on files that are not in a repository:

git diff --no-index --word-diff old.txt new.txt

Git's documentation says --no-index compares two paths on the filesystem and works outside a Git working tree. In --word-diff mode, removed words appear as [-removed-] and added words as {+added+}, so "£49 a month" becoming "£59 a month" reads as £[-49-]{+59+} a month. Add --color-words for colour instead of brackets. Git works by computing word changes within each line-by-line hunk, so the line diff is still your overview and the word diff your close reading.

Clean out the noise before you conclude anything

Pages change without anyone changing them. Expect these in any pair of captures:

• Dates and times, such as "Updated 12 October" or a copyright year.
• Counters and live figures: basket counts, stock levels, "47 people are viewing this".
• Rotating content: promotional banners, "recommended for you" panels, cookie notices.
• Navigation and footers, which can run to hundreds of lines and are the same on every page of the site.

A diff full of these hides the change you want. Delete the repeated navigation and footer from both text files before comparing, or look only at the section you are interested in. Keep the originals untouched and work on copies, so you can show someone the complete files if they ask.

Then ask of every difference that remains: did the page's meaning change, or only its wording? A reworded sentence and a changed obligation look the same to a diff. Reading the surrounding lines settles it, and it is the part no tool does for you.

Check the comparison itself

Before you rely on a diff, check three things.

1. Are these the right two copies? Note the address and the capture time for each. A page served at example.com/terms in one copy and example.com/uk/terms in the other will differ for reasons that have nothing to do with a change.
2. Is the order right? Older first, newer second.
3. Is each copy complete? A copy taken while the page was half loaded, or one that lost its content to a cookie notice, makes the other copy look as though it gained text. Open both and read the top and bottom before you diff them.

Write up the difference so someone can check it

A diff on your screen is not yet something you can hand to a colleague, a regulator or a court. Record:

• The address, and the date and time of each copy.
• Where each copy came from, and how you saved the text from it.
• The exact command or tool, and its version, if you used one.
• The output, unedited, and any lines you removed beforehand and why.
• Your reading of it, kept apart from the output: "the second copy removes the sentence beginning 'Refunds are available'".

A hash of each copy, taken when you saved it, lets a reader confirm they are looking at the same files you compared; what a SHA-256 hash does and does not prove is worth reading first so you do not claim more than it supports.

How Archivepanel handles it

Archivepanel captures public pages, one page at a time, without logging in. Every capture is stored with a plain-text version beside the saved page, and any two captures of an address can be compared as text.

• From a record, Compare with previous opens it against the capture of the same address taken just before. In the archive list, tick two rows and choose Compare the two. The older snapshot is always the "before", whichever order you ticked them in.
• Removed lines are marked with a minus and a red tint, added lines with a plus and a green tint, and the words that changed within an edited line are highlighted. Long unchanged stretches are folded, and the n and p keys step between changes.
• It compares text only, so changes to layout, images or scripts do not appear. Each side is compared up to its first 20,000 lines or 2 MB, and the page says when it stopped early. Snapshots saved before the text view existed have no text and cannot be compared.
• A schedule on the paid plan builds the series of captures for you, so the "before" exists when you need it. The free tier gives you 100 MB of storage and captures by hand.
• Each capture carries a DigiCert RFC 3161 timestamp, which you can check with OpenSSL. It is not an eIDAS qualified timestamp.

The documentation describes the comparison view in full.

Sources

• Internet Archive, The Wayback Machine: Fighting Digital Extinction in New Ways, 18 October 2019.
• Internet Archive Developer Portal, Compare two versions of a website.
• GNU Diffutils manual, Unified Format and Detailed Description of Unified Format.
• Git documentation, git-diff.