1. Lock the queries before the first run

Every engine gets the same eight queries. Each one stands for a type of search: knowledge, how-to, health, product, person, news, comparison, and one image search. Seven are web searches, and the AI results are counted on those. The image search is there to check for AI image filters.

The set was locked on October 6, 2026, before round 1. Any later edit goes in the changelog with a date. The easiest way to fool yourself is to swap a query after seeing what it returns. With a fixed set, a change shows up as "the health query now gets an AI answer," not as a general impression.

2. Decide what counts before you look

The definition is written down and published:

• Counts: AI answers, summaries and Overviews that appear on their own; AI Mode, chat or an assistant panel; AI-generated images in the results.
• Doesn't count: machine learning used only for ranking or spam fighting, spelling correction, autocomplete, knowledge panels.
• Recorded as a caveat: an "Ask AI" button that only runs after a click.

The caveat line does a lot of work. On DuckDuckGo's no-AI address, a Duck.ai link stays in the header. On Google's Web filter, the AI Mode tab stays in the menu. Neither is an AI answer on the page, so neither changes the verdict, but both are written next to each result ("Duck.ai link in the header only", "AI Mode tab in the menu only") so a reader can disagree with the call.

3. Fix the browser state, and write it down

The default state is signed out, from a US IP address. Round 1 used Chrome 154 on a macOS desktop. The test log records the full build number, not only the major version: 154.0.8037.98 for the October 6 run, and 154.0.8037.97 for an October 14 check on Linux.

Some fields will be missing, and the log should say so. The October 14 check lists its country as "Not recorded" rather than a guessed "US".

4. Use status labels with strict definitions

Each engine gets one of four labels:

• No AI: with default settings at the tested address, no generative AI appeared on any test query.
• AI optional: AI appears by default on some queries, and an official setting turns it off.
• Workaround: no official off switch; AI can only be avoided with an unofficial method that may stop working.
• Pending test: no finished hands-on test yet, so no result is shown.

The rule that matters most: if a device wasn't tested, its cell stays pending. A result on desktop Chrome says nothing about Safari on an iPhone, so the site doesn't infer one from the other. Most of the phone steps on the site are still marked pending for that reason.

5. Record the same fields every time

For each query: whether AI appeared and what kind, whether it appeared by default, whether it can be turned off, whether the setting survives closing and reopening the browser, what happens on a phone, and a screenshot. Those fields map to the columns of the comparison table, so each cell traces back to a recorded observation.

6. Date the evidence, and keep it close

Each finished test cites a screenshot and, when a claim depends on one, an official document. Both are dated. The screenshots are hosted on the site itself, so the evidence doesn't depend on an image host staying online.

Official documents need dates too, because they change. Google's help page on these settings currently says "We're currently updating settings for Google Search services," so it gets re-read every month.

7. Label each claim by its evidence

Not every check produces a new screenshot; say which is which. The October 14 check confirmed that Chrome's address bar opened Google's Web filter with no AI Overview after the udm=14 URL was made the default. The log entry reads "No new screenshot" and points back to the October 6 screenshots.

Reports from other people go in a separate state. Users say that adding "-ai" to a Google query is no longer reliable. The site lists that with a link to the report and marks it "Pending test" until we run it ourselves.

8. Only log a change when something shows it

The AI change log has one rule: a row is added only when a dated screenshot or an official page shows the change. October 6, 2026 is the baseline row for each engine. Retests run once a month, in the first week, and again when a reader reports a change. The next one is the first week of November 2026.

What this doesn't solve

So far it's one country and one desktop browser. Phones, Firefox and Edge are pending. SearXNG results depend on the instance, and only baresearch.org was tested. Signed-in results may differ.

The full rules, the query set and both changelogs are here: How we test search engines for AI. If you run similar tests and handle any of these steps differently, I'd like to hear how.

I drafted this article with help from an AI assistant. The method, dates and log entries come from my own test records, and I checked them before publishing.