Why We Removed 5,210 Misleading lastmod Values from Our Sitemap

An XML sitemap should help a crawler decide which URLs may be worth revisiting. During a maintenance audit of RevealTheme, we found that our sitemap was answering a different question: when was the sitemap requested?

The live document contained 5,679 URLs at the time of the audit. For 5,210 of them, the generator supplied the current request time as lastmod, even though the underlying page had not necessarily changed. The XML was valid, but the freshness signal was not defensible.

We removed those generated dates instead of replacing them with another estimate. A live read-back on 30 August 2026 found 5,683 sitemap URLs: 473 with a persisted, valid modification date; 5,210 with no lastmod; and zero invalid lastmod values.

The failure mode

RevealTheme combines detector pages, developer tools, comparison pages, localized routes, and a Mongo-backed blog in one Next.js sitemap. The old generator created a request-time value once and attached it to static and programmatic entries:

const now = new Date().toISOString();
const urls = STATIC_ROUTES.map((route) => ({
  ...route,
  lastmod: now,
}));

Blog entries had a similar fallback. If neither updatedAt nor date was available, they received now. Invalid stored dates were also replaced with the current time. Every branch therefore produced a plausible ISO date, even when the application had no evidence that the page changed.

This is why schema validation alone did not catch the problem. A validator can confirm that a timestamp is formatted correctly. It cannot confirm that the timestamp describes a meaningful modification.

The rule we adopted

Each URL now falls into one of two practical cases:

1. A reliable content timestamp exists. Normalize it and emit lastmod.
2. No reliable timestamp exists. Omit the optional field.

For code-backed routes, we removed the blanket timestamp:

const urls = STATIC_ROUTES.map((route) => ({ ...route }));

For Mongo-backed blog posts, the entry keeps a persisted date but has no request-time fallback:

urls.push({
  loc: "/blog/" + post.permalink,
  lastmod: post.updatedAt || post.date || null,
  priority: "0.6",
  changefreq: "monthly",
});

Normalization also returns null for missing or invalid values. The XML renderer writes a lastmod line only when normalization succeeds:

function toW3CDate(value) {
  if (!value) return null;
  const date = value instanceof Date ? value : new Date(value);
  return Number.isNaN(date.getTime()) ? null : date.toISOString();
}

function lastmodLine(value) {
  const normalized = toW3CDate(value);
  return normalized
    ? "    <lastmod>" + normalized + "</lastmod>\n"
    : "";
}

The important behavior is the empty branch. When we cannot prove a modification date, we do not manufacture one.

Removing a self-request from sitemap generation

The same maintenance change updated a legacy sitemap API that was requesting blog data through the public application. The server already had direct access to the Mongo collection, so that loop introduced a network and edge layer into an internal data read.

The revised code queries the blog collection directly and projects only the fields the sitemap needs. This is not an SEO shortcut. It simply removes an avoidable dependency from a crawler-facing endpoint.

Regression checks that matter

We added an end-to-end check for a representative code-backed route, /tools. The test extracts that URL's XML block and asserts that it has no lastmod. That protects the exact semantic rule instead of merely checking that the sitemap parses.

Our verification sequence was:

1. Run lint and the production build. The recorded build generated 237 pages.
2. Read the production sitemap rather than trusting the local result.
3. Count total URL entries, entries with lastmod, entries without it, and invalid dates.
4. Inspect a representative static/tool entry and a Mongo-backed blog entry separately.
5. Fetch the sitemap again without changing content and confirm that thousands of dates do not move with the request time.

Here is the core of the read-back check:

const entries = [...xml.matchAll(/<url>([\s\S]*?)<\/url>/g)]
  .map((match) => match[1]);

const dated = entries.filter((entry) => entry.includes("<lastmod>"));
const invalid = dated.filter((entry) => {
  const value = entry.match(/<lastmod>([^<]+)<\/lastmod>/)?.[1];
  return !value || Number.isNaN(Date.parse(value));
});

What this change does not prove

We do not treat lastmod as a ranking switch. Search visibility depends on content quality, internal architecture, crawl demand, competition, and many other signals. Removing false dates improves the truthfulness and observability of the sitemap; it does not establish that rankings, clicks, or revenue will increase.

That distinction matters because a technically correct cleanup can still coincide with unrelated search movement. We will evaluate search performance only over comparable windows and will not attribute a later change to this single deployment without stronger evidence.

A practical sitemap freshness checklist

1. Does lastmod change when the page content does not?
2. Can each value be traced to content data, a versioned source, or another trustworthy record?
3. Are creation time, deployment time, request time, and modification time being confused?
4. Do stable utility pages receive a timestamp only because the sitemap was regenerated?
5. Can the generator omit the field instead of inventing a fallback?
6. Does a second fetch produce a meaningful diff?
7. Did you inspect the deployed XML, not only the build output?

The broader lesson

SEO metadata is production data. Optional fields should not be populated merely because a library makes that convenient. For RevealTheme, omitting 5,210 unsupported dates created a more truthful sitemap than generating 5,210 replacements.

For protocol details, see Google's sitemap guidance and the Sitemaps XML protocol.