Google wants link-shared pages out of its index. Which user-created pages to noindex, and how

KEY TAKEAWAYS

1. Google now says pages made for sharing, collaboration or utility should be out of the index by default. Pages made for discovery can stay in.
2. Use noindex in the page or the response header and keep the URL crawlable. A robots.txt block stops Google reading the noindex.
3. Fix already indexed share pages in order. Remove any robots.txt block, add noindex, hide the prefix with the Removals tool, then wait for recrawls.
4. Alert from your own creation and log data. The Page Indexing report has gone 10 to 18 days without new data this year.

On 26 October 2026 Google rewrote part of its guide to preventing user-generated spam. The guide used to be about comment forms and free hosting. It now also covers tools that let users publish shared pages, generated content or web apps. A new section, titled "Only index content meant for Search", sets a default: pages made for direct sharing, collaboration or utility should stay out of the index.

That reaches far past forums. AI chat share links, generated reports, public app previews, quote and proposal pages, saved carts and shared dashboards all fit the description. The coverage so far reports the change and one technical caveat. This piece covers the decision underneath it: which of your page types belong in Search, how to switch the rest off without making things worse, and what to watch while Google catches up.

What did Google change on 26 October?

Three things, all in one help page whose previous version dated from 10 December 2025, according to Search Engine Journal and Relevant Audience.

• Scope. The opening now names hosting platforms and tools that publish shared pages, generated content or web apps. Google also widened the unit of abuse from whole sites to single pages as well.
• A default. Services should decide whether user-made pages are meant for Search or only for people holding the link. Pages for sharing, collaboration or utility should be blocked from indexing by default, through a robots meta tag, an X-Robots-Tag header or a robots.txt rule.
• Monitoring. Verify the site or subdomain in Search Console so Google can send manual action and security alerts. Watch the Page Indexing and Performance reports for spikes. Track creation and indexing at the platform level, and alert when new public pages surge or a small group of users produces a large share of indexable URLs.

The changelog entry describes the aim as keeping user-created pages that are not meant for Search out of the index and catching spam earlier. It gives no thresholds and names no companies.

Is this a penalty? What the text supports

No new penalty is described. This is a help document update, not a change to the spam policies, and none of the coverage found an enforcement date. Marie Haynes and Lily Ray both raised the possibility of action against AI tools and app builders on X, and both framed it as a question or speculation.

The existing rules matter more than the speculation. Google's manual actions report already includes a spammy free host action, which can apply to a whole hosting service when a significant fraction of its pages are spam. That is the real exposure for a platform that lets anyone publish on its domain. By contrast, sites built for user content, such as forums and comment sections, are explicitly outside the site reputation policy, which we covered in the EEA split piece.

The privacy history explains why Google wrote this down. In July 2025 Fast Company found nearly 4,500 ChatGPT conversations in a Google site search, and OpenAI pulled its opt-in discoverability setting a day later. In August 2025 Forbes reported that a Google search showed more than 370,000 Grok conversations. Neither count says how many pages were truly indexed, only what a site search returned.

Robots.txt or noindex? Google's own pages disagree

The new section lists robots.txt as an acceptable way to keep share pages out. Google's other documentation says otherwise, and the other documentation is right for this use.

• Google's robots.txt guide says the file is used to manage crawl traffic and is not a way to keep a page out of Google. A blocked URL can still be indexed, without a description, when other pages link to it.
• The noindex guide says the rule only works if robots.txt does not block the page, because the crawler never sees it.
• The content controls page, updated on 21 October, lists robots.txt for images and video only, and noindex for all content types.
• John Mueller gave the same advice in 2018: use noindex instead of disallowing.

Share links are built to be pasted into chats, emails and forums. That is exactly the case where a robots.txt block fails, because the links Google needs to index a bare URL are guaranteed to exist. Search Engine Journal's Matt Southern found Claude's share path both disallowed in robots.txt and sending a noindex-equivalent header in an earlier test, a pairing in which the header cannot be read. He could not establish what Google saw.

The verdict: send noindex in the page or the X-Robots-Tag header, and keep the URL crawlable. Save robots.txt for crawl load and media files. If you also want the page gone after a date, Google supports unavailable_after in the same tag or header. If the content is private, put it behind a login, which Google's content controls page treats as the right tool for confidential material. We cover the crawler side of robots.txt in which AI crawlers to allow or block.

Which pages are meant for Search? A triage by page type

Google leaves the line to each platform. The useful test is who the page was made for. If it exists so one person can send it to another, it is not meant for Search. If it exists so strangers can find it, it may be.

The last two rows are where the counter-argument lives. Pierre-Jean Hillion argued in February 2025 that user-made templates and community pages can be a strong acquisition loop for software products. That case holds, and nothing in Google's guide forbids it. What changes is the default. Notion offers one working model: a published site gets a separate "Discoverable on the web" toggle, and the owner has to turn it on. Google's older advice in the same guide, noindex for new users until they earn reputation, is still there and fits on top.

Already indexed: how to switch without making it worse

Most platforms reading this have share pages in Google now. Order matters, because the obvious first move, a robots.txt disallow, locks the problem in.

1. List every route that creates a public URL from user input. Include generated PDFs and images, which need the header form, since they have no HTML head.
2. Verify a domain property in Search Console. It covers every subdomain and protocol, so user subdomains are not missed.
3. Remove any robots.txt rule covering share paths. Google cannot drop what it cannot recrawl.
4. Send noindex on every non-search route. X-Robots-Tag: noindex works for HTML and files alike. Google treats none as noindex plus nofollow.
5. Hide the worst prefixes with the Removals tool. A temporary removal can cover a whole URL prefix and lasts about six months. It only affects Google and only buys time.
6. Wait for recrawls, then confirm. Google warns rarely visited pages can take months. Check samples with URL Inspection, and look for "Indexed, though blocked by robots.txt", which Google says to fix by removing the block and using noindex.
7. Build opt-in for the pages that should rank. Tie it to a review or reputation rule, not a checkbox alone.

Why the Page Indexing report cannot be your alarm

Google tells platforms to watch the Page Indexing report for spikes. The report has not been reliable enough to alert from. Search Engine Roundtable logged three stretches with no new data this summer: 13 to 30 June, 1 to 10 July and 11 to 24 July, per Brodie Clark's count. In a Reddit thread Search Engine Roundtable covered on 27 October, a site owner said their report had not updated since 16 October. Mueller replied that indexing changes slowly and shows up in the Performance reports first. When the thread was covered, his advice on chasing report freshness was "save your energy".

Both points are fair. The report is fine for direction and slow for detection. Its examples table also stops at 1,000 rows, which is small for a platform. So the alerts Google recommends have to run on your own data: daily count of new public pages, share of indexable URLs created by the top 1% of accounts, and Googlebot hits on share paths in server logs. Search Console then confirms what your logs already showed.

A worked example with invented numbers

This is hypothetical. A design tool has 200,000 public share links, a site: search returns about 20,000 of them, and robots.txt disallows /s/. It also has 5,000 reviewed templates it wants to rank.

The team removes the disallow, sends X-Robots-Tag: noindex on /s/, and files a prefix removal for /s/. Templates move to a curated /templates/ hub with no noindex. Accounts under 30 days old publish templates as noindex until a reviewer approves them. Logs then show Googlebot re-fetching share URLs and reading the header. The prefix removal covers the gap while those recrawls run. The result to check is Google showing the templates and not the share links, not the size of any single report.

If your content lives on someone else's share pages

This also runs in reverse. Teams sometimes rank a vendor-hosted page, such as a published doc, a generated microsite or a link-in-bio page, because it was quick. If that vendor adopts the new default, those pages can drop out without anyone on your side changing a thing. DuckDuckGo shows where products are heading. Its Duck.ai share pages, launched on 7 October 2026, send noindex in the page and the header and expire after 48 hours by default.

Anything you want found should live on your own domain, inside your own internal links. Our piece on the UGC Fresh Data Program covers the other side: user content that Google does want, and how it gets in.

What is still unknown

Google has not said whether it will act on platforms that ignore the default, how many indexed share pages counts as too many, or whether its own app builders follow the guide. Watch the spam policies page and the manual actions list, not the help guide, for any change with consequences.

If you want a check of which of your URLs are in Google and in AI answers, join the waitlist or write to esh@ethosengine.io.

Questions

Is Google penalising platforms whose share pages are indexed?

Nothing in the update says so. It is a help document change, not a spam policy change, and it names no products, dates or thresholds. The existing manual actions for user-generated spam and spammy free hosts still apply, and the second can cover a whole hosting service.

Should I block share links in robots.txt or use noindex?

Use noindex and keep the URL crawlable. Google's robots.txt guide says the file is not a way to keep a page out of Google, and a blocked URL that is linked elsewhere can still be indexed without a description. Google's own content controls page limits robots.txt to images and video.

How long does it take for noindexed share pages to drop out?

Each URL has to be recrawled first, and Google says that can take months for pages it rarely visits. The Removals tool hides URLs or a whole prefix for about six months while that happens, but it is temporary and only affects Google.

Does this mean user-generated pages are bad for SEO?

No. The default covers pages made to be sent to one person. Pages built for discovery, such as a curated template gallery, can still be indexed. The safer model is an explicit opt-in for indexing, backed by a quality or reputation check before the noindex comes off.

Sources

1. Google Search Central, Prevent user-generated spam on your site and platform developers.google.com
2. Google Search Central, Latest Google Search documentation updates developers.google.com
3. Google Search Central, Block Search indexing with noindex developers.google.com
4. Google Search Central, Introduction to robots.txt developers.google.com
5. Google Search Central, Control what you share with Google Search developers.google.com
6. Google Search Central, Robots meta tag, data-nosnippet, and X-Robots-Tag specifications developers.google.com
7. Google Search Central, Spam policies for Google web search developers.google.com
8. Search Console Help, Manual actions report support.google.com
9. Search Console Help, Removals and SafeSearch reports tool support.google.com
10. Search Console Help, Page indexing report support.google.com
11. Search Console Help, Add a website property to Search Console support.google.com
12. Search Engine Journal, Google updates spam guide for shared pages, generated content and apps searchenginejournal.com
13. Search Engine Roundtable, Google updates user-generated spam guidance to specify tools seroundtable.com
14. Search Engine Roundtable, Google downplays the importance of fresh index reporting seroundtable.com
15. Search Engine Roundtable, Google Page Indexing report delays providing static data seroundtable.com
16. PPC Land, Google tells platforms to noindex link-shared pages by default ppc.land
17. PPC Land, Duck.ai share links expire after 48 hours by default ppc.land
18. Relevant Audience, Google noindex link-shared pages spam guide relevantaudience.com
19. Fast Company, Google is indexing ChatGPT conversations fastcompany.com
20. Search Engine Journal, OpenAI is pulling shared ChatGPT chats from Google search searchenginejournal.com
21. Forbes Australia, xAI published hundreds of thousands of Grok chatbot conversations forbes.com.au
22. Search Engine Journal, Google: pages blocked by robots.txt will get indexed if they are linked to searchenginejournal.com
23. Notion Help, Publish a Notion Site notion.com
24. The Growth Mind, The new SEO playbook. UGC-driven assets thegrowthmind.substack.com

NEXT STEP

See where your brand stands.

Your Digital Authority Score, who AI recommends for your target questions, and the placements that close each gap. Your first report is free.

Get your free report →