next.js: Use of useSearchParams is causing the addition of meta robots with noindex in the production build

Link to the code that reproduces this issue

https://github.com/FrancisGregori/nextjs-noindex-issue

To Reproduce

Use Next.js version 13.5.6-canary.3 or higher.
Implement the useSearchParams hook in any page.
Build the application for production.
Notice that the <meta name="robots" content="noindex"/> tag is automatically added to the generated HTML, even with the correct robots configuration in place.

Current vs. Expected behavior

Current Behavior: When using useSearchParams in Next.js version 13 or higher, the <meta name="robots" content="noindex"/> tag is automatically added to pages in the production build. This occurs even if the robots meta tag is already configured manually, leading to erroneous addition and potential duplication of the robots meta tag.

Expected Behavior: The <meta name="robots" content="noindex"/> tag should only be added in preview versions, not affecting the production build. Manual configurations of the robots meta tags should be respected to avoid duplication and ensure correct indexing of pages by search engines in production.

Verify canary release

I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 23.1.0: Mon Oct  9 21:27:24 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6000
Binaries:
  Node: 18.17.0
  npm: 9.6.7
  Yarn: 1.22.19
  pnpm: N/A
Relevant Packages:
  next: 14.0.4-canary.2
  eslint-config-next: 14.0.1
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.2.2
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

Metadata (metadata, generateMetadata, next/head), Routing (next/router, next/navigation, next/link)

Additional context

I have encountered this problem both locally and when deploying on Vercel. Upon further investigation, it seems that this issue started with version 13.5.6-canary.3. This observation leads me to believe that the erroneous addition of the <meta name="robots" content="noindex"/> tag in the production build when using useSearchParams might be specific to this and subsequent versions. Prior versions, including 13.5.6-canary.2 and earlier, do not exhibit this issue.

_NEXT-1853

About this issue

Original URL
State: closed
Created 7 months ago
Reactions: 6
Comments: 28 (4 by maintainers)

Most upvoted comments

For the last 5 days I have wondered why this meta was added while everything else was working fine. This is really concerning, especially for a framework that is supposed to be SEO-friendly. Even if it’s minor, it has an impact. Don’t hesitate to let us know when this is fixed. In my case, I need to use the useSearchParams in some components. Thanks @FrancisGregori for raising this issue and making me feel less alone. I’m surprised that so few people are concerned

atrhacker on Dec 9, 2023

Thank you! We have a PR going https://github.com/vercel/next.js/pull/59531.

leerob on Dec 12, 2023

Setting the dynamic route config option to force-dynamic on the pages where you use useSearchParams() will prevent Nest.js from adding the <meta name="robots" content="noindex"/>. Basically, the page will load dynamically for each request and the query params will be accessible to the server.

@haginus, using force-dynamic in Next.js for server access to query parameters on pages with useSearchParams does have a few notable trade-offs:

Increased Server Load: It leads to server-side rendering on every request, which can gently heighten server workload and extend response times.

Caching Trade-offs: This approach bypasses some of Next.js’s caching advantages, which might slightly impact site performance.

SEO Considerations: While it avoids adding a noindex meta tag, there’s a subtle possibility that slower load times could affect SEO, but this can be managed.

In essence, force-dynamic is a helpful tool for a specific need, but it’s good to weigh its impacts on performance and SEO against your site’s overall goals.

FrancisGregori on Dec 5, 2023

I had to use useSearchParams in my project as well, had the same issue of duplicated meta tag robots with noindex value, but even removing useSearchParams from a project and deleting .next folder to rebuild the app again didn’t solve the issue.

My bad, I didn’t double check if I removed useSearchParams completly from my app, I found one more place where I used it, after removing it the issue with duplicated metatag gone. This temporary solution

  const searchParams = new URLSearchParams(
    typeof window !== 'undefined' ? window.location.search : '',
  );

Is indeed good.

ZakharPilipenko on Nov 30, 2023

@marduc812 , as a workaround I ended up removing the use of useSearchParams and using window.location.search with URLSearchParams.

  const searchParams = new URLSearchParams(
    typeof window !== 'undefined' ? window.location.search : '',
  );

Maybe this will help you temporarily.

FrancisGregori on Nov 21, 2023

Awesome @leerob! Thanks for the heads up. Much appreciated!

FrancisGregori on Dec 12, 2023

Same issue. <meta name="robots" content="noindex"/> automatically added.

prettydev on Dec 4, 2023