*

mocody

  • ****
  • 139 posts
  • Only Marketing matters
Google Search Console
« on: June 07, 2021, 01:52:31 PM »
@mb themes,
I want to remove these folders from the search engine.
will this be the right method.

Googlebot indexed a blocked url, I already blocked the directories, like vendor,apps etc. but it's scanning.
can we just add noindex metatag, for all the pages in those directories.



*

MB Themes

Re: Google Search Console
« Reply #1 on: June 07, 2021, 03:04:23 PM »
@mocody
You have robots.txt for that.
  To get fast support, we need following details: Detail description, URL to reproduce problem, Screenshots

*

mocody

  • ****
  • 139 posts
  • Only Marketing matters
Re: Google Search Console
« Reply #2 on: June 07, 2021, 06:43:19 PM »
@mb themes
how would you remove that from the search console.

*

MB Themes

Re: Google Search Console
« Reply #3 on: June 07, 2021, 07:09:55 PM »
@mocody
You have robots.txt for that.
  To get fast support, we need following details: Detail description, URL to reproduce problem, Screenshots

*

mocody

  • ****
  • 139 posts
  • Only Marketing matters
Re: Google Search Console
« Reply #4 on: June 07, 2021, 07:16:16 PM »

You should not use a robots.txt file as a means to hide your web pages from Google Search results. This is because, if other pages point to your page with descriptive text, your page could still be indexed without visiting the page. If you want to block your page from search results, use another method such as password protection or a noindex directive.

https://developers.google.com/search/docs/advanced/robots/intro

google bot scanned through blocked content,

*

MB Themes

Re: Google Search Console
« Reply #5 on: June 07, 2021, 07:22:30 PM »
Your question: I want to remove these folders from the search engine.
You complain: You should not use a robots.txt file as a means to hide your web pages from Google Search results.

So you want to remove pages from console, but complains that suggested steps will remove pages from console.

Does it make sense? No it does not.

You can add noindex to files in vendor folder and do it on each update as those are external libraries updated from github.
  To get fast support, we need following details: Detail description, URL to reproduce problem, Screenshots