Osclass Support Forums

General osclass questions => Report bug => Topic started by: mocody on March 01, 2021, 01:14:24 PM

Title: Top Warnings - Indexed, though blocked by robots.txt
Post by: mocody on March 01, 2021, 01:14:24 PM

@mb themes

oc-admin/index.php has been indexed by google search engine.
indicates that Google indexed URLs even though they were blocked by your robots.txt

so can you please add this option  add to index / exclude from index that allows you to manage which pages can be indexed by google (search engines) (like static pages.)

or add some No-follow - inside the index.php.

whats the way forward for this.

Thanks

Title: Re: Top Warnings - Indexed, though blocked by robots.txt
Post by: MB Themes on March 01, 2021, 02:11:19 PM

@mocody
Download osclass 4.3 and check robots.txt

Title: Re: Top Warnings - Indexed, though blocked by robots.txt
Post by: Sanjay Srivastava on November 16, 2021, 12:15:01 PM

Hello ,

I am also facing same issue - Top Warnings - Indexed, though blocked by robots.txt

while I am using Osclass v4.4

Here is the robot.txt , I am using all in one seo tool.

User-agent: *
Disallow: /oc-admin/
Disallow: /oc-content/downloads/
Disallow: /oc-content/languages/
Disallow: /oc-content/uploads/HTML/
Disallow: /oc-content/uploads/minify/
Disallow: /oc-content/uploads/temp/
Disallow: /oc-includes/osclass/
Disallow: /oc-includes/vendor/

also I install plugin external links for Osclass . check the screen shot. here one option Add a rel="nofollow" attribute to outbound links. - that is checked by default. am i getting error because of this ??

Title: Re: Top Warnings - Indexed, though blocked by robots.txt
Post by: MB Themes on November 16, 2021, 01:33:21 PM

@Sanjay
I am not sure, have you identified what "Top Warnings - Indexed, though blocked by robots.txt" means and if it is issue or just notice?

Title: Re: Top Warnings - Indexed, though blocked by robots.txt
Post by: Sanjay Srivastava on November 17, 2021, 08:11:31 AM

I am getting this error  Top Warnings - Indexed, though blocked by robots.txt  in Semrush site Audit. (225 issues with blocked internal resources in robots.txt)

and here is one more error . by google search console - Submitted URL marked noindex.

Please check google search console screenshot.

   

Title: Re: Top Warnings - Indexed, though blocked by robots.txt
Post by: Sanjay Srivastava on November 17, 2021, 08:14:35 AM

Here is robot.txt error in bing webmaster tools

pls check screenshot.

Title: Re: Top Warnings - Indexed, though blocked by robots.txt
Post by: MB Themes on November 17, 2021, 04:02:39 PM

No issue there.
You a tually complain, that search engines does not crawl pages, those you do not want to index because it is duplicated content (due to sOrder parameter)

Title: Re: Top Warnings - Indexed, though blocked by robots.txt
Post by: Sanjay Srivastava on November 17, 2021, 06:22:59 PM

Means i dont have to worry about this. And i dont need to do anything.

Thanks for your reply.

Title: Re: Top Warnings - Indexed, though blocked by robots.txt
Post by: MB Themes on November 17, 2021, 06:37:30 PM

I would say so  :-\