*

Own Petz

  • ****
  • 171 posts
  • OwnPetz - Free Classified Ads. Buy, Sell or Adopt
How to stop indexing unwanted file in robot.txt
« on: December 08, 2023, 07:13:08 AM »
Hi... Thanks in Advance..

Can you please tell me How to stop indexing unwanted file in search console by robot.txt

For examble this type of file..

/item/send-friend/7053?type=comment
/item/send-friend/7053?type=friend
/user/profile/740628
item...../.../Number

« Last Edit: December 08, 2023, 07:19:48 AM by Own Petz »
OwnPetz - Free Classified Ads. Buy, Sell or Adopt Online. https://ownpetz.com

*

MB Themes

Re: How to stop indexing unwanted file in robot.txt
« Reply #1 on: December 09, 2023, 08:50:25 AM »
This path could work
Code: [Select]
/item/send-friend/
  To get fast support, we need following details: Detail description, URL to reproduce problem, Screenshots

*

Own Petz

  • ****
  • 171 posts
  • OwnPetz - Free Classified Ads. Buy, Sell or Adopt
Re: How to stop indexing unwanted file in robot.txt
« Reply #2 on: December 10, 2023, 08:03:39 AM »
Thanks for reply..

means I have to Disallow: /item/send-friend/ and Disallow: /user/profile/ (Should I have to block user profile) because many user register for backlinks.

and What for number many link is indexing with product link /number

like this https://ownpetz.com/for-sale/promotion/best-java-training-course-in-aligarh-with-uncodemy/op7159/Number

how to stop this..


And now I have robots.txt file is like below (Is this correct)

User-agent: Googlebot
User-agent: Bingbot
User-agent: Yandex
Disallow: /nogooglebot/
User-agent: *
Disallow: /oc-admin/
Disallow: /oc-content/downloads/
Disallow: /oc-content/languages/
Disallow: /oc-content/uploads/HTML/
Disallow: /oc-content/uploads/minify/
Disallow: /oc-content/uploads/temp/
Disallow: /oc-includes/osclass/
Disallow: /oc-includes/vendor/
Disallow: /item/send-friend/
Disallow: /user/profile/
Allow: /
Sitemap: https://ownpetz.com/sitemap.xml
Sitemap: https://ownpetz.com/sitemap_blog.xml

OwnPetz - Free Classified Ads. Buy, Sell or Adopt Online. https://ownpetz.com

*

MB Themes

Re: How to stop indexing unwanted file in robot.txt
« Reply #3 on: December 10, 2023, 01:09:09 PM »
You can validate robots.txt using online validators.
Looks ok.
  To get fast support, we need following details: Detail description, URL to reproduce problem, Screenshots