Hi everyone, I'm configuring the robots.txt file for my classifieds website, but I don't really know what I should block and what I shouldn't. I'm currently using: "User-agent: *
Disallow: /" to disable all indexing.
Instead of disabling everything, I would like to disable only the following data:
- Users
- /oc-admin
- All search and filter pages (for example: /search/pattern, car)
- index.php?page=ajax&action=runhook&hook=ur_ajax_list&userId=1&itemId= (I don't know why this page was indexed)
- All sections and files of osclass that shouldn't be indexed (I don't know).
To index only the following:
- User posts
- Categories
- Locations
- Static pages (Login, Register, Contact, etc.)
I would really appreciate it if someone could help me create the file, as I don't know much about osclass.
Thanks.