เพิ่มข้อความด้านล่างใน robots.txt
cr.
Code:
User-agent: PetalBot
User-agent: AspiegelBot
User-agent: AhrefsBot
User-agent: SemrushBot
User-agent: SemRush
User-agent: DotBot
User-agent: MauiBot
User-agent: MJ12bot
Disallow: /
User-agent: Amazonbot
Disallow: /threads/*/reply
User-agent: *
Disallow: /whats-new/
Disallow: /account/
Disallow: /members/
Disallow: /attachments/
Disallow: /goto/
Disallow: /help/
Disallow: /posts/
Disallow: /login/
Disallow: /search/
Disallow: /admin.php
Allow: /
Sitemap: https://yourforum.com/sitemap.xml
cr.
XF 2.2 - Recommended robots.txt
Hello, I've wanted ask, what nowadays is recommended to setup robots.txt for XF2.2 installation. Any recommendations are very appreciated, thanks!
xenforo.com