วิธีการทํา Recommended robots.txt

[thxf.org]

เพิ่มข้อความด้านล่างใน robots.txt

User-agent: PetalBot
User-agent: AspiegelBot
User-agent: AhrefsBot
User-agent: SemrushBot
User-agent: SemRush
User-agent: DotBot
User-agent: MauiBot
User-agent: MJ12bot
Disallow: /

User-agent: Amazonbot
Disallow: /threads/*/reply

User-agent: *
Disallow: /whats-new/
Disallow: /account/
Disallow: /members/
Disallow: /attachments/
Disallow: /goto/
Disallow: /help/
Disallow: /posts/
Disallow: /login/
Disallow: /search/
Disallow: /admin.php
Allow: /

Sitemap: https://yourforum.com/sitemap.xml

cr.

https://xenforo.com/community/threads/recommended-robots-txt.209086/