how to make robot.txt file for my website? - Stack Of Codes

Breaking

Thursday, 25 January 2018

how to make robot.txt file for my website?

how to make robot.txt file for my website?

Robots.txt Generator


Filename: Robots.txt

CONTENTS LIKE:

User-Agent: *
Disallow:

User-Agent: Googlebot
Allow: /

User-Agent: Googlebot-Mobile
Allow: /

User-Agent: Googlebot-Image
Allow: /

User-Agent: Mediapartners-Google
Allow: /

User-Agent: Adsbot-Google
Allow: /




you can make robots.txt file from :  http://tools.seobook.com/robots-txt/generator/


1 comment:

Topics

PHP (27) CodeIgniter (22) SQL (4) Facebook (3) HTML (3) Blogger (2) Constructor (2) Destructor (2) Google (2) Aadhaar (1) Agent (1) Browser (1) CSS (1) Cakephp (1) Constants (1) How to (1) India (1) Ip address (1) JS (1) Jquery (1) Meta Tags (1) Robots (1) Scraping Data (1) escape_str (1) htaccess (1) iS mobile (1) javascript (1) mysqli (1)