Manage robots.txt file in the root directory

Last modified: 2011-11-12 Key words: block crawler

More Topics	Source Code
Check page called from the main frame Creating graphics stripcslashes clear a string form and / session start and how to work with it AJAX call a function when responseText arrived open a new window from javascript check if field input have content message asking if you are sure to do something disable a link from being active using a function the first page meta section calculate date different Find the diferent between dates str_replace string strings read a file to an arrya Creat a zip file using php in the host computer START TRANSACTION COMMIT ROLLBACK stripcslashes clear a string form and / short way to write an if sentanse php function global Variable scope php function Passing by Reference array_count_values() PHP switch case Create php Requests From HTML form update a parent from javascript window dates between today redirection in php update iframe from another ifram List sort files in directory list of files with wildcard and delete the file save array to file php resize picture htmlspecialchars htmlspecialchars_decode The top of each page load common inc TAB in textarea set header to a file from external header file prepare bind_param execute error handling array sort functions image create Encoding decoding URI in PHP and Javascript html / uri conversion SUMMARY Check if the parameter is heacking AJAX send parameter to a PHP page using JavaScript Communication from a PHP server to external FTP server Check if the page is HTTPS SSL. Detect Hebrew characters in a given string and change the dir for PHP and JavaScript Go to last page PHP previous -1 Get current directory on the server file chnage modify on server Connect to MySQLi database Show files in directory include connect to db Responsive media Files list in directory JSON function return multi files upload to server save file as pdf using header page redirect in php from url location to disk real directory path unzip download to server or client remove all files in a directory recursively hebrew character set for mysql DNS get record Mail cc and bcc PHP $_REQUEST vs $_SESSION link external php file MySQL get last autoincrement ID Get the SSL certificate details using php How to handle JSON objects array in PHP Read JSON using JavaScript and PHP decoding json_decode to array or object Pass variables and data from PHP to JavaScript Write JavaScript process using php Create an Onclick event using php Read JSON object from database using JavaScript Reading MySQL table to perform JSON dynamic report in JavaScript PHP show JSON content php show url parameters An AJAX example with a call to a function when loaded Get Jerusalem time in PHP AJAX using post and form to transfer all characters to php page Get all selected values of a multiple Select Read JSON file using PHP and convert it to JavaScript JSON	in: robots.txt User-agent: dotbot Disallow: / Alternatively you can explicitly disallow all disallowed pages: User-agent: * Disallow: /~joe/junk.html Disallow: /~joe/foo.html Disallow: /~joe/bar.html User-agent: * Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /~joe/ User-agent: Google Disallow:	Check page called from the main frame Creating graphics stripcslashes clear a string form and / session start and how to work with it AJAX call a function when responseText arrived open a new window from javascript check if field input have content message asking if you are sure to do something disable a link from being active using a function the first page meta section calculate date different Find the diferent between dates str_replace string strings read a file to an arrya Creat a zip file using php in the host computer START TRANSACTION COMMIT ROLLBACK stripcslashes clear a string form and / short way to write an if sentanse php function global Variable scope php function Passing by Reference array_count_values() PHP switch case Create php Requests From HTML form update a parent from javascript window dates between today redirection in php update iframe from another ifram List sort files in directory list of files with wildcard and delete the file save array to file php resize picture htmlspecialchars htmlspecialchars_decode The top of each page load common inc TAB in textarea set header to a file from external header file prepare bind_param execute error handling array sort functions image create Encoding decoding URI in PHP and Javascript html / uri conversion SUMMARY Check if the parameter is heacking AJAX send parameter to a PHP page using JavaScript Communication from a PHP server to external FTP server Check if the page is HTTPS SSL. Detect Hebrew characters in a given string and change the dir for PHP and JavaScript Go to last page PHP previous -1 Get current directory on the server file chnage modify on server Connect to MySQLi database Show files in directory include connect to db Responsive media Files list in directory JSON function return multi files upload to server save file as pdf using header page redirect in php from url location to disk real directory path unzip download to server or client remove all files in a directory recursively hebrew character set for mysql DNS get record Mail cc and bcc PHP $_REQUEST vs $_SESSION link external php file MySQL get last autoincrement ID Get the SSL certificate details using php How to handle JSON objects array in PHP Read JSON using JavaScript and PHP decoding json_decode to array or object Pass variables and data from PHP to JavaScript Write JavaScript process using php Create an Onclick event using php Read JSON object from database using JavaScript Reading MySQL table to perform JSON dynamic report in JavaScript PHP show JSON content php show url parameters An AJAX example with a call to a function when loaded Get Jerusalem time in PHP AJAX using post and form to transfer all characters to php page Get all selected values of a multiple Select Read JSON file using PHP and convert it to JavaScript JSON

All rights reserved to Gideon Koch 1998-2021