Web crawlers can often cause resource problems for WordPress sites. The first tool for protecting your site against web crawlers is a robots.txt file. Please review the instructions below for assistance with creating a robots.txt file to improve the WordPress site's performance.
These are the step-by-step instructions for creating a file called robots.txt in the document root for your WordPress site. The file should contain the following text:
To create the file:
- Log in to cPanel.
- Click on the File Manager icon.
- Navigate to your site's document root.
- Click on the + File icon.
- Name the file robots.txt, then click Create New File.
- Do a right-click to the robots.txt file, then select Edit.
- In the new pop-up dialog box, click Edit.
- In the new window, enter the code below, then click Save Changes.
User-agent: * Disallow: /cgi-bin/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /wp-content/ Disallow: /trackback/ Disallow: /index.php Disallow: /xmlrpc.com Disallow: /wp-login.php Disallow: /wp-content/plugins/ Disallow: /comments/feed/ User-agent: Yandex Disallow: / User-agent: Baiduspider Disallow: /
In addition to robots.txt, the Bing Network and Google Webmaster accounts offer tools to help reduce the rate that crawlers initiate requests with your site, reduce the resources, and allow for more legitimate traffic to be served.
Check Your Theme
Sometimes, the coding of a theme can cause resource issues on a WordPress site. Fortunately, this is simple to test for. Please note that this method will not work for sites that are currently access restricted. The site must be able to have traffic to test resource usage accurately.
- Familiarize yourself with how to monitor resource usage.
- Set your WordPress to a default theme.
- After several hours of usage, check your resources to see how your WordPress functions with the default theme.
- Activate your theme of choice.
- After several hours of usage, check your resources to see how the new theme has impacted the resource usage of your site.
If your theme is causing resource issues for your site, you need to contact your theme developer or select a new theme.
These changes are not guaranteed fixes but provide a starting point for correcting high load issues on WordPress sites. These recommendations are provided in order of complexity and will have varying results depending on the specific cause of the resource usage of your WordPress site.
