How to preload your page cache using this handy little script.
It’s great for any scenario where you need automated cache-warming:
- Sites using cache without a built-in cache prebuild function.
- Large sites (with many pages) and want to prebuild more often.
- Server admins cache warming for many sites.
- Those of you on restricted webhosts using proprietary caching (Kinsta, WPEngine, etc) can definitely use this to precache your sites.
The only requirements are that you have server access or cron jobs, and that your site has an XML sitemap.
Optimus Cache Prime (by Patrick Mylund Nielson)
Before we start, let’s give a special thanks to Patrick for writing this awesome script many years ago. Also a special thanks to Fabio Tielen of https://codeagency.be/ for answering n00b questions I had.
OCP basically crawls all the pages in your XML sitemap to prebuild your cache. This helps avoid the slow “uncached hits” effect on sites without cache prebuild function, or sites with too many pages to prebuild quick enough.
Installing and testing OCP:
- Download OCP from https://patrickmn.com/projects/ocp/
- Extract on your server. (Can be root for self-managed servers, and user directory for shared servers.)
- Test it with
/path/to/ocp -v https://yoursite.com/sitemap.xmland see if verbose output is correct.
- If working, you can set a cron job for it to
- If you’re curious, try
tail -f /var/log/cronto see if it’s runs.
I was using CentOS 7, WHM/cPanel. Fabio was on Ubuntu and using something like
cd /var/www/ocp/; ./ocp -v=true https://yoursite.com/sitemap.xml but I found it unnecessary that way.
FYI: the commands above are only possible for those with SSH access to the server. If you don’t have SSH access, you can still use this script as a cron job.
Running OCP as cron job:
Sure, you can simply cron the command above but it’s not recommended for large sites that don’t finish pre-caching before the next cron interval. I like to do a check every 5 or 10 minutes but some sites might take an hour or more to pre-cache, which means you run the risk of multiple cron jobs hitting the same script.
So to get around this, we use one.sh (another neat little script by Patrick). It will run commands but first checks to make sure they’re not already running…which is ideal for triggering frequent cron runs without the risk of multiple executions. It’s super handy not only for OCP but other scripts as well.)
- Download one.sh from https://patrickmn.com/projects/one/
- Extract on your server. (I like to put it in the same OCP directory.)
- Set a cron job for
/path/to/one.sh /path/to/ocp https://yoursite.com/sitemap.xml
- If your script doesn’t run, make sure you have the right path and also proper permissions (755).
- You should read the documentation to see the different commands available.
- I listed the commands using HTTPS domain path to make this guide easier for non-Linux people, but large sites should go the local file path (“/path/to/sitemap.xml”) for better performance since it negates DNS lookup and HTTPS handshake.
- To prevent high server load, use the
-coption to limit how many pages are crawled at once. It will look something like
ocp -c 5 https://yoursite.com/sitemap.xmlwhich means 5 pages at a time.
- Thoughts on running cron as root.