To enable our crawlers to access the website you want to monitor, you may need to whitelist the User Agent (UA) depending on your Cloudflare settings.
We recommend whitelisting the Hexowatch User Agent to prevent blocking your IP or any other unwanted situation while monitoring websites.
This will allow the crawlers to access the site and prevent false incident alerts from being generated by Cloudflare. It is important to take this step to ensure proper archiving of the site.
Below, we will demonstrate the steps to whitelist the Hexowatch User Agent in Cloudflare.
Step 1: Log in to the Cloudflare account and sellect the Security tab from the left menu.
Step 2: Next, select WAF. It will show the Web Application Firewall security settings.
Step 3: Then, navigate to the Firewall rules tab.
Step 4: Click on the Create firewall Rule.
Step 5: The next step is giving a name to the Firewall rule.
Step 6: Set Field to User Agent, Contains as the Operator, and Value to Hexowatch
Or
Set Field to User Agent, Equals as the Operator, and Value to:
Mozilla/5.0 (X11; Linux i686; rv:78.0) Gecko/20100101 Firefox/84.0 | Hexowatch
Then, turn on the Allow.
Step 7: As a final step, click Deploy in the bottom-right corner after completing the form.
Automate page change monitoring with Hexowatch
Hexowatch is your AI sidekick to monitor any website for visual, content, source code, technology, availability, or price changes.
Hexowatch works 24/7 to help spot trends, spy on your competitors, visually check your website, keep an archive of every change and turn any website into your own private data source accessing changes as a downloadable CSV file, google sheets, or via Zapier.
Monitor any website and get started in minutes – no software, proxies, or programming required
