Website Crawling Requirements for Partners

As a partner, you may need to take certain steps such as configuring User-Agent identification to ensure that impact.com Regulated Compliance crawlers can programmatically access and retrieve data from your website without interruption and with proper authorization. This guide is primarily intended for developers, IT administrators, and technical teams responsible for managing website access and security settings.

How to allow impact.com’s Regulated Compliance Bot

As a partner, your website may be monitored by impact.com’s Regulated Compliance Bot to ensure specific content aligns with your agreement with the brand(s). Each monitored item requires a separate visit to your site, so websites with a high volume of monitored content will receive proportionally more requests.

If impact.com’s Regulated Compliance Bot is restricted (e.g., returns HTTP 429 – Too Many Requests or HTTP 403 – Forbidden), we may be unable to review and verify your content, which could affect compliance visibility.

To ensure uninterrupted monitoring:

• Allowlist the impact.com Regulated Compliance Bot using the provided User-Agent.

• Avoid blocking or throttling legitimate requests from the impact.com Regulated Compliance Bot.

• Verify the impact.com Regulated Compliance Bot using the signature included in the request headers to confirm authenticity and prevent spoofing.

• By allowing and verifying the impact.com Regulated Compliance Bot, you help ensure your content is accurately reviewed and reflected in the impact.com system.

User-Agent identification

Our crawler uses a custom User-Agent to identify itself during web requests. This allows you, as a partner, to distinguish impact.com compliance traffic from other bots and crawlers.

User-Agent String to allowlist (use exactly as shown):

Mozilla/5.0 (compatible;Impact.com Agent) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36

Verify impact.com crawler requests

To confirm that incoming traffic is from the impact.com crawler, each request includes a custom HMAC signature header:

X-Impact-Crawler-Signature: <hmac signature>

This signature allows you to authenticate the request using a known secret and a shared algorithm. It ensures the request originated from impact.com and has not been tampered with.

Signature construction

Signature construction

We compute the signature using the following process:

• Secret: /impact/crawler/signature/

• Data to sign: User-Agent

• HMAC Algorithm: HMAC-SHA256

• Signature Output: Hexadecimal string

How to verify (pseudocode)

const crypto = require('crypto');

// Extract headers from incoming request
const userAgent = req.headers['user-agent'];
const signature = req.headers['x-impact-crawler-signature'];

const secret = '/impact/crawler/signature/';

// Recompute signature
const expectedSignature = crypto
  .createHmac('sha256', secret)
  .update(userAgent)
  .digest('hex');

// Compare signatures
if (signature === expectedSignature) {
  // ✅ Verified
} else {
  // ❌ Reject or log for review
}

Warning: Be sure to use the exact User-Agent value as received in the request. Any modifications such as trimming, decoding, or reformatting will result in a signature mismatch.

IP allowlist for regulated compliance monitoring

To support ongoing Regulated Compliance monitoring, please allowlist the following IP address ranges. This ensures that traffic from impact.com is not unintentionally blocked and that site monitoring can proceed without interruption.

Allow the following IP ranges:

• 163.116.128.0/17

• 162.10.0.0/17

• 31.186.239.0/24

• 8.39.144.0/24

• 8.36.116.0/24

• 34.145.188.137

• 35.245.212.57

• 34.21.56.213

• 34.48.90.233

• 35.245.229.167

• 34.150.151.151

• 35.245.185.65

• 35.188.242.212

• 34.86.33.122

• 34.145.224.193

• 35.245.135.218

• 35.221.1.178

• 34.150.217.77

We advise updating your firewall or access control lists to permit incoming traffic from the following impact.com IP address ranges.

Recommended: For technical coordination or questions related to Regulated Compliance monitoring and bot access, contact support.