This article applies to organization admins and third-party users in the PI2 software, as well as IT admins at your organization.
When adding organization context for Obi, you may encounter an error that your website page cannot be read. In this scenario, it's possible your organization may need to allow Obi access to read this webpage.
In this article, we'll explain the bot request that PI makes on behalf of Obi in order to read your webpage. Please share this resource with your IT admin so they can decide whether to grant access.
What is PI-ContextExtractor?
PI-ContextExtractor is an automated request made by Obi on behalf of our customers. When a PI customer sets up Obi for their organization, they can provide their company's public website URL so Obi can use publicly available information about the company as context in its responses. PI-ContextExtractor is the bot that retrieves that page.
What this request does
It fetches a single, customer-specified public webpage — the one the customer entered in their Obi settings.
It extracts the visible text content of that page (removing scripts, navigation, and similar non-content elements).
It runs only on demand (i.e., when a customer adds or updates their company website URL in Obi settings).
What this request does not do
It does not crawl your site, follow links, or fetch additional pages beyond the one URL provided.
It does not submit forms, execute JavaScript, or interact with the page in any way beyond a standard HTTP GET request.
It does not run on a recurring schedule. (It only runs when a customer adds or updates their company website URL in Obi settings.)
How to identify requests from Obi by PI
Requests from this bot use the following User-Agent string:
Mozilla/5.0 (compatible; PI-ContextExtractor/1.0; +https://docs.predictiveindex.com/en/articles/15888132-allowing-obi-to-read-your-website-it-admins)
Allowing or blocking this bot
We honor robots.txt. If your site blocks automated access generally but you'd like to allow PI-ContextExtractor specifically (if your organization uses Obi), add:
User-agent: PI-ContextExtractor
Allow: /
To disallow this bot specifically while allowing other bots, add:
User-agent: PI-ContextExtractor
Disallow: /
If your firewall or bot-management provider (Cloudflare, Akamai, etc.) filters by User-Agent rather than robots.txt, allowlist the exact string above.
Data handling
The text extracted from your webpage is used solely to give the requesting customer's instance of Obi context about their own organization. This text is not used to train models, shared with other customers, or used for any purpose beyond that customer's account.
Questions or concerns
If you believe this bot is behaving unexpectedly, or you have questions about a request from PI-ContextExtractor, please submit a PI Support ticket.
Additional support