FoundryResearchBot
How Oryxs Foundry reads a company's public pages, and how to ask it to stop.
What to expect from it.
What it is
The program Oryxs Foundry uses to read a company's public web pages when a customer asks for research on that company. It reads what any visitor could read.
How it names itself
Its User-Agent is FoundryResearchBot, followed by this site's address and info@oryxs.in. It does not present itself as a browser.
How it behaves
It checks your robots.txt for the name FoundryResearchBot before it requests any page other than your home page. It waits at least a quarter of a second between requests to the same site, reads up to a dozen pages, and stops when its time is up. It does not sign in, fill in forms or open pages behind a login.
What it does not do
It does not crawl a whole site, and it does not read private pages. Text it reads is used to write that customer's research notes, each claim next to the sentence it came from.
A fallback you should know about
When a site refuses our request for its home page, or a page shows almost no text without running JavaScript, we may ask a third-party reading service (Jina Reader) to fetch it for us: the home page, and at most two other pages in one piece of research. That request comes from the service, carries its name, and is not covered by our robots.txt check.
How to stop it
Add a rule for FoundryResearchBot to your robots.txt, for example a Disallow of / under User-agent: FoundryResearchBot. Or write to us and we will stop reading your site.
Write to us.
Questions, complaints or a block request: info@oryxs.in. For a security problem, see security.txt.
Rather talk it through with a person?
Book a call