Product onboarding
Read a customer-provided website to establish product context, positioning, public evidence, navigation, and brand cues.
Crawler transparency
About BloggedBot
BloggedBot is Blogged's identified crawler. It visits publicly accessible pages only when a customer-directed workflow needs product context, approved source evidence, competitor context, or reusable visual candidates.
Identified on every request Bounded crawl scope Public pages only
Why it visits
A crawler attached to an explicit customer workflow.
BloggedBot does not crawl the web to build a general-purpose search index or model training corpus. A customer action or configured product workflow provides the domain, purpose, and bounded scope.
Knowledge refresh
Revisit approved public sources so product and competitor context can be checked against current evidence.
Competitor context
Review approved competitor sites and public content to support market-aware research and monitoring.
Access boundaries
Public access is still technically constrained.
The crawler validates destinations and redirects, caps response sizes, and uses page budgets. It does not bypass authentication, paywalls, private-network protections, or other technical access controls.
robots.txt policy
BloggedBot reads and records robots.txt. For customer-directed work, it may continue reading publicly accessible pages when robots.txt discourages crawling because robots directives are treated as crawler guidance rather than access control. Requests remain identified and technically bounded.
User agent:
Information URL:
BloggedBot/0.1Information URL:
https://blogged.dev/botFAQ
Questions about BloggedBot.
What is BloggedBot?
BloggedBot is Blogged's identified crawler. It visits publicly accessible pages only when a customer-directed workflow needs product context, approved source evidence, competitor context, or reusable visual candidates.
What can BloggedBot access?
BloggedBot can request public web pages, robots.txt, sitemaps, feeds, and linked public assets within bounded crawl scopes. It does not bypass authentication, paywalls, private-network protections, or other technical access controls.
How does BloggedBot handle robots.txt?
BloggedBot reads and records robots.txt. For customer-directed work, it may continue reading publicly accessible pages when robots.txt discourages crawling because robots directives are treated as crawler guidance rather than access control. Requests remain identified and technically bounded.
How can a site owner request an exclusion?
Email hi@blogged.dev with the domain and enough information to verify the request. Blogged can add the domain and its subdomains to the crawler exclusion policy.
Request a crawler exclusion.
Email the domain and enough information to verify the request. Blogged can add the domain and its subdomains to the crawler exclusion policy.
Email hi@blogged.dev