about fedi.li

what
fediverse instance scanner. crawls nodeinfo + instance apis, builds a public database of instances, software and versions — in the spirit of fedidb.
user-agent
fedi.li-scanner/0.1.0 (+https://fedi.li/about)
version
0.1.0
crawl policy
one connection per instance, few requests per second globally, retry-after honored, exponential backoff on errors (up to 7 days between retries), instances that keep failing are retired.
discovery
public peer lists of already known instances. no content is fetched, only public server metadata (nodeinfo, instance endpoints).
cdn detection
instances behind known shields (cloudflare, fastly, akamai, cloudfront, sucuri, imperva, bunnycdn, prot.li shield) are labeled with the shield name instead of a meaningless anycast datacenter location.
opt-out
block the user-agent above or serve nodeinfo with crawler directives; to be removed entirely, contact the operator.