What is a content delivery network?
A CDN is a distributed network of servers that delivers website content from locations closer to its readers. CDN services may also provide firewall and bot controls.
The original hosting system is the origin. An edge server can fetch content from the origin and keep a cached copy, then reuse that copy for later requests when its caching rules permit.
How caching changes what a reader receives
A missing or expired edge copy may require another retrieval from the origin. After a website update, an older cached response can therefore remain visible until the relevant cache changes.
A CDN does not cache every page simply because it is enabled. In Cloudflare's documented response headers, CF-Cache-Status: HIT indicates a response served from cache; MISS indicates that the requested content was not in that cache.
The browser has a separate cache. Updating one layer does not establish that every other layer has updated. Public images and personalized account responses also have different sharing requirements; personalized content should not be treated as an ordinary reusable public response.
Delivery and bot protection are separate questions
A CDN can reduce delivery distance and help absorb traffic spikes. Some services can provide a cached response when the origin has problems. The benefit depends on configuration and the content involved.
A CDN is not a promise of search visibility. Overly strict firewall or bot rules can block intended crawler access. After changing protections, check actual retrieval, including the available Search Console inspection tools.
Why robots.txt alone cannot explain every blocked request
Bot controls can act at the network layer, separately from the file's published instructions. If an allowed agent still cannot fetch a page, inspect both layers.
Cloudflare's July 2025 announcement describes AI crawling controls and a Pay Per Crawl experiment. Availability and settings for a particular account need a separate check. The crawler policy guide keeps historical observations separate from current access decisions.
Sources and scope
- Cloudflare: What is a CDN?
- Cloudflare: Content Independence Day, July 2025
- Cloudflare: cache response status
- MDN: HTTP caching
Source-language entry records a source check on 2026-10-14. This adaptation preserves that provenance and does not represent a new audit of the cited pages.