Origin<>SERA<>Crawler request flow

Prerequisite

1. You must have an account created on Nitrogen dashboard to view the SERA statistics

2. SERA must be configured to receive the requests from bots

Flow

A typical request flow with SERA integration looks like this :

Page serving workflow :
(This is shown at left-hand side of flow in Yellow color)

1. Crawler requests are received on N7 CDN
2. If the N7 CDN has page available in cache/storage - it is returned back to crawler
   1. If the page page is unavailable in cache/storage :
      1. If a failover is not configured - a 404 is returned to crawler
      2. If a failover is configured - the request is forwarded (proxied) to origin server
         1. Origin server sends the response
         2. SERA receives it, and sends it to crawler

Page rendering workflow :
(This is shown at right-hand side of flow in Yellow color)

1. SERA reads the configured sitemaps at configured schedules, and makes respective page requests to the origin server
2. Origin server serves the page
3. SERA :
   1. Renders the page in the same manner browser renders it for users.
   2. Then optimizes it from bot perspective, keeping only the content relevant for bots.
   3. And then caches it on the CDN POP closest to GoogleBot origin

Notes

Please also go through the different scenarios mentioned here