Skip to content

SERA Scenarios

This document specifies scenarios that can affect the bot results.

Prerequisite

  1. Your must have a domain on which SERA is configured.

  2. You must have an account on Nitrogen to view the dashboard.

SERA workflow

SERA (Search Engine Rank Accelerator) is a tool that serves the 'optimized' version of 'pre-processed' page to the bots.
As bots need less time to process and render the SERA pages, they provide a good CWV score to the pages, and also effectively utilize the crawl budget.
All the render and crawl related stats can be checked in Dash.
To process the pages, SERA uses :

  1. sitemap.xml files
  2. the URLs submitted by on-demand render API
  3. URLs provided in uploaded file

Different Scenarios to consider with bots

  1. robots.txt is not configured to prevent bots from accessing irrelevant pages

    Description : Bots can request pages through sitemap, metadata on page, links present on page, external links, merchant feeds, backtracking etc.
    Their access needs to be managed in robots.txt file to avoid getting unintended pages getting indexed.
    e.g. These pages should not be indexed :
    /page-not-found
    /error
    /login
    /account/
    /cart/
    /wishlist/
    /_next/
    /*.json
    /[slug]/
    /.well-known/

    Impact : As these pages are not present on SERA, bots will receive 404 for them.
    If failover is configured, those requests will be served by origin with respective response code.
    Note : Failover functionality is currently under development.

    Recommendation : Access to sensitive pages such as account, wishlist, cart, checkout should be disallowed.
    You can refer this link for reference.

  2. Page requested by bot is not present on SERA

    1. Due to page not present in sitemap

      Description : If pages are not present in sitemap, or were not requested by render API, those will not be rendered and available at SERA end.

      Impact : Bots will receive 404 for the pages not present in SERA.
      If failover is configured, those requests will be served by origin with respective response code.
      Note : Failover functionality is currently under development.

      Recommendation : Keep adding newly added pages to sitemap, to get picked for rendering at the configured schedule.
      For urgent renderings, use SERA render API.

    2. Page URL has querystring

      Description : SERA renders the pages with the URL available in sitemap or received through API.
      e.g.
      Sitemap contains : https://www.domain.com/c/150/new-arrivals
      Bot requested : https://www.domain.com/c/150/new-arrivals?sort_by=price_desc

      Impact : Bots will receive 404 for the pages not present in SERA.
      If failover is configured, those requests will be served by origin with respective response code.
      Note : Failover functionality is currently under development.

      Recommendation : SERA has ability to ignore the given query parameters e.g. campaign/tracking params like utm, _ga etc. This can be used to reduce the probability of URL mismatches.
      In most cases it is already configured by the N7 team, and can be checked in Dash. For the SERA routings configured at the origin end, it can be managed in the rewrite rules.

  3. Sitemap contains invalid URLs

    Description : This is the case when sitemap/render-API-payload contain invalid/irrelevant/improper URLs (non-existent, malformed, redirecting)

    Impact : SERA will rely on origin for these requests, and will process those if received with 200 response code.
    For any other response codes, SERA too will return 404/7XX code.

    Recommendation : All render statuses can be checked in Dash. You can view/download the respective log, and make changes at your end.

  4. Origin was down when SERA accessed it

    Impact : SERA will not be able to render the respective pages.
    If it has the older version of page in storage, it will be served to bots.
    If failover is configured, those requests will be served by origin with respective response code.
    Note : Failover functionality is currently under development.

    Recommendation : The failed render requests can be checked in Dash, and can be processed again using API or by creating temporary page-group.

  5. Origin was down when bots requested some page

    Impact :

    • If page is present on SERA : it will be served from there

    • If page is not present on SERA : SERA will return 404 to requesting bot
      If failover is configured, those requests will be served by origin with respective response code.
      Note : Failover functionality is currently under development.

    Recommendation : The failed crawl requests can be checked in Dash, and can be indexed again in respective search-engine dashboards.

  6. SERA served old version of page to bot

    Description : This can happen if the page got updated, and the SERA schedule has not yet kicked in to process the updated/new pages.

    Impact : Bots will receive old content.

    Recommendation : You can run SERA render API to render the pages on demand basis.
    You can view the crawl details in SERA dashboard for more details.

  7. Page is present in sitemap, but was not served to bots by SERA

    Description : This can happen if the page was added to sitemap, but the SERA schedule has not yet kicked in to process the new pages.

    Impact : SERA will return 404 to requesting bot
    If failover is configured, those requests will be served by origin with respective response code.
    Note : Failover functionality is currently under development.

    Recommendation : You can run SERA render API to render the pages on demand basis.
    You can view the crawl details in SERA dashboard for more details.

  8. Sometimes Google Search Console shows spike in page load time

    Description : This can happen only for the domains configured on N7 CDN. It happens in case the full site's cache is purged.

    Impact : You will see a spike with increase of few milliseconds in page load time chart. It will be for first visit by bots after the purge. The pages will then get put in cache.

    Recommendation : Avoid full site purge whenever possible. Only purge the resources that have got updated.

  9. SERA was down when bots requested some page

    Description : This is highly unlikely to happen, and have not been reported yet.
    Our monitoring teams monitors the jobs 24x7 for 365 days. And have configured respective alerts and failovers set to handle this scenario.

    Impact : The SERA would not be able to process the pages, and serve the processed pages.

    Recommendation : The SERA routing can be disabled till notified by our team. For that period, bots will receive the original pages.