Problem

There are no records or records missing from your collections index.

Solutions for no records:

  • Use our page debug tool to see if our crawler encounters an error on your homepage

  • Has the domain been added correctly to the Domains section of our platform?

  • If the entered domain has a www prefix the crawler will not visit any subdomains. See here

  • Is the site password protected? See here

Solutions for records missing from your collection:

  • Use our page debug tool to see if our crawler encounters an error on any of the URL’s that are missing

  • If the content was added / altered recently our crawler may not have visited your site according to its schedule. See here

  • Are you using our ping-back / instant indexing code and have met the conditions documented here?

  • Do you have any exclusion rules set up in the Rules section preventing the pages from being crawled?

  • Do your pages have the noindex or data-sj-noindex tags preventing crawling?

  • Is any of your content only accessible via query parameters i.e. www.mysite/news-articles/press-releases?page=2? If so please contact our support team here.

  • Are any of your pages orphaned i.e. have no links from other webpages allowing the crawler to find it?

  • Do you have canonical tags pointing our crawler to a different page?

  • Do you have any canonical loops i.e. homepage redirects to a different page, which redirects back to the homepage

  • Are you using path-relative URLs instead of root-relative URLs? The crawler does not follow path-relative URLs unless there is a base path on the page <a href="example-page">). For further reading on URL types please see here

  • Do you have a sitemap? This page describes the benefits