How does the crawler handle 301 or 302 redirect?

Answer

Search.io Site Search handles HTTP redirects automatically when a page is reindexed. The redirected pages are removed from the Collection(index) when they have a 301 or 302 HTTP status code.

The destination page of the redirect gets added to the Collection and will be shown in the search results.

Note: Any page that doesn’t have any meta changes detected will be re-crawled after 3-6 days in any case. The redirected pages might still appear in the search results until the next re-crawl takes place.

How to remove redirected pages immediately from a Collection:

You can also manually trigger the crawler to remove the page if you don't want to wait by following these steps:

  1. Log in to your search.io account

  2. Select the relevant Collection

  3. Navigate to 'Domains' and click on 'Diagnose'

  4. Enter the URL that you have removed/redirected and press "Diagnose"

  5. The result and details of the record would be returned. Press "Add to Index"

The page will be removed from the index in a few minutes, and the "State" would change to "Redirect" the next time it is diagnosed. See a screenshot below: