{% if doc and not doc.crawl_last %}

This page has not been crawled yet.

{% elif doc and doc.robotstxt_rejected %}

Crawling this page was rejected by a robots.txt rule.

You can ignore this robots.txt policy in the Domain settings for {{ doc.default_domain_setting.domain }}.

{% elif doc and doc.error %}

An error occured while crawling this page:

            {{ doc.error }}
        

{% endif %}