Real Time Crawler Addon for the Cache Warmer is for stores with many URLs.

Imagine a store crawling 20,000 URLs. The normal crawler will crawl all 20,000 URLs every time it runs. That can be inefficient.

With the Real Time Crawler Addon for Cache Warmer, only the URLs that have expired and been flushed in the page cache (since the last crawl) are crawled.

As a result, fewer URLs need to be crawled every job - which means lower server load, faster crawling, faster caching of flushed pages, etc.

Interested in adding Real Time Crawler to your Magneto store? Learn how to do it at "How do I purchase Addons for my store"?