This software can generate various sitemaps for search engines such as Google and Yahoo, including text, HTML, RSS, and XML. It features templates for HTML sitemaps, with support for splitting and compressing XML sitemaps. It offers multiple website crawl options and allows users to view all broken and redirected links within their site.
If you want to ensure that search engines like Google and Yahoo can crawl and index your website effectively, look no further than Crawler. This software allows you to create text, HTML, RSS, and XML sitemaps to improve your site's visibility to search engines.
Crawler is packed with features that make it easy to customize your website crawling options. You can configure the number of simultaneous connections you want to use, set filters, specify custom connection and read timeout values, remove session IDs, scan JavaScript and CSS files, set up proxies, and more.
One particularly useful feature is the ability to alias paths during the scan. If your site uses multiple domain names but has the same content, this can simplify the indexing process. You can also scan websites from multiple start paths, which is useful for sites that aren't fully crosslinked.
Crawler supports scanning both static and dynamic websites, including portals, online stores, blogs, and forums. You can generate reports on broken and redirected links and use rich templates for HTML sitemaps. In addition, you can generate sitemap files for ASP.Net controls, split and compress XML sitemaps, and set priorities, change frequencies, and last modified dates.
If you need to change the root path used in your sitemaps, Crawler makes it easy. You can also upload sitemaps via FTP and ping search engines when changes are made. Finally, if you prefer to work from the command line, Crawler offers support for loading projects, scanning websites, building sitemaps, uploading via FTP, and pinging search engines.
Crawler is packed with features that make it easy to customize your website crawling options. You can configure the number of simultaneous connections you want to use, set filters, specify custom connection and read timeout values, remove session IDs, scan JavaScript and CSS files, set up proxies, and more.
One particularly useful feature is the ability to alias paths during the scan. If your site uses multiple domain names but has the same content, this can simplify the indexing process. You can also scan websites from multiple start paths, which is useful for sites that aren't fully crosslinked.
Crawler supports scanning both static and dynamic websites, including portals, online stores, blogs, and forums. You can generate reports on broken and redirected links and use rich templates for HTML sitemaps. In addition, you can generate sitemap files for ASP.Net controls, split and compress XML sitemaps, and set priorities, change frequencies, and last modified dates.
If you need to change the root path used in your sitemaps, Crawler makes it easy. You can also upload sitemaps via FTP and ping search engines when changes are made. Finally, if you prefer to work from the command line, Crawler offers support for loading projects, scanning websites, building sitemaps, uploading via FTP, and pinging search engines.
What's New
Version 12.0.0: Each version contains improvements in functionality.
Version 11.0.0: Each version contains improvements in functionality.