Heatmap Web Crawler with Visual Site Mapping
The Heatmap Web Crawler with Visual Site Mapping tool is a desktop application that crawls websites and generates visual site maps with screenshots. Built with Electron, Express, and Mermaid.js. by Emma Dickson
Features
- Crawl websites and generate interactive site maps
- Capture full-page screenshots of each page
- Highlight specific HTML elements during crawl
- Configurable crawl depth
- Visual flowchart representation of site structure
- Customizable storage location for crawl data
Find the project here: https://github.com/emmadickson/heatmap
This tool is developed as part of an ongoing COSE research project into the structures of internet-based artworks. Feedback, suggestions, and collaborations are welcome.
Tool in action:
The Heatmap Web Crawler was used in the Second Order Visitor’s Guide to London model complex.
Visualisation of an interlinked webpage complex (right), and the individual paths taken by three volunteers.