You can choose to extract them via ‘Config > Spider > Extraction’ and selecting ‘HTTP Headers’. This can be useful when analysing various scenarios which are not covered by the default headers extracted, such as details of caching status, set-cookie, content-language, feature policies, security headers etc. You can now store, view and query full HTTP headers. This should be helpful when sharing data in teams, with clients, or for Google Data Studio reporting. You’re able to choose to create a timestamped folder in Google Drive, or overwrite an existing file. This means you can schedule a crawl, which automatically exports any tabs, filters, exports or reports to a Sheet within Google Drive. This has also been integrated into scheduling and the command line. We had started work on writing to multiple sheets, but really, Sheets shouldn’t be used in that way. If you need to export more, use a different export format that’s built for the size (or reduce your number of columns). This sounds like a lot, but when you have 55 columns by default in the Internal tab (which can easily triple depending on your config), it means you can only export around 90k rows (55 x 90,000 = 4,950,000 cells). Many of you will already be aware that Google Sheets isn’t really built for scale and has a 5m cell limit. You can add multiple Google accounts and connect to any, quickly, to save your crawl data which will appear in Google Drive within a ‘Screaming Frog SEO Spider’ folder, and be accessible via Sheets. You’re now able to export directly to Google Sheets.

The non-eye-strained among you may notice we’ve also tweaked some other styling elements and graphs, such as those in the right-hand overview and site structure tabs. Not only will this help reduce eye strain for those that work in low light (everyone living in the UK right now), it also looks super cool – and is speculated (by me now) to increase your technical SEO skills significantly. You can now switch to dark mode, via ‘Config > User Interface > Theme > Dark’.

While arguably not the most significant feature in this release, it is used throughout the screenshots – so it makes sense to talk about first. Let’s talk about what’s new in this release.

Since the release of version 13 in July, we’ve been busy working on the next round of features for version 14, based upon user feedback and as always, a little internal steer. We are pleased to launch Screaming Frog SEO Spider version 14.0, codenamed internally as ‘megalomaniac’.

Screaming frog seo spider download