Wow, well that took longer than expected! [44 days ago I blogged](/posts/tinkering-with-typescript/) that I had started work on a second version of my Chrome Crawler extension and have only just managed to get it to a state I was happy with enough to release it. To be fair I had been on [a trip to New York](/posts/new-york-new-york/) during that period so perhaps I can be excused. Having said that however I think the time has been well spent and am fairly proud of the result.



### TL;DR

Recursive is an experimental tool for visualising the world wide web. Given a URL it downloads the page search for links and then recursively downloads those. The information is then displayed in a node-based graph.



### The Name

So what's this all about? Why is it called 'Recursive', why not 'Chrome Crawler 2'?

Although I would like to have called the spiritual successor to ['Chrome Crawler'](/posts/chrome-crawler-a-web-crawler-written-in-javascript/), 'Chrome Crawler 2' Chrome's [branding guidelines](https://developers.google.com/chrome/web-store/branding) forbid using the Chrome name or logo (they brought this in since the launch of Chrome Crawler 1).

With that in mind I decided that rather bend Chrome Crawler's name and logo to fit the guidelines I would create a whole new logo, name and app. The app is a total rewrite from the previous iteration anyway so I thought it justified.

According to dictionary.com there is no definition for "Recursive" or "Recurse" but there is one for "Recursion":

> 2\. the application of a function to its own values to generate an infinite sequence of values.
> So a tool that downloads pages, follows the links on that page to download other pages seemed like a rather apt description of something that is "Recursive".

### Video

Before I go much further, I put together this little video demonstrating some of the extensions core functionality:

<iframe width="640" height="360" src="https://www.youtube.com/embed/oNdpoM5Vhsc" frameborder="0" allowfullscreen></iframe>

### Installing

Installing and upgrading is dead simple thanks to how Google Chrome's extension system works. Just head over to this link and hit install:

[https://chrome.google.com/webstore/detail/recursive/hbgbcmcmpiiciafmolmoapfgegbhbmcc](https://chrome.google.com/webstore/detail/recursive/hbgbcmcmpiiciafmolmoapfgegbhbmcc)

Then to launch it visit any website and hit the little icon in the Omnibox:

[![](/wp-content/uploads/2012/12/Screenshot_008.png "Screenshot_008")](/wp-content/uploads/2012/12/Screenshot_008.png)

### How it works

Recursive works by taking in a starting URL which it uses to download the page it points to:

[![](/wp-content/uploads/2012/12/03.jpg "03")](/wp-content/uploads/2012/12/03.jpg)

Once that page is downloaded Recursive parses it looking for links and files. If it finds things it thinks are files then it records them against that URL. It then proceeds to visit all the links in turn, downloading the page then parsing the for yet more files and links.

This cycle continues until a certain "depth" is reached which is the maximum number of links away from the starting URL. You can set the maximum depth allowed in the settings:

[![](/wp-content/uploads/2012/12/04.jpg "04")](/wp-content/uploads/2012/12/04.jpg)

One of the key improvements of Recursive over Chrome Crawler is the way it visualises the data as it is returned:

[![](/wp-content/uploads/2012/12/05.jpg "05")](/wp-content/uploads/2012/12/05.jpg)

Every page is grouped by its domain and is represented by a circular "node".

[![](/wp-content/uploads/2012/12/Screenshot_013.png "Screenshot_013")](/wp-content/uploads/2012/12/Screenshot_013.png)

So for example "/posts/tinkering-with-typescript/" would be grouped under the "mikecann.blog" domain. Any other pages found while running that match this domain are added as little page icons inside the host node.

[![](/wp-content/uploads/2012/12/Screenshot_014.png "Screenshot_014")](/wp-content/uploads/2012/12/Screenshot_014.png)

Any files that are found on a given page are given an appropriate icon and added to that page's domain node.

[![](/wp-content/uploads/2012/12/Screenshot_015.png "Screenshot_015")](/wp-content/uploads/2012/12/Screenshot_015.png)

As Recursive downloads pages and follows links it records the path it takes. It then draws lines between the nodes that are linked:

[![](/wp-content/uploads/2012/12/06.jpg "06")](/wp-content/uploads/2012/12/06.jpg)

### Interacting

Using the mouse wheel you can zoom in and out to get a better perspective. Click and drag to move about the recursive space. You can also run the app in fullscreen if you so desire.

[![](/wp-content/uploads/2012/12/Screenshot_017.png "Screenshot_017")](/wp-content/uploads/2012/12/Screenshot_017.png)

If you click on a node it tells Recursive to explore that node for one extra level depth.

Right clicking a node opens a menu that lets you either open all the pages contained in that node or view the files for that node.

[![](/wp-content/uploads/2012/12/Screenshot_019.png "Screenshot_019")](/wp-content/uploads/2012/12/Screenshot_019.png)

### Files

By using the context menu for a node you can checkout all the files that Recursive found for that node. The files are separated into various categories which you can toggle on or off:

[![](/wp-content/uploads/2012/12/Screenshot_020.png "Screenshot_020")](/wp-content/uploads/2012/12/Screenshot_020.png)

Then if you wish you can download all the files as a zip.

### More

If you would like to read more on recursive, then checkout these posts:
[catlist id=30 numberposts=100]

[![](/wp-content/uploads/2010/12/700.jpg "700")](/wp-content/uploads/2010/12/700.jpg)

**EDIT: I now have a newer, better version of this called "[Recursive](/posts/recursive/)"**

Depending on your level of geekness you may or may not enjoy this one.



I proudly present [Chrome Crawler](https://chrome.google.com/extensions/detail/amjiobljggbfblhmiadbhpjbjakbkldd), my latest Google Chrome extension:

[![](/wp-content/uploads/2010/12/Shot_002.png "Shot_002")](/wp-content/uploads/2010/12/Shot_002.png)

The idea is simple really. You just give it a URL, it then goes off and finds all the links on that page then follows them to more pages then gets all the links and follows them and so on and so on.

Along the way it checks each page to see if there are any 'interesting' files linked there, if it finds an interesting link it will flag it for you so you can check it out.

Theres an options page that lets you customise the way it all works:

[![](/wp-content/uploads/2010/12/Shot_004.png "Shot_004")](/wp-content/uploads/2010/12/Shot_004.png)

If you are still confused check out the video below:

<iframe width="100%" height="400" src="https://www.youtube.com/embed/ceeQEMTSD0I" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>

So why did I make this? Well to be frank, I made it mostly "just 'cause I can"!

Also having learned from my last Chrome Extension project [PostToTumblr](/posts/my-first-chrome-extension-post-to-tumblr/) I realised the Chrome API allowed you to do some things that you wouldn't normally be allowed to do on a website (nameley the [Cross-Origin XHR](https://code.google.com/chrome/extensions/xhr.html)) and I wanted to do something to take advantage of it.

It didnt take me long to knock out this project, one lazy Saturday for the majority of the code and today for a quick fix or two and to write this post and make the video. As such I expect there to be many bugs and problems so if you encounter one drop me an email (my address is found in the options page).

Oh finally, I wouldnt try using this on a google page as you will likely end up seeing this quite often:

[![](/wp-content/uploads/2010/12/Shot_003.png "Shot_003")](/wp-content/uploads/2010/12/Shot_003.png)

Anyways you can grab it over on the [Chrome extensions gallery here](https://chrome.google.com/extensions/detail/amjiobljggbfblhmiadbhpjbjakbkldd). If you enjoy it please leave me a review / comment, much love!