Project Detail

Description

Spider friends pages for a list of named twitter accounts. e.g.

http://twitter.com/briansolis/friends?page=1
http://twitter.com/briansolis/friends?page=2
...
http://twitter.com/briansolis/friends?page=n

The spider will output a simple Pajek .net format file

where vertices (named accounts) are stored as

<id> "<label>"

and edges are stored as:

<from_id> <to_id> <weight>

(weight is a count of the number of links between vertices -- in this instance weight should ALWAYS be 1)

Here is an example format:

*vertices <# of vertices>
1 "http://twitter.com/briansolis"
2 "http://twitter.com/another"
3 "http://twitter.com/another"
*edges
1 2 1
1 3 1
2 3 1

May not need MySQL back end -- depends on memory handling.

There will be two other closely related projects that follow this one. A satisfactory result will lead to automatic selection for the next project(s)

Spider can run from the command line (I use Mac OS X by preference, and PREFER cross-platform tools)


Additional files submitted:
pajek100.net