Project Detail

Description

I am an economics professor who is interested in hiring someone to write a program to scrape diamond data from 9 different diamond sites. I'm doing a follow-up to an academic study I did earlier (http://gatton.uky.edu/Faculty/yelowitz/diamonds/). For each of the sites below, the program would extract the data on diamond prices and characteristics into either an excel spreadsheet, a text file, or a comma delimited file. The program would gather data for all diamond shapes (round, radiant, princess, pear, etc.). Note that different diamond characteristics are available based on the diamond's shape, so the program would need to take that into account.

The 9 sites are:

http://diamonds.com/Main/AdvanceSearch.aspx
variables: shape,carats,color,clarity,report,cut,price,diamond id (which is obtained from the URL when you click "view")

http://www.uniondiamond.com/search-loose-diamonds.html
variables: id,size,color,clarity,cut,wire price,price,polish/symmetry,certificate

http://www.adiamor.com/diamondsearch.aspx
variables: shape,ID No.,carat,color,clarity,depth,table,cut grade,report,price

http://www.abazias.com/builder/builder.asp?view=diamondsearch
variables: shape,carat,color,clarity,cut,certificate,price

http://www.diamondsonweb.com/diamond/searchform.mhtml
variables: shape,carat weight,color,clarity,lab,inscribed,depth,table,fluor,pol/sym,cut,price,stock id,measurements

http://mondera.com/
http://www.mondera.com/round-cut-diamonds
http://www.mondera.com/princess-cut-diamonds
http://www.mondera.com/emerald-cut-diamonds
etc.
variables: shape,carat,color,clarity,cut,polish/symmetry,certificate,price,product id

http://www.brilliance.com/diamonds/search/search.aspx
variables: shape,carat,cut,color,clarity,report,polish,symmetry,price,stock number,measurements,depth,table

http://www.bluenile.com/diamond-search?basic=1&filter_id=0
variables: shape,carat,cut,color,clarity,polish/symmetry,report,price,stock number

http://www.amazon.com/Loose-Diamonds-Diamond-Engagement-Rings/loosediamonds?%5Fencoding=UTF8&productGroupID=loose%5Fdiamonds
variables: shape,carat,cut,color,clarity,price

I've written web scraping programs myself in the past (using Lencom's Visual Web Task), but I'm hoping to get someone to write a more efficient program for gathering the data. I've set this listing to run for 15 days, but I'm willing to award this sooner if someone gives a very competitive proposal.