Contract PHP Developer for Long-Term Contract to Develop and Support Web Crawlers or Spiders

Contract PHP Developer for Long-Term Contract to Develop and Support Web Crawlers or Spiders is project number 43796 posted at GetAFreelancer.com. Click here to post your own project.

Status:	Closed (Project expired)
Selected Providers:	-
Budget:	$30-100
Created:	02/10/2006 at 10:00 EST
Bidding Ends:	03/17/2006 at 10:00 EST
Project Creator:	PSOutsource   Buyer Rating: (5 reviews)
Description:	Note: You are bidding on the development of one (1) crawled site which will be specified later. If we like the final script, we will contract to have additional scripts created for a set agreed-upon price per script. Please put your bid in for the initial script and mention the price for any additional scripts. Final selection will be made on the bid for this initial script as well as the price of each incremental script. Note: Also note that existing crawler scripts (there are 2 available) will be provided to the winner of the bid for this project to expedite the development process. Please consider this in your bids. We are interested in hiring PHP developers/team on a long-term contract basis who are knowledgeable about the design and development of web crawlers/spiders using PHP. We currently have crawlers developed for two retail sites—-one with local store information and the other with online store information—-and are interested in significantly expanding the number of sites crawled (see listings below). The crawlers extract various information (product, store, and pricing) from the sites. Database used is MySQL database (see attachment for db schema). The current code is written in PHP and are of two varieties: 1. “On-Demand” Scripts – utilized to crawl sites on-the-fly with specific information and populating a table with that information 2. Nightly Batch Scripts – similar to On-Demand scripts but executed nightly via batch processes to extract data from websites and populating tables with information We also are looking for a development team for various admin tasks (not included in the scope of this contract) which we have listed in another project located here: http://www.getafreelancer.com/projects/43792.html The preliminary listing of sites to be crawled are as follows (the final site may or may not be selected from the following listing): Local Shopping Sites: 1. Cairo.com (crawler already developed for this site) 2. ShopLocal.com 3. Froogle.com 4. StepUp.com Online Shopping Sites: 1. [Trademark masked].com 2. Amazon.com 3. Dealtime.com 4. Buy.com (crawler already developed for this site) 5. MySimon.com 6. eBay.com 7. Crutchfield.com 8. PCMall.com/MacMall.com 9. TechBargains.com 10. Outpost.com ** Note: Furthermore, the Scripts should be flexible enough (extensible) to crawl all subcategories (e.g. Electronics, Computers, Housewares, Toys, etc.) available on the site. Brick-and-Mortar Companies which have an Online Presence: 1. BestBuy.com 2. CircuitCity.com 3. CompUSA 4. OfficeDepot.com 5. Staples.com 6. Frys.com 7. B&HPhoto.com 8. J&R 9. PC Richards ** Note: For local stores and pricing of products above, you may need to specify a zip code. The script should be able to take a text file of zip codes and run the search accordingly. The script can run in a batch manner with zip-code text file as a feed. NOTES: 1. For the local online sites, the Script should read zip codes in from a text file and search within a certain radius from that zip code. The text file will contain zip codes delimited by a separator. 2. The Script should crawl all subcategories under the primary categories on each website. This hierarchy should be retained in the database. The script should be intelligent enough to search down each hierarchy of items categorized under the parent category as specified by each individual site. Hierarchical format will be specified in the tables. 3. Data will need to be captured in the respective product tables, including the category, product, and brand tables without duplication of records. See each table for more information. 4. The script should enable another PHP page to call it On-Demand (i.e., initiated when required) and the data parsed into the appropriate tables, without duplication of records. 5. Contractor will agree to and sign the Service Agreement contract. 6. Contractual rate is negotiated for each script/website. Each payment will be contingent upon meeting milestones throughout the project (i.e., delivering working code). The final payment will be made at the end of the entire project dependent on meeting all requirements and conclusion of testing (see Service Agreement for more detail). 7. Contractor must participate in bi-weekly meetings and provide daily status updates. Once a project starts, you must provide daily updates and must be available on a ad-hoc basis for conference calls via Skype and GotoMeeting (already paid for). 8. Scripts must be periodically backed up, uploaded and tested on our servers. Final acceptance of a script is contingent on testing and approval by management. 9. Contractor will provide project timeline, noting milestones of when various scripts and web crawlers will be delivered (unless specified in the Service Agreement). If the contractor anticipates that it will miss the deadline, management must be notified or risk breach of contract. Any delays in the project timeline must be agreed by both parties. 10. Contractor will work closely with PeerShopper™ team members in testing and integrating scripts into existing code. 11. Contractor will provided evidence of previous work using PHP, preferably for developing web crawlers. 12. You must include 1 month of assistance and upgrades on a weekly basis. Additional files submitted: psdbschema9_1.sql
Job Type:	Data Processing PHP Script Installation
Database:	MySQL
Operating system:	MS Windows
Bid count:	8
Average bid:	N/A