Objective:
PHP base scrapper program built to scrape Trustpilot reviews based on the domain names provided. The client wanted to start similar website and required some data for start. The program allow to submit one or more URLs and process every URL to receive the review information from Trustpilot.
Strategy:
The program creates Review lists with all the following information.
- User
- Rating Value
- Date Reviewed
- Title
- Review
Advance Features:
Add Domains
The program support adding domains in bulk or one by one. URL s can be enter one by one in new lines to process
Data Export
The program support export data into excel files by each domain
Manage Random proxies for scrapping
Scrapper supports proxies and random proxies will be picked at the scraping process to make sure the ips will not be detected or blocked by the other server.
Multi Threaded processing
Processor can scrape more than 300 reviews per every minute. Even though keep the number below the 100 is optimum as more it scrape, more chances to getting the IP s blacklisted.
