This scrapper built as a wordpress plugin and for scraping the eBook information from Frasesbonitas.biz and store in a wordpress custom post type called “EBooks”. The eBook categories are stored in wordpress taxonomy called “Ebook_Categories” and all the eBooks are stored in their original category as in the first site.
Plugin itself has few pages for manage and displaying the processing information of the eBooks. All the additional information about eBooks are saved in post Meta options of each post.
Following data scrapped for ebooks from the main site and stored as WordPress posts.
- Ebook Title
- Cover image
- About Author
- Author URL
- Publisher URL
Plugin supports displaying eBooks using short codes and comes with two short codes.
The settings panel support for proxies and other general settings for scrapping and displaying data in wordpress pages. And upon installation of the plugin it will create two pages for display all ebooks (archive) and the single ebook view page which can change from settings panel later.
All the scrapped data are stored in wordpress database and deliver as posts to site visitors. The Meta options used to store additional details including movie watch links and etc.
Manage Random proxies for scrapping
The plugin support proxies in its settings panel and loads the proxes randomly in a way to make sure that the ips are won’t get blacklist.
Multi Treaded processing
Processor can scrape nearly 10 eBooks in every minute. Due to complexity of data the processing and as the data are gathered through different pages for a eBook, it is slower but still support the multi treaded for faster processing