06 Jul Web
2. A readme file with complete description of used software, installation, compilation and execution instructions
3. A document with the results for the questions below.
Task:
Develop a specialized Web crawler.
Test your crawler only on the data in:
Make sure that your crawler is not allowed to get out of this directory!!! Yes, there is a robots.txt file that must be used. Note that it is in a non-standard location.
The required input to your program is N, the limit on the number of pages to retrieve and a list of stop words (of your choosing) to exclude.
Perform case insensitive matching.
You can assume that there are no errors in the input. Your code should be robust under errors in the Web pages you’re searching. If an error is encountered, feel free, if necessary, just to skip the page where it is encountered.
Efficiency: Don’t be ridiculously inefficient. There’s no need to deliver turbo-charged algorithms or implementations. You don’t need to worry about memory constraints; if your program runs out of space and dies on encountering a large file, that’s OK. You do not have to use multiple threads; sequential downloading is OK.
Our website has a team of professional writers who can help you write any of your homework. They will write your papers from scratch. We also have a team of editors just to make sure all papers are of HIGH QUALITY & PLAGIARISM FREE. To make an Order you only need to click Ask A Question and we will direct you to our Order Page at WriteDemy. Then fill Our Order Form with all your assignment instructions. Select your deadline and pay for your paper. You will get it few hours before your set deadline.
Fill in all the assignment paper details that are required in the order form with the standard information being the page count, deadline, academic level and type of paper. It is advisable to have this information at hand so that you can quickly fill in the necessary information needed in the form for the essay writer to be immediately assigned to your writing project. Make payment for the custom essay order to enable us to assign a suitable writer to your order. Payments are made through Paypal on a secured billing page. Finally, sit back and relax.
About Writedemy
We are a professional paper writing website. If you have searched a question and bumped into our website just know you are in the right place to get help in your coursework. We offer HIGH QUALITY & PLAGIARISM FREE Papers.
How It Works
To make an Order you only need to click on “Order Now” and we will direct you to our Order Page. Fill Our Order Form with all your assignment instructions. Select your deadline and pay for your paper. You will get it few hours before your set deadline.
Are there Discounts?
All new clients are eligible for 20% off in their first Order. Our payment method is safe and secure.