Michael Posner

I currently work as a System Software Engineer. I earned a Masters degree in Robotics as well as Bachelors degrees in Computer Science and in Mechanical Engineering from the University of Pennsylvania. My interests include robotics, embedded systems, aerospace, machine learning, artificial intelligence, distributed systems, and data analysis.

 

Web Crawler

CIS 455: Internet and Web Systems

University of Pennsylvania

Spring 2013

Class Website


As part of my final project, I created a multithreaded, distributed web crawler written in Java. Each node ran several threads to visit and download web pages, communicated with each other in order to divide up found URLs, and stored data on local BerkeleyDB storage.

Objectives

Design

Distributed Network

Web Pages

Multi-Threading

Storage

Results


Gear image source. Logos copyright GitHub and LinkedIn.

Developed using Twitter's Bootstrap

Creative Commons License

This site's content is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 United States License unless otherwise noted.