PHP Classes

What is the best PHP url crawler class?: Crawler to extract all metadata and URLs of Web site

Recommend this page to a friend!
  All requests RSS feed  >  What is the best PHP url crawler class?  >  Request new recommendation  >  A request is featured when there is no good recommended package on the site when it is posted. Featured requests  >  No recommendations No recommendations  

What is the best PHP url crawler class?

Edit

Picture of Dan Beck by Dan Beck - 8 years ago (2016-02-21)

Crawler to extract all metadata and URLs of Web site

This request is clear and relevant.
This request is not clear or is not relevant.

+4

I need a crawler which can extract all pages metadata and URL into a database.

Ask clarification

1 Recommendation

PHP Search Engine Crawler: Crawl pages and extract links, images and metadata

This recommendation solves the problem.
This recommendation does not solve the problem.

+3

Picture of Manuel Lemos by Manuel Lemos Reputation 23770 - 8 years ago (2016-02-25) Comment

This class does exactly what you ask. It can crawl Web site pages and extract links and metadata.

It does not crawl the whole site automatically but the example script demonstrates how to do it calling the class recursively.

There are other classes that crawl the site recursively but they do not extract the pages meta data.

Recommend package
: 
: