PHP Classes

Web Crawler using MySQL DB: Retrieve Web pages and store links in a database

Recommend this page to a friend!

  Detailed description   Download Download .zip .tar.gz   Install with Composer Install with Composer  
This class can be used to retrieve Web pages and store the URLs links in a MySQL database.

It retrieves a given Web page and parses its HTML content to extract the URLs of links and frames.

The URLs that are crawled are stored in a MySQL database table if the URL was not yet stored previously.

The class can also display in a Web page the list of URLs already stored from a given domain.

  Author Author  
Picture of vivek
Name: vivek <contact>
Classes: 1 package by
Country: India India
Age: ???
All time rank: 94755 in India India
Week rank: 587 Up47 in India India Up

  Classes of vivek  >  Web Crawler using MySQL DB  >  Download Download .zip .tar.gz  >  Support forum Support forum (2)  >  Blog Blog  
Name: Web Crawler using MySQL DB
Base name: dbcrawler
Description: Retrieve Web pages and store links in a database
Version: 1.0.0
PHP version: 4.0
Licenses: BSD License
GNU Free Documentation License (FDL)
Free For Educational Use
GNU General Public License (GPL)
All time users: 3386 users
All time rank: 1015
Week users: 0 users
Week rank: 106 Equal
 
  Groups   Rate classes User ratings   Applications   Files Files  

  Groups  
Group folder image HTML HTML generation and processing View top rated classes
Group folder image Databases Database management, accessing and searching View top rated classes
Group folder image Searching Search engines, crawling and indexing View top rated classes


  User ratings  
RatingsUtility Consistency Documentation Examples Tests Videos Overall Rank
All time: Good (81%) Good (81%) - Good (81%) - - Not sure (56%) 1696
Month: Not yet rated by the users

  Applications that use this package  
No pages of applications that use this class were specified.

Add link image If you know an application of this package, send a message to the author to add a link here.

  Files folder image Files  
File Role Description
Plain text file spider_advanced.class.php Class advanced spider
Plain text file spider_advanced_example.php Example advanced class example file
Accessible without login Plain text file spider_simple.class.php Class DBCrawler
Plain text file spider_simple_example.php Example example

Install with Composer Install with Composer - Download Download all files: dbcrawler.tar.gz dbcrawler.zip
NOTICE: if you are using a download manager program like 'GetRight', please Login before trying to download this archive.
  Files folder image Files  
File Role Description
Plain text file spider_advanced.class.php Class advanced spider
Plain text file spider_advanced_example.php Example advanced class example file
Accessible without login Plain text file spider_simple.class.php Class DBCrawler
Plain text file spider_simple_example.php Example example

Install with Composer Install with Composer - Download Download all files: dbcrawler.tar.gz dbcrawler.zip
NOTICE: if you are using a download manager program like 'GetRight', please Login before trying to download this archive.