• About
  • Policies
  • What is open access
  • Library
  • Contact
Advanced search
      View Item 
      •   BUIR Home
      • Scholarly Publications
      • Faculty of Engineering
      • Department of Computer Engineering
      • View Item
      •   BUIR Home
      • Scholarly Publications
      • Faculty of Engineering
      • Department of Computer Engineering
      • View Item
      JavaScript is disabled for your browser. Some features of this site may not work without it.

      Data-parallel web crawling models

      Thumbnail
      View / Download
      175.9 Kb
      Author(s)
      Cambazoglu, B. B.
      Turk, A.
      Aykanat, Cevdet
      Date
      2004
      Source Title
      Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
      Print ISSN
      0302-9743
       
      1611-3349
       
      Publisher
      Springer
      Volume
      3280
      Pages
      801 - 809
      Language
      English
      Type
      Article
      Item Usage Stats
      195
      views
      196
      downloads
      Abstract
      The need to quickly locate, gather, and store the vast amount of material in the Web necessitates parallel computing. In this paper, we propose two models, based on multi-constraint graph-partitioning, for efficient data-parallel Web crawling. The models aim to balance the amount of data downloaded and stored by each processor as well as balancing the number of page requests made by the processors. The models also minimize the total volume of communication during the link exchange between the processors. To evaluate the performance of the models, experimental results are presented on a sample Web repository containing around 915,000 pages. © Springer-Verlag 2004.
      Keywords
      Artificial intelligence
      Computers
      Data parallel
      Multi-constraints
      Web Crawling
      Web repositories
      Parallel processing systems
      Permalink
      http://hdl.handle.net/11693/24172
      Published Version (Please cite this version)
      https://doi.org/10.1007/978-3-540-30182-0_80
      Collections
      • Department of Computer Engineering 1561
      Show full item record

      Browse

      All of BUIRCommunities & CollectionsTitlesAuthorsAdvisorsBy Issue DateKeywordsTypeDepartmentsCoursesThis CollectionTitlesAuthorsAdvisorsBy Issue DateKeywordsTypeDepartmentsCourses

      My Account

      Login

      Statistics

      View Usage StatisticsView Google Analytics Statistics

      Bilkent University

      If you have trouble accessing this page and need to request an alternate format, contact the site administrator. Phone: (312) 290 2976
      © Bilkent University - Library IT

      Contact Us | Send Feedback | Off-Campus Access | Admin | Privacy