Uses of Class
org.apache.nutch.util.NutchTool
Packages that use NutchTool
Package
Description
Crawl control code and tools to run the crawler.
The Nutch multi-threaded fetching module
Index content, configure and run indexing and cleaning jobs to
add, update, and delete documents from an index.
The
Parse interface and related classes.Miscellaneous tools.
-
Uses of NutchTool in org.apache.nutch.crawl
Subclasses of NutchTool in org.apache.nutch.crawlModifier and TypeClassDescriptionclassThis class takes the output of the fetcher and updates the crawldb accordingly.classGeneric deduplicator which groups fetched URLs with the same digest and marks all of them as duplicate except the one with the highest score (based on the score in the crawldb, which is not necessarily the same as the score indexed).classGenerates a subset of a CrawlDb to fetch.classInjector takes a flat text file of URLs (or a folder containing text files) and merges ("injects") these URLs into the CrawlDb.classMaintains an inverted link map, listing incoming links for each url. -
Uses of NutchTool in org.apache.nutch.fetcher
Subclasses of NutchTool in org.apache.nutch.fetcher -
Uses of NutchTool in org.apache.nutch.indexer
Subclasses of NutchTool in org.apache.nutch.indexerModifier and TypeClassDescriptionclassGeneric indexer which relies on the plugins implementing IndexWriter -
Uses of NutchTool in org.apache.nutch.parse
Subclasses of NutchTool in org.apache.nutch.parse -
Uses of NutchTool in org.apache.nutch.tools
Subclasses of NutchTool in org.apache.nutch.toolsModifier and TypeClassDescriptionclassThe Common Crawl Data Dumper tool enables one to reverse generate the raw content from Nutch segment data directories into a common crawling data format, consumed by many applications.