Package org.apache.nutch.scoring.webgraph
package org.apache.nutch.scoring.webgraph
-
ClassesClassDescriptionA class for holding link information including the url, anchor text, a score, the timestamp of the link and a link type.The LinkDumper tool creates a database of node to inlink information that can be read using the nested Reader class.Inverts outlinks from the WebGraph to inlinks and attaches node information.Wraps all values in ObjectWritables.Inverts outlinks to inlinks while attaching node information to the outlink.Bean class which holds url to node information.Writable class which holds an array of LinkNode objects.Merges LinkNode objects into a single array value per url.Reader class which will print out the url and all of its inlinks to system out.A class which holds the number of inlinks and outlinks for a given url along with an inlink score from a link analysis program and any metadata.A tools that dumps out the top urls by number of inlinks, number of outlinks, or by score, to a text file.Outputs the hosts or domains with an associated value.Outputs the host or domain as key for this record and numInlinks, numOutlinks or score as the value.Outputs either the sum or the top value for this record.Outputs the top urls sorted in descending order.Outputs the url with the appropriate number of inlinks, outlinks, or for score.Flips and collects the url and numeric sort value.Reads and prints to system out information for a single node from the NodeDb in the WebGraph.Updates the score from the WebGraph node database into the crawl database.Changes input into ObjectWritables.Creates new CrawlDatum objects with the updated score from the NodeDb or with a cleared score.Creates three databases, one for inlinks, one for outlinks, and a node database that holds the number of in and outlinks to a url and the current score for the url.The OutlinkDb creates a database of all outlinks.Passes through existing LinkDatum objects from an existing OutlinkDb and maps out new LinkDatum objects from new crawls ParseData.