public class ParserJob extends NutchTool implements Tool
| Modifier and Type | Class and Description |
|---|---|
static class |
ParserJob.ParserMapper |
| Modifier and Type | Field and Description |
|---|---|
static java.lang.String |
SKIP_TRUNCATED |
currentJob, currentJobNum, numJobs, results, status| Constructor and Description |
|---|
ParserJob() |
ParserJob(Configuration conf) |
| Modifier and Type | Method and Description |
|---|---|
Configuration |
getConf() |
java.util.Collection<WebPage.Field> |
getFields(Job job) |
static boolean |
isTruncated(java.lang.String url,
WebPage page)
Checks if the page's content is truncated.
|
static void |
main(java.lang.String[] args) |
int |
parse(java.lang.String batchId,
boolean shouldResume,
boolean force) |
int |
parse(java.lang.String batchId,
boolean shouldResume,
boolean force,
boolean sitemap) |
java.util.Map<java.lang.String,java.lang.Object> |
run(java.util.Map<java.lang.String,java.lang.Object> args)
Runs the tool, using a map of arguments.
|
int |
run(java.lang.String[] args) |
void |
setConf(Configuration conf) |
getProgress, getStatus, killJob, stopJobpublic static final java.lang.String SKIP_TRUNCATED
public ParserJob()
public ParserJob(Configuration conf)
public static boolean isTruncated(java.lang.String url,
WebPage page)
url - page - true. When it is not, or when
it could be determined, false.public java.util.Collection<WebPage.Field> getFields(Job job)
public Configuration getConf()
getConf in interface ConfigurablegetConf in class Configuredpublic void setConf(Configuration conf)
setConf in interface ConfigurablesetConf in class Configuredpublic java.util.Map<java.lang.String,java.lang.Object> run(java.util.Map<java.lang.String,java.lang.Object> args)
throws java.lang.Exception
NutchToolpublic int parse(java.lang.String batchId,
boolean shouldResume,
boolean force)
throws java.lang.Exception
java.lang.Exceptionpublic int parse(java.lang.String batchId,
boolean shouldResume,
boolean force,
boolean sitemap)
throws java.lang.Exception
java.lang.Exceptionpublic int run(java.lang.String[] args)
throws java.lang.Exception
public static void main(java.lang.String[] args)
throws java.lang.Exception
java.lang.ExceptionCopyright © 2019 The Apache Software Foundation