nutch 1.2问题求解

13/02/28 14:15:32 INFO crawl.Crawl: crawl started in: test
13/02/28 14:15:32 INFO crawl.Crawl: rootUrlDir = testData/urls.txt
13/02/28 14:15:32 INFO crawl.Crawl: threads = 10
13/02/28 14:15:32 INFO crawl.Crawl: depth = 3
13/02/28 14:15:32 INFO crawl.Crawl: indexer=lucene
13/02/28 14:15:32 INFO crawl.Crawl: topN = 10
13/02/28 14:15:32 INFO crawl.Injector: Injector: starting at 2013-02-28 14:15:32
13/02/28 14:15:32 INFO crawl.Injector: Injector: crawlDb: test/crawldb
13/02/28 14:15:32 INFO crawl.Injector: Injector: urlDir: testData/urls.txt
13/02/28 14:15:32 INFO crawl.Injector: Injector: Converting injected urls to crawl db entries.
13/02/28 14:15:32 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
13/02/28 14:15:32 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/02/28 14:15:32 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/02/28 14:15:32 INFO mapred.FileInputFormat: Total input paths to process : 1
13/02/28 14:15:33 INFO mapred.JobClient: Running job: job_local_0001
13/02/28 14:15:33 INFO mapred.FileInputFormat: Total input paths to process : 1
13/02/28 14:15:33 INFO mapred.MapTask: numReduceTasks: 1
13/02/28 14:15:33 INFO mapred.MapTask: io.sort.mb = 100
13/02/28 14:15:33 INFO mapred.MapTask: data buffer = 79691776/99614720
13/02/28 14:15:33 INFO mapred.MapTask: record buffer = 262144/327680
13/02/28 14:15:33 INFO plugin.PluginRepository: Plugins: looking in: D:\workspace\Nutch1.2\.\plugins
13/02/28 14:15:33 INFO plugin.PluginRepository: Plugin Auto-activation mode: [true]
13/02/28 14:15:33 INFO plugin.PluginRepository: Registered Plugins:
13/02/28 14:15:33 INFO plugin.PluginRepository:  the nutch core extension points (nutch-extensionpoints)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Basic Query Filter (query-basic)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Basic URL Normalizer (urlnormalizer-basic)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Html Parse Plug-in (parse-html)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Basic Indexing Filter (index-basic)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Site Query Filter (query-site)
13/02/28 14:15:33 INFO plugin.PluginRepository:  HTTP Framework (lib-http)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Text Parse Plug-in (parse-text)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Regex URL Filter (urlfilter-regex)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Pass-through URL Normalizer (urlnormalizer-pass)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Http Protocol Plug-in (protocol-http)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Regex URL Normalizer (urlnormalizer-regex)
13/02/28 14:15:33 INFO plugin.PluginRepository:  OPIC Scoring Plug-in (scoring-opic)
13/02/28 14:15:33 INFO plugin.PluginRepository:  CyberNeko HTML Parser (lib-nekohtml)
13/02/28 14:15:33 INFO plugin.PluginRepository:  JavaScript Parser (parse-js)
13/02/28 14:15:33 INFO plugin.PluginRepository:  URL Query Filter (query-url)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Lucene Highlighter Summary Plug-in (summary-lucene)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Regex URL Filter Framework (lib-regex-filter)
13/02/28 14:15:33 INFO plugin.PluginRepository: Registered Extension-Points:
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Summarizer (org.apache.nutch.searcher.Summarizer)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Protocol (org.apache.nutch.protocol.Protocol)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Segment Merge Filter (org.apache.nutch.segment.SegmentMergeFilter)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Analysis (org.apache.nutch.analysis.NutchAnalyzer)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Field Filter (org.apache.nutch.indexer.field.FieldFilter)
13/02/28 14:15:33 INFO plugin.PluginRepository:  HTML Parse Filter (org.apache.nutch.parse.HtmlParseFilter)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Query Filter (org.apache.nutch.searcher.QueryFilter)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Search Results Response Writer (org.apache.nutch.searcher.response.ResponseWriter)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch URL Normalizer (org.apache.nutch.net.URLNormalizer)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch URL Filter (org.apache.nutch.net.URLFilter)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Online Search Results Clustering Plugin (org.apache.nutch.clustering.OnlineClusterer)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Indexing Filter (org.apache.nutch.indexer.IndexingFilter)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Content Parser (org.apache.nutch.parse.Parser)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Nutch Scoring (org.apache.nutch.scoring.ScoringFilter)
13/02/28 14:15:33 INFO plugin.PluginRepository:  Ontology Model Loader (org.apache.nutch.ontology.Ontology)
13/02/28 14:15:33 INFO conf.Configuration: found resource regex-urlfilter.txt at file:/D:/workspace/Nutch1.2/bin/regex-urlfilter.txt
13/02/28 14:15:33 WARN regex.RegexURLNormalizer: can't find rules for scope 'inject', using default
13/02/28 14:15:33 INFO mapred.MapTask: Starting flush of map output
13/02/28 14:15:34 INFO mapred.JobClient:  map 0% reduce 0%
13/02/28 14:15:34 INFO mapred.MapTask: Finished spill 0
13/02/28 14:15:34 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting
13/02/28 14:15:34 INFO mapred.LocalJobRunner: file:/D:/workspace/Nutch1.2/testData/urls.txt:0+21
13/02/28 14:15:34 INFO mapred.TaskRunner: Task 'attempt_local_0001_m_000000_0' done.
13/02/28 14:15:34 INFO mapred.LocalJobRunner:
13/02/28 14:15:34 INFO mapred.Merger: Merging 1 sorted segments
13/02/28 14:15:34 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 53 bytes
13/02/28 14:15:34 INFO mapred.LocalJobRunner:
13/02/28 14:15:34 INFO mapred.TaskRunner: Task:attempt_local_0001_r_000000_0 is done. And is in the process of commiting
13/02/28 14:15:34 INFO mapred.LocalJobRunner:
13/02/28 14:15:34 INFO mapred.TaskRunner: Task attempt_local_0001_r_000000_0 is allowed to commit now
13/02/28 14:15:34 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0001_r_000000_0' to file:/tmp/hadoop-Administrator/mapred/temp/inject-temp-861851334
13/02/28 14:15:34 INFO mapred.LocalJobRunner: reduce > reduce
13/02/28 14:15:34 INFO mapred.TaskRunner: Task 'attempt_local_0001_r_000000_0' done.
13/02/28 14:15:35 INFO mapred.JobClient:  map 100% reduce 100%
13/02/28 14:15:35 INFO mapred.JobClient: Job complete: job_local_0001
13/02/28 14:15:35 INFO mapred.JobClient: Counters: 13
13/02/28 14:15:35 INFO mapred.JobClient:   FileSystemCounters
13/02/28 14:15:35 INFO mapred.JobClient:     FILE_BYTES_READ=47717
13/02/28 14:15:35 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=96315
13/02/28 14:15:35 INFO mapred.JobClient:   Map-Reduce Framework
13/02/28 14:15:35 INFO mapred.JobClient:     Reduce input groups=1
13/02/28 14:15:35 INFO mapred.JobClient:     Combine output records=0
13/02/28 14:15:35 INFO mapred.JobClient:     Map input records=1
13/02/28 14:15:35 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/02/28 14:15:35 INFO mapred.JobClient:     Reduce output records=1
13/02/28 14:15:35 INFO mapred.JobClient:     Spilled Records=2
13/02/28 14:15:35 INFO mapred.JobClient:     Map output bytes=49
13/02/28 14:15:35 INFO mapred.JobClient:     Map input bytes=21
13/02/28 14:15:35 INFO mapred.JobClient:     Combine input records=0
13/02/28 14:15:35 INFO mapred.JobClient:     Map output records=1
13/02/28 14:15:35 INFO mapred.JobClient:     Reduce input records=1
13/02/28 14:15:35 INFO crawl.Injector: Injector: Merging injected urls into crawl db.
13/02/28 14:15:35 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
13/02/28 14:15:35 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/02/28 14:15:35 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/02/28 14:15:35 INFO mapred.FileInputFormat: Total input paths to process : 2
13/02/28 14:15:35 INFO mapred.JobClient: Running job: job_local_0002
13/02/28 14:15:35 INFO mapred.FileInputFormat: Total input paths to process : 2
13/02/28 14:15:35 INFO mapred.MapTask: numReduceTasks: 1
13/02/28 14:15:35 INFO mapred.MapTask: io.sort.mb = 100
13/02/28 14:15:35 INFO mapred.MapTask: data buffer = 79691776/99614720
13/02/28 14:15:35 INFO mapred.MapTask: record buffer = 262144/327680
13/02/28 14:15:35 INFO mapred.MapTask: Starting flush of map output
13/02/28 14:15:35 INFO mapred.MapTask: Finished spill 0
13/02/28 14:15:35 INFO mapred.TaskRunner: Task:attempt_local_0002_m_000000_0 is done. And is in the process of commiting
13/02/28 14:15:35 INFO mapred.LocalJobRunner: file:/D:/workspace/Nutch1.2/test/crawldb/current/part-00000/data:0+91920
13/02/28 14:15:35 INFO mapred.TaskRunner: Task 'attempt_local_0002_m_000000_0' done.
13/02/28 14:15:35 INFO mapred.MapTask: numReduceTasks: 1
13/02/28 14:15:35 INFO mapred.MapTask: io.sort.mb = 100
13/02/28 14:15:50 INFO mapred.JobClient:  map 100% reduce 0%
13/02/28 14:15:53 INFO mapred.LocalJobRunner: file:/tmp/hadoop-Administrator/mapred/temp/inject-temp-861851334/part-00000:0+143
13/02/28 14:15:54 INFO mapred.JobClient:  map 50% reduce 0%
13/02/28 14:15:56 INFO mapred.MapTask: data buffer = 79691776/99614720
13/02/28 14:15:56 INFO mapred.MapTask: record buffer = 262144/327680
13/02/28 14:16:00 INFO mapred.MapTask: Starting flush of map output
13/02/28 14:16:01 INFO mapred.MapTask: Finished spill 0
13/02/28 14:16:02 INFO mapred.TaskRunner: Task:attempt_local_0002_m_000001_0 is done. And is in the process of commiting
13/02/28 14:16:03 INFO mapred.LocalJobRunner: file:/tmp/hadoop-Administrator/mapred/temp/inject-temp-861851334/part-00000:0+143
13/02/28 14:16:03 INFO mapred.TaskRunner: Task 'attempt_local_0002_m_000001_0' done.
13/02/28 14:16:03 INFO mapred.JobClient:  map 100% reduce 0%
13/02/28 14:16:04 INFO mapred.LocalJobRunner:
13/02/28 14:16:04 INFO mapred.Merger: Merging 2 sorted segments
13/02/28 14:16:04 INFO mapred.Merger: Down to the last merge-pass, with 2 segments left of total size: 85148 bytes
13/02/28 14:16:04 INFO mapred.LocalJobRunner:
13/02/28 14:16:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
13/02/28 14:16:17 INFO compress.CodecPool: Got brand-new compressor
13/02/28 14:16:17 INFO mapred.TaskRunner: Task:attempt_local_0002_r_000000_0 is done. And is in the process of commiting
13/02/28 14:16:17 INFO mapred.LocalJobRunner:
13/02/28 14:16:17 INFO mapred.TaskRunner: Task attempt_local_0002_r_000000_0 is allowed to commit now
13/02/28 14:16:17 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0002_r_000000_0' to file:/D:/workspace/Nutch1.2/test/crawldb/1817072542
13/02/28 14:16:17 INFO mapred.LocalJobRunner: reduce > reduce
13/02/28 14:16:17 INFO mapred.TaskRunner: Task 'attempt_local_0002_r_000000_0' done.
13/02/28 14:16:18 INFO mapred.JobClient:  map 100% reduce 100%
13/02/28 14:16:18 INFO mapred.JobClient: Job complete: job_local_0002
13/02/28 14:16:18 INFO mapred.JobClient: Counters: 13
13/02/28 14:16:18 INFO mapred.JobClient:   FileSystemCounters
13/02/28 14:16:18 INFO mapred.JobClient:     FILE_BYTES_READ=507077
13/02/28 14:16:18 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=639545
13/02/28 14:16:18 INFO mapred.JobClient:   Map-Reduce Framework
13/02/28 14:16:18 INFO mapred.JobClient:     Reduce input groups=983
13/02/28 14:16:18 INFO mapred.JobClient:     Combine output records=0
13/02/28 14:16:18 INFO mapred.JobClient:     Map input records=984
13/02/28 14:16:18 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/02/28 14:16:18 INFO mapred.JobClient:     Reduce output records=983
13/02/28 14:16:18 INFO mapred.JobClient:     Spilled Records=1968
13/02/28 14:16:18 INFO mapred.JobClient:     Map output bytes=83139
13/02/28 14:16:18 INFO mapred.JobClient:     Map input bytes=91891
13/02/28 14:16:18 INFO mapred.JobClient:     Combine input records=0
13/02/28 14:16:18 INFO mapred.JobClient:     Map output records=984
13/02/28 14:16:18 INFO mapred.JobClient:     Reduce input records=984
13/02/28 14:16:18 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
13/02/28 14:16:18 INFO crawl.Injector: Injector: finished at 2013-02-28 14:16:18, elapsed: 00:00:46
13/02/28 14:16:20 INFO crawl.Generator: Generator: starting at 2013-02-28 14:16:20
13/02/28 14:16:20 INFO crawl.Generator: Generator: Selecting best-scoring urls due for fetch.
13/02/28 14:16:20 INFO crawl.Generator: Generator: filtering: true
13/02/28 14:16:20 INFO crawl.Generator: Generator: normalizing: true
13/02/28 14:16:20 INFO crawl.Generator: Generator: topN: 10
13/02/28 14:16:20 INFO crawl.Generator: Generator: jobtracker is 'local', generating exactly one partition.
13/02/28 14:16:20 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
13/02/28 14:16:20 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/02/28 14:16:21 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/02/28 14:16:21 INFO mapred.FileInputFormat: Total input paths to process : 1
13/02/28 14:16:22 INFO mapred.JobClient: Running job: job_local_0003
13/02/28 14:16:22 INFO mapred.FileInputFormat: Total input paths to process : 1
13/02/28 14:16:22 INFO mapred.MapTask: numReduceTasks: 1
13/02/28 14:16:22 INFO mapred.MapTask: io.sort.mb = 100
13/02/28 14:16:23 INFO mapred.JobClient:  map 0% reduce 0%
13/02/28 14:16:30 INFO mapred.MapTask: data buffer = 79691776/99614720
13/02/28 14:16:30 INFO mapred.MapTask: record buffer = 262144/327680
13/02/28 14:16:32 INFO plugin.PluginRepository: Plugins: looking in: D:\workspace\Nutch1.2\.\plugins
13/02/28 14:16:33 INFO mapred.LocalJobRunner: file:/D:/workspace/Nutch1.2/test/crawldb/current/part-00000/data:0+91920
13/02/28 14:16:39 INFO plugin.PluginRepository: Plugin Auto-activation mode: [true]
13/02/28 14:16:39 INFO plugin.PluginRepository: Registered Plugins:
13/02/28 14:16:39 INFO plugin.PluginRepository:  the nutch core extension points (nutch-extensionpoints)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Basic Query Filter (query-basic)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Basic URL Normalizer (urlnormalizer-basic)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Html Parse Plug-in (parse-html)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Basic Indexing Filter (index-basic)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Site Query Filter (query-site)
13/02/28 14:16:39 INFO plugin.PluginRepository:  HTTP Framework (lib-http)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Text Parse Plug-in (parse-text)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Regex URL Filter (urlfilter-regex)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Pass-through URL Normalizer (urlnormalizer-pass)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Http Protocol Plug-in (protocol-http)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Regex URL Normalizer (urlnormalizer-regex)
13/02/28 14:16:39 INFO plugin.PluginRepository:  OPIC Scoring Plug-in (scoring-opic)
13/02/28 14:16:39 INFO plugin.PluginRepository:  CyberNeko HTML Parser (lib-nekohtml)
13/02/28 14:16:39 INFO plugin.PluginRepository:  JavaScript Parser (parse-js)
13/02/28 14:16:39 INFO plugin.PluginRepository:  URL Query Filter (query-url)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Lucene Highlighter Summary Plug-in (summary-lucene)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Regex URL Filter Framework (lib-regex-filter)
13/02/28 14:16:39 INFO plugin.PluginRepository: Registered Extension-Points:
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Summarizer (org.apache.nutch.searcher.Summarizer)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Protocol (org.apache.nutch.protocol.Protocol)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Segment Merge Filter (org.apache.nutch.segment.SegmentMergeFilter)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Analysis (org.apache.nutch.analysis.NutchAnalyzer)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Field Filter (org.apache.nutch.indexer.field.FieldFilter)
13/02/28 14:16:39 INFO plugin.PluginRepository:  HTML Parse Filter (org.apache.nutch.parse.HtmlParseFilter)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Query Filter (org.apache.nutch.searcher.QueryFilter)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Search Results Response Writer (org.apache.nutch.searcher.response.ResponseWriter)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch URL Normalizer (org.apache.nutch.net.URLNormalizer)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch URL Filter (org.apache.nutch.net.URLFilter)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Online Search Results Clustering Plugin (org.apache.nutch.clustering.OnlineClusterer)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Indexing Filter (org.apache.nutch.indexer.IndexingFilter)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Content Parser (org.apache.nutch.parse.Parser)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Nutch Scoring (org.apache.nutch.scoring.ScoringFilter)
13/02/28 14:16:39 INFO plugin.PluginRepository:  Ontology Model Loader (org.apache.nutch.ontology.Ontology)
13/02/28 14:16:39 INFO conf.Configuration: found resource regex-urlfilter.txt at file:/D:/workspace/Nutch1.2/bin/regex-urlfilter.txt
13/02/28 14:16:40 INFO crawl.FetchScheduleFactory: Using FetchSchedule impl: org.apache.nutch.crawl.DefaultFetchSchedule
13/02/28 14:16:40 INFO crawl.AbstractFetchSchedule: defaultInterval=2592000
13/02/28 14:16:40 INFO crawl.AbstractFetchSchedule: maxInterval=7776000
13/02/28 14:16:40 WARN regex.RegexURLNormalizer: can't find rules for scope 'partition', using default
13/02/28 14:16:41 INFO mapred.MapTask: Starting flush of map output
13/02/28 14:16:45 INFO mapred.LocalJobRunner: file:/D:/workspace/Nutch1.2/test/crawldb/current/part-00000/data:0+91920
13/02/28 14:16:45 INFO mapred.MapTask: Finished spill 0
13/02/28 14:16:45 INFO mapred.TaskRunner: Task:attempt_local_0003_m_000000_0 is done. And is in the process of commiting
13/02/28 14:16:45 INFO mapred.JobClient:  map 100% reduce 0%
13/02/28 14:16:45 INFO mapred.LocalJobRunner: file:/D:/workspace/Nutch1.2/test/crawldb/current/part-00000/data:0+91920
13/02/28 14:16:45 INFO mapred.TaskRunner: Task 'attempt_local_0003_m_000000_0' done.
13/02/28 14:16:45 INFO mapred.LocalJobRunner:
13/02/28 14:16:45 INFO mapred.Merger: Merging 1 sorted segments
13/02/28 14:16:45 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 63503 bytes
13/02/28 14:16:45 INFO mapred.LocalJobRunner:
13/02/28 14:16:45 INFO plugin.PluginRepository: Plugins: looking in: D:\workspace\Nutch1.2\.\plugins
13/02/28 14:16:47 INFO plugin.PluginRepository: Plugin Auto-activation mode: [true]
13/02/28 14:16:47 INFO plugin.PluginRepository: Registered Plugins:
13/02/28 14:16:47 INFO plugin.PluginRepository:  the nutch core extension points (nutch-extensionpoints)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Basic Query Filter (query-basic)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Basic URL Normalizer (urlnormalizer-basic)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Html Parse Plug-in (parse-html)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Basic Indexing Filter (index-basic)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Site Query Filter (query-site)
13/02/28 14:16:47 INFO plugin.PluginRepository:  HTTP Framework (lib-http)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Text Parse Plug-in (parse-text)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Regex URL Filter (urlfilter-regex)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Pass-through URL Normalizer (urlnormalizer-pass)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Http Protocol Plug-in (protocol-http)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Regex URL Normalizer (urlnormalizer-regex)
13/02/28 14:16:47 INFO plugin.PluginRepository:  OPIC Scoring Plug-in (scoring-opic)
13/02/28 14:16:47 INFO plugin.PluginRepository:  CyberNeko HTML Parser (lib-nekohtml)
13/02/28 14:16:47 INFO plugin.PluginRepository:  JavaScript Parser (parse-js)
13/02/28 14:16:47 INFO plugin.PluginRepository:  URL Query Filter (query-url)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Lucene Highlighter Summary Plug-in (summary-lucene)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Regex URL Filter Framework (lib-regex-filter)
13/02/28 14:16:47 INFO plugin.PluginRepository: Registered Extension-Points:
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Summarizer (org.apache.nutch.searcher.Summarizer)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Protocol (org.apache.nutch.protocol.Protocol)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Segment Merge Filter (org.apache.nutch.segment.SegmentMergeFilter)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Analysis (org.apache.nutch.analysis.NutchAnalyzer)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Field Filter (org.apache.nutch.indexer.field.FieldFilter)
13/02/28 14:16:47 INFO plugin.PluginRepository:  HTML Parse Filter (org.apache.nutch.parse.HtmlParseFilter)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Query Filter (org.apache.nutch.searcher.QueryFilter)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Search Results Response Writer (org.apache.nutch.searcher.response.ResponseWriter)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch URL Normalizer (org.apache.nutch.net.URLNormalizer)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch URL Filter (org.apache.nutch.net.URLFilter)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Online Search Results Clustering Plugin (org.apache.nutch.clustering.OnlineClusterer)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Indexing Filter (org.apache.nutch.indexer.IndexingFilter)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Content Parser (org.apache.nutch.parse.Parser)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Nutch Scoring (org.apache.nutch.scoring.ScoringFilter)
13/02/28 14:16:47 INFO plugin.PluginRepository:  Ontology Model Loader (org.apache.nutch.ontology.Ontology)
13/02/28 14:16:47 INFO conf.Configuration: found resource regex-urlfilter.txt at file:/D:/workspace/Nutch1.2/bin/regex-urlfilter.txt
13/02/28 14:16:47 INFO crawl.FetchScheduleFactory: Using FetchSchedule impl: org.apache.nutch.crawl.DefaultFetchSchedule
13/02/28 14:16:47 INFO crawl.AbstractFetchSchedule: defaultInterval=2592000
13/02/28 14:16:47 INFO crawl.AbstractFetchSchedule: maxInterval=7776000
13/02/28 14:16:47 WARN regex.RegexURLNormalizer: can't find rules for scope 'generate_host_count', using default
13/02/28 14:16:47 INFO mapred.TaskRunner: Task:attempt_local_0003_r_000000_0 is done. And is in the process of commiting
13/02/28 14:16:47 INFO mapred.LocalJobRunner:
13/02/28 14:16:47 INFO mapred.TaskRunner: Task attempt_local_0003_r_000000_0 is allowed to commit now
13/02/28 14:16:47 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0003_r_000000_0' to file:/tmp/hadoop-Administrator/mapred/temp/generate-temp-1362032179375
13/02/28 14:16:47 INFO mapred.LocalJobRunner: reduce > reduce
13/02/28 14:16:47 INFO mapred.TaskRunner: Task 'attempt_local_0003_r_000000_0' done.
13/02/28 14:16:47 INFO mapred.JobClient:  map 100% reduce 100%
13/02/28 14:16:47 INFO mapred.JobClient: Job complete: job_local_0003
13/02/28 14:16:47 INFO mapred.JobClient: Counters: 13
13/02/28 14:16:47 INFO mapred.JobClient:   FileSystemCounters
13/02/28 14:16:47 INFO mapred.JobClient:     FILE_BYTES_READ=749901
13/02/28 14:16:47 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=778794
13/02/28 14:16:47 INFO mapred.JobClient:   Map-Reduce Framework
13/02/28 14:16:47 INFO mapred.JobClient:     Reduce input groups=43
13/02/28 14:16:47 INFO mapred.JobClient:     Combine output records=0
13/02/28 14:16:47 INFO mapred.JobClient:     Map input records=983
13/02/28 14:16:47 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/02/28 14:16:47 INFO mapred.JobClient:     Reduce output records=10
13/02/28 14:16:47 INFO mapred.JobClient:     Spilled Records=1226
13/02/28 14:16:47 INFO mapred.JobClient:     Map output bytes=62261
13/02/28 14:16:47 INFO mapred.JobClient:     Map input bytes=91834
13/02/28 14:16:47 INFO mapred.JobClient:     Combine input records=0
13/02/28 14:16:47 INFO mapred.JobClient:     Map output records=613
13/02/28 14:16:47 INFO mapred.JobClient:     Reduce input records=10
13/02/28 14:16:47 INFO crawl.Generator: Generator: Partitioning selected urls for politeness.
13/02/28 14:16:48 INFO crawl.Generator: Generator: segment: test/segments/20130228141648
13/02/28 14:16:48 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
13/02/28 14:16:49 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/02/28 14:16:49 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/02/28 14:16:49 INFO mapred.FileInputFormat: Total input paths to process : 1
13/02/28 14:16:50 INFO mapred.JobClient: Running job: job_local_0004
13/02/28 14:16:50 INFO mapred.FileInputFormat: Total input paths to process : 1
13/02/28 14:16:50 INFO mapred.MapTask: numReduceTasks: 1
13/02/28 14:16:50 INFO mapred.MapTask: io.sort.mb = 100
13/02/28 14:16:51 INFO mapred.JobClient:  map 0% reduce 0%
13/02/28 14:17:01 INFO mapred.MapTask: data buffer = 79691776/99614720
13/02/28 14:17:02 INFO mapred.MapTask: record buffer = 262144/327680
13/02/28 14:17:02 INFO mapred.LocalJobRunner: file:/tmp/hadoop-Administrator/mapred/temp/generate-temp-1362032179375/fetchlist-1/part-00000:0+1196
13/02/28 14:17:09 INFO plugin.PluginRepository: Plugins: looking in: D:\workspace\Nutch1.2\.\plugins
13/02/28 14:17:22 INFO plugin.PluginRepository: Plugin Auto-activation mode: [true]
13/02/28 14:17:22 INFO plugin.PluginRepository: Registered Plugins:
13/02/28 14:17:22 INFO plugin.PluginRepository:  the nutch core extension points (nutch-extensionpoints)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Basic Query Filter (query-basic)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Basic URL Normalizer (urlnormalizer-basic)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Html Parse Plug-in (parse-html)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Basic Indexing Filter (index-basic)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Site Query Filter (query-site)
13/02/28 14:17:22 INFO plugin.PluginRepository:  HTTP Framework (lib-http)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Text Parse Plug-in (parse-text)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Regex URL Filter (urlfilter-regex)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Pass-through URL Normalizer (urlnormalizer-pass)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Http Protocol Plug-in (protocol-http)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Regex URL Normalizer (urlnormalizer-regex)
13/02/28 14:17:22 INFO plugin.PluginRepository:  OPIC Scoring Plug-in (scoring-opic)
13/02/28 14:17:22 INFO plugin.PluginRepository:  CyberNeko HTML Parser (lib-nekohtml)
13/02/28 14:17:22 INFO plugin.PluginRepository:  JavaScript Parser (parse-js)
13/02/28 14:17:22 INFO plugin.PluginRepository:  URL Query Filter (query-url)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Lucene Highlighter Summary Plug-in (summary-lucene)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Regex URL Filter Framework (lib-regex-filter)
13/02/28 14:17:22 INFO plugin.PluginRepository: Registered Extension-Points:
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Summarizer (org.apache.nutch.searcher.Summarizer)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Protocol (org.apache.nutch.protocol.Protocol)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Segment Merge Filter (org.apache.nutch.segment.SegmentMergeFilter)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Analysis (org.apache.nutch.analysis.NutchAnalyzer)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Field Filter (org.apache.nutch.indexer.field.FieldFilter)
13/02/28 14:17:22 INFO plugin.PluginRepository:  HTML Parse Filter (org.apache.nutch.parse.HtmlParseFilter)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Query Filter (org.apache.nutch.searcher.QueryFilter)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Search Results Response Writer (org.apache.nutch.searcher.response.ResponseWriter)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch URL Normalizer (org.apache.nutch.net.URLNormalizer)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch URL Filter (org.apache.nutch.net.URLFilter)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Online Search Results Clustering Plugin (org.apache.nutch.clustering.OnlineClusterer)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Indexing Filter (org.apache.nutch.indexer.IndexingFilter)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Content Parser (org.apache.nutch.parse.Parser)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Nutch Scoring (org.apache.nutch.scoring.ScoringFilter)
13/02/28 14:17:22 INFO plugin.PluginRepository:  Ontology Model Loader (org.apache.nutch.ontology.Ontology)
13/02/28 14:17:27 WARN regex.RegexURLNormalizer: can't find rules for scope 'partition', using default
13/02/28 14:17:27 INFO mapred.MapTask: Starting flush of map output
13/02/28 14:17:29 INFO mapred.MapTask: Finished spill 0
13/02/28 14:17:29 INFO mapred.TaskRunner: Task:attempt_local_0004_m_000000_0 is done. And is in the process of commiting
13/02/28 14:17:29 INFO mapred.LocalJobRunner: file:/tmp/hadoop-Administrator/mapred/temp/generate-temp-1362032179375/fetchlist-1/part-00000:0+1196
13/02/28 14:17:29 INFO mapred.TaskRunner: Task 'attempt_local_0004_m_000000_0' done.
13/02/28 14:17:30 INFO mapred.LocalJobRunner:
13/02/28 14:17:30 INFO mapred.Merger: Merging 1 sorted segments
13/02/28 14:17:30 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 1418 bytes
13/02/28 14:17:30 INFO mapred.LocalJobRunner:
13/02/28 14:17:30 INFO mapred.JobClient:  map 100% reduce 0%
13/02/28 14:17:30 INFO mapred.TaskRunner: Task:attempt_local_0004_r_000000_0 is done. And is in the process of commiting
13/02/28 14:17:30 INFO mapred.LocalJobRunner:
13/02/28 14:17:30 INFO mapred.TaskRunner: Task attempt_local_0004_r_000000_0 is allowed to commit now
13/02/28 14:17:30 INFO mapred.FileOutputCommitter: Saved output of task 'attempt_local_0004_r_000000_0' to file:/D:/workspace/Nutch1.2/test/segments/20130228141648/crawl_generate
13/02/28 14:17:30 INFO mapred.LocalJobRunner: reduce > reduce
13/02/28 14:17:30 INFO mapred.TaskRunner: Task 'attempt_local_0004_r_000000_0' done.
13/02/28 14:17:31 INFO mapred.JobClient:  map 100% reduce 100%
13/02/28 14:17:31 INFO mapred.JobClient: Job complete: job_local_0004
13/02/28 14:17:31 INFO mapred.JobClient: Counters: 13
13/02/28 14:17:31 INFO mapred.JobClient:   FileSystemCounters
13/02/28 14:17:31 INFO mapred.JobClient:     FILE_BYTES_READ=866456
13/02/28 14:17:31 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=883116
13/02/28 14:17:31 INFO mapred.JobClient:   Map-Reduce Framework
13/02/28 14:17:31 INFO mapred.JobClient:     Reduce input groups=10
13/02/28 14:17:31 INFO mapred.JobClient:     Combine output records=0
13/02/28 14:17:31 INFO mapred.JobClient:     Map input records=10
13/02/28 14:17:31 INFO mapred.JobClient:     Reduce shuffle bytes=0
13/02/28 14:17:31 INFO mapred.JobClient:     Reduce output records=10
13/02/28 14:17:31 INFO mapred.JobClient:     Spilled Records=20
13/02/28 14:17:31 INFO mapred.JobClient:     Map output bytes=1396
13/02/28 14:17:31 INFO mapred.JobClient:     Map input bytes=1088
13/02/28 14:17:31 INFO mapred.JobClient:     Combine input records=0
13/02/28 14:17:31 INFO mapred.JobClient:     Map output records=10
13/02/28 14:17:31 INFO mapred.JobClient:     Reduce input records=10
13/02/28 14:17:31 INFO crawl.Generator: Generator: finished at 2013-02-28 14:17:31, elapsed: 00:01:11
13/02/28 14:17:31 WARN fetcher.Fetcher: Fetcher: Your 'http.agent.name' value should be listed first in 'http.robots.agents' property.
13/02/28 14:17:31 INFO fetcher.Fetcher: Fetcher: starting at 2013-02-28 14:17:31
13/02/28 14:17:31 INFO fetcher.Fetcher: Fetcher: segment: test/segments/20130228141648
13/02/28 14:17:31 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
13/02/28 14:17:31 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/02/28 14:17:31 WARN mapred.JobClient: No job jar file set.  User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/02/28 14:17:31 INFO mapred.FileInputFormat: Total input paths to process : 1
13/02/28 14:17:32 INFO mapred.JobClient: Running job: job_local_0005
13/02/28 14:17:32 INFO mapred.FileInputFormat: Total input paths to process : 1
13/02/28 14:17:32 INFO mapred.MapTask: numReduceTasks: 1
13/02/28 14:17:32 INFO mapred.MapTask: io.sort.mb = 100
13/02/28 14:17:33 INFO mapred.JobClient:  map 0% reduce 0%
13/02/28 14:17:40 INFO mapred.MapTask: data buffer = 79691776/99614720
13/02/28 14:17:40 INFO mapred.MapTask: record buffer = 262144/327680
13/02/28 14:17:40 INFO mapred.LocalJobRunner: file:/D:/workspace/Nutch1.2/test/segments/20130228141648/crawl_generate/part-00000:0+1094
13/02/28 14:17:44 INFO fetcher.Fetcher: Fetcher: threads: 10
13/02/28 14:17:45 INFO fetcher.Fetcher: QueueFeeder finished: total 10 records + hit by time limit :0
13/02/28 14:17:45 INFO plugin.PluginRepository: Plugins: looking in: D:\workspace\Nutch1.2\.\plugins
13/02/28 14:17:49 INFO plugin.PluginRepository: Plugin Auto-activation mode: [true]
13/02/28 14:17:49 INFO plugin.PluginRepository: Registered Plugins:
13/02/28 14:17:49 INFO plugin.PluginRepository:  the nutch core extension points (nutch-extensionpoints)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Basic Query Filter (query-basic)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Basic URL Normalizer (urlnormalizer-basic)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Html Parse Plug-in (parse-html)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Basic Indexing Filter (index-basic)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Site Query Filter (query-site)
13/02/28 14:17:49 INFO plugin.PluginRepository:  HTTP Framework (lib-http)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Text Parse Plug-in (parse-text)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Regex URL Filter (urlfilter-regex)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Pass-through URL Normalizer (urlnormalizer-pass)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Http Protocol Plug-in (protocol-http)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Regex URL Normalizer (urlnormalizer-regex)
13/02/28 14:17:49 INFO plugin.PluginRepository:  OPIC Scoring Plug-in (scoring-opic)
13/02/28 14:17:49 INFO plugin.PluginRepository:  CyberNeko HTML Parser (lib-nekohtml)
13/02/28 14:17:49 INFO plugin.PluginRepository:  JavaScript Parser (parse-js)
13/02/28 14:17:49 INFO plugin.PluginRepository:  URL Query Filter (query-url)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Lucene Highlighter Summary Plug-in (summary-lucene)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Regex URL Filter Framework (lib-regex-filter)
13/02/28 14:17:49 INFO plugin.PluginRepository: Registered Extension-Points:
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Summarizer (org.apache.nutch.searcher.Summarizer)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Protocol (org.apache.nutch.protocol.Protocol)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Segment Merge Filter (org.apache.nutch.segment.SegmentMergeFilter)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Analysis (org.apache.nutch.analysis.NutchAnalyzer)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Field Filter (org.apache.nutch.indexer.field.FieldFilter)
13/02/28 14:17:49 INFO plugin.PluginRepository:  HTML Parse Filter (org.apache.nutch.parse.HtmlParseFilter)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Query Filter (org.apache.nutch.searcher.QueryFilter)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Search Results Response Writer (org.apache.nutch.searcher.response.ResponseWriter)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch URL Normalizer (org.apache.nutch.net.URLNormalizer)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch URL Filter (org.apache.nutch.net.URLFilter)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Online Search Results Clustering Plugin (org.apache.nutch.clustering.OnlineClusterer)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Indexing Filter (org.apache.nutch.indexer.IndexingFilter)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Content Parser (org.apache.nutch.parse.Parser)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Nutch Scoring (org.apache.nutch.scoring.ScoringFilter)
13/02/28 14:17:49 INFO plugin.PluginRepository:  Ontology Model Loader (org.apache.nutch.ontology.Ontology)
13/02/28 14:17:50 INFO mapred.LocalJobRunner: file:/D:/workspace/Nutch1.2/test/segments/20130228141648/crawl_generate/part-00000:0+1094
13/02/28 14:17:50 INFO conf.Configuration: found resource regex-urlfilter.txt at file:/D:/workspace/Nutch1.2/bin/regex-urlfilter.txt
13/02/28 14:17:50 INFO mapred.JobClient:  map 100% reduce 0%
13/02/28 14:17:51 INFO conf.Configuration: parse-plugins.xml not found
13/02/28 14:17:51 WARN parse.ParsePluginsReader: Unable to parse [null].Reason is [java.net.MalformedURLException]
13/02/28 14:17:51 WARN mapred.LocalJobRunner: job_local_0005
java.lang.RuntimeException: Parse Plugins preferences could not be loaded.
 at org.apache.nutch.parse.ParserFactory.<init>(ParserFactory.java:73)
 at org.apache.nutch.parse.ParseUtil.<init>(ParseUtil.java:53)
 at org.apache.nutch.fetcher.Fetcher$FetcherThread.<init>(Fetcher.java:551)
 at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:1026)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
 at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
13/02/28 14:17:52 INFO mapred.JobClient: Job complete: job_local_0005
13/02/28 14:17:52 INFO mapred.JobClient: Counters: 9
13/02/28 14:17:52 INFO mapred.JobClient:   FileSystemCounters
13/02/28 14:17:52 INFO mapred.JobClient:     FILE_BYTES_READ=459000
13/02/28 14:17:52 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=490325
13/02/28 14:17:52 INFO mapred.JobClient:   Map-Reduce Framework
13/02/28 14:17:52 INFO mapred.JobClient:     Combine output records=0
13/02/28 14:17:52 INFO mapred.JobClient:     Map input records=10
13/02/28 14:17:52 INFO mapred.JobClient:     Spilled Records=0
13/02/28 14:17:52 INFO mapred.JobClient:     Map output bytes=0
13/02/28 14:17:52 INFO mapred.JobClient:     Map input bytes=1008
13/02/28 14:17:52 INFO mapred.JobClient:     Map output records=0
13/02/28 14:17:52 INFO mapred.JobClient:     Combine input records=0
Exception in thread "main" java.io.IOException: Job failed!
 at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
 at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:1107)
 at org.apache.nutch.crawl.Crawl.main(Crawl.java:133)