Hadoop - Avro : java.lang.NoClassDefFoundError : org/apache/avro/hadoop/io/AvroKeyComparator

Question

Map Reduce 프로그램을 개발하여 고정 길이 레코드를 읽고 avro 파일로 작성합니다. 이것은 초안 양식입니다. 실행하는 동안, 나는 아래의 오류 :

java.lang.Exception: java.lang.NoClassDefFoundError: org/apache/avro/hadoop/io/AvroKeyComparator 
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:406) 
Caused by: java.lang.NoClassDefFoundError: org/apache/avro/hadoop/io/AvroKeyComparator 
    at org.apache.avro.mapreduce.AvroJob.setMapOutputKeySchema(AvroJob.java:93) 
    at org.apache.avro.mapreduce.AvroMultipleOutputs.setSchema(AvroMultipleOutputs.java:511) 
    at org.apache.avro.mapreduce.AvroMultipleOutputs.getContext(AvroMultipleOutputs.java:547) 
    at org.apache.avro.mapreduce.AvroMultipleOutputs.write(AvroMultipleOutputs.java:399) 
    at org.apache.avro.mapreduce.AvroMultipleOutputs.write(AvroMultipleOutputs.java:378) 
    at com.visa.edp.common.vssParser.CustomTestMapper.map(CustomTestMapper.java:98) 
    at com.visa.edp.common.vssParser.CustomTestMapper.map(CustomTestMapper.java:32) 
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140) 
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) 
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) 
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:268) 
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
    at java.lang.Thread.run(Thread.java:748) 
Caused by: java.lang.ClassNotFoundException: org.apache.avro.hadoop.io.AvroKeyComparator 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 
    ... 16 more 
17/11/10 18:13:28 INFO mapred.JobClient: map 0% reduce 0% 

내 프로젝트에 다음과 같은 종속성이 있습니다

<dependency> 
    <groupId>org.apache.hadoop</groupId> 
    <artifactId>hadoop-client</artifactId> 
    <version>2.6.0-mr1-cdh5.8.3</version> 
    <scope>provided</scope> 
</dependency> 

<dependency> 
    <groupId>org.apache.hadoop</groupId> 
    <artifactId>hadoop-core</artifactId> 
    <version>2.6.0-mr1-cdh5.8.3</version> 
</dependency> 

<dependency> 
    <groupId>org.apache.avro</groupId> 
    <artifactId>avro</artifactId> 
    <version>1.7.6-cdh5.8.3</version> 
</dependency> 

<dependency> 
    <groupId>org.apache.avro</groupId> 
    <artifactId>avro-mapred</artifactId> 
    <version>1.7.6-cdh5.8.3</version> 
</dependency> 

을 그리고 프로젝트 종속성에서 패키지를 org.apache.avro.hadoop 찾을 수 없습니다. 종속성이 누락 되었습니까?

감사합니다.

Answer 1

모듈에 종속성을 추가하고 정상적으로 작동합니다.