예제 map-reduce oozie 프로그램이 CDH에서 작동하지 않음 4.5

필자는 지금까지 Mac에서 Hadoop (CDH 4.5)을 사용하고 있으며 맵 축소 작업을 정기적으로 실행합니다. 나는 최근에 (다시, CDH4.5)를 다음의 지시에 따라 설치했다 : http://archive.cloudera.com/cdh4/cdh/4/oozie-3.3.2-cdh4.5.0/DG_QuickStart.html 그리고 제공된 샘플 프로그램을 실행하려고 시도했다. 그러나 다음 오류로 인해 항상 실패합니다. 워크 플로우가 전혀 작동하지 않는 것 같습니다. 작업 정보의 콘솔 URL 필드도 비어 있습니다.예제 map-reduce oozie 프로그램이 CDH에서 작동하지 않음 4.5

누군가가 도와 주시겠습니까?

Oozie 작업 로그의 관련 스 니펫이 이어집니다.

2014-06-10 17:27:18,414 INFO ActionStartXCommand:539 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[[email protected]:start:] Start action [[email protected]:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2014-06-10 17:27:18,417 WARN ActionStartXCommand:542 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action status=DONE 
2014-06-10 17:27:18,417 WARN ActionStartXCommand:542 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action updated in DB! 
2014-06-10 17:27:18,576 INFO ActionStartXCommand:539 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2014-06-10 17:27:19,188 WARN MapReduceActionExecutor:542 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[[email protected]] credentials is null for the action 
2014-06-10 17:27:19,423 WARN ActionStartXCommand:542 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[[email protected]] Error starting action [mr-node]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: Unknown rpc kind RPC_WRITABLE] 
org.apache.oozie.action.ActionExecutorException: JA009: Unknown rpc kind RPC_WRITABLE 
at org.apache.oozie.action.ActionExecutor.convertExceptionHelper(ActionExecutor.java:418) 
at org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:392) 
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:773) 
at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:927) 
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:211) 
at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:59) 
at org.apache.oozie.command.XCommand.call(XCommand.java:277) 
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326) 
at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255) 
at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:744) 
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): Unknown rpc kind RPC_WRITABLE 
at org.apache.hadoop.ipc.Client.call(Client.java:1238) 
at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:225) 
at org.apache.hadoop.mapred.$Proxy30.getDelegationToken(Unknown Source) 
at org.apache.hadoop.mapred.JobClient.getDelegationToken(JobClient.java:2125) 
at org.apache.oozie.service.HadoopAccessorService.createJobClient(HadoopAccessorService.java:372) 
at org.apache.oozie.action.hadoop.JavaActionExecutor.createJobClient(JavaActionExecutor.java:970) 
at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:723) 
... 10 more 

2014-06-10 17:27:19,426 INFO ActionStartXCommand:539 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[[email protected]] Next Retry, Attempt Number [1] in [60,000] milliseconds 
2014-06-10 17:28:19,468 INFO ActionStartXCommand:539 - USER[userXXX] GROUP[-] TOKEN[] APP[map-reduce-wf] JOB[0000000-140610172702069-oozie-usrX-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]

출처

2014-06-10 user2002748

MapReduce 또는 YARN을 사용하여 hadoop을 설정 했습니까? 당신이 cdh4 분포를 가지고 있기 때문에 당신이 당신의 job.properties에, 원사가있는 경우,이 링크 http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_17.html

다음 oozie를 설치

는

Configuring which Hadoop Version to Use 
The Oozie client does not interact directly with Hadoop MapReduce, and so it does not require any MapReduce configuration. 

The Oozie server can work with either MRv1 or YARN. It cannot work with both simultaneously. 

The MapReduce version Oozie server works with is determined by the CATALINA_BASE variable in the /etc/oozie/conf/oozie-env.sh file. By default, CATALINA_BASE is set to /usr/lib/oozie/oozie-server-0.20. This setting configures the Oozie server to work with MRv1. 

To configure the Oozie server to work with YARN instead, set CATALINA_BASE to /usr/lib/oozie/oozie-server.

대신 포트 8021의

nameNode=hdfs://localhost:8020 
jobTracker=localhost:8021

JobTracker에

의 포트 (JobTracker를 변경), 자원 관리자의 포트를 사용하십시오 : 8032

jobTracker=localhost:8032

https://groups.google.com/a/cloudera.org/forum/#!topic/cdh-user/MlsqeUdMpgA

출처

2014-06-12 11:12:56

안녕 Ramanan, 나는 YARN을 사용하고 있고 CATALINA_BASE도 oozie/oozie-server이고 oozie-server-0.20는 아닙니다. 또한 제공된 링크에서 설치를 시도했지만 행운이 없습니다 : (. – user2002748

실행중인 샘플 프로그램은 무엇입니까? –

업데이트 된 답변 확인 –

예제 map-reduce oozie 프로그램이 CDH에서 작동하지 않음 4.5

답변

관련 문제