Spark API를 사용하여 스칼라에서 독립 실행 형 응용 프로그램 예제를 실행하는 동안 오류가 발생했습니다.

Question

Iam이 스파크 예제를 실행할 수 없습니다. (https://spark.incubator.apache.org/docs/latest/quick-start.html#a-standalone-app-in-scala).

sbt 패키지 명령이 성공적으로 실행되었습니다. 그러나 sbt 실행 명령은 오류를 발생시킵니다. SBT 패키지

출력

[info] Set current project to Simple Project (in build file:/home/raghuveer/Spark/) 
[info] Updating {file:/home/raghuveer/Spark/}default-13c61e... 
[info] Resolving com.codahale.metrics#metrics-graphite;3.0.0 ... 
[info] Done updating. 
[info] Compiling 1 Scala source to /home/raghuveer/Spark/target/scala-2.10/classes... 
[info] Packaging /home/raghuveer/Spark/target/scala-2.10/simple-project_2.10-1.0.jar ... 
[info] Done packaging. 
[success] Total time: 16 s, completed Feb 27, 2014 6:19:14 PM 

SBT 실행

ERROR executor.Executor: Exception in task ID 0 java.io.IOException: Server returned HTTP response code: 504 for URL:http://10.135.217.189:49650/jars/simple-project_2.10-1.0.jar 
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1403) 
at java.net.URL.openStream(URL.java:1031) 
at org.apache.spark.util.Utils$.fetchFile(Utils.scala:253) 
at org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Executor$$updateDependencies$6.apply(Executor.scala:345) 
at org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Executor$$updateDependencies$6.apply(Executor.scala:343) 
at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772) 
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) 
at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98) 
at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226) 
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) 
at scala.collection.mutable.HashMap.foreach(HashMap.scala:98) 
at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771) 
at org.apache.spark.executor.Executor.org$apache$spark$executor$Executor$$updateDependencies(Executor.scala:343) 
at org.apache.spark.executor.Executor$TaskRunner$$anonfun$run$1.apply$mcV$sp(Executor.scala:194) 
at org.apache.spark.deploy.SparkHadoopUtil.runAsUser(SparkHadoopUtil.scala:49) 
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:178) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:701) 

[error] (run-main) org.apache.spark.SparkException: Job aborted: Task 0.0:0 failed 1 times (most recent failure: 
Exception failure: java.io.IOException: Server returned HTTP response code: 504 for URL: http://10.135.217.189:49650/jars/simple-project_2.10-1.0.jar)org.apache.spark.SparkException: Job aborted: Task 0.0:0 failed 1 times (most recent failure: 
Exception failure: java.io.IOException: Server returned HTTP response code: 504 for URL: http://10.135.217.189:49650/jars/simple-project_2.10-1.0.jar) 

그리고 추적에 대한 오류

[trace] Stack trace suppressed: run last compile:run for the full output. 
14/02/27 18:20:58 INFO network.ConnectionManager: Selector thread was interrupted! 
java.lang.RuntimeException: Nonzero exit code: 1 
at scala.sys.package$.error(package.scala:27) 
[trace] Stack trace suppressed: run last compile:run for the full output. 
[error] (compile:run) Nonzero exit code: 1 
[error] Total time: 36 s, completed Feb 27, 2014 6:20:58 PM 

편집이다 : 나는 인터넷 연결 및 현재 java.io를 분리 .IOException : 서버가 HTTP 응답 코드를 반환했습니다 : 504가 오지 않아 성공적으로 실행되어 출력을 보여줍니다.하지만 나는 왜 그런 일이 일어나는 지 알 수 없었다.

Answer 1

이 게시물은 Spark-streaming 독립 실행 형 응용 프로그램을 만드는 방법과 scala-SDK (Eclipse IDE)에서 Spark 응용 프로그램을 실행하는 방법을 공유합니다.

check this