Spark 어플리케이션이 "IOException : (null) command string : null chmod 0644"항목과 함께 실패하는 이유는 무엇입니까?

Question

나는 JAVA

dataset.write().mode(SaveMode.Overwrite).option("header",true).csv("C:\\tmp\\csvs"); 

를 사용하여 아래 사용하여 단일 CSV로 데이터 세트의 결과를 작성하는 중이 야하지만 시간이 초과 파일이 작성되지 않는 간다.

예외 org.apache.spark.SparkException: Job aborted.

오류 :

org.apache.spark.SparkException: Job aborted due to stage failure: 

Task 0 in stage 13.0 failed 1 times, most recent failure: Lost task 0.0 in stage 13.0 (TID 16, localhost): java.io.IOException: (null) entry in command string: null chmod 0644 C:\tmp\12333333testSpark\_temporary\0\_temporary\attempt_201712282255_0013_m_000000_0\part-r-00000-229fd1b6-ffb9-4ba1-9dc9-89dfdbd0be43.csv 
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:770) 
at org.apache.hadoop.util.Shell.execCommand(Shell.java:866) 
at org.apache.hadoop.util.Shell.execCommand(Shell.java:849) 
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:733) 
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:225) 
at org.apache.hadoop.fs.RawLocalFileSystem$LocalFSFileOutputStream.<init>(RawLocalFileSystem.java:209) 
at org.apache.hadoop.fs.RawLocalFileSystem.createOutputStreamWithMode(RawLocalFileSystem.java:307) 
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:296) 
at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:328) 
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:398) 
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:461) 
at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:440) 
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:911) 
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:892) 
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:789) 
at org.apache.hadoop.mapreduce.lib.output.TextOutputFormat.getRecordWriter(TextOutputFormat.java:132) 
at org.apache.spark.sql.execution.datasources.csv.CsvOutputWriter.<init>(CSVRelation.scala:200) 
at org.apache.spark.sql.execution.datasources.csv.CSVOutputWriterFactory.newInstance(CSVRelation.scala:170) 
at org.apache.spark.sql.execution.datasources.BaseWriterContainer.newOutputWriter(WriterContainer.scala:131) 
at org.apache.spark.sql.execution.datasources.DefaultWriterContainer.writeRows(WriterContainer.scala:247) 
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(InsertIntoHadoopFsRelationCommand.scala:143) 
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand$$anonfun$run$1$$anonfun$apply$mcV$sp$1.apply(InsertIntoHadoopFsRelationCommand.scala:143) 
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70) 
at org.apache.spark.scheduler.Task.run(Task.scala:86) 
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745) 

Answer 1

당신은 다음과 같은 예외가 고정으로 좁혀 할 수 있습니다 :이에보고

java.io.IOException: (null) entry in command string: null chmod 0644 

가 bin\winuitls.exe와 하위 디렉토리에 HADOOP_HOME 설정 시도 SO question. 그래도 도움이되지 않으면 다른 SO link에 해결 방법이 있습니다.