HDInsight 클라이언트가 누락되었습니다.

Azure에서 실행되는 HDinsight 클러스터가 있습니다. HDInsight에 연결할 클라이언트 컴퓨터를 만들려고했습니다. Hortonworks 설치 안내서의 지시에 따라 모든 클라이언트 구성 요소를 설치 한 다음/etc/hadoop/conf 파일을 HDInsight 노드 중 하나에서이 새 노드로 복사했습니다. 내가 하둡 때 fs -ls하여 클러스터에 액세스하려고 할 때 나는 내가 여기에 실종 된 패키지 모르는HDInsight 클라이언트가 누락되었습니다.

[email protected]:~# hadoop fs -ls/
log4j:ERROR Could not instantiate class [com.microsoft.log4jappender.EtwAppender]. 
java.lang.ClassNotFoundException: com.microsoft.log4jappender.EtwAppender 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358) 
    at java.lang.Class.forName0(Native Method) 
    at java.lang.Class.forName(Class.java:195) 
    at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198) 
    at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327) 
    at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124) 
    at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785) 
    at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) 
    at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648) 
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514) 
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) 
    at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) 
    at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) 
    at org.apache.log4j.Logger.getLogger(Logger.java:104) 
    at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262) 
    at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:108) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
    at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1025) 
    at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:790) 
    at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:541) 
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:292) 
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:269) 
    at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:657) 
    at org.apache.hadoop.fs.FsShell.<clinit>(FsShell.java:43) 
log4j:ERROR Could not instantiate appender named "ETW". 
log4j:ERROR Could not instantiate class [com.microsoft.log4jappender.FilterLogAppender]. 
java.lang.ClassNotFoundException: com.microsoft.log4jappender.FilterLogAppender 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358) 
    at java.lang.Class.forName0(Native Method) 
    at java.lang.Class.forName(Class.java:195) 
    at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198) 
    at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327) 
    at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124) 
    at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785) 
    at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) 
    at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648) 
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514) 
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) 
    at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) 
    at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) 
    at org.apache.log4j.Logger.getLogger(Logger.java:104) 
    at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262) 
    at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:108) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
    at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1025) 
    at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:790) 
    at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:541) 
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:292) 
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:269) 
    at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:657) 
    at org.apache.hadoop.fs.FsShell.<clinit>(FsShell.java:43) 
log4j:ERROR Could not instantiate appender named "FilterLog". 
log4j:ERROR Could not instantiate class [com.microsoft.log4jappender.EtwAppender]. 
java.lang.ClassNotFoundException: com.microsoft.log4jappender.EtwAppender 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358) 
    at java.lang.Class.forName0(Native Method) 
    at java.lang.Class.forName(Class.java:195) 
    at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198) 
    at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327) 
    at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124) 
    at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785) 
    at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) 
    at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672) 
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516) 
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) 
    at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) 
    at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) 
    at org.apache.log4j.Logger.getLogger(Logger.java:104) 
    at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262) 
    at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:108) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
    at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1025) 
    at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:790) 
    at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:541) 
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:292) 
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:269) 
    at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:657) 
    at org.apache.hadoop.fs.FsShell.<clinit>(FsShell.java:43) 
log4j:ERROR Could not instantiate appender named "ETW". 
log4j:ERROR Could not instantiate class [com.microsoft.log4jappender.FilterLogAppender]. 
java.lang.ClassNotFoundException: com.microsoft.log4jappender.FilterLogAppender 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358) 
    at java.lang.Class.forName0(Native Method) 
    at java.lang.Class.forName(Class.java:195) 
    at org.apache.log4j.helpers.Loader.loadClass(Loader.java:198) 
    at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:327) 
    at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:124) 
    at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:785) 
    at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) 
    at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672) 
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516) 
    at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) 
    at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) 
    at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) 
    at org.apache.log4j.Logger.getLogger(Logger.java:104) 
    at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:262) 
    at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:108) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
    at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1025) 
    at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:790) 
    at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:541) 
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:292) 
    at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:269) 
    at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:657) 
    at org.apache.hadoop.fs.FsShell.<clinit>(FsShell.java:43) 
log4j:ERROR Could not instantiate appender named "AMFilterLog". 
16/06/23 16:57:20 WARN impl.MetricsSystemImpl: Error creating sink 'azurefs2' 
org.apache.hadoop.metrics2.impl.MetricsConfigException: Error creating plugin: org.apache.hadoop.metrics2.sink.WasbAzureIaasSink 
    at org.apache.hadoop.metrics2.impl.MetricsConfig.getPlugin(MetricsConfig.java:203) 
    at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.newSink(MetricsSystemImpl.java:528) 
    at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configureSinks(MetricsSystemImpl.java:500) 
    at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.configure(MetricsSystemImpl.java:479) 
    at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:189) 
    at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:164) 
    at org.apache.hadoop.fs.azure.metrics.AzureFileSystemMetricsSystem.fileSystemStarted(AzureFileSystemMetricsSystem.java:41) 
    at org.apache.hadoop.fs.azure.NativeAzureFileSystem.initialize(NativeAzureFileSystem.java:1153) 
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653) 
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92) 
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687) 
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669) 
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371) 
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:170) 
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:355) 
    at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) 
    at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:325) 
    at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:235) 
    at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:218) 
    at org.apache.hadoop.fs.shell.Command.processRawArguments(Command.java:201) 
    at org.apache.hadoop.fs.shell.Command.run(Command.java:165) 
    at org.apache.hadoop.fs.FsShell.run(FsShell.java:287) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) 
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90) 
    at org.apache.hadoop.fs.FsShell.main(FsShell.java:340) 
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.metrics2.sink.WasbAzureIaasSink 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358) 
    at java.lang.Class.forName0(Native Method) 
    at java.lang.Class.forName(Class.java:278) 
    at org.apache.hadoop.metrics2.impl.MetricsConfig.getPlugin(MetricsConfig.java:196) 
    ... 24 more 
ls: org.apache.hadoop.fs.azure.KeyProviderException: java.io.IOException: Cannot run program "/usr/lib/python2.7/dist-packages/hdinsight_common/decrypt.sh": error=2, No such file or directory

, 사람이 여기에 도움이 될 수 있습니다 오류 다음 얻을?

감사

출처

2016-06-23 roy

다음 복사/등/하둡/conf의 HDInsight 노드 중 하나 ...
ClassNotFoundException: com.microsoft.log4jappender.EtwAppender 

이 로그 구성을 복사 나타내고 HDInsight가 나타납니다에서 Microsoft 특정 log4j 확장자는 ETW입니다. 이 로깅이 필요합니까/필요합니까? 그렇다면 HDInsight 라이브러리에서 관련 JAR을 복사해야합니다. 그렇지 않은 경우 노드 로깅 구성을이 클래스를 사용하지 않는 것으로 변경하십시오.

출처

2016-06-23 18:23:29

ETW 참조를 제거했지만 여전히 을 가져옵니다. org.apache.hadoop.fs.azure.KeyProviderException : java.io.IOException : "/usr/lib/python2.7/dist-packages/hdinsight_common 프로그램을 실행할 수 없습니다. /decrypt.sh ": error = 2, No such file or directory – roy

패키지가'/ usr/lib/python2.7/dist-packages/hdinsight_common'을 설치할 때 어떤 아이디어가 있습니까? – roy

HDInsight 클라이언트가 누락되었습니다.

답변

관련 문제