Alma istisna: java.lang.NoSuchMethodError: scala.reflect.api.JavaUniverse.runtimeMirror (Ljava/lang/ClassLoader;) verileri kullanılarak

Question

çerçeveleri ederken java.lang.NoSuchMethodError" alıyorum: scala.reflect.api.JavaUniverse .RuntimeMirror (Ljava/lang/ClassLoader;) "scala uygulamasında dataframes kullanırken ve spark kullanarak çalıştırırken hata. Ancak, sadece RDD'leri ve veri çerçevelerini kullanmazsam, aynı pompa ve ayarlarla böyle bir hata olmaz. Ayrıca aynı hata ile diğer mesajlar üzerinden geçerken, 2.11 scala ile uyumlu değildir ve 2.0.0 kıvılcım ile 2.10 scala versiyonunu kullanıyorum diye scala versiyonunun 2.10 olması gerektiği belirtilmektedir.

Below is the snip from pom: 



<properties> 
     <spark-assembly>/usr/lib/spark/lib/spark-assembly.jar</spark-assembly> 
     <encoding>UTF-8</encoding> 
     <hadoop.version>2.7.1</hadoop.version> 
     <hbase.version>1.1.1</hbase.version> 
     <scala.version>2.10.5</scala.version> 
     <scala.tools.version>2.10</scala.tools.version> 
     <spark.version>2.0.0</spark.version> 
     <phoenix.version>4.7.0-HBase-1.1</phoenix.version> 
    </properties> 

    <dependencies> 
     <dependency> 
      <groupId>org.scala-lang</groupId> 
      <artifactId>scala-library</artifactId> 
      <version>${scala.version}</version> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.hadoop</groupId> 
      <artifactId>hadoop-client</artifactId> 
      <version>${hadoop.version}</version> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.hbase</groupId> 
      <artifactId>hbase-client</artifactId> 
      <version>${hbase.version}</version> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.hbase</groupId> 
      <artifactId>hbase-server</artifactId> 
      <version>${hbase.version}</version> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-core_${scala.tools.version}</artifactId> 
      <version>${spark.version}</version> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-sql_${scala.tools.version}</artifactId> 
      <version>${spark.version}</version> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-hive_${scala.tools.version}</artifactId> 
      <version>${spark.version}</version> 
     </dependency> 

    </dependencies> 

Hata:

16/10/19 02:57:26 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.NoSuchMethodError: scala.reflect.api.JavaUniverse.runtimeMirror(Ljava/lang/ClassLoader;)Lscala/reflect/api/JavaMirrors$JavaMirror; 
java.lang.NoSuchMethodError: scala.reflect.api.JavaUniverse.runtimeMirror(Ljava/lang/ClassLoader;)Lscala/reflect/api/JavaMirrors$JavaMirror; 
     at com.abc.xyz.Compare$.main(Compare.scala:64) 
     at com.abc.xyz.Compare.main(Compare.scala) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:498) 
     at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:627) 
16/10/19 02:57:26 INFO yarn.ApplicationMaster: Final app status: FAILED, exitCode: 15, (reason: User class threw exception: java.lang.NoSuchMethodError: scala.reflect.api.JavaUniverse.runtimeMirror(Ljava/lang/ClassLoader;)Lscala/reflect/api/JavaMirrors$JavaMirror;) 
16/10/19 02:57:26 INFO spark.SparkContext: Invoking stop() from shutdown hook 

Answer 1

Değişim scala versiyonu

<scala.version>2.11.8</scala.version>

<scala.tools.version>2.11</scala.tools.version> 

ve eklemek

<dependency> 
    <groupId>org.scala-lang</groupId> 
    <artifactId>scala-reflect</artifactId> 
    <version>${scala.version}</version> 
    </dependency>