kıvılcım kabuk ilk başlatma hata

CentOS 7.2 bu talimatları kıvılcım kabuk ilk başlatma hata

http://davidssysadminnotes.blogspot.com/2016/01/installing-spark-centos-7.html http://tecadmin.net/setup-hadoop-2-4-single-node-cluster-on-linux/#

izledi.

[[email protected] ~]$ pyspark 
Python 2.7.11 |Anaconda 4.0.0 (64-bit)| (default, Dec 6 2015, 18:08:32) 
Type "copyright", "credits" or "license" for more information. 

IPython 4.1.2 -- An enhanced Interactive Python. 
?   -> Introduction and overview of IPython's features. 
%quickref -> Quick reference. 
help  -> Python's own help system. 
object? -> Details about 'object', use 'object??' for extra details. 
16/04/01 16:02:34 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
Welcome to 
     ____    __ 
    /__/__ ___ _____/ /__ 
    _\ \/ _ \/ _ `/ __/ '_/ 
    /__/.__/\_,_/_/ /_/\_\ version 1.6.1 
     /_/ 

Using Python version 2.7.11 (default, Dec 6 2015 18:08:32) 
SparkContext available as sc, HiveContext available as sqlContext. 

In [1]:

Ama kıvılcım-kabuk başlattığınızda, ben hatalar olsun: Ben pyspark başlattığınızda, herşey yolunda görünüyor

[[email protected] ~]$ spark-shell 
16/04/01 15:59:27 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
Welcome to 
     ____    __ 
    /__/__ ___ _____/ /__ 
    _\ \/ _ \/ _ `/ __/ '_/ 
    /___/ .__/\_,_/_/ /_/\_\ version 1.6.1 
     /_/ 

Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_45) 
Type in expressions to have them evaluated. 
Type :help for more information. 
Spark context available as sc. 
16/04/01 15:59:42 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark-latest/lib/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar." 
16/04/01 15:59:42 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark-latest/lib/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar." 
16/04/01 15:59:42 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark-latest/lib/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar." 
16/04/01 15:59:42 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) 
Fri Apr 01 15:59:42 EDT 2016 Thread[main,5,main] java.io.FileNotFoundException: derby.log (Permission denied) 
16/04/01 15:59:43 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) 
---------------------------------------------------------------- 
Fri Apr 01 15:59:43 EDT 2016: 
Booting Derby version The Apache Software Foundation - Apache Derby - 10.10.1.1 - (1458268): instance a816c00e-0153-d368-eecc-000031edf8d8 
on database directory /tmp/spark-f22a65d6-4bda-4426-8b80-0f34b39b28dd/metastore with class loader [email protected] 
Loaded from file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar 
java.vendor=Oracle Corporation 
java.runtime.version=1.8.0_45-b14 
user.dir=/home/idf 
os.name=Linux 
os.arch=amd64 
os.version=3.10.0-327.10.1.el7.x86_64 
derby.system.home=null 
Database Class Loader started - derby.database.classpath='' 
16/04/01 16:00:02 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0 
16/04/01 16:00:02 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException 
16/04/01 16:00:06 WARN General: Plugin (Bundle) "org.datanucleus" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark-latest/lib/datanucleus-core-3.2.10.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar." 
16/04/01 16:00:06 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark-latest/lib/datanucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar." 
16/04/01 16:00:06 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL "file:/opt/spark-latest/lib/datanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to register an identical plugin located at URL "file:/opt/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar." 
16/04/01 16:00:06 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] java.io.FileNotFoundException: derby.log (Permission denied) 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.dataDictionary in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.lockManagerJ1 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.dvfJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.javaCompiler in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.replication.slave in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.rawStore.transactionJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.ef in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.rawStore.transactionJ1 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.database in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.NoneAuthentication in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.netServer.autoStart in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.dvfJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.mgmt.null in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.nativeAuthentication in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.lockManagerJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.replication.master in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.dvfCDC in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.access.btree in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.lockManagerJ1 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.uuidJ1 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.cryptographyJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.rawStore.data.genericJ4 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.rawStore.data.genericJ1 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.access in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.jdbc169 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.cryptographyJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.optimizer in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.mgmt.jmx in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.dvfJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.specificAuthentication in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.JNDIAuthentication in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.basicAuthentication in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.data.genericJ4 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.validation in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.data.genericJ1 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.classManagerJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.streams in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.classManagerJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.resourceAdapterJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.rawStore.transactionJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.jdbcJ8 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.jdbcJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.classManagerJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.jdbcJ4 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.log in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.log.readonly in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.access.heap in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.jdbcJ8 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.daemon in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.cacheManagerJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.tcf in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.jdbcJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.jdbcJ4 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.access.uniquewithduplicatenullssort in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.cacheManagerJ1 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.cacheManagerJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.resultSetStatisticsFactory in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.cacheManagerJ1 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.cryptographyJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.rawStore.data.genericJ4 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.database.slave in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.XPLAINFactory in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.resourceAdapterJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.access.sort in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.mgmt.jmx in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.transactionJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.JNDIAuthentication in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.rawStore.transactionJ1 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.timer in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.lcf in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.rawStore in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.resourceAdapterJ2 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.classes.jdbc169 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.lf in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.jdbcJ8 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.jdbcJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.env.jdk.jdbcJ4 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.nodeFactory in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] Ignored duplicate property derby.module.lockManagerJ6 in jar:file:/opt/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/derby/modules.properties 
Fri Apr 01 16:00:06 EDT 2016 Thread[main,5,main] 
16/04/01 16:00:06 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) 
Fri Apr 01 16:00:07 EDT 2016 Thread[main,5,main] Cleanup action starting 
java.sql.SQLException: Failed to start database 'metastore_db' with class loader [email protected]31509c, see the next exception for details. 
     at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)

kaynak

2016-04-01 Ivan

Sınıf yolunu kıvılcımlamak için herhangi bir sepcial hive-site.xml eklediniz mi? veya sistem geneli? – user1314742

kıvılcım kabuklu alreeady sizin için Hive-destekli bir SqlContext sağlar:

[[email protected]: ~/spark-1.6.1-bin-hadoop2.6] # ./bin/spark-shell --master local[4] --packages com.databricks:spark-csv_2.10:1.4.0 
Ivy Default Cache set to: /home/meox/.ivy2/cache 
The jars for the packages stored in: /home/meox/.ivy2/jars 
:: loading settings :: url = jar:file:/home/meox/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/ivy/core/settings/ivysettings.xml 
com.databricks#spark-csv_2.10 added as a dependency 
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0 
     confs: [default] 
     found com.databricks#spark-csv_2.10;1.4.0 in central 
     found org.apache.commons#commons-csv;1.1 in list 
     found com.univocity#univocity-parsers;1.5.1 in list 
:: resolution report :: resolve 429ms :: artifacts dl 21ms 
     :: modules in use: 
     com.databricks#spark-csv_2.10;1.4.0 from central in [default] 
     com.univocity#univocity-parsers;1.5.1 from list in [default] 
     org.apache.commons#commons-csv;1.1 from list in [default] 
     --------------------------------------------------------------------- 
     |     |   modules   || artifacts | 
     |  conf  | number| search|dwnlded|evicted|| number|dwnlded| 
     --------------------------------------------------------------------- 
     |  default  | 3 | 0 | 0 | 0 || 3 | 0 | 
     --------------------------------------------------------------------- 
:: retrieving :: org.apache.spark#spark-submit-parent 
     confs: [default] 
     0 artifacts copied, 3 already retrieved (0kB/10ms) 
16/04/02 11:02:35 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
16/04/02 11:02:35 INFO SecurityManager: Changing view acls to: meox 
16/04/02 11:02:35 INFO SecurityManager: Changing modify acls to: meox 
16/04/02 11:02:35 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(meox); users with modify permissions: Set(meox) 
16/04/02 11:02:35 INFO HttpServer: Starting HTTP Server 
16/04/02 11:02:35 INFO Utils: Successfully started service 'HTTP class server' on port 40697. 
Welcome to 
     ____    __ 
    /__/__ ___ _____/ /__ 
    _\ \/ _ \/ _ `/ __/ '_/ 
    /___/ .__/\_,_/_/ /_/\_\ version 1.6.1 
     /_/ 

Using Scala version 2.10.5 (OpenJDK 64-Bit Server VM, Java 1.7.0_95) 
Type in expressions to have them evaluated. 
Type :help for more information. 
16/04/02 11:02:41 INFO SparkContext: Running Spark version 1.6.1 
16/04/02 11:02:41 INFO SecurityManager: Changing view acls to: meox 
16/04/02 11:02:41 INFO SecurityManager: Changing modify acls to: meox 
16/04/02 11:02:41 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(meox); users with modify permissions: Set(meox) 
16/04/02 11:02:42 INFO Utils: Successfully started service 'sparkDriver' on port 41544. 
16/04/02 11:02:42 INFO Slf4jLogger: Slf4jLogger started 
16/04/02 11:02:42 INFO Remoting: Starting remoting 
16/04/02 11:02:42 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:37295] 
16/04/02 11:02:42 INFO Utils: Successfully started service 'sparkDriverActorSystem' on port 37295. 
16/04/02 11:02:42 INFO SparkEnv: Registering MapOutputTracker 
16/04/02 11:02:42 INFO SparkEnv: Registering BlockManagerMaster 
16/04/02 11:02:42 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-e5d301cf-bb94-4738-aecd-9d97f0f6174a 
16/04/02 11:02:42 INFO MemoryStore: MemoryStore started with capacity 511.5 MB 
16/04/02 11:02:42 INFO SparkEnv: Registering OutputCommitCoordinator 
16/04/02 11:02:43 INFO Utils: Successfully started service 'SparkUI' on port 4040. 
16/04/02 11:02:43 INFO SparkUI: Started SparkUI at http://192.168.0.50:4040 
16/04/02 11:02:43 INFO HttpFileServer: HTTP File server directory is /tmp/spark-af3ded9a-8e45-4d5a-8d14-a3114dd899aa/httpd-90b07dcf-a8bc-4c32-9e0a-4d698033c7aa 
16/04/02 11:02:43 INFO HttpServer: Starting HTTP Server 
16/04/02 11:02:43 INFO Utils: Successfully started service 'HTTP file server' on port 45007. 
16/04/02 11:02:43 INFO SparkContext: Added JAR file:/home/meox/.ivy2/jars/com.databricks_spark-csv_2.10-1.4.0.jar at http://192.168.0.50:45007/jars/com.databricks_spark-csv_2.10-1.4.0.jar with timestamp 1459587763133 
16/04/02 11:02:43 INFO SparkContext: Added JAR file:/home/meox/.ivy2/jars/org.apache.commons_commons-csv-1.1.jar at http://192.168.0.50:45007/jars/org.apache.commons_commons-csv-1.1.jar with timestamp 1459587763134 
16/04/02 11:02:43 INFO SparkContext: Added JAR file:/home/meox/.ivy2/jars/com.univocity_univocity-parsers-1.5.1.jar at http://192.168.0.50:45007/jars/com.univocity_univocity-parsers-1.5.1.jar with timestamp 1459587763135 
16/04/02 11:02:43 INFO Executor: Starting executor ID driver on host localhost 
16/04/02 11:02:43 INFO Executor: Using REPL class URI: http://192.168.0.50:40697 
16/04/02 11:02:43 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 58692. 
16/04/02 11:02:43 INFO NettyBlockTransferService: Server created on 58692 
16/04/02 11:02:43 INFO BlockManagerMaster: Trying to register BlockManager 
16/04/02 11:02:43 INFO BlockManagerMasterEndpoint: Registering block manager localhost:58692 with 511.5 MB RAM, BlockManagerId(driver, localhost, 58692) 
16/04/02 11:02:43 INFO BlockManagerMaster: Registered BlockManager 
16/04/02 11:02:43 INFO SparkILoop: Created spark context.. 
Spark context available as sc. 
16/04/02 11:02:44 INFO HiveContext: Initializing execution hive, version 1.2.1 
16/04/02 11:02:44 INFO ClientWrapper: Inspected Hadoop version: 2.6.0 
16/04/02 11:02:44 INFO ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.6.0 
16/04/02 11:02:44 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 
16/04/02 11:02:44 INFO ObjectStore: ObjectStore, initialize called 
16/04/02 11:02:44 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored 
16/04/02 11:02:44 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 
16/04/02 11:02:45 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) 
16/04/02 11:02:45 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) 
16/04/02 11:02:47 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 
16/04/02 11:02:48 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:48 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:51 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:51 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:51 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 
16/04/02 11:02:51 INFO ObjectStore: Initialized ObjectStore 
16/04/02 11:02:51 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0 
16/04/02 11:02:52 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException 
16/04/02 11:02:52 INFO HiveMetaStore: Added admin role in metastore 
16/04/02 11:02:52 INFO HiveMetaStore: Added public role in metastore 
16/04/02 11:02:52 INFO HiveMetaStore: No user is added in admin role, since config is empty 
16/04/02 11:02:52 INFO HiveMetaStore: 0: get_all_databases 
16/04/02 11:02:52 INFO audit: ugi=meox ip=unknown-ip-addr  cmd=get_all_databases 
16/04/02 11:02:52 INFO HiveMetaStore: 0: get_functions: db=default pat=* 
16/04/02 11:02:52 INFO audit: ugi=meox ip=unknown-ip-addr  cmd=get_functions: db=default pat=* 
16/04/02 11:02:52 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:53 INFO SessionState: Created local directory: /tmp/c9aee4a3-05b4-4606-8cc9-eb6e3af28fa1_resources 
16/04/02 11:02:53 INFO SessionState: Created HDFS directory: /tmp/hive/meox/c9aee4a3-05b4-4606-8cc9-eb6e3af28fa1 
16/04/02 11:02:53 INFO SessionState: Created local directory: /tmp/meox/c9aee4a3-05b4-4606-8cc9-eb6e3af28fa1 
16/04/02 11:02:53 INFO SessionState: Created HDFS directory: /tmp/hive/meox/c9aee4a3-05b4-4606-8cc9-eb6e3af28fa1/_tmp_space.db 
16/04/02 11:02:53 INFO HiveContext: default warehouse location is /user/hive/warehouse 
16/04/02 11:02:53 INFO HiveContext: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes. 
16/04/02 11:02:53 INFO ClientWrapper: Inspected Hadoop version: 2.6.0 
16/04/02 11:02:53 INFO ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.6.0 
16/04/02 11:02:54 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 
16/04/02 11:02:54 INFO ObjectStore: ObjectStore, initialize called 
16/04/02 11:02:54 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored 
16/04/02 11:02:54 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 
16/04/02 11:02:54 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) 
16/04/02 11:02:54 WARN Connection: BoneCP specified but not present in CLASSPATH (or one of dependencies) 
16/04/02 11:02:55 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 
16/04/02 11:02:56 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:56 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:56 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:56 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:56 INFO Query: Reading in results for query "[email protected]" since the connection used is closing 
16/04/02 11:02:56 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 
16/04/02 11:02:56 INFO ObjectStore: Initialized ObjectStore 
16/04/02 11:02:57 INFO HiveMetaStore: Added admin role in metastore 
16/04/02 11:02:57 INFO HiveMetaStore: Added public role in metastore 
16/04/02 11:02:57 INFO HiveMetaStore: No user is added in admin role, since config is empty 
16/04/02 11:02:57 INFO HiveMetaStore: 0: get_all_databases 
16/04/02 11:02:57 INFO audit: ugi=meox ip=unknown-ip-addr  cmd=get_all_databases 
16/04/02 11:02:57 INFO HiveMetaStore: 0: get_functions: db=default pat=* 
16/04/02 11:02:57 INFO audit: ugi=meox ip=unknown-ip-addr  cmd=get_functions: db=default pat=* 
16/04/02 11:02:57 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table. 
16/04/02 11:02:57 INFO SessionState: Created local directory: /tmp/2d5b72da-368c-4c00-8d14-ca8c5cfbf66c_resources 
16/04/02 11:02:57 INFO SessionState: Created HDFS directory: /tmp/hive/meox/2d5b72da-368c-4c00-8d14-ca8c5cfbf66c 
16/04/02 11:02:57 INFO SessionState: Created local directory: /tmp/meox/2d5b72da-368c-4c00-8d14-ca8c5cfbf66c 
16/04/02 11:02:57 INFO SessionState: Created HDFS directory: /tmp/hive/meox/2d5b72da-368c-4c00-8d14-ca8c5cfbf66c/_tmp_space.db 
16/04/02 11:02:57 INFO SparkILoop: Created sql context (with Hive support).. 
SQL context available as sqlContext.

son satırları görürseniz diyor: "16/04/02 11:02:57 BİLGİ SparkILoop: oluşturuldu sql bağlam (kovan desteği ile) .."

unutmayın:

günlüğünü etkinleştirmek (cp conf/log4j.properties.template conf/log4j.properties)
(sağ izinlerle) klasör/kullanıcı/kovan/depo/

oluşturmak

kaynak

2016-04-02 09:10:39

kıvılcım kabuk ilk başlatma hata

cevap

İlgili konular