You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Guru Medasani <gd...@gmail.com> on 2015/08/04 05:08:29 UTC
Re: Spark-Submit error
Hi Satish,
Can you add more error or log info to the email?
Guru Medasani
gdmeda@gmail.com
> On Jul 31, 2015, at 1:06 AM, satish chandra j <js...@gmail.com> wrote:
>
> HI,
> I have submitted a Spark Job with options jars,class,master as local but i am getting an error as below
>
> dse spark-submit spark error exception in thread main java.io.ioexception: Invalid Request Exception(Why you have not logged in)
>
> Note: submitting datastax spark node
>
> please let me know if anybody have a solutions for this issue
>
>
>
> Regards,
> Saish Chandra
Re: Spark-Submit error
Posted by Guru Medasani <gd...@gmail.com>.
Thanks Satish. I only see the INFO messages and don’t see any error messages in the output you pasted.
Can you paste the log with the error messages?
Guru Medasani
gdmeda@gmail.com
> On Aug 3, 2015, at 11:12 PM, satish chandra j <js...@gmail.com> wrote:
>
> Hi Guru,
> I am executing this on DataStax Enterprise Spark node and ~/.dserc file exists which consists Cassandra credentials but still getting the error
>
> Below is the given command
>
> dse spark-submit --master spark://10.246.43.15:7077 <http://10.246.43.15:7077/> --class HelloWorld --jars ///home/missingmerch/postgresql-9.4-1201.jdbc41.jar ///home/missingmerch/etl-0.0.1-SNAPSHOT.jar
>
>
> Please find the error log details below
>
> INFO 2015-07-31 05:15:17 org.apache.spark.executor.CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT] INFO 2015-07-31 05:15:18 org.apache.spark.SecurityManager: Changing view acls to: cassandra,missingmerch INFO 2015-07-31 05:15:18 org.apache.spark.SecurityManager: Changing modify acls to: cassandra,missingmerch INFO 2015-07-31 05:15:18 org.apache.spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(cassandra, missingmerch); users with modify permissions: Set(cassandra, missingmerch) INFO 2015-07-31 05:15:22 akka.event.slf4j.Slf4jLogger: Slf4jLogger started INFO 2015-07-31 05:15:22 Remoting: Starting remoting INFO 2015-07-31 05:15:22 Remoting: Remoting started; listening on addresses :[akka.tcp://driverPropsFetcher@10.246.43.14 <ma...@10.246.43.14><mailto://driverPropsFetcher@10.246.43.14 <ma...@10.246.43.14>>:48952]
> INFO 2015-07-31 05:15:22 org.apache.spark.util.Utils: Successfully started service 'driverPropsFetcher' on port 48952.
> INFO 2015-07-31 05:15:24 akka.remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
> INFO 2015-07-31 05:15:24 akka.remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
> INFO 2015-07-31 05:15:24 org.apache.spark.SecurityManager: Changing view acls to: cassandra,missingmerch INFO 2015-07-31 05:15:24 org.apache.spark.SecurityManager: Changing modify acls to: cassandra,missingmerch INFO 2015-07-31 05:15:24 org.apache.spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(cassandra, missingmerch); users with modify permissions: Set(cassandra, missingmerch) INFO 2015-07-31 05:15:24 akka.event.slf4j.Slf4jLogger: Slf4jLogger started INFO 2015-07-31 05:15:24 akka.remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
> INFO 2015-07-31 05:15:24 Remoting: Starting remoting INFO 2015-07-31 05:15:24 Remoting: Remoting started; listening on addresses :[akka.tcp://sparkExecutor@10.246.43.14 <ma...@10.246.43.14><mailto://sparkExecutor@10.246.43.14 <ma...@10.246.43.14>>:56358]
> INFO 2015-07-31 05:15:24 org.apache.spark.util.Utils: Successfully started service 'sparkExecutor' on port 56358.
> INFO 2015-07-31 05:15:24 org.apache.spark.executor.CoarseGrainedExecutorBackend: Connecting to driver: akka.tcp://sparkDriver@tstl400029.wal-mart.com <ma...@tstl400029.wal-mart.com><mailto://sparkDriver@tstl400029.wal-mart.com <ma...@tstl400029.wal-mart.com>>:60525/user/CoarseGrainedScheduler
> INFO 2015-07-31 05:15:24 org.apache.spark.deploy.worker.WorkerWatcher: Connecting to worker akka.tcp://sparkWorker@10.246.43.14 <ma...@10.246.43.14><mailto://sparkWorker@10.246.43.14 <ma...@10.246.43.14>>:51552/user/Worker
> INFO 2015-07-31 05:15:24 org.apache.spark.deploy.worker.WorkerWatcher: Successfully connected to akka.tcp://sparkWorker@10.246.43.14 <ma...@10.246.43.14><mailto://sparkWorker@10.246.43.14 <ma...@10.246.43.14>>:51552/user/Worker
> INFO 2015-07-31 05:15:24 org.apache.spark.executor.CoarseGrainedExecutorBackend: Successfully registered with driver INFO 2015-07-31 05:15:24 org.apache.spark.executor.Executor: Starting executor ID 0 on host 10.246.43.14 INFO 2015-07-31 05:15:24 org.apache.spark.SecurityManager: Changing view acls to: cassandra,missingmerch INFO 2015-07-31 05:15:24 org.apache.spark.SecurityManager: Changing modify acls to: cassandra,missingmerch INFO 2015-07-31 05:15:24 org.apache.spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(cassandra, missingmerch); users with modify permissions: Set(cassandra, missingmerch) INFO 2015-07-31 05:15:24 org.apache.spark.util.AkkaUtils: Connecting to MapOutputTracker: akka.tcp://sparkDriver@tstl400029.wal-mart.com <ma...@tstl400029.wal-mart.com><mailto://sparkDriver@tstl400029.wal-mart.com <ma...@tstl400029.wal-mart.com>>:60525/user/MapOutputTracker
> INFO 2015-07-31 05:15:25 org.apache.spark.util.AkkaUtils: Connecting to BlockManagerMaster: akka.tcp://sparkDriver@tstl400029.wal-mart.com <ma...@tstl400029.wal-mart.com><mailto://sparkDriver@tstl400029.wal-mart.com <ma...@tstl400029.wal-mart.com>>:60525/user/BlockManagerMaster
> INFO 2015-07-31 05:15:25 org.apache.spark.storage.DiskBlockManager: Created local directory at /var/lib/spark/rdd/spark-2ab3a70e-2bdd-4ab5-9aff-fa48224f14d4/spark-c1430eaf-272d-41d5-acd1-ac88dbca4698/spark-b102f621-e107-4684-96cd-910ed4f2c0cf/spark-7e5ab6f6-3433-4e69-adcc-318083c37c4a
> INFO 2015-07-31 05:15:25 org.apache.spark.storage.MemoryStore: MemoryStore started with capacity 265.4 MB INFO 2015-07-31 05:15:31 org.apache.spark.network.netty.NettyBlockTransferService: Server created on 60707 INFO 2015-07-31 05:15:31 org.apache.spark.storage.BlockManagerMaster: Trying to register BlockManager INFO 2015-07-31 05:15:31 org.apache.spark.storage.BlockManagerMaster: Registered BlockManager INFO 2015-07-31 05:15:31 org.apache.spark.executor.Executor: Using REPL class URI: http://10.246.43.15:48903 <http://10.246.43.15:48903/> INFO 2015-07-31 05:15:31 org.apache.spark.util.AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver@tstl400029.wal-mart.com <ma...@tstl400029.wal-mart.com><mailto://sparkDriver@tstl400029.wal-mart.com <ma...@tstl400029.wal-mart.com>>:60525/user/HeartbeatReceiver
>
>
> On Tue, Aug 4, 2015 at 8:38 AM, Guru Medasani <gdmeda@gmail.com <ma...@gmail.com>> wrote:
> Hi Satish,
>
> Can you add more error or log info to the email?
>
>
> Guru Medasani
> gdmeda@gmail.com <ma...@gmail.com>
>
>
>
>> On Jul 31, 2015, at 1:06 AM, satish chandra j <jsatishchandra@gmail.com <ma...@gmail.com>> wrote:
>>
>> HI,
>> I have submitted a Spark Job with options jars,class,master as local but i am getting an error as below
>>
>> dse spark-submit spark error exception in thread main java.io.ioexception: Invalid Request Exception(Why you have not logged in)
>>
>> Note: submitting datastax spark node
>>
>> please let me know if anybody have a solutions for this issue
>>
>>
>>
>> Regards,
>> Saish Chandra
>
>
Re: Spark-Submit error
Posted by satish chandra j <js...@gmail.com>.
Hi Guru,
I am executing this on DataStax Enterprise Spark node and ~/.dserc file
exists which consists Cassandra credentials but still getting the error
Below is the given command
dse spark-submit --master spark://10.246.43.15:7077 --class HelloWorld
--jars ///home/missingmerch/postgresql-9.4-1201.jdbc41.jar
///home/missingmerch/etl-0.0.1-SNAPSHOT.jar
Please find the error log details below
INFO 2015-07-31 05:15:17
org.apache.spark.executor.CoarseGrainedExecutorBackend: Registered signal
handlers for [TERM, HUP, INT] INFO 2015-07-31 05:15:18
org.apache.spark.SecurityManager: Changing view acls to:
cassandra,missingmerch INFO 2015-07-31 05:15:18
org.apache.spark.SecurityManager: Changing modify acls to:
cassandra,missingmerch INFO 2015-07-31 05:15:18
org.apache.spark.SecurityManager: SecurityManager: authentication disabled;
ui acls disabled; users with view permissions: Set(cassandra,
missingmerch); users with modify permissions: Set(cassandra, missingmerch)
INFO 2015-07-31 05:15:22 akka.event.slf4j.Slf4jLogger: Slf4jLogger started
INFO 2015-07-31 05:15:22 Remoting: Starting remoting INFO 2015-07-31
05:15:22 Remoting: Remoting started; listening on addresses :[akka.tcp://
driverPropsFetcher@10.246.43.14<mailto://driverPropsFetcher@10.246.43.14
>:48952]
INFO 2015-07-31 05:15:22 org.apache.spark.util.Utils: Successfully started
service 'driverPropsFetcher' on port 48952.
INFO 2015-07-31 05:15:24
akka.remote.RemoteActorRefProvider$RemotingTerminator: Shutting down remote
daemon.
INFO 2015-07-31 05:15:24
akka.remote.RemoteActorRefProvider$RemotingTerminator: Remote daemon shut
down; proceeding with flushing remote transports.
INFO 2015-07-31 05:15:24 org.apache.spark.SecurityManager: Changing view
acls to: cassandra,missingmerch INFO 2015-07-31 05:15:24
org.apache.spark.SecurityManager: Changing modify acls to:
cassandra,missingmerch INFO 2015-07-31 05:15:24
org.apache.spark.SecurityManager: SecurityManager: authentication disabled;
ui acls disabled; users with view permissions: Set(cassandra,
missingmerch); users with modify permissions: Set(cassandra, missingmerch)
INFO 2015-07-31 05:15:24 akka.event.slf4j.Slf4jLogger: Slf4jLogger started
INFO 2015-07-31 05:15:24
akka.remote.RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
INFO 2015-07-31 05:15:24 Remoting: Starting remoting INFO 2015-07-31
05:15:24 Remoting: Remoting started; listening on addresses :[akka.tcp://
sparkExecutor@10.246.43.14<ma...@10.246.43.14>:56358]
INFO 2015-07-31 05:15:24 org.apache.spark.util.Utils: Successfully started
service 'sparkExecutor' on port 56358.
INFO 2015-07-31 05:15:24
org.apache.spark.executor.CoarseGrainedExecutorBackend: Connecting to
driver: akka.tcp://sparkDriver@tstl400029.wal-mart.com<mailto://
sparkDriver@tstl400029.wal-mart.com>:60525/user/CoarseGrainedScheduler
INFO 2015-07-31 05:15:24 org.apache.spark.deploy.worker.WorkerWatcher:
Connecting to worker akka.tcp://sparkWorker@10.246.43.14<mailto://
sparkWorker@10.246.43.14>:51552/user/Worker
INFO 2015-07-31 05:15:24 org.apache.spark.deploy.worker.WorkerWatcher:
Successfully connected to akka.tcp://sparkWorker@10.246.43.14<mailto://
sparkWorker@10.246.43.14>:51552/user/Worker
INFO 2015-07-31 05:15:24
org.apache.spark.executor.CoarseGrainedExecutorBackend: Successfully
registered with driver INFO 2015-07-31 05:15:24
org.apache.spark.executor.Executor: Starting executor ID 0 on host
10.246.43.14 INFO 2015-07-31 05:15:24 org.apache.spark.SecurityManager:
Changing view acls to: cassandra,missingmerch INFO 2015-07-31 05:15:24
org.apache.spark.SecurityManager: Changing modify acls to:
cassandra,missingmerch INFO 2015-07-31 05:15:24
org.apache.spark.SecurityManager: SecurityManager: authentication disabled;
ui acls disabled; users with view permissions: Set(cassandra,
missingmerch); users with modify permissions: Set(cassandra, missingmerch)
INFO 2015-07-31 05:15:24 org.apache.spark.util.AkkaUtils: Connecting to
MapOutputTracker: akka.tcp://sparkDriver@tstl400029.wal-mart.com<mailto://
sparkDriver@tstl400029.wal-mart.com>:60525/user/MapOutputTracker
INFO 2015-07-31 05:15:25 org.apache.spark.util.AkkaUtils: Connecting to
BlockManagerMaster: akka.tcp://sparkDriver@tstl400029.wal-mart.com<mailto://
sparkDriver@tstl400029.wal-mart.com>:60525/user/BlockManagerMaster
INFO 2015-07-31 05:15:25 org.apache.spark.storage.DiskBlockManager:
Created local directory at
/var/lib/spark/rdd/spark-2ab3a70e-2bdd-4ab5-9aff-fa48224f14d4/spark-c1430eaf-272d-41d5-acd1-ac88dbca4698/spark-b102f621-e107-4684-96cd-910ed4f2c0cf/spark-7e5ab6f6-3433-4e69-adcc-318083c37c4a
INFO 2015-07-31 05:15:25 org.apache.spark.storage.MemoryStore: MemoryStore
started with capacity 265.4 MB INFO 2015-07-31 05:15:31
org.apache.spark.network.netty.NettyBlockTransferService: Server created on
60707 INFO 2015-07-31 05:15:31
org.apache.spark.storage.BlockManagerMaster: Trying to register
BlockManager INFO 2015-07-31 05:15:31
org.apache.spark.storage.BlockManagerMaster: Registered BlockManager INFO
2015-07-31 05:15:31 org.apache.spark.executor.Executor: Using REPL class
URI: http://10.246.43.15:48903 INFO 2015-07-31 05:15:31
org.apache.spark.util.AkkaUtils: Connecting to HeartbeatReceiver:
akka.tcp://sparkDriver@tstl400029.wal-mart.com<mailto://
sparkDriver@tstl400029.wal-mart.com>:60525/user/HeartbeatReceiver
On Tue, Aug 4, 2015 at 8:38 AM, Guru Medasani <gd...@gmail.com> wrote:
> Hi Satish,
>
> Can you add more error or log info to the email?
>
>
> Guru Medasani
> gdmeda@gmail.com
>
>
>
> On Jul 31, 2015, at 1:06 AM, satish chandra j <js...@gmail.com>
> wrote:
>
> HI,
> I have submitted a Spark Job with options jars,class,master as *local*
> but i am getting an error as below
>
> *dse spark-submit spark error exception in thread main
> java.io.ioexception: Invalid Request Exception(Why you have not logged in)*
>
> *Note: submitting datastax spark node*
>
> please let me know if anybody have a solutions for this issue
>
>
>
> Regards,
> Saish Chandra
>
>
>