You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jem Tucker <je...@gmail.com> on 2015/07/03 11:02:10 UTC
Accessing the console from spark
Hi,
We have an application that requires a username/password to be entered from
the command line. To screen a password in java you need to use
System.console.readPassword however when running with spark System.console
returns null?? Any ideas on how to get the console from spark?
Thanks,
Jem
Re: Accessing the console from spark
Posted by Jem Tucker <je...@gmail.com>.
Thanks for spending the time to answer this question! I understand that the
console in not available when running through eclipse etc, the issue I have
been having is accessing it when running my code through spark-submit. If
you were to run the code above via spark submit you will see the same error
as when running through intelliJ. I was wondering if spark holds a
reference to the console that I can access at all?
Thanks,
Jem
On Sat, Jul 4, 2015 at 6:58 AM Akhil Das <ak...@sigmoidanalytics.com> wrote:
> Well, System.console returns null if there's no console. (while you run it
> in eclipse, intellij etc)
> [image: Inline image 1]
>
> If you run the same program in the terminal, it will work.
>
> [image: Inline image 2]
>
>
> Thanks
> Best Regards
>
> On Fri, Jul 3, 2015 at 4:10 PM, Jem Tucker <je...@gmail.com> wrote:
>
>> I have shown two senarios below:
>>
>> // setup spark context
>>
>> val user = readLine("username: ")
>> val pass = System.console.readPassword("password: ") <- null pointer
>> exception here
>>
>> and....
>>
>> // setup spark context
>>
>> val user = readLine("username: ")
>> val console = System.console <- null pointer exception here
>> val pass = console.readPassword("password: ")
>>
>>
>> thanks,
>>
>> Jem
>>
>>
>> On Fri, Jul 3, 2015 at 11:04 AM Akhil Das <ak...@sigmoidanalytics.com>
>> wrote:
>>
>>> Can you paste the code? Something is missing
>>>
>>> Thanks
>>> Best Regards
>>>
>>> On Fri, Jul 3, 2015 at 3:14 PM, Jem Tucker <je...@gmail.com> wrote:
>>>
>>>> In the driver when running spark-submit with --master yarn-client
>>>>
>>>> On Fri, Jul 3, 2015 at 10:23 AM Akhil Das <ak...@sigmoidanalytics.com>
>>>> wrote:
>>>>
>>>>> Where does it returns null? Within the driver or in the executor? I
>>>>> just tried System.console.readPassword in spark-shell and it worked.
>>>>>
>>>>> Thanks
>>>>> Best Regards
>>>>>
>>>>> On Fri, Jul 3, 2015 at 2:32 PM, Jem Tucker <je...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We have an application that requires a username/password to be
>>>>>> entered from the command line. To screen a password in java you need to use
>>>>>> System.console.readPassword however when running with spark System.console
>>>>>> returns null?? Any ideas on how to get the console from spark?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Jem
>>>>>>
>>>>>
>>>>>
>>>
>
Re: Accessing the console from spark
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Well, System.console returns null if there's no console. (while you run it
in eclipse, intellij etc)
[image: Inline image 1]
If you run the same program in the terminal, it will work.
[image: Inline image 2]
Thanks
Best Regards
On Fri, Jul 3, 2015 at 4:10 PM, Jem Tucker <je...@gmail.com> wrote:
> I have shown two senarios below:
>
> // setup spark context
>
> val user = readLine("username: ")
> val pass = System.console.readPassword("password: ") <- null pointer
> exception here
>
> and....
>
> // setup spark context
>
> val user = readLine("username: ")
> val console = System.console <- null pointer exception here
> val pass = console.readPassword("password: ")
>
>
> thanks,
>
> Jem
>
>
> On Fri, Jul 3, 2015 at 11:04 AM Akhil Das <ak...@sigmoidanalytics.com>
> wrote:
>
>> Can you paste the code? Something is missing
>>
>> Thanks
>> Best Regards
>>
>> On Fri, Jul 3, 2015 at 3:14 PM, Jem Tucker <je...@gmail.com> wrote:
>>
>>> In the driver when running spark-submit with --master yarn-client
>>>
>>> On Fri, Jul 3, 2015 at 10:23 AM Akhil Das <ak...@sigmoidanalytics.com>
>>> wrote:
>>>
>>>> Where does it returns null? Within the driver or in the executor? I
>>>> just tried System.console.readPassword in spark-shell and it worked.
>>>>
>>>> Thanks
>>>> Best Regards
>>>>
>>>> On Fri, Jul 3, 2015 at 2:32 PM, Jem Tucker <je...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We have an application that requires a username/password to be entered
>>>>> from the command line. To screen a password in java you need to use
>>>>> System.console.readPassword however when running with spark System.console
>>>>> returns null?? Any ideas on how to get the console from spark?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Jem
>>>>>
>>>>
>>>>
>>
Re: Accessing the console from spark
Posted by Jem Tucker <je...@gmail.com>.
I have shown two senarios below:
// setup spark context
val user = readLine("username: ")
val pass = System.console.readPassword("password: ") <- null pointer
exception here
and....
// setup spark context
val user = readLine("username: ")
val console = System.console <- null pointer exception here
val pass = console.readPassword("password: ")
thanks,
Jem
On Fri, Jul 3, 2015 at 11:04 AM Akhil Das <ak...@sigmoidanalytics.com>
wrote:
> Can you paste the code? Something is missing
>
> Thanks
> Best Regards
>
> On Fri, Jul 3, 2015 at 3:14 PM, Jem Tucker <je...@gmail.com> wrote:
>
>> In the driver when running spark-submit with --master yarn-client
>>
>> On Fri, Jul 3, 2015 at 10:23 AM Akhil Das <ak...@sigmoidanalytics.com>
>> wrote:
>>
>>> Where does it returns null? Within the driver or in the executor? I just
>>> tried System.console.readPassword in spark-shell and it worked.
>>>
>>> Thanks
>>> Best Regards
>>>
>>> On Fri, Jul 3, 2015 at 2:32 PM, Jem Tucker <je...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> We have an application that requires a username/password to be entered
>>>> from the command line. To screen a password in java you need to use
>>>> System.console.readPassword however when running with spark System.console
>>>> returns null?? Any ideas on how to get the console from spark?
>>>>
>>>> Thanks,
>>>>
>>>> Jem
>>>>
>>>
>>>
>
Re: Accessing the console from spark
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Can you paste the code? Something is missing
Thanks
Best Regards
On Fri, Jul 3, 2015 at 3:14 PM, Jem Tucker <je...@gmail.com> wrote:
> In the driver when running spark-submit with --master yarn-client
>
> On Fri, Jul 3, 2015 at 10:23 AM Akhil Das <ak...@sigmoidanalytics.com>
> wrote:
>
>> Where does it returns null? Within the driver or in the executor? I just
>> tried System.console.readPassword in spark-shell and it worked.
>>
>> Thanks
>> Best Regards
>>
>> On Fri, Jul 3, 2015 at 2:32 PM, Jem Tucker <je...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> We have an application that requires a username/password to be entered
>>> from the command line. To screen a password in java you need to use
>>> System.console.readPassword however when running with spark System.console
>>> returns null?? Any ideas on how to get the console from spark?
>>>
>>> Thanks,
>>>
>>> Jem
>>>
>>
>>
Re: Accessing the console from spark
Posted by Jem Tucker <je...@gmail.com>.
In the driver when running spark-submit with --master yarn-client
On Fri, Jul 3, 2015 at 10:23 AM Akhil Das <ak...@sigmoidanalytics.com>
wrote:
> Where does it returns null? Within the driver or in the executor? I just
> tried System.console.readPassword in spark-shell and it worked.
>
> Thanks
> Best Regards
>
> On Fri, Jul 3, 2015 at 2:32 PM, Jem Tucker <je...@gmail.com> wrote:
>
>> Hi,
>>
>> We have an application that requires a username/password to be entered
>> from the command line. To screen a password in java you need to use
>> System.console.readPassword however when running with spark System.console
>> returns null?? Any ideas on how to get the console from spark?
>>
>> Thanks,
>>
>> Jem
>>
>
>
Re: Accessing the console from spark
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Where does it returns null? Within the driver or in the executor? I just
tried System.console.readPassword in spark-shell and it worked.
Thanks
Best Regards
On Fri, Jul 3, 2015 at 2:32 PM, Jem Tucker <je...@gmail.com> wrote:
> Hi,
>
> We have an application that requires a username/password to be entered
> from the command line. To screen a password in java you need to use
> System.console.readPassword however when running with spark System.console
> returns null?? Any ideas on how to get the console from spark?
>
> Thanks,
>
> Jem
>