You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Tomas Zubiri <tz...@prokarma.com> on 2018/05/04 20:15:49 UTC
RV: Unintelligible warning arose out of the blue.
________________________________
De: Tomas Zubiri
Enviado: viernes, 04 de mayo de 2018 04:23 p.m.
Para: user@spark.apache.org
Asunto: Unintelligible warning arose out of the blue.
My setup is as follows:
Windows 10
Python 3.6.5
Spark 2.3.0
The latest java jdk
winutils/hadoop installed from this github page https://github.com/steveloughran/winutils
I initialize spark from the pyspark shell as follows:
df = spark.read.csv('mvce.csv')
the mvce is a small file with 3 lines and 2 columns.
The warning I receive is:
2018-05-04 16:17:44 WARN ObjectStore:568 - Failed to get database global_temp, returning NoSuchObjectException
What does this mean? I think it has something to do with YARN, but being an internal technology I have no clue about it. It doesn't seem to be causing any trouble, but I don't want to add the uncertainty that this might be causing an bug in future diagnosing of issues.
Thank you for your help!
Re: RV: Unintelligible warning arose out of the blue.
Posted by Marco Mistroni <mm...@gmail.com>.
Hi
i think it has to do with spark configuration, dont think the standard
configuration is geared up to be running in local mode on windows
your dataframe is ok, you can check out that you have read it successfully
by printing out df.count() and you will see your code is reading the
dataframe successfully
hth
m
On Fri, May 4, 2018 at 9:15 PM, Tomas Zubiri <tz...@prokarma.com> wrote:
>
>
>
> ------------------------------
> *De:* Tomas Zubiri
> *Enviado:* viernes, 04 de mayo de 2018 04:23 p.m.
> *Para:* user@spark.apache.org
> *Asunto:* Unintelligible warning arose out of the blue.
>
>
> My setup is as follows:
> Windows 10
> Python 3.6.5
> Spark 2.3.0
> The latest java jdk
> winutils/hadoop installed from this github page https://github.com/
> steveloughran/winutils
>
> I initialize spark from the pyspark shell as follows:
> df = spark.read.csv('mvce.csv')
>
>
> the mvce is a small file with 3 lines and 2 columns.
>
>
> The warning I receive is:
>
> 2018-05-04 16:17:44 WARN ObjectStore:568 - Failed to get database
> global_temp, returning NoSuchObjectException
>
> What does this mean? I think it has something to do with YARN, but being
> an internal technology I have no clue about it. It doesn't seem to be
> causing any trouble, but I don't want to add the uncertainty that this
> might be causing an bug in future diagnosing of issues.
>
> Thank you for your help!
>
>
>