You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Vajiha Begum S A <va...@maestrowiz.com> on 2023/01/04 07:26:07 UTC
Got Error Creating permanent view in Postgresql through Pyspark code
I have tried to Create a permanent view in Postgresql DB through Pyspark
code, but I have received the below error message. Kindly help me to create
a permanent view table in the database.How shall create permanent view
using Pyspark code. Please do reply.
*Error Message::*
*Exception has occurred: Analysis Exception*
Not allowed to create a permanent view `default`.`TEMP1` by referencing a
temporary view TEMP_VIEW. Please create a temp view instead by CREATE TEMP
VIEW
Regards,
Vajiha
Research Analyst
MW Solutions
Re: Got Error Creating permanent view in Postgresql through Pyspark code
Posted by Stelios Philippou <st...@gmail.com>.
Vajiha,
I believe that you might be confusing stuff ?
Permanent View in PSQL is a standard view.
Temp view or Global View is the Spark View that is internal for Spark.
Can we get a snippet of the code please.
On Wed, 4 Jan 2023 at 15:10, Vajiha Begum S A <va...@maestrowiz.com>
wrote:
>
> I have tried to Create a permanent view in Postgresql DB through Pyspark
> code, but I have received the below error message. Kindly help me to create
> a permanent view table in the database.How shall create permanent view
> using Pyspark code. Please do reply.
>
> *Error Message::*
> *Exception has occurred: Analysis Exception*
> Not allowed to create a permanent view `default`.`TEMP1` by referencing a
> temporary view TEMP_VIEW. Please create a temp view instead by CREATE TEMP
> VIEW
>
>
> Regards,
> Vajiha
> Research Analyst
> MW Solutions
>
Re: Got Error Creating permanent view in Postgresql through Pyspark code
Posted by ayan guha <gu...@gmail.com>.
Hi
What you are trying to do does not make sense. I suggest you to understand
how Views work in SQL. IMHO you are better off creating a table.
Ayan
On Fri, 6 Jan 2023 at 12:20 am, Stelios Philippou <st...@gmail.com>
wrote:
> Vajiha,
>
> I dont see your query working as you hope it will.
>
> spark.sql will execute a query on a database level
>
> to retrieve the temp view you need to go from the sessions.
> i.e
>
> session.sql("SELECT * FROM TEP_VIEW")
>
> You might need to retrieve the data in a collection and iterate over them
> to do batch insertion using spark.sql("INSERt ...");
>
> Hope this helps
>
> Stelios
>
>
> --------------
> Hi Stelios Philippou,
> I need to create a view table in Postgresql DB using pyspark code. But I'm
> unable to create a view table, I can able to create table through pyspark
> code.
> I need to know Whether through Pyspark code can I create view table in
> postgresql database or not. Thanks for you reply
>
> Pyspark Code:
> df.createOrReplaceTempView("TEMP_VIEW")
> spark.sql("CREATE VIEW TEMP1 AS SELECT * FROM TEMP_VIEW")
>
> On Wed, 4 Jan 2023 at 15:10, Vajiha Begum S A <
> vajihabegum.s@maestrowiz.com> wrote:
>
>>
>> I have tried to Create a permanent view in Postgresql DB through Pyspark
>> code, but I have received the below error message. Kindly help me to create
>> a permanent view table in the database.How shall create permanent view
>> using Pyspark code. Please do reply.
>>
>> *Error Message::*
>> *Exception has occurred: Analysis Exception*
>> Not allowed to create a permanent view `default`.`TEMP1` by referencing a
>> temporary view TEMP_VIEW. Please create a temp view instead by CREATE TEMP
>> VIEW
>>
>>
>> Regards,
>> Vajiha
>> Research Analyst
>> MW Solutions
>>
> --
Best Regards,
Ayan Guha
Re: Got Error Creating permanent view in Postgresql through Pyspark code
Posted by Stelios Philippou <st...@gmail.com>.
Vajiha,
I dont see your query working as you hope it will.
spark.sql will execute a query on a database level
to retrieve the temp view you need to go from the sessions.
i.e
session.sql("SELECT * FROM TEP_VIEW")
You might need to retrieve the data in a collection and iterate over them
to do batch insertion using spark.sql("INSERt ...");
Hope this helps
Stelios
--------------
Hi Stelios Philippou,
I need to create a view table in Postgresql DB using pyspark code. But I'm
unable to create a view table, I can able to create table through pyspark
code.
I need to know Whether through Pyspark code can I create view table in
postgresql database or not. Thanks for you reply
Pyspark Code:
df.createOrReplaceTempView("TEMP_VIEW")
spark.sql("CREATE VIEW TEMP1 AS SELECT * FROM TEMP_VIEW")
On Wed, 4 Jan 2023 at 15:10, Vajiha Begum S A <va...@maestrowiz.com>
wrote:
>
> I have tried to Create a permanent view in Postgresql DB through Pyspark
> code, but I have received the below error message. Kindly help me to create
> a permanent view table in the database.How shall create permanent view
> using Pyspark code. Please do reply.
>
> *Error Message::*
> *Exception has occurred: Analysis Exception*
> Not allowed to create a permanent view `default`.`TEMP1` by referencing a
> temporary view TEMP_VIEW. Please create a temp view instead by CREATE TEMP
> VIEW
>
>
> Regards,
> Vajiha
> Research Analyst
> MW Solutions
>