SPARK CREATING EXTERNAL TABLE

[Trinadh Kaja <kt...@gmail.com>]

Hi All,

I am facing different problem using spark,

i am using spark-sql.

below are the details,

sqlcontext.sql("""create external table XXXX location '/xxxx' as select *
from XXX""" )

this is my query
table success fully done but in hive command

describe formatted XXXX

showing MANAGETABLE, totally confused,

data loaded in hdfs path successfully,

why hive showing manage table,

i am missing any thing,plz tell me

-- 
Thanks&Regards
K.Trinadh
Ph-7348826118

Re: SPARK CREATING EXTERNAL TABLE

[Mich Talebzadeh <mi...@gmail.com>]

This should work

Spark 2.0.0,  Hive 2.0.1

//create external table in a Hive database with CTAS

scala> spark.sql(""" CREATE EXTERNAL TABLE test.extPrices LOCATION
"/tmp/extPrices" AS SELECT * FROM test.prices LIMIT 5""")
res4: org.apache.spark.sql.DataFrame = []


Now if I go to Hive and look at that table, I see

hive> describe formatted test.extPrices;
OK
# col_name              data_type               comment
timeinserted            string
index                   int
Location:               hdfs://rhes564:9000/tmp/extPrices
Table Type:             EXTERNAL_TABLE
Table Parameters:
        EXTERNAL                TRUE
        numFiles                1
        totalSize               292
        transient_lastDdlTime   1475243290
# Storage Information
SerDe Library:          org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
InputFormat:            org.apache.hadoop.mapred.TextInputFormat
OutputFormat:
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Compressed:             No
Num Buckets:            -1
Bucket Columns:         []
Sort Columns:           []
Storage Desc Params:


Defined as external.

HTH





Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.



On 30 September 2016 at 12:40, Trinadh Kaja <kt...@gmail.com> wrote:

> Hi All,
>
> I am facing different problem using spark,
>
> i am using spark-sql.
>
> below are the details,
>
> sqlcontext.sql("""create external table XXXX location '/xxxx' as select *
> from XXX""" )
>
> this is my query
> table success fully done but in hive command
>
> describe formatted XXXX
>
> showing MANAGETABLE, totally confused,
>
> data loaded in hdfs path successfully,
>
> why hive showing manage table,
>
> i am missing any thing,plz tell me
>
> --
> Thanks&Regards
> K.Trinadh
> Ph-7348826118
>