[jira] [Resolved] (SPARK-7476) Dynamic partitioning random behaviour

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-7476.
------------------------------
    Resolution: Invalid

I think this is at best a question for user@. I don't think this relates to dynamic partition discovery if that's what you mean, nor is it random.

> Dynamic partitioning random behaviour
> -------------------------------------
>
>                 Key: SPARK-7476
>                 URL: https://issues.apache.org/jira/browse/SPARK-7476
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.2.0
>         Environment: Spark SQL in standalone mode on CDH 5.3
>            Reporter: Eswara Reddy Adapa
>
> According documentation spark sql 1.2 supports dynamic partitioning.
> But, we see below
> Expected output - 
> !http://ibin.co/20zI4242Ur1h!
> Output seen – 
> !http://ibin.co/20zILW5LQ5nT!
> It is generating only one partition (on a random value each time)
> Query:
> USE miah_ga;
> SET hive.exec.dynamic.partition=true;
> SET hive.exec.dynamic.partition.mode = nonstrict;
> DROP TABLE sem_stg_tmp_part;
> CREATE TABLE sem_stg_tmp_part
> (chnl_nm string
> ,cmpgn_yr_nbr string
> ,cmpgn_qtr_nbr string
> ,actv_mo_nm string
> ,actv_wk_end_dt string
> ,actv_dt string
> ,seg_nm string
> ,bdgt_node_nm string
> ,ad_grp_nm string
> ,kywrd_txt string
> ,srch_engn_nm string
> ,engn_cmpgn_nm string
> ,publ_ctry_nm string
> ,publ_geo_cd string
> ,last_dest_url_txt string
> ,dvc_cat_nm string
> ,mdia_propty_nm string
> ,mdia_type_nm string
> ,audnc_nm string
> ,sub_aud_nm string
> ,prog_nm string
> ,org_intv_nm string
> ,sub_org_intv_nm string
> ,prd_nm string
> ,prim_engg_dest_nm string
> ,imprsn_cnt string
> ,click_cnt string
> ,tot_cost_amt string
> ,ctr_pct string
> ,cpc_amt string
> ,vist_cnt string
> ,paid_per_vist_cnt string
> ,paid_pir_vist_cnt string
> ,cost_per_intel_per_vist_amt string)
> partitioned by (ddate string)
> ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
> STORED AS TEXTFILE;
> INSERT overwrite TABLE sem_stg_tmp_part PARTITION (ddate)
> SELECT *, concat(substr(actv_dt,1,2),substr(actv_dt,4,2),substr(actv_dt,7,4)) as ddate
> FROM miah_ga.sem_stg_tmp;



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org