You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by Zhangshunyu <zh...@126.com> on 2016/09/28 07:11:04 UTC
[discussion]When table properties is repeated it only set the last
one
When table properties is repeated it only set the last one, for example,
CREATE TABLE IF NOT EXISTS carbontable
(ID Int, date Timestamp, country String,
name String, phonetype String, serialname String, salary Int)
STORED BY 'carbondata'
TBLPROPERTIES('DICTIONARY_EXCLUDE'='country','DICTIONARY_INCLUDE'='ID',
'DICTIONARY_EXCLUDE'='phonetype', 'DICTIONARY_INCLUDE'='salary')
As we use map to store the properties, only salary is set to
DICTIONARY_INCLUDE and only phonetype is set to DICTIONARY_EXCLUDE.
I think we can fix this pbm in 2 way:
1. Do restrict syntax check: When tblProperty is repeated, throw an
MalformedCarbonCommandException to tell the user that table properties is
repeated, so that the user would not perform error operation.
2. Let the properties recombining, and set the table properties for all the
specified columns.
What's your opinion?
--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/discussion-When-table-properties-is-repeated-it-only-set-the-last-one-tp1539.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.
Re: [discussion]When table properties is repeated it only set the
last one
Posted by "bill.zhou" <zg...@163.com>.
+1
--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/discussion-When-table-properties-is-repeated-it-only-set-the-last-one-tp1539p1559.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.
RE: [discussion]When table properties is repeated it only set the
last one
Posted by Jihong Ma <Ji...@huawei.com>.
We have start the discussion with Spark guys to open the door for pluggable parser going forward, and some of features we have now would require syntax extension as well, for now, we are ok.
Regards.
Jenny
-----Original Message-----
From: Ravindra Pesala [mailto:ravi.pesala@gmail.com]
Sent: Wednesday, September 28, 2016 8:42 PM
To: dev
Subject: Re: [discussion]When table properties is repeated it only set the last one
I am rethinking, As per the hive behavior it only takes the last one if
tableproperties are repeated. We can add the validation now in carbondata
as we control the ddl parsing. But going forward in spark 2.0 we cannot
control the ddl parsing so it is not possible to add validations there. So
I guess we better stick to the hive behavior to avoid future problems.
Regards,
Ravindra.
On 29 September 2016 at 08:04, Aniket Adnaik <an...@gmail.com>
wrote:
> +1 for option-1- should throw exception..
> Regards,
> Aniket
>
> On 28 Sep 2016 7:01 p.m., "Ravindra Pesala" <ra...@gmail.com> wrote:
>
> > +1 for option 1
> >
> > On Thu, 29 Sep 2016 02:52 Jihong Ma, <Ji...@huawei.com> wrote:
> >
> > > Would prefer error out, vote for option 1.
> > >
> > > Jenny
> > >
> > > -----Original Message-----
> > > From: Zhangshunyu [mailto:zhangshunyu1990@126.com]
> > > Sent: Wednesday, September 28, 2016 12:11 AM
> > > To: dev@carbondata.incubator.apache.org
> > > Subject: [discussion]When table properties is repeated it only set the
> > > last one
> > >
> > > When table properties is repeated it only set the last one, for
> example,
> > >
> > > CREATE TABLE IF NOT EXISTS carbontable
> > > (ID Int, date Timestamp, country String,
> > > name String, phonetype String, serialname String, salary Int)
> > > STORED BY 'carbondata'
> > > TBLPROPERTIES('DICTIONARY_EXCLUDE'='country','
> DICTIONARY_INCLUDE'='ID',
> > > 'DICTIONARY_EXCLUDE'='phonetype', 'DICTIONARY_INCLUDE'='salary')
> > >
> > > As we use map to store the properties, only salary is set to
> > > DICTIONARY_INCLUDE and only phonetype is set to DICTIONARY_EXCLUDE.
> > >
> > > I think we can fix this pbm in 2 way:
> > >
> > > 1. Do restrict syntax check: When tblProperty is repeated, throw an
> > > MalformedCarbonCommandException to tell the user that table properties
> > is
> > > repeated, so that the user would not perform error operation.
> > > 2. Let the properties recombining, and set the table properties for all
> > the
> > > specified columns.
> > >
> > > What's your opinion?
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > > http://apache-carbondata-mailing-list-archive.1130556.
> > n5.nabble.com/discussion-When-table-properties-is-repeated-
> > it-only-set-the-last-one-tp1539.html
> > > Sent from the Apache CarbonData Mailing List archive mailing list
> archive
> > > at Nabble.com.
> > >
> >
>
--
Thanks & Regards,
Ravi
Re: [discussion]When table properties is repeated it only set the
last one
Posted by Ravindra Pesala <ra...@gmail.com>.
I am rethinking, As per the hive behavior it only takes the last one if
tableproperties are repeated. We can add the validation now in carbondata
as we control the ddl parsing. But going forward in spark 2.0 we cannot
control the ddl parsing so it is not possible to add validations there. So
I guess we better stick to the hive behavior to avoid future problems.
Regards,
Ravindra.
On 29 September 2016 at 08:04, Aniket Adnaik <an...@gmail.com>
wrote:
> +1 for option-1- should throw exception..
> Regards,
> Aniket
>
> On 28 Sep 2016 7:01 p.m., "Ravindra Pesala" <ra...@gmail.com> wrote:
>
> > +1 for option 1
> >
> > On Thu, 29 Sep 2016 02:52 Jihong Ma, <Ji...@huawei.com> wrote:
> >
> > > Would prefer error out, vote for option 1.
> > >
> > > Jenny
> > >
> > > -----Original Message-----
> > > From: Zhangshunyu [mailto:zhangshunyu1990@126.com]
> > > Sent: Wednesday, September 28, 2016 12:11 AM
> > > To: dev@carbondata.incubator.apache.org
> > > Subject: [discussion]When table properties is repeated it only set the
> > > last one
> > >
> > > When table properties is repeated it only set the last one, for
> example,
> > >
> > > CREATE TABLE IF NOT EXISTS carbontable
> > > (ID Int, date Timestamp, country String,
> > > name String, phonetype String, serialname String, salary Int)
> > > STORED BY 'carbondata'
> > > TBLPROPERTIES('DICTIONARY_EXCLUDE'='country','
> DICTIONARY_INCLUDE'='ID',
> > > 'DICTIONARY_EXCLUDE'='phonetype', 'DICTIONARY_INCLUDE'='salary')
> > >
> > > As we use map to store the properties, only salary is set to
> > > DICTIONARY_INCLUDE and only phonetype is set to DICTIONARY_EXCLUDE.
> > >
> > > I think we can fix this pbm in 2 way:
> > >
> > > 1. Do restrict syntax check: When tblProperty is repeated, throw an
> > > MalformedCarbonCommandException to tell the user that table properties
> > is
> > > repeated, so that the user would not perform error operation.
> > > 2. Let the properties recombining, and set the table properties for all
> > the
> > > specified columns.
> > >
> > > What's your opinion?
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > > http://apache-carbondata-mailing-list-archive.1130556.
> > n5.nabble.com/discussion-When-table-properties-is-repeated-
> > it-only-set-the-last-one-tp1539.html
> > > Sent from the Apache CarbonData Mailing List archive mailing list
> archive
> > > at Nabble.com.
> > >
> >
>
--
Thanks & Regards,
Ravi
Re: [discussion]When table properties is repeated it only set the
last one
Posted by Aniket Adnaik <an...@gmail.com>.
+1 for option-1- should throw exception..
Regards,
Aniket
On 28 Sep 2016 7:01 p.m., "Ravindra Pesala" <ra...@gmail.com> wrote:
> +1 for option 1
>
> On Thu, 29 Sep 2016 02:52 Jihong Ma, <Ji...@huawei.com> wrote:
>
> > Would prefer error out, vote for option 1.
> >
> > Jenny
> >
> > -----Original Message-----
> > From: Zhangshunyu [mailto:zhangshunyu1990@126.com]
> > Sent: Wednesday, September 28, 2016 12:11 AM
> > To: dev@carbondata.incubator.apache.org
> > Subject: [discussion]When table properties is repeated it only set the
> > last one
> >
> > When table properties is repeated it only set the last one, for example,
> >
> > CREATE TABLE IF NOT EXISTS carbontable
> > (ID Int, date Timestamp, country String,
> > name String, phonetype String, serialname String, salary Int)
> > STORED BY 'carbondata'
> > TBLPROPERTIES('DICTIONARY_EXCLUDE'='country','DICTIONARY_INCLUDE'='ID',
> > 'DICTIONARY_EXCLUDE'='phonetype', 'DICTIONARY_INCLUDE'='salary')
> >
> > As we use map to store the properties, only salary is set to
> > DICTIONARY_INCLUDE and only phonetype is set to DICTIONARY_EXCLUDE.
> >
> > I think we can fix this pbm in 2 way:
> >
> > 1. Do restrict syntax check: When tblProperty is repeated, throw an
> > MalformedCarbonCommandException to tell the user that table properties
> is
> > repeated, so that the user would not perform error operation.
> > 2. Let the properties recombining, and set the table properties for all
> the
> > specified columns.
> >
> > What's your opinion?
> >
> >
> >
> > --
> > View this message in context:
> > http://apache-carbondata-mailing-list-archive.1130556.
> n5.nabble.com/discussion-When-table-properties-is-repeated-
> it-only-set-the-last-one-tp1539.html
> > Sent from the Apache CarbonData Mailing List archive mailing list archive
> > at Nabble.com.
> >
>
Re: [discussion]When table properties is repeated it only set the
last one
Posted by Ravindra Pesala <ra...@gmail.com>.
+1 for option 1
On Thu, 29 Sep 2016 02:52 Jihong Ma, <Ji...@huawei.com> wrote:
> Would prefer error out, vote for option 1.
>
> Jenny
>
> -----Original Message-----
> From: Zhangshunyu [mailto:zhangshunyu1990@126.com]
> Sent: Wednesday, September 28, 2016 12:11 AM
> To: dev@carbondata.incubator.apache.org
> Subject: [discussion]When table properties is repeated it only set the
> last one
>
> When table properties is repeated it only set the last one, for example,
>
> CREATE TABLE IF NOT EXISTS carbontable
> (ID Int, date Timestamp, country String,
> name String, phonetype String, serialname String, salary Int)
> STORED BY 'carbondata'
> TBLPROPERTIES('DICTIONARY_EXCLUDE'='country','DICTIONARY_INCLUDE'='ID',
> 'DICTIONARY_EXCLUDE'='phonetype', 'DICTIONARY_INCLUDE'='salary')
>
> As we use map to store the properties, only salary is set to
> DICTIONARY_INCLUDE and only phonetype is set to DICTIONARY_EXCLUDE.
>
> I think we can fix this pbm in 2 way:
>
> 1. Do restrict syntax check: When tblProperty is repeated, throw an
> MalformedCarbonCommandException to tell the user that table properties is
> repeated, so that the user would not perform error operation.
> 2. Let the properties recombining, and set the table properties for all the
> specified columns.
>
> What's your opinion?
>
>
>
> --
> View this message in context:
> http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/discussion-When-table-properties-is-repeated-it-only-set-the-last-one-tp1539.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>
RE: [discussion]When table properties is repeated it only set the
last one
Posted by Jihong Ma <Ji...@huawei.com>.
Would prefer error out, vote for option 1.
Jenny
-----Original Message-----
From: Zhangshunyu [mailto:zhangshunyu1990@126.com]
Sent: Wednesday, September 28, 2016 12:11 AM
To: dev@carbondata.incubator.apache.org
Subject: [discussion]When table properties is repeated it only set the last one
When table properties is repeated it only set the last one, for example,
CREATE TABLE IF NOT EXISTS carbontable
(ID Int, date Timestamp, country String,
name String, phonetype String, serialname String, salary Int)
STORED BY 'carbondata'
TBLPROPERTIES('DICTIONARY_EXCLUDE'='country','DICTIONARY_INCLUDE'='ID',
'DICTIONARY_EXCLUDE'='phonetype', 'DICTIONARY_INCLUDE'='salary')
As we use map to store the properties, only salary is set to
DICTIONARY_INCLUDE and only phonetype is set to DICTIONARY_EXCLUDE.
I think we can fix this pbm in 2 way:
1. Do restrict syntax check: When tblProperty is repeated, throw an
MalformedCarbonCommandException to tell the user that table properties is
repeated, so that the user would not perform error operation.
2. Let the properties recombining, and set the table properties for all the
specified columns.
What's your opinion?
--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/discussion-When-table-properties-is-repeated-it-only-set-the-last-one-tp1539.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.
Re: [discussion]When table properties is repeated it only set the
last one
Posted by Zhangshunyu <zh...@126.com>.
So what's the conclusion of this problem? Should we keep the same as hive or
throw an error?
-----
My English name is Sunday
--
View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/discussion-When-table-properties-is-repeated-it-only-set-the-last-one-tp1539p1671.html
Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.