You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by akshay_nuthala <ak...@gmail.com> on 2021/03/18 16:30:24 UTC
[DISCUSSION] Support alter schema for complex types
Hi everyone,
Currently carbon only supports altering schema for primitive types.
We can extend it to complex columns too -
1) alter add complex column(array/struct)
2) alter drop complex column(array/struct)
3) alter change datatype/rename complex column(array/struct)
Here phase-1 focuses only on non-nested complex columns.
Phase-2 will later target nested ones.
Please let me know your valid inputs about the same.
Following is the link to the design document.
https://docs.google.com/document/d/1DhhkVXM8rMvOuKDZeccJpFEfO3VkA9C0c7JHCV88NXI/edit
Thanks,
Akshay
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [DISCUSSION] Support alter schema for complex types
Posted by akshay_nuthala <ak...@gmail.com>.
Handled comments
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [DISCUSSION] Support alter schema for complex types
Posted by akashrn5 <ak...@gmail.com>.
Hi,
+1 for feature. Thanks for proposing it as now most of the use case from
user perspective involves complex columns.
I have reviewed the doc and given comments, please work on it, then can be
reviewed again.
Regards,
Akash R
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [DISCUSSION] Support alter schema for complex types
Posted by Ajantha Bhat <aj...@gmail.com>.
Hi Akshay,
The mail description and document content are not matching. For
single-level struct also document says cannot support.
So, please list down all the work that need to be done in points and
then divide which is supported in phase1 and which is supported in phase 2
clearly in the summary section of the document.
Also in the query flow, after adding the column, for previously loaded
segments what will be the output NULL or empty complex type ?
you can refer hive behavior for this. Hope schema evolution (column drift)
also intact with complex column support.
Thanks,
Ajantha
On Tue, Mar 30, 2021 at 11:18 AM Kunal Kapoor <ku...@gmail.com>
wrote:
> +1
>
> On Fri, Mar 26, 2021 at 6:19 PM akshay_nuthala <ak...@gmail.com>
> wrote:
>
> > No, these and other nested level operations will be taken care in the
> next
> > phase.
> >
> >
> >
> > --
> > Sent from:
> > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
> >
>
Re: [DISCUSSION] Support alter schema for complex types
Posted by Kunal Kapoor <ku...@gmail.com>.
+1
On Fri, Mar 26, 2021 at 6:19 PM akshay_nuthala <ak...@gmail.com>
wrote:
> No, these and other nested level operations will be taken care in the next
> phase.
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>
Re: [DISCUSSION] Support alter schema for complex types
Posted by akshay_nuthala <ak...@gmail.com>.
No, these and other nested level operations will be taken care in the next
phase.
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [DISCUSSION] Support alter schema for complex types
Posted by Indhumathi <in...@gmail.com>.
+1. It is an important feature needs to be supported.
Since, you have mentioned alter drop struct child column, are you gonna
handle below scenarios also in phase 1?
1. support rename/change struct primitive child members ?
2. add new primitive child members in an existing struct ?
If so, please update design for those DDL commands.
Regards,
Indhumathi M
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Re: [DISCUSSION] Support alter schema for complex types
Posted by Mahesh Raju Somalaraju <ma...@gmail.com>.
Hi,
+1 For the feature. Agreed with Ajantha.
Thanks & Regards
Mahesh Raju Somalaraju
On Thu, Mar 18, 2021 at 10:00 PM akshay_nuthala <ak...@gmail.com>
wrote:
> Hi everyone,
>
> Currently carbon only supports altering schema for primitive types.
> We can extend it to complex columns too -
> 1) alter add complex column(array/struct)
> 2) alter drop complex column(array/struct)
> 3) alter change datatype/rename complex column(array/struct)
>
> Here phase-1 focuses only on non-nested complex columns.
> Phase-2 will later target nested ones.
>
> Please let me know your valid inputs about the same.
> Following is the link to the design document.
>
>
> https://docs.google.com/document/d/1DhhkVXM8rMvOuKDZeccJpFEfO3VkA9C0c7JHCV88NXI/edit
>
> Thanks,
> Akshay
>
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>