You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hudi.apache.org by vino yang <ya...@gmail.com> on 2020/01/16 05:56:02 UTC
[DISCUSS] Rename the name of hudi-hadoop-mr module
Hi guys,
I want to start a proposal about refactoring the naming of the
"hudi-hadoop-mr" module.
IMHO, this module name is not user-friendly. It may make users confused.
Because it looks like that it's about integrating with MapReduce( although
I know it referenced parquet-mr[1] project).
Based on the purpose of this module (contains InputFormat implementations
for ReadOptimized, Incremental, Realtime views).
I suggest that we can rename it to "*hudi-query-common*". Then, we can also
rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
What do you think?
Any thoughts and suggestions are welcome and appreciated.
Best,
Vino
[1]: https://github.com/apache/parquet-mr
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by Vinoth Chandar <vi...@apache.org>.
Great! HUDI-572 filed!
On Sun, Jan 19, 2020 at 6:09 PM vino yang <ya...@gmail.com> wrote:
> Hi Vinoth,
>
> OK, I find that our main difference lies in our understanding of
> "integration".
>
> In general, my understanding is that the integration module is a separate
> module. But it seems that "hudi-hadoop-mr" just contains some basic class
> files, and the module will be used as a public library by other modules.
>
> But on the other hand, hudi itself exists as a library. Therefore, there
> seems to be no problem in understanding this way. The "integration" I
> understand is still stuck in the model where the main framework can
> function as a single individual (for example Flink, Presto and so on).
>
> In short, now I have no problem, +1 to rename "hudi-hadoop-mr" to
> "hudi-hive".
>
> Best,
> Vino
>
> Vinoth Chandar <vi...@apache.org> 于2020年1月19日周日 上午1:41写道：
>
> > >I suggest that we use the "hudi-{another bigdata framework}" naming
> > pattern more carefully
> >
> > Fully understand your concern. But thats exactly what hudi-hadoop-mr is
> > doing, is it not? :) InputFormats are how you integrate to Hive.
> >
> > hudi-spark, hudi-presto etc have their own integrations, but they both
> can
> > fallback to the hive integration.
> >
> > On Thu, Jan 16, 2020 at 8:36 PM vino yang <ya...@gmail.com> wrote:
> >
> > > Hi Vionth, Bhavani,
> > >
> > > +1 for renaming hudi-hive -> hudi-hive-sync
> > >
> > > About "hudi-hadoopm-mr -> hudi-hive", I suggest that we use the
> > > "hudi-{another bigdata framework}" naming pattern more carefully. On a
> > > superficial level of understanding. It is very easy for users to
> > > misunderstand that the module is doing ecosystem integration.
> Especially
> > > those who have seen the source code of mainstream projects, such as
> > > presto.[1]
> > >
> > > When we go to check out the hardi-hadoop-mr, it actually just contains
> > some
> > > InputFormat.
> > >
> > > If we do want to mention other frameworks without letting users
> > > misunderstand that we are doing ecosystem integration. Then, we need to
> > add
> > > additional information, for example: "hudi- {another bigdata framework}
> > > -xxx" or "hudi-xxx- {another bigdata framework}".
> > >
> > > [1]: https://github.com/prestodb/presto
> > >
> > > Best,
> > > Vino
> > >
> > > Bhavani Sudha <bh...@gmail.com> 于2020年1月17日周五 上午5:42写道：
> > >
> > > > Thanks @vinoth for giving a overall picture. I think I can relate
> > better
> > > > with the name changes you proposed.
> > > >
> > > > +1 for renaming hudi-hive -> hudi-hive-sync and hudi-hadoopm-mr ->
> > > > hudi-hive
> > > >
> > > > On Thu, Jan 16, 2020 at 1:33 PM Vinoth Chandar <vi...@apache.org>
> > > wrote:
> > > >
> > > > > First let me share the context for the existing name.. We saw how
> > > Parquet
> > > > > hands out the InputFormat and named it similar to parquet-mr.
> > > > > InputFormat is indeed a MapReduce class.. I know we live in the age
> > of
> > > > > Flink and Spark.. But its true :)
> > > > >
> > > > > I think this is the crux of the "understandability" issue..
> > > > >
> > > > > Here are my thoughts..
> > > > >
> > > > > - +0 (neutral) on the rename to hudi-query-common., (whatever we
> > > decide,
> > > > > we need to rename the bundle accordingly)
> > > > > - On hudi-query-bundle being confusing with hive/spark/presto
> > > bundles, I
> > > > > don't feel its more confusing than it is today
> > > > >
> > > > > Real issue IMO, is hudi-hive, which is really about syncing to
> hive,
> > > not
> > > > > querying Hive.
> > > > > Then, may be we can rename
> > > > > - hudi-hadoop-mr to hudi-hive (more understandable, Hive does use
> > > > > InputFormat as the abstraction)
> > > > > - current hudi-hive to hudi-hive-sync
> > > > > (bundles renamed accordingly)
> > > > >
> > > > > I know this hijacks the conversation. Apologize :). But thought I'd
> > > > present
> > > > > a broader take
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jan 16, 2020 at 11:26 AM Bhavani Sudha Saktheeswaran
> > > > > <bh...@uber.com.invalid> wrote:
> > > > >
> > > > > > +1 to generally renaming the packages. Since this is about
> renaming
> > > for
> > > > > the
> > > > > > purpose of making it user friendly, I am concerned if we make
> this
> > as
> > > > > > hudi-query-bundle, users might get confused with other modules
> like
> > > > > > hudi-hive and hudi-spark. And inside packaging module, we further
> > > have
> > > > > > bundles specific to spark, hive and presto.
> > > > > >
> > > > > > Any suggestions on how to rename broadly to avoid these
> confusions?
> > > Let
> > > > > me
> > > > > > also think and get back.
> > > > > >
> > > > > > Thanks,
> > > > > > Sudha
> > > > > >
> > > > > > On Wed, Jan 15, 2020 at 9:56 PM vino yang <yanghua1127@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > > > Hi guys,
> > > > > > >
> > > > > > > I want to start a proposal about refactoring the naming of the
> > > > > > > "hudi-hadoop-mr" module.
> > > > > > >
> > > > > > > IMHO, this module name is not user-friendly. It may make users
> > > > > confused.
> > > > > > > Because it looks like that it's about integrating with
> MapReduce(
> > > > > > although
> > > > > > > I know it referenced parquet-mr[1] project).
> > > > > > >
> > > > > > > Based on the purpose of this module (contains InputFormat
> > > > > implementations
> > > > > > > for ReadOptimized, Incremental, Realtime views).
> > > > > > >
> > > > > > > I suggest that we can rename it to "*hudi-query-common*". Then,
> > we
> > > > can
> > > > > > also
> > > > > > > rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
> > > > > > >
> > > > > > > What do you think?
> > > > > > >
> > > > > > > Any thoughts and suggestions are welcome and appreciated.
> > > > > > >
> > > > > > > Best,
> > > > > > > Vino
> > > > > > >
> > > > > > > [1]:
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_parquet-2Dmr&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=dmZJgDEuo5sZCNsoyMRQUpiJoBP7u4r2i8cdHDMmQic&s=4CnBhu54QxDqAWdCb3NXUdQg9beV2xEmgx-N0yhTr9Y&e=
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by vino yang <ya...@gmail.com>.
Hi Vinoth,
OK, I find that our main difference lies in our understanding of
"integration".
In general, my understanding is that the integration module is a separate
module. But it seems that "hudi-hadoop-mr" just contains some basic class
files, and the module will be used as a public library by other modules.
But on the other hand, hudi itself exists as a library. Therefore, there
seems to be no problem in understanding this way. The "integration" I
understand is still stuck in the model where the main framework can
function as a single individual (for example Flink, Presto and so on).
In short, now I have no problem, +1 to rename "hudi-hadoop-mr" to
"hudi-hive".
Best,
Vino
Vinoth Chandar <vi...@apache.org> 于2020年1月19日周日 上午1:41写道：
> >I suggest that we use the "hudi-{another bigdata framework}" naming
> pattern more carefully
>
> Fully understand your concern. But thats exactly what hudi-hadoop-mr is
> doing, is it not? :) InputFormats are how you integrate to Hive.
>
> hudi-spark, hudi-presto etc have their own integrations, but they both can
> fallback to the hive integration.
>
> On Thu, Jan 16, 2020 at 8:36 PM vino yang <ya...@gmail.com> wrote:
>
> > Hi Vionth, Bhavani,
> >
> > +1 for renaming hudi-hive -> hudi-hive-sync
> >
> > About "hudi-hadoopm-mr -> hudi-hive", I suggest that we use the
> > "hudi-{another bigdata framework}" naming pattern more carefully. On a
> > superficial level of understanding. It is very easy for users to
> > misunderstand that the module is doing ecosystem integration. Especially
> > those who have seen the source code of mainstream projects, such as
> > presto.[1]
> >
> > When we go to check out the hardi-hadoop-mr, it actually just contains
> some
> > InputFormat.
> >
> > If we do want to mention other frameworks without letting users
> > misunderstand that we are doing ecosystem integration. Then, we need to
> add
> > additional information, for example: "hudi- {another bigdata framework}
> > -xxx" or "hudi-xxx- {another bigdata framework}".
> >
> > [1]: https://github.com/prestodb/presto
> >
> > Best,
> > Vino
> >
> > Bhavani Sudha <bh...@gmail.com> 于2020年1月17日周五 上午5:42写道：
> >
> > > Thanks @vinoth for giving a overall picture. I think I can relate
> better
> > > with the name changes you proposed.
> > >
> > > +1 for renaming hudi-hive -> hudi-hive-sync and hudi-hadoopm-mr ->
> > > hudi-hive
> > >
> > > On Thu, Jan 16, 2020 at 1:33 PM Vinoth Chandar <vi...@apache.org>
> > wrote:
> > >
> > > > First let me share the context for the existing name.. We saw how
> > Parquet
> > > > hands out the InputFormat and named it similar to parquet-mr.
> > > > InputFormat is indeed a MapReduce class.. I know we live in the age
> of
> > > > Flink and Spark.. But its true :)
> > > >
> > > > I think this is the crux of the "understandability" issue..
> > > >
> > > > Here are my thoughts..
> > > >
> > > > - +0 (neutral) on the rename to hudi-query-common., (whatever we
> > decide,
> > > > we need to rename the bundle accordingly)
> > > > - On hudi-query-bundle being confusing with hive/spark/presto
> > bundles, I
> > > > don't feel its more confusing than it is today
> > > >
> > > > Real issue IMO, is hudi-hive, which is really about syncing to hive,
> > not
> > > > querying Hive.
> > > > Then, may be we can rename
> > > > - hudi-hadoop-mr to hudi-hive (more understandable, Hive does use
> > > > InputFormat as the abstraction)
> > > > - current hudi-hive to hudi-hive-sync
> > > > (bundles renamed accordingly)
> > > >
> > > > I know this hijacks the conversation. Apologize :). But thought I'd
> > > present
> > > > a broader take
> > > >
> > > >
> > > >
> > > > On Thu, Jan 16, 2020 at 11:26 AM Bhavani Sudha Saktheeswaran
> > > > <bh...@uber.com.invalid> wrote:
> > > >
> > > > > +1 to generally renaming the packages. Since this is about renaming
> > for
> > > > the
> > > > > purpose of making it user friendly, I am concerned if we make this
> as
> > > > > hudi-query-bundle, users might get confused with other modules like
> > > > > hudi-hive and hudi-spark. And inside packaging module, we further
> > have
> > > > > bundles specific to spark, hive and presto.
> > > > >
> > > > > Any suggestions on how to rename broadly to avoid these confusions?
> > Let
> > > > me
> > > > > also think and get back.
> > > > >
> > > > > Thanks,
> > > > > Sudha
> > > > >
> > > > > On Wed, Jan 15, 2020 at 9:56 PM vino yang <ya...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi guys,
> > > > > >
> > > > > > I want to start a proposal about refactoring the naming of the
> > > > > > "hudi-hadoop-mr" module.
> > > > > >
> > > > > > IMHO, this module name is not user-friendly. It may make users
> > > > confused.
> > > > > > Because it looks like that it's about integrating with MapReduce(
> > > > > although
> > > > > > I know it referenced parquet-mr[1] project).
> > > > > >
> > > > > > Based on the purpose of this module (contains InputFormat
> > > > implementations
> > > > > > for ReadOptimized, Incremental, Realtime views).
> > > > > >
> > > > > > I suggest that we can rename it to "*hudi-query-common*". Then,
> we
> > > can
> > > > > also
> > > > > > rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > Any thoughts and suggestions are welcome and appreciated.
> > > > > >
> > > > > > Best,
> > > > > > Vino
> > > > > >
> > > > > > [1]:
> > > > > >
> > > > >
> > > >
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_parquet-2Dmr&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=dmZJgDEuo5sZCNsoyMRQUpiJoBP7u4r2i8cdHDMmQic&s=4CnBhu54QxDqAWdCb3NXUdQg9beV2xEmgx-N0yhTr9Y&e=
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by Vinoth Chandar <vi...@apache.org>.
>I suggest that we use the "hudi-{another bigdata framework}" naming
pattern more carefully
Fully understand your concern. But thats exactly what hudi-hadoop-mr is
doing, is it not? :) InputFormats are how you integrate to Hive.
hudi-spark, hudi-presto etc have their own integrations, but they both can
fallback to the hive integration.
On Thu, Jan 16, 2020 at 8:36 PM vino yang <ya...@gmail.com> wrote:
> Hi Vionth, Bhavani,
>
> +1 for renaming hudi-hive -> hudi-hive-sync
>
> About "hudi-hadoopm-mr -> hudi-hive", I suggest that we use the
> "hudi-{another bigdata framework}" naming pattern more carefully. On a
> superficial level of understanding. It is very easy for users to
> misunderstand that the module is doing ecosystem integration. Especially
> those who have seen the source code of mainstream projects, such as
> presto.[1]
>
> When we go to check out the hardi-hadoop-mr, it actually just contains some
> InputFormat.
>
> If we do want to mention other frameworks without letting users
> misunderstand that we are doing ecosystem integration. Then, we need to add
> additional information, for example: "hudi- {another bigdata framework}
> -xxx" or "hudi-xxx- {another bigdata framework}".
>
> [1]: https://github.com/prestodb/presto
>
> Best,
> Vino
>
> Bhavani Sudha <bh...@gmail.com> 于2020年1月17日周五 上午5:42写道：
>
> > Thanks @vinoth for giving a overall picture. I think I can relate better
> > with the name changes you proposed.
> >
> > +1 for renaming hudi-hive -> hudi-hive-sync and hudi-hadoopm-mr ->
> > hudi-hive
> >
> > On Thu, Jan 16, 2020 at 1:33 PM Vinoth Chandar <vi...@apache.org>
> wrote:
> >
> > > First let me share the context for the existing name.. We saw how
> Parquet
> > > hands out the InputFormat and named it similar to parquet-mr.
> > > InputFormat is indeed a MapReduce class.. I know we live in the age of
> > > Flink and Spark.. But its true :)
> > >
> > > I think this is the crux of the "understandability" issue..
> > >
> > > Here are my thoughts..
> > >
> > > - +0 (neutral) on the rename to hudi-query-common., (whatever we
> decide,
> > > we need to rename the bundle accordingly)
> > > - On hudi-query-bundle being confusing with hive/spark/presto
> bundles, I
> > > don't feel its more confusing than it is today
> > >
> > > Real issue IMO, is hudi-hive, which is really about syncing to hive,
> not
> > > querying Hive.
> > > Then, may be we can rename
> > > - hudi-hadoop-mr to hudi-hive (more understandable, Hive does use
> > > InputFormat as the abstraction)
> > > - current hudi-hive to hudi-hive-sync
> > > (bundles renamed accordingly)
> > >
> > > I know this hijacks the conversation. Apologize :). But thought I'd
> > present
> > > a broader take
> > >
> > >
> > >
> > > On Thu, Jan 16, 2020 at 11:26 AM Bhavani Sudha Saktheeswaran
> > > <bh...@uber.com.invalid> wrote:
> > >
> > > > +1 to generally renaming the packages. Since this is about renaming
> for
> > > the
> > > > purpose of making it user friendly, I am concerned if we make this as
> > > > hudi-query-bundle, users might get confused with other modules like
> > > > hudi-hive and hudi-spark. And inside packaging module, we further
> have
> > > > bundles specific to spark, hive and presto.
> > > >
> > > > Any suggestions on how to rename broadly to avoid these confusions?
> Let
> > > me
> > > > also think and get back.
> > > >
> > > > Thanks,
> > > > Sudha
> > > >
> > > > On Wed, Jan 15, 2020 at 9:56 PM vino yang <ya...@gmail.com>
> > wrote:
> > > >
> > > > > Hi guys,
> > > > >
> > > > > I want to start a proposal about refactoring the naming of the
> > > > > "hudi-hadoop-mr" module.
> > > > >
> > > > > IMHO, this module name is not user-friendly. It may make users
> > > confused.
> > > > > Because it looks like that it's about integrating with MapReduce(
> > > > although
> > > > > I know it referenced parquet-mr[1] project).
> > > > >
> > > > > Based on the purpose of this module (contains InputFormat
> > > implementations
> > > > > for ReadOptimized, Incremental, Realtime views).
> > > > >
> > > > > I suggest that we can rename it to "*hudi-query-common*". Then, we
> > can
> > > > also
> > > > > rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
> > > > >
> > > > > What do you think?
> > > > >
> > > > > Any thoughts and suggestions are welcome and appreciated.
> > > > >
> > > > > Best,
> > > > > Vino
> > > > >
> > > > > [1]:
> > > > >
> > > >
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_parquet-2Dmr&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=dmZJgDEuo5sZCNsoyMRQUpiJoBP7u4r2i8cdHDMmQic&s=4CnBhu54QxDqAWdCb3NXUdQg9beV2xEmgx-N0yhTr9Y&e=
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by vino yang <ya...@gmail.com>.
Hi Vionth, Bhavani,
+1 for renaming hudi-hive -> hudi-hive-sync
About "hudi-hadoopm-mr -> hudi-hive", I suggest that we use the
"hudi-{another bigdata framework}" naming pattern more carefully. On a
superficial level of understanding. It is very easy for users to
misunderstand that the module is doing ecosystem integration. Especially
those who have seen the source code of mainstream projects, such as
presto.[1]
When we go to check out the hardi-hadoop-mr, it actually just contains some
InputFormat.
If we do want to mention other frameworks without letting users
misunderstand that we are doing ecosystem integration. Then, we need to add
additional information, for example: "hudi- {another bigdata framework}
-xxx" or "hudi-xxx- {another bigdata framework}".
[1]: https://github.com/prestodb/presto
Best,
Vino
Bhavani Sudha <bh...@gmail.com> 于2020年1月17日周五 上午5:42写道：
> Thanks @vinoth for giving a overall picture. I think I can relate better
> with the name changes you proposed.
>
> +1 for renaming hudi-hive -> hudi-hive-sync and hudi-hadoopm-mr ->
> hudi-hive
>
> On Thu, Jan 16, 2020 at 1:33 PM Vinoth Chandar <vi...@apache.org> wrote:
>
> > First let me share the context for the existing name.. We saw how Parquet
> > hands out the InputFormat and named it similar to parquet-mr.
> > InputFormat is indeed a MapReduce class.. I know we live in the age of
> > Flink and Spark.. But its true :)
> >
> > I think this is the crux of the "understandability" issue..
> >
> > Here are my thoughts..
> >
> > - +0 (neutral) on the rename to hudi-query-common., (whatever we decide,
> > we need to rename the bundle accordingly)
> > - On hudi-query-bundle being confusing with hive/spark/presto bundles, I
> > don't feel its more confusing than it is today
> >
> > Real issue IMO, is hudi-hive, which is really about syncing to hive, not
> > querying Hive.
> > Then, may be we can rename
> > - hudi-hadoop-mr to hudi-hive (more understandable, Hive does use
> > InputFormat as the abstraction)
> > - current hudi-hive to hudi-hive-sync
> > (bundles renamed accordingly)
> >
> > I know this hijacks the conversation. Apologize :). But thought I'd
> present
> > a broader take
> >
> >
> >
> > On Thu, Jan 16, 2020 at 11:26 AM Bhavani Sudha Saktheeswaran
> > <bh...@uber.com.invalid> wrote:
> >
> > > +1 to generally renaming the packages. Since this is about renaming for
> > the
> > > purpose of making it user friendly, I am concerned if we make this as
> > > hudi-query-bundle, users might get confused with other modules like
> > > hudi-hive and hudi-spark. And inside packaging module, we further have
> > > bundles specific to spark, hive and presto.
> > >
> > > Any suggestions on how to rename broadly to avoid these confusions? Let
> > me
> > > also think and get back.
> > >
> > > Thanks,
> > > Sudha
> > >
> > > On Wed, Jan 15, 2020 at 9:56 PM vino yang <ya...@gmail.com>
> wrote:
> > >
> > > > Hi guys,
> > > >
> > > > I want to start a proposal about refactoring the naming of the
> > > > "hudi-hadoop-mr" module.
> > > >
> > > > IMHO, this module name is not user-friendly. It may make users
> > confused.
> > > > Because it looks like that it's about integrating with MapReduce(
> > > although
> > > > I know it referenced parquet-mr[1] project).
> > > >
> > > > Based on the purpose of this module (contains InputFormat
> > implementations
> > > > for ReadOptimized, Incremental, Realtime views).
> > > >
> > > > I suggest that we can rename it to "*hudi-query-common*". Then, we
> can
> > > also
> > > > rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
> > > >
> > > > What do you think?
> > > >
> > > > Any thoughts and suggestions are welcome and appreciated.
> > > >
> > > > Best,
> > > > Vino
> > > >
> > > > [1]:
> > > >
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_parquet-2Dmr&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=dmZJgDEuo5sZCNsoyMRQUpiJoBP7u4r2i8cdHDMmQic&s=4CnBhu54QxDqAWdCb3NXUdQg9beV2xEmgx-N0yhTr9Y&e=
> > > >
> > >
> >
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by Bhavani Sudha <bh...@gmail.com>.
Thanks @vinoth for giving a overall picture. I think I can relate better
with the name changes you proposed.
+1 for renaming hudi-hive -> hudi-hive-sync and hudi-hadoopm-mr -> hudi-hive
On Thu, Jan 16, 2020 at 1:33 PM Vinoth Chandar <vi...@apache.org> wrote:
> First let me share the context for the existing name.. We saw how Parquet
> hands out the InputFormat and named it similar to parquet-mr.
> InputFormat is indeed a MapReduce class.. I know we live in the age of
> Flink and Spark.. But its true :)
>
> I think this is the crux of the "understandability" issue..
>
> Here are my thoughts..
>
> - +0 (neutral) on the rename to hudi-query-common., (whatever we decide,
> we need to rename the bundle accordingly)
> - On hudi-query-bundle being confusing with hive/spark/presto bundles, I
> don't feel its more confusing than it is today
>
> Real issue IMO, is hudi-hive, which is really about syncing to hive, not
> querying Hive.
> Then, may be we can rename
> - hudi-hadoop-mr to hudi-hive (more understandable, Hive does use
> InputFormat as the abstraction)
> - current hudi-hive to hudi-hive-sync
> (bundles renamed accordingly)
>
> I know this hijacks the conversation. Apologize :). But thought I'd present
> a broader take
>
>
>
> On Thu, Jan 16, 2020 at 11:26 AM Bhavani Sudha Saktheeswaran
> <bh...@uber.com.invalid> wrote:
>
> > +1 to generally renaming the packages. Since this is about renaming for
> the
> > purpose of making it user friendly, I am concerned if we make this as
> > hudi-query-bundle, users might get confused with other modules like
> > hudi-hive and hudi-spark. And inside packaging module, we further have
> > bundles specific to spark, hive and presto.
> >
> > Any suggestions on how to rename broadly to avoid these confusions? Let
> me
> > also think and get back.
> >
> > Thanks,
> > Sudha
> >
> > On Wed, Jan 15, 2020 at 9:56 PM vino yang <ya...@gmail.com> wrote:
> >
> > > Hi guys,
> > >
> > > I want to start a proposal about refactoring the naming of the
> > > "hudi-hadoop-mr" module.
> > >
> > > IMHO, this module name is not user-friendly. It may make users
> confused.
> > > Because it looks like that it's about integrating with MapReduce(
> > although
> > > I know it referenced parquet-mr[1] project).
> > >
> > > Based on the purpose of this module (contains InputFormat
> implementations
> > > for ReadOptimized, Incremental, Realtime views).
> > >
> > > I suggest that we can rename it to "*hudi-query-common*". Then, we can
> > also
> > > rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
> > >
> > > What do you think?
> > >
> > > Any thoughts and suggestions are welcome and appreciated.
> > >
> > > Best,
> > > Vino
> > >
> > > [1]:
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_parquet-2Dmr&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=dmZJgDEuo5sZCNsoyMRQUpiJoBP7u4r2i8cdHDMmQic&s=4CnBhu54QxDqAWdCb3NXUdQg9beV2xEmgx-N0yhTr9Y&e=
> > >
> >
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by Vinoth Chandar <vi...@apache.org>.
First let me share the context for the existing name.. We saw how Parquet
hands out the InputFormat and named it similar to parquet-mr.
InputFormat is indeed a MapReduce class.. I know we live in the age of
Flink and Spark.. But its true :)
I think this is the crux of the "understandability" issue..
Here are my thoughts..
- +0 (neutral) on the rename to hudi-query-common., (whatever we decide,
we need to rename the bundle accordingly)
- On hudi-query-bundle being confusing with hive/spark/presto bundles, I
don't feel its more confusing than it is today
Real issue IMO, is hudi-hive, which is really about syncing to hive, not
querying Hive.
Then, may be we can rename
- hudi-hadoop-mr to hudi-hive (more understandable, Hive does use
InputFormat as the abstraction)
- current hudi-hive to hudi-hive-sync
(bundles renamed accordingly)
I know this hijacks the conversation. Apologize :). But thought I'd present
a broader take
On Thu, Jan 16, 2020 at 11:26 AM Bhavani Sudha Saktheeswaran
<bh...@uber.com.invalid> wrote:
> +1 to generally renaming the packages. Since this is about renaming for the
> purpose of making it user friendly, I am concerned if we make this as
> hudi-query-bundle, users might get confused with other modules like
> hudi-hive and hudi-spark. And inside packaging module, we further have
> bundles specific to spark, hive and presto.
>
> Any suggestions on how to rename broadly to avoid these confusions? Let me
> also think and get back.
>
> Thanks,
> Sudha
>
> On Wed, Jan 15, 2020 at 9:56 PM vino yang <ya...@gmail.com> wrote:
>
> > Hi guys,
> >
> > I want to start a proposal about refactoring the naming of the
> > "hudi-hadoop-mr" module.
> >
> > IMHO, this module name is not user-friendly. It may make users confused.
> > Because it looks like that it's about integrating with MapReduce(
> although
> > I know it referenced parquet-mr[1] project).
> >
> > Based on the purpose of this module (contains InputFormat implementations
> > for ReadOptimized, Incremental, Realtime views).
> >
> > I suggest that we can rename it to "*hudi-query-common*". Then, we can
> also
> > rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
> >
> > What do you think?
> >
> > Any thoughts and suggestions are welcome and appreciated.
> >
> > Best,
> > Vino
> >
> > [1]:
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_parquet-2Dmr&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=dmZJgDEuo5sZCNsoyMRQUpiJoBP7u4r2i8cdHDMmQic&s=4CnBhu54QxDqAWdCb3NXUdQg9beV2xEmgx-N0yhTr9Y&e=
> >
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by Bhavani Sudha Saktheeswaran <bh...@uber.com.INVALID>.
+1 to generally renaming the packages. Since this is about renaming for the
purpose of making it user friendly, I am concerned if we make this as
hudi-query-bundle, users might get confused with other modules like
hudi-hive and hudi-spark. And inside packaging module, we further have
bundles specific to spark, hive and presto.
Any suggestions on how to rename broadly to avoid these confusions? Let me
also think and get back.
Thanks,
Sudha
On Wed, Jan 15, 2020 at 9:56 PM vino yang <ya...@gmail.com> wrote:
> Hi guys,
>
> I want to start a proposal about refactoring the naming of the
> "hudi-hadoop-mr" module.
>
> IMHO, this module name is not user-friendly. It may make users confused.
> Because it looks like that it's about integrating with MapReduce( although
> I know it referenced parquet-mr[1] project).
>
> Based on the purpose of this module (contains InputFormat implementations
> for ReadOptimized, Incremental, Realtime views).
>
> I suggest that we can rename it to "*hudi-query-common*". Then, we can also
> rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
>
> What do you think?
>
> Any thoughts and suggestions are welcome and appreciated.
>
> Best,
> Vino
>
> [1]:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_parquet-2Dmr&d=DwIBaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=oyPDRKU5b-LuEWWyf8gacx4mFFydIGdyS50OKdxizX0&m=dmZJgDEuo5sZCNsoyMRQUpiJoBP7u4r2i8cdHDMmQic&s=4CnBhu54QxDqAWdCb3NXUdQg9beV2xEmgx-N0yhTr9Y&e=
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by Pratyaksh Sharma <pr...@gmail.com>.
Hi Vino,
I understand your point. I was just putting forward my opinion as I was
able to relate to previous module name i.e hudi-hadoop-mr as well. :)
On Thu, Jan 16, 2020 at 4:31 PM vino yang <ya...@gmail.com> wrote:
> Hi Pratyaksh,
>
> It would make the module name more understandable (and may not be
> confusing).
>
> Best,
> Vino
>
> Pratyaksh Sharma <pr...@gmail.com> 于2020年1月16日周四 下午5:47写道：
>
> > Hi,
> >
> > It does not make much of a difference IMO.
> >
> > On Thu, Jan 16, 2020 at 12:20 PM leesf <le...@gmail.com> wrote:
> >
> > > Thanks vino for bringing up the discussion.
> > >
> > > +1 to rename the module name as the name is not very suitable, and
> > > hudi-query-common/hudi-query-bundle sounds resonable.
> > >
> > > Best,
> > > Leesf
> > >
> > > vino yang <ya...@gmail.com> 于2020年1月16日周四 下午1:56写道：
> > >
> > > > Hi guys,
> > > >
> > > > I want to start a proposal about refactoring the naming of the
> > > > "hudi-hadoop-mr" module.
> > > >
> > > > IMHO, this module name is not user-friendly. It may make users
> > confused.
> > > > Because it looks like that it's about integrating with MapReduce(
> > > although
> > > > I know it referenced parquet-mr[1] project).
> > > >
> > > > Based on the purpose of this module (contains InputFormat
> > implementations
> > > > for ReadOptimized, Incremental, Realtime views).
> > > >
> > > > I suggest that we can rename it to "*hudi-query-common*". Then, we
> can
> > > also
> > > > rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
> > > >
> > > > What do you think?
> > > >
> > > > Any thoughts and suggestions are welcome and appreciated.
> > > >
> > > > Best,
> > > > Vino
> > > >
> > > > [1]: https://github.com/apache/parquet-mr
> > > >
> > >
> >
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by vino yang <ya...@gmail.com>.
Hi Pratyaksh,
It would make the module name more understandable (and may not be
confusing).
Best,
Vino
Pratyaksh Sharma <pr...@gmail.com> 于2020年1月16日周四 下午5:47写道：
> Hi,
>
> It does not make much of a difference IMO.
>
> On Thu, Jan 16, 2020 at 12:20 PM leesf <le...@gmail.com> wrote:
>
> > Thanks vino for bringing up the discussion.
> >
> > +1 to rename the module name as the name is not very suitable, and
> > hudi-query-common/hudi-query-bundle sounds resonable.
> >
> > Best,
> > Leesf
> >
> > vino yang <ya...@gmail.com> 于2020年1月16日周四 下午1:56写道：
> >
> > > Hi guys,
> > >
> > > I want to start a proposal about refactoring the naming of the
> > > "hudi-hadoop-mr" module.
> > >
> > > IMHO, this module name is not user-friendly. It may make users
> confused.
> > > Because it looks like that it's about integrating with MapReduce(
> > although
> > > I know it referenced parquet-mr[1] project).
> > >
> > > Based on the purpose of this module (contains InputFormat
> implementations
> > > for ReadOptimized, Incremental, Realtime views).
> > >
> > > I suggest that we can rename it to "*hudi-query-common*". Then, we can
> > also
> > > rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
> > >
> > > What do you think?
> > >
> > > Any thoughts and suggestions are welcome and appreciated.
> > >
> > > Best,
> > > Vino
> > >
> > > [1]: https://github.com/apache/parquet-mr
> > >
> >
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by Pratyaksh Sharma <pr...@gmail.com>.
Hi,
It does not make much of a difference IMO.
On Thu, Jan 16, 2020 at 12:20 PM leesf <le...@gmail.com> wrote:
> Thanks vino for bringing up the discussion.
>
> +1 to rename the module name as the name is not very suitable, and
> hudi-query-common/hudi-query-bundle sounds resonable.
>
> Best,
> Leesf
>
> vino yang <ya...@gmail.com> 于2020年1月16日周四 下午1:56写道：
>
> > Hi guys,
> >
> > I want to start a proposal about refactoring the naming of the
> > "hudi-hadoop-mr" module.
> >
> > IMHO, this module name is not user-friendly. It may make users confused.
> > Because it looks like that it's about integrating with MapReduce(
> although
> > I know it referenced parquet-mr[1] project).
> >
> > Based on the purpose of this module (contains InputFormat implementations
> > for ReadOptimized, Incremental, Realtime views).
> >
> > I suggest that we can rename it to "*hudi-query-common*". Then, we can
> also
> > rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
> >
> > What do you think?
> >
> > Any thoughts and suggestions are welcome and appreciated.
> >
> > Best,
> > Vino
> >
> > [1]: https://github.com/apache/parquet-mr
> >
>
Re: [DISCUSS] Rename the name of hudi-hadoop-mr module
Posted by leesf <le...@gmail.com>.
Thanks vino for bringing up the discussion.
+1 to rename the module name as the name is not very suitable, and
hudi-query-common/hudi-query-bundle sounds resonable.
Best,
Leesf
vino yang <ya...@gmail.com> 于2020年1月16日周四 下午1:56写道：
> Hi guys,
>
> I want to start a proposal about refactoring the naming of the
> "hudi-hadoop-mr" module.
>
> IMHO, this module name is not user-friendly. It may make users confused.
> Because it looks like that it's about integrating with MapReduce( although
> I know it referenced parquet-mr[1] project).
>
> Based on the purpose of this module (contains InputFormat implementations
> for ReadOptimized, Incremental, Realtime views).
>
> I suggest that we can rename it to "*hudi-query-common*". Then, we can also
> rename "hudi-hadoop-mr-bundle" to "*hudi-query-bundle*".
>
> What do you think?
>
> Any thoughts and suggestions are welcome and appreciated.
>
> Best,
> Vino
>
> [1]: https://github.com/apache/parquet-mr
>