You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metamodel.apache.org by Kasper Sørensen <i....@gmail.com> on 2015/11/04 10:57:20 UTC
[DISCUSS] ElasticSearch and MongoDB granularity
Hi everybody,
Wanted to bring up a discussion topic that I feel is important for two
current pull requests regarding supporting ElasticSearch REST API [1] and
MongoDB 3.x API [2]
The topic is about granularity of modules and how that sometimes determines
the amount of separation but also the amount of code reuse we can have.
In the MongoDB case we have two APIs - one which is legacy and deprecated
(but still working and compatible with older MongoDB installations) and a
new API. Both of them can be used with the same dependency, we'll just have
to live with a bunch of deprecation warnings.
In the ElasticSearch (ES) case we have the official ES jar which includes
"Transport" and "Node" client support. In addition ES provides a JSON
protocol which has some benefits - particularly that as a Java client you
have less dependencies and don't need to sync with the server. So for the
REST protocol we now have the PR which uses the library "Jest" that
integrates with ES that way.
Normally I would then say: Create four modules - one for MongoDB 2.x, one
for MongoDB 3.x, one for ES transport+node client and one for ES REST
client.
But the trouble is that there is also a lot of duplicated code then. So I
think we have in total 3 options to go for:
1) Bundle the modules together but make dependencies optional. There is a
slight risk here that e.g. a future MongoDB version will delete the API
which is right now deprecated. That would make our module un-compileable.
But OTOH at that time we might decide to delete it as well ;-)
2) Create a 5th and 6th module: One module for shared ES code and one
module for shared MongoDB code.
3) Don't worry about the duplicated code.
Let's discuss or raise your favourite approaches.
Best regards,
Kasper
[1] https://github.com/apache/metamodel/pull/67
[2] https://github.com/apache/metamodel/pull/68
Re: [DISCUSS] ElasticSearch and MongoDB granularity
Posted by Alberto Rodriguez <ar...@stratio.com>.
Sure! the names were just an example ;)
Alberto Rodriguez
Vía de las dos Castillas, 33, Ática 4, 3ª Planta
28224 Pozuelo de Alarcón, Madrid
Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
<https://twitter.com/StratioBD>*
2015-11-04 15:57 GMT+01:00 Kasper Sørensen <i....@gmail.com>:
> I like that approach too. It's a bit more work than clubbing the modules
> together, but it's the proper thing to do I think.
>
> Small remark on the example names there: I would prefer to rename
> "estransportnode" to "esnative" or something like that. Not that it has
> _anything_ to do with the thing we where discussing ;-)
>
> Anyone else?
>
> 2015-11-04 13:13 GMT+01:00 Alberto Rodriguez <ar...@stratio.com>:
>
> > Hi all,
> >
> > this is a very interesting topic indeed.
> >
> > Having a look at your options Kasper I'd discard the third one "Don't
> worry
> > about the duplicated code", this is not an option for me :)
> >
> > I'd create kind of a "commons" package within the ES and MongoDB module
> and
> > then specific modules for each version of the connector, the structure
> > would like something like this:
> >
> > metamodel
> > .......
> > |-- mongodb
> > |-- commons
> > |-- mongo2
> > |-- mongo3
> > |-- elasticsearch
> > | -- commons
> > | -- estransportnode
> > | -- esrest
> > .....
> >
> > Kind regards,
> >
> >
> > Alberto Rodriguez
> >
> > Vía de las dos Castillas, 33, Ática 4, 3ª Planta
> > 28224 Pozuelo de Alarcón, Madrid
> > Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
> > <https://twitter.com/StratioBD>*
> >
> > 2015-11-04 10:57 GMT+01:00 Kasper Sørensen <
> i.am.kasper.sorensen@gmail.com
> > >:
> >
> > > Hi everybody,
> > >
> > > Wanted to bring up a discussion topic that I feel is important for two
> > > current pull requests regarding supporting ElasticSearch REST API [1]
> and
> > > MongoDB 3.x API [2]
> > >
> > > The topic is about granularity of modules and how that sometimes
> > determines
> > > the amount of separation but also the amount of code reuse we can have.
> > >
> > > In the MongoDB case we have two APIs - one which is legacy and
> deprecated
> > > (but still working and compatible with older MongoDB installations)
> and a
> > > new API. Both of them can be used with the same dependency, we'll just
> > have
> > > to live with a bunch of deprecation warnings.
> > >
> > > In the ElasticSearch (ES) case we have the official ES jar which
> includes
> > > "Transport" and "Node" client support. In addition ES provides a JSON
> > > protocol which has some benefits - particularly that as a Java client
> you
> > > have less dependencies and don't need to sync with the server. So for
> the
> > > REST protocol we now have the PR which uses the library "Jest" that
> > > integrates with ES that way.
> > >
> > > Normally I would then say: Create four modules - one for MongoDB 2.x,
> one
> > > for MongoDB 3.x, one for ES transport+node client and one for ES REST
> > > client.
> > >
> > > But the trouble is that there is also a lot of duplicated code then.
> So I
> > > think we have in total 3 options to go for:
> > >
> > > 1) Bundle the modules together but make dependencies optional. There
> is a
> > > slight risk here that e.g. a future MongoDB version will delete the API
> > > which is right now deprecated. That would make our module
> un-compileable.
> > > But OTOH at that time we might decide to delete it as well ;-)
> > > 2) Create a 5th and 6th module: One module for shared ES code and one
> > > module for shared MongoDB code.
> > > 3) Don't worry about the duplicated code.
> > >
> > > Let's discuss or raise your favourite approaches.
> > >
> > > Best regards,
> > > Kasper
> > >
> > > [1] https://github.com/apache/metamodel/pull/67
> > > [2] https://github.com/apache/metamodel/pull/68
> > >
> >
>
Re: [DISCUSS] ElasticSearch and MongoDB granularity
Posted by Kasper Sørensen <i....@gmail.com>.
I like that approach too. It's a bit more work than clubbing the modules
together, but it's the proper thing to do I think.
Small remark on the example names there: I would prefer to rename
"estransportnode" to "esnative" or something like that. Not that it has
_anything_ to do with the thing we where discussing ;-)
Anyone else?
2015-11-04 13:13 GMT+01:00 Alberto Rodriguez <ar...@stratio.com>:
> Hi all,
>
> this is a very interesting topic indeed.
>
> Having a look at your options Kasper I'd discard the third one "Don't worry
> about the duplicated code", this is not an option for me :)
>
> I'd create kind of a "commons" package within the ES and MongoDB module and
> then specific modules for each version of the connector, the structure
> would like something like this:
>
> metamodel
> .......
> |-- mongodb
> |-- commons
> |-- mongo2
> |-- mongo3
> |-- elasticsearch
> | -- commons
> | -- estransportnode
> | -- esrest
> .....
>
> Kind regards,
>
>
> Alberto Rodriguez
>
> Vía de las dos Castillas, 33, Ática 4, 3ª Planta
> 28224 Pozuelo de Alarcón, Madrid
> Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
> <https://twitter.com/StratioBD>*
>
> 2015-11-04 10:57 GMT+01:00 Kasper Sørensen <i.am.kasper.sorensen@gmail.com
> >:
>
> > Hi everybody,
> >
> > Wanted to bring up a discussion topic that I feel is important for two
> > current pull requests regarding supporting ElasticSearch REST API [1] and
> > MongoDB 3.x API [2]
> >
> > The topic is about granularity of modules and how that sometimes
> determines
> > the amount of separation but also the amount of code reuse we can have.
> >
> > In the MongoDB case we have two APIs - one which is legacy and deprecated
> > (but still working and compatible with older MongoDB installations) and a
> > new API. Both of them can be used with the same dependency, we'll just
> have
> > to live with a bunch of deprecation warnings.
> >
> > In the ElasticSearch (ES) case we have the official ES jar which includes
> > "Transport" and "Node" client support. In addition ES provides a JSON
> > protocol which has some benefits - particularly that as a Java client you
> > have less dependencies and don't need to sync with the server. So for the
> > REST protocol we now have the PR which uses the library "Jest" that
> > integrates with ES that way.
> >
> > Normally I would then say: Create four modules - one for MongoDB 2.x, one
> > for MongoDB 3.x, one for ES transport+node client and one for ES REST
> > client.
> >
> > But the trouble is that there is also a lot of duplicated code then. So I
> > think we have in total 3 options to go for:
> >
> > 1) Bundle the modules together but make dependencies optional. There is a
> > slight risk here that e.g. a future MongoDB version will delete the API
> > which is right now deprecated. That would make our module un-compileable.
> > But OTOH at that time we might decide to delete it as well ;-)
> > 2) Create a 5th and 6th module: One module for shared ES code and one
> > module for shared MongoDB code.
> > 3) Don't worry about the duplicated code.
> >
> > Let's discuss or raise your favourite approaches.
> >
> > Best regards,
> > Kasper
> >
> > [1] https://github.com/apache/metamodel/pull/67
> > [2] https://github.com/apache/metamodel/pull/68
> >
>
Re: [DISCUSS] ElasticSearch and MongoDB granularity
Posted by Alberto Rodriguez <ar...@stratio.com>.
Hi all,
this is a very interesting topic indeed.
Having a look at your options Kasper I'd discard the third one "Don't worry
about the duplicated code", this is not an option for me :)
I'd create kind of a "commons" package within the ES and MongoDB module and
then specific modules for each version of the connector, the structure
would like something like this:
metamodel
.......
|-- mongodb
|-- commons
|-- mongo2
|-- mongo3
|-- elasticsearch
| -- commons
| -- estransportnode
| -- esrest
.....
Kind regards,
Alberto Rodriguez
Vía de las dos Castillas, 33, Ática 4, 3ª Planta
28224 Pozuelo de Alarcón, Madrid
Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
<https://twitter.com/StratioBD>*
2015-11-04 10:57 GMT+01:00 Kasper Sørensen <i....@gmail.com>:
> Hi everybody,
>
> Wanted to bring up a discussion topic that I feel is important for two
> current pull requests regarding supporting ElasticSearch REST API [1] and
> MongoDB 3.x API [2]
>
> The topic is about granularity of modules and how that sometimes determines
> the amount of separation but also the amount of code reuse we can have.
>
> In the MongoDB case we have two APIs - one which is legacy and deprecated
> (but still working and compatible with older MongoDB installations) and a
> new API. Both of them can be used with the same dependency, we'll just have
> to live with a bunch of deprecation warnings.
>
> In the ElasticSearch (ES) case we have the official ES jar which includes
> "Transport" and "Node" client support. In addition ES provides a JSON
> protocol which has some benefits - particularly that as a Java client you
> have less dependencies and don't need to sync with the server. So for the
> REST protocol we now have the PR which uses the library "Jest" that
> integrates with ES that way.
>
> Normally I would then say: Create four modules - one for MongoDB 2.x, one
> for MongoDB 3.x, one for ES transport+node client and one for ES REST
> client.
>
> But the trouble is that there is also a lot of duplicated code then. So I
> think we have in total 3 options to go for:
>
> 1) Bundle the modules together but make dependencies optional. There is a
> slight risk here that e.g. a future MongoDB version will delete the API
> which is right now deprecated. That would make our module un-compileable.
> But OTOH at that time we might decide to delete it as well ;-)
> 2) Create a 5th and 6th module: One module for shared ES code and one
> module for shared MongoDB code.
> 3) Don't worry about the duplicated code.
>
> Let's discuss or raise your favourite approaches.
>
> Best regards,
> Kasper
>
> [1] https://github.com/apache/metamodel/pull/67
> [2] https://github.com/apache/metamodel/pull/68
>