You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@geode.apache.org by Dan Smith <ds...@pivotal.io> on 2016/03/18 23:56:05 UTC
[DISCUSS] What to do with the partial HDFS related code on develop?
When geode was donated to apache, there was a partially completed
integration with HDFS built in. See GEODE-10.
Since that initial donation, we have removed the APIs related to creating
HDFS regions because they were not yet stable. See GEODE-429.
The problem is that we still have HDFS related code sprinkled throughout
geode-core. This causes us to pull in a bunch of compile dependencies on
hbase and hadoop.
I already created an issue, GEODE-1072, to move the HDFS code it it's own
subproject, but that will take a lot of work.
In the mean time, I propose we go ahead and create a separate branch for
the HDFS related changes and remove the HDFS related code from develop. The
work to refactor the HDFS integration to be a subproject can take place on
that branch and geode-core on develop will be cleaner in the mean time.
What do you folks think?
-Dan
Re: [DISCUSS] What to do with the partial HDFS related code on
develop?
Posted by Udo Kohlmeyer <uk...@pivotal.io>.
+1
On 28/04/2016 7:16 am, Dan Smith wrote:
> This is done. The HDFS related changes are gone from develop and are now on
> their own feature branch - feature/GEODE-10.
>
> -Dan
>
> On Sat, Mar 19, 2016 at 2:05 AM, <ma...@daimler.com> wrote:
>
>> +1
>> a possible HDFS integration should also be an optional jar plugin package
>> in the distribution as Fred started the discussion to split jars anyway.
>>
>> Best Regards
>> Manuel Birke
>>
>> Mercedes-Benz Research & Development North America, Inc.
>> Address: 309 N Pastoria Ave | Sunnyvale, CA 94085
>> Mobile: +1 408-203-6026
>> Phone: +1 408-991-6502
>> ________________________________
>> From: Anilkumar Gingade<ma...@pivotal.io>
>> Sent: 3/19/2016 0:05
>> To: dev@geode.incubator.apache.org<ma...@geode.incubator.apache.org>
>> Subject: Re: [DISCUSS] What to do with the partial HDFS related code on
>> develop?
>>
>> +1
>> Good idea...It will allow us to re-visit the HDFS implementation and right
>> way to provide this support...
>>
>> -Anil.
>>
>>
>>
>>
>>
>>
>> On Fri, Mar 18, 2016 at 3:56 PM, Dan Smith <ds...@pivotal.io> wrote:
>>
>>> When geode was donated to apache, there was a partially completed
>>> integration with HDFS built in. See GEODE-10.
>>>
>>> Since that initial donation, we have removed the APIs related to creating
>>> HDFS regions because they were not yet stable. See GEODE-429.
>>>
>>> The problem is that we still have HDFS related code sprinkled throughout
>>> geode-core. This causes us to pull in a bunch of compile dependencies on
>>> hbase and hadoop.
>>>
>>> I already created an issue, GEODE-1072, to move the HDFS code it it's own
>>> subproject, but that will take a lot of work.
>>>
>>> In the mean time, I propose we go ahead and create a separate branch for
>>> the HDFS related changes and remove the HDFS related code from develop.
>> The
>>> work to refactor the HDFS integration to be a subproject can take place
>> on
>>> that branch and geode-core on develop will be cleaner in the mean time.
>>>
>>> What do you folks think?
>>>
>>> -Dan
>>>
>> If you are not the addressee, please inform us immediately that you have
>> received this e-mail by mistake, and delete it. We thank you for your
>> support.
>>
>>
Re: [DISCUSS] What to do with the partial HDFS related code on develop?
Posted by Gregory Chase <gc...@pivotal.io>.
This is a topic we didn't cover at Geode Summit. We could bring it up at a
Geode Clubhouse if anyone is interested.
On Wed, Apr 27, 2016 at 2:16 PM, Dan Smith <ds...@pivotal.io> wrote:
> This is done. The HDFS related changes are gone from develop and are now on
> their own feature branch - feature/GEODE-10.
>
> -Dan
>
> On Sat, Mar 19, 2016 at 2:05 AM, <ma...@daimler.com> wrote:
>
> > +1
> > a possible HDFS integration should also be an optional jar plugin package
> > in the distribution as Fred started the discussion to split jars anyway.
> >
> > Best Regards
> > Manuel Birke
> >
> > Mercedes-Benz Research & Development North America, Inc.
> > Address: 309 N Pastoria Ave | Sunnyvale, CA 94085
> > Mobile: +1 408-203-6026
> > Phone: +1 408-991-6502
> > ________________________________
> > From: Anilkumar Gingade<ma...@pivotal.io>
> > Sent: 3/19/2016 0:05
> > To: dev@geode.incubator.apache.org<mailto:dev@geode.incubator.apache.org
> >
> > Subject: Re: [DISCUSS] What to do with the partial HDFS related code on
> > develop?
> >
> > +1
> > Good idea...It will allow us to re-visit the HDFS implementation and
> right
> > way to provide this support...
> >
> > -Anil.
> >
> >
> >
> >
> >
> >
> > On Fri, Mar 18, 2016 at 3:56 PM, Dan Smith <ds...@pivotal.io> wrote:
> >
> > > When geode was donated to apache, there was a partially completed
> > > integration with HDFS built in. See GEODE-10.
> > >
> > > Since that initial donation, we have removed the APIs related to
> creating
> > > HDFS regions because they were not yet stable. See GEODE-429.
> > >
> > > The problem is that we still have HDFS related code sprinkled
> throughout
> > > geode-core. This causes us to pull in a bunch of compile dependencies
> on
> > > hbase and hadoop.
> > >
> > > I already created an issue, GEODE-1072, to move the HDFS code it it's
> own
> > > subproject, but that will take a lot of work.
> > >
> > > In the mean time, I propose we go ahead and create a separate branch
> for
> > > the HDFS related changes and remove the HDFS related code from develop.
> > The
> > > work to refactor the HDFS integration to be a subproject can take place
> > on
> > > that branch and geode-core on develop will be cleaner in the mean time.
> > >
> > > What do you folks think?
> > >
> > > -Dan
> > >
> >
> > If you are not the addressee, please inform us immediately that you have
> > received this e-mail by mistake, and delete it. We thank you for your
> > support.
> >
> >
>
--
Greg Chase
Global Head, Big Data Communities
http://www.pivotal.io/big-data
Pivotal Software
http://www.pivotal.io/
650-215-0477
@GregChase
Blog: http://geekmarketing.biz/
Re: [DISCUSS] What to do with the partial HDFS related code on develop?
Posted by Dan Smith <ds...@pivotal.io>.
This is done. The HDFS related changes are gone from develop and are now on
their own feature branch - feature/GEODE-10.
-Dan
On Sat, Mar 19, 2016 at 2:05 AM, <ma...@daimler.com> wrote:
> +1
> a possible HDFS integration should also be an optional jar plugin package
> in the distribution as Fred started the discussion to split jars anyway.
>
> Best Regards
> Manuel Birke
>
> Mercedes-Benz Research & Development North America, Inc.
> Address: 309 N Pastoria Ave | Sunnyvale, CA 94085
> Mobile: +1 408-203-6026
> Phone: +1 408-991-6502
> ________________________________
> From: Anilkumar Gingade<ma...@pivotal.io>
> Sent: 3/19/2016 0:05
> To: dev@geode.incubator.apache.org<ma...@geode.incubator.apache.org>
> Subject: Re: [DISCUSS] What to do with the partial HDFS related code on
> develop?
>
> +1
> Good idea...It will allow us to re-visit the HDFS implementation and right
> way to provide this support...
>
> -Anil.
>
>
>
>
>
>
> On Fri, Mar 18, 2016 at 3:56 PM, Dan Smith <ds...@pivotal.io> wrote:
>
> > When geode was donated to apache, there was a partially completed
> > integration with HDFS built in. See GEODE-10.
> >
> > Since that initial donation, we have removed the APIs related to creating
> > HDFS regions because they were not yet stable. See GEODE-429.
> >
> > The problem is that we still have HDFS related code sprinkled throughout
> > geode-core. This causes us to pull in a bunch of compile dependencies on
> > hbase and hadoop.
> >
> > I already created an issue, GEODE-1072, to move the HDFS code it it's own
> > subproject, but that will take a lot of work.
> >
> > In the mean time, I propose we go ahead and create a separate branch for
> > the HDFS related changes and remove the HDFS related code from develop.
> The
> > work to refactor the HDFS integration to be a subproject can take place
> on
> > that branch and geode-core on develop will be cleaner in the mean time.
> >
> > What do you folks think?
> >
> > -Dan
> >
>
> If you are not the addressee, please inform us immediately that you have
> received this e-mail by mistake, and delete it. We thank you for your
> support.
>
>
RE: [DISCUSS] What to do with the partial HDFS related code on develop?
Posted by ma...@daimler.com.
+1
a possible HDFS integration should also be an optional jar plugin package in the distribution as Fred started the discussion to split jars anyway.
Best Regards
Manuel Birke
Mercedes-Benz Research & Development North America, Inc.
Address: 309 N Pastoria Ave | Sunnyvale, CA 94085
Mobile: +1 408-203-6026
Phone: +1 408-991-6502
________________________________
From: Anilkumar Gingade<ma...@pivotal.io>
Sent: 3/19/2016 0:05
To: dev@geode.incubator.apache.org<ma...@geode.incubator.apache.org>
Subject: Re: [DISCUSS] What to do with the partial HDFS related code on develop?
+1
Good idea...It will allow us to re-visit the HDFS implementation and right
way to provide this support...
-Anil.
On Fri, Mar 18, 2016 at 3:56 PM, Dan Smith <ds...@pivotal.io> wrote:
> When geode was donated to apache, there was a partially completed
> integration with HDFS built in. See GEODE-10.
>
> Since that initial donation, we have removed the APIs related to creating
> HDFS regions because they were not yet stable. See GEODE-429.
>
> The problem is that we still have HDFS related code sprinkled throughout
> geode-core. This causes us to pull in a bunch of compile dependencies on
> hbase and hadoop.
>
> I already created an issue, GEODE-1072, to move the HDFS code it it's own
> subproject, but that will take a lot of work.
>
> In the mean time, I propose we go ahead and create a separate branch for
> the HDFS related changes and remove the HDFS related code from develop. The
> work to refactor the HDFS integration to be a subproject can take place on
> that branch and geode-core on develop will be cleaner in the mean time.
>
> What do you folks think?
>
> -Dan
>
If you are not the addressee, please inform us immediately that you have received this e-mail by mistake, and delete it. We thank you for your support.
Re: [DISCUSS] What to do with the partial HDFS related code on develop?
Posted by Anilkumar Gingade <ag...@pivotal.io>.
+1
Good idea...It will allow us to re-visit the HDFS implementation and right
way to provide this support...
-Anil.
On Fri, Mar 18, 2016 at 3:56 PM, Dan Smith <ds...@pivotal.io> wrote:
> When geode was donated to apache, there was a partially completed
> integration with HDFS built in. See GEODE-10.
>
> Since that initial donation, we have removed the APIs related to creating
> HDFS regions because they were not yet stable. See GEODE-429.
>
> The problem is that we still have HDFS related code sprinkled throughout
> geode-core. This causes us to pull in a bunch of compile dependencies on
> hbase and hadoop.
>
> I already created an issue, GEODE-1072, to move the HDFS code it it's own
> subproject, but that will take a lot of work.
>
> In the mean time, I propose we go ahead and create a separate branch for
> the HDFS related changes and remove the HDFS related code from develop. The
> work to refactor the HDFS integration to be a subproject can take place on
> that branch and geode-core on develop will be cleaner in the mean time.
>
> What do you folks think?
>
> -Dan
>