You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Jason Jeong <cu...@gmail.com> on 2016/07/14 00:32:14 UTC
IMRU Map codec configuration is not passed?
Hi,
Find Usages on
`IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration` gives
me nothing, implying that the map input codec is not being passed to Map
and Update Tasks.
Is this a bug, or am I making some obvious mistake?
Thanks,
Jason
Re: IMRU Map codec configuration is not passed?
Posted by Jason Jeong <cu...@gmail.com>.
Thanks Dhruv, for looking in to it.
Glad to here the fix doesn't involve anything too complicated.
Jason
On Wed, Jul 13, 2016 at 6:40 PM, Dhruv Mahajan <dh...@gmail.com>
wrote:
> Jason
>
> i looked at the code. Here is what happened:
>
> During one of Pull requests, I decided to use only one codec configuration
> for both map and update tasks. However, I forgot to change
> IMRUJobDefinition so that instead of two functions
> (SetMapInputCodecConfiguration and SetMapFunctionCodecConfiguration) we
> simply have one SetCodecConfiguration. So this is sort of a bug which does
> not impact the IMRU run the reason being that
> UpdateFunctionCodeConfiguration is subset of MapFunctionCodecs.
>
> This needs to be fixed. Markus/Julia do we need to go through deprecation
> process for this?
>
> Dhruv
>
> On Wed, Jul 13, 2016 at 5:32 PM, Jason Jeong <cu...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Find Usages on
> > `IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration`
> gives
> > me nothing, implying that the map input codec is not being passed to Map
> > and Update Tasks.
> > Is this a bug, or am I making some obvious mistake?
> >
> > Thanks,
> > Jason
> >
>
Re: IMRU Map codec configuration is not passed?
Posted by Jason Jeong <cu...@gmail.com>.
Sure, I'll gladly file a JIRA.
Thanks,
Jason
On Fri, Jul 15, 2016 at 11:55 AM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com> wrote:
> This is basically code cleanup and deprecation thing. In the current
> usage, it is used in both IMRU client and tests, we set the same values to
> both, if one of them is used, the result is the same.
>
> Making the change is simple. My concern is the conflict with the IMRU
> fault tolerant work. If you change now, later I have to do a lot of manual
> merge as IMRU fault tolerant change is huge. Can we open a Jira and
> postpone the work until we merge REEF-1251?
>
> Thanks,
> Julia
>
> -----Original Message-----
> From: Jason Jeong [mailto:culex496782@gmail.com]
> Sent: Friday, July 15, 2016 11:33 AM
> To: dev@reef.apache.org
> Subject: Re: IMRU Map codec configuration is not passed?
>
> The IMRUJobDefinitionBuilder class includes the method
> SetMapInputCodecConfiguration(), which is 1) public and 2) included in
> release 0.15.
> The problem is, deprecation implies that the functionality is still
> active, but currently
> IMRUJobDefinitionBuilder.SetMapInputCodecConfiguration() does nothing; as
> Dhruv said, this is a bug.
> I think the most quickest way to solve this would be to remove
> SetMapInputCodecConfiguration() and rename
> SetUpdateFunctionCodecConfiguration() to SetCodecConfiguration(), but if
> deleting a public interface is more problematic then I guess we deprecate
> both SetMapInputCodecConfiguration() and SetUpdateInputCodecConfiguration(),
> and add a new SetCodecConfiguration() method.
>
> Jason
>
> On Thu, Jul 14, 2016 at 10:54 AM, Mariia Mykhailova <
> mamykhai@microsoft.com>
> wrote:
>
> > > This needs to be fixed. Markus/Julia do we need to go through
> > deprecation process for this?
> >
> > We need deprecation process if 1) we delete a publically accessible
> > interface, and 2) it's been available in release 0.15 or earlier (so
> > it's part of our contract).
> > IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration
> > itself is internal, so can be deleted safely. For other functions you
> > want to modify, we might have to go through deprecation.
> >
> > -Mariia
> >
> > -----Original Message-----
> > From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> > Sent: Wednesday, July 13, 2016 6:41 PM
> > To: dev@reef.apache.org
> > Subject: Re: IMRU Map codec configuration is not passed?
> >
> > Jason
> >
> > i looked at the code. Here is what happened:
> >
> > During one of Pull requests, I decided to use only one codec
> > configuration for both map and update tasks. However, I forgot to
> > change IMRUJobDefinition so that instead of two functions
> > (SetMapInputCodecConfiguration and SetMapFunctionCodecConfiguration)
> > we simply have one SetCodecConfiguration. So this is sort of a bug
> > which does not impact the IMRU run the reason being that
> > UpdateFunctionCodeConfiguration is subset of MapFunctionCodecs.
> >
> > This needs to be fixed. Markus/Julia do we need to go through
> > deprecation process for this?
> >
> > Dhruv
> >
> > On Wed, Jul 13, 2016 at 5:32 PM, Jason Jeong <cu...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > Find Usages on
> > > `IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration`
> > > gives me nothing, implying that the map input codec is not being
> > > passed to Map and Update Tasks.
> > > Is this a bug, or am I making some obvious mistake?
> > >
> > > Thanks,
> > > Jason
> > >
> >
>
RE: IMRU Map codec configuration is not passed?
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com>.
This is basically code cleanup and deprecation thing. In the current usage, it is used in both IMRU client and tests, we set the same values to both, if one of them is used, the result is the same.
Making the change is simple. My concern is the conflict with the IMRU fault tolerant work. If you change now, later I have to do a lot of manual merge as IMRU fault tolerant change is huge. Can we open a Jira and postpone the work until we merge REEF-1251?
Thanks,
Julia
-----Original Message-----
From: Jason Jeong [mailto:culex496782@gmail.com]
Sent: Friday, July 15, 2016 11:33 AM
To: dev@reef.apache.org
Subject: Re: IMRU Map codec configuration is not passed?
The IMRUJobDefinitionBuilder class includes the method SetMapInputCodecConfiguration(), which is 1) public and 2) included in release 0.15.
The problem is, deprecation implies that the functionality is still active, but currently IMRUJobDefinitionBuilder.SetMapInputCodecConfiguration() does nothing; as Dhruv said, this is a bug.
I think the most quickest way to solve this would be to remove
SetMapInputCodecConfiguration() and rename
SetUpdateFunctionCodecConfiguration() to SetCodecConfiguration(), but if deleting a public interface is more problematic then I guess we deprecate both SetMapInputCodecConfiguration() and SetUpdateInputCodecConfiguration(),
and add a new SetCodecConfiguration() method.
Jason
On Thu, Jul 14, 2016 at 10:54 AM, Mariia Mykhailova <ma...@microsoft.com>
wrote:
> > This needs to be fixed. Markus/Julia do we need to go through
> deprecation process for this?
>
> We need deprecation process if 1) we delete a publically accessible
> interface, and 2) it's been available in release 0.15 or earlier (so
> it's part of our contract).
> IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration
> itself is internal, so can be deleted safely. For other functions you
> want to modify, we might have to go through deprecation.
>
> -Mariia
>
> -----Original Message-----
> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> Sent: Wednesday, July 13, 2016 6:41 PM
> To: dev@reef.apache.org
> Subject: Re: IMRU Map codec configuration is not passed?
>
> Jason
>
> i looked at the code. Here is what happened:
>
> During one of Pull requests, I decided to use only one codec
> configuration for both map and update tasks. However, I forgot to
> change IMRUJobDefinition so that instead of two functions
> (SetMapInputCodecConfiguration and SetMapFunctionCodecConfiguration)
> we simply have one SetCodecConfiguration. So this is sort of a bug
> which does not impact the IMRU run the reason being that
> UpdateFunctionCodeConfiguration is subset of MapFunctionCodecs.
>
> This needs to be fixed. Markus/Julia do we need to go through
> deprecation process for this?
>
> Dhruv
>
> On Wed, Jul 13, 2016 at 5:32 PM, Jason Jeong <cu...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Find Usages on
> > `IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration`
> > gives me nothing, implying that the map input codec is not being
> > passed to Map and Update Tasks.
> > Is this a bug, or am I making some obvious mistake?
> >
> > Thanks,
> > Jason
> >
>
Re: IMRU Map codec configuration is not passed?
Posted by Jason Jeong <cu...@gmail.com>.
The IMRUJobDefinitionBuilder class includes the method
SetMapInputCodecConfiguration(), which is 1) public and 2) included in
release 0.15.
The problem is, deprecation implies that the functionality is still active,
but currently IMRUJobDefinitionBuilder.SetMapInputCodecConfiguration() does
nothing; as Dhruv said, this is a bug.
I think the most quickest way to solve this would be to remove
SetMapInputCodecConfiguration() and rename
SetUpdateFunctionCodecConfiguration() to SetCodecConfiguration(), but if
deleting a public interface is more problematic then I guess we deprecate
both SetMapInputCodecConfiguration() and SetUpdateInputCodecConfiguration(),
and add a new SetCodecConfiguration() method.
Jason
On Thu, Jul 14, 2016 at 10:54 AM, Mariia Mykhailova <ma...@microsoft.com>
wrote:
> > This needs to be fixed. Markus/Julia do we need to go through
> deprecation process for this?
>
> We need deprecation process if 1) we delete a publically accessible
> interface, and 2) it's been available in release 0.15 or earlier (so it's
> part of our contract).
> IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration itself
> is internal, so can be deleted safely. For other functions you want to
> modify, we might have to go through deprecation.
>
> -Mariia
>
> -----Original Message-----
> From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
> Sent: Wednesday, July 13, 2016 6:41 PM
> To: dev@reef.apache.org
> Subject: Re: IMRU Map codec configuration is not passed?
>
> Jason
>
> i looked at the code. Here is what happened:
>
> During one of Pull requests, I decided to use only one codec configuration
> for both map and update tasks. However, I forgot to change
> IMRUJobDefinition so that instead of two functions
> (SetMapInputCodecConfiguration and SetMapFunctionCodecConfiguration) we
> simply have one SetCodecConfiguration. So this is sort of a bug which does
> not impact the IMRU run the reason being that
> UpdateFunctionCodeConfiguration is subset of MapFunctionCodecs.
>
> This needs to be fixed. Markus/Julia do we need to go through deprecation
> process for this?
>
> Dhruv
>
> On Wed, Jul 13, 2016 at 5:32 PM, Jason Jeong <cu...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Find Usages on
> > `IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration`
> > gives me nothing, implying that the map input codec is not being
> > passed to Map and Update Tasks.
> > Is this a bug, or am I making some obvious mistake?
> >
> > Thanks,
> > Jason
> >
>
RE: IMRU Map codec configuration is not passed?
Posted by Mariia Mykhailova <ma...@microsoft.com>.
> This needs to be fixed. Markus/Julia do we need to go through deprecation process for this?
We need deprecation process if 1) we delete a publically accessible interface, and 2) it's been available in release 0.15 or earlier (so it's part of our contract).
IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration itself is internal, so can be deleted safely. For other functions you want to modify, we might have to go through deprecation.
-Mariia
-----Original Message-----
From: Dhruv Mahajan [mailto:dhruv.mahajan@gmail.com]
Sent: Wednesday, July 13, 2016 6:41 PM
To: dev@reef.apache.org
Subject: Re: IMRU Map codec configuration is not passed?
Jason
i looked at the code. Here is what happened:
During one of Pull requests, I decided to use only one codec configuration for both map and update tasks. However, I forgot to change IMRUJobDefinition so that instead of two functions (SetMapInputCodecConfiguration and SetMapFunctionCodecConfiguration) we simply have one SetCodecConfiguration. So this is sort of a bug which does not impact the IMRU run the reason being that UpdateFunctionCodeConfiguration is subset of MapFunctionCodecs.
This needs to be fixed. Markus/Julia do we need to go through deprecation process for this?
Dhruv
On Wed, Jul 13, 2016 at 5:32 PM, Jason Jeong <cu...@gmail.com> wrote:
> Hi,
>
> Find Usages on
> `IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration`
> gives me nothing, implying that the map input codec is not being
> passed to Map and Update Tasks.
> Is this a bug, or am I making some obvious mistake?
>
> Thanks,
> Jason
>
Re: IMRU Map codec configuration is not passed?
Posted by Dhruv Mahajan <dh...@gmail.com>.
Jason
i looked at the code. Here is what happened:
During one of Pull requests, I decided to use only one codec configuration
for both map and update tasks. However, I forgot to change
IMRUJobDefinition so that instead of two functions
(SetMapInputCodecConfiguration and SetMapFunctionCodecConfiguration) we
simply have one SetCodecConfiguration. So this is sort of a bug which does
not impact the IMRU run the reason being that
UpdateFunctionCodeConfiguration is subset of MapFunctionCodecs.
This needs to be fixed. Markus/Julia do we need to go through deprecation
process for this?
Dhruv
On Wed, Jul 13, 2016 at 5:32 PM, Jason Jeong <cu...@gmail.com> wrote:
> Hi,
>
> Find Usages on
> `IMRU.OnREEF.Driver.ConfigurationManager.MapInputCodecConfiguration` gives
> me nothing, implying that the map input codec is not being passed to Map
> and Update Tasks.
> Is this a bug, or am I making some obvious mistake?
>
> Thanks,
> Jason
>