You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@wayang.apache.org by Dimitris Theodosakis <di...@gmail.com> on 2023/06/02 06:47:56 UTC
Code restructure
Hi *! :)
I have removed GraphChi dep from everywhere and now I am working on the
code restructure.
I'd just like to ask a bit about the task of moving code from */code/* to
the usual maven structure.
I saw there are multiple versions of scala etc and there are some
duplicated packages (1 per version) in some cases that can lead to code
duplication. Also I saw here:
https://github.com/apache/incubator-wayang/issues/283 the discussion about
the old version and the decommissioning. So should I proceed by removing
the old version?
Also, is there any JIRA ticket for this task?
Re: Code restructure
Posted by Dimitris Theodosakis <di...@gmail.com>.
Sure, we can take a look and see if the pattern they propose can be applied
in our case
--Dimitris
On Fri, Jun 2, 2023 at 10:49 AM Alexander Alten <al...@databloom.ai> wrote:
> Good point, dbt has a great strategy:
>
>
> https://stackoverflow.com/questions/13872226/how-to-support-multiple-scala-versions-in-a-library/13880582#13880582
>
> Maybe we can learn from them? Not sure if it would work for the optimizer
> and wayangSubmit(), tbh.
>
> —Alex
>
>
> > On 2. Jun 2023, at 09:45, Dimitris Theodosakis <
> dimitris.theodosakis@gmail.com> wrote:
> >
> > Hi Alex,
> >
> > Thank you for your prompt reply!
> >
> > If we need to support multiple versions, my feeling is that it is
> difficult
> > to do within a single branch, as code duplication may arise due to
> > the modules dependencies. So maybe we need a different strategy to
> support
> > it like 1 branch/version or something similar.
> >
> > Sure, I'll also add a comment in GitHub to discuss further.
> >
> > Regards,
> > Dimitris
> >
> > On Fri, Jun 2, 2023 at 9:57 AM Alexander Alten <alo@databloom.ai
> <ma...@databloom.ai>> wrote:
> >
> >> Hi Dimitris,
> >>
> >> Great, thank you!
> >>
> >> For Scala - we support multiple versions, depends on Spark. Not sure
> >> about if we should deprecate older Scala versions, and if so we’d need
> to
> >> display a message that we do so in upcoming releases. And yes, I’d be in
> >> favor to move /code to a clean maven structure. Just my opinion ;)
> >>
> >> We don’t have a Jira, we work with GutHub issues. Can you please add
> >> comments to the referenced issue? Would be great!
> >>
> >> Thank you,
> >> —Alex
> >>
> >> —
> >> Alexander Alten
> >> PPMC Apache Wayang
> >> aloalt@apache.org
> >>
> >> @apachewayang
> >>
> >>> On 2. Jun 2023, at 08:47, Dimitris Theodosakis <
> >> dimitris.theodosakis@gmail.com> wrote:
> >>>
> >>> Hi *! :)
> >>>
> >>> I have removed GraphChi dep from everywhere and now I am working on the
> >>> code restructure.
> >>>
> >>> I'd just like to ask a bit about the task of moving code from */code/*
> to
> >>> the usual maven structure.
> >>>
> >>> I saw there are multiple versions of scala etc and there are some
> >>> duplicated packages (1 per version) in some cases that can lead to code
> >>> duplication. Also I saw here:
> >>>
> >>
> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/incubator-wayang/issues/283%26source%3Dgmail-imap%26ust%3D1686293319000000%26usg%3DAOvVaw3dYGUtLkaKPXDLyup8XJ-1&source=gmail-imap&ust=1686296758000000&usg=AOvVaw2v5ZKORHQjxX9KLIZzv2AV
> >> the discussion about
> >>> the old version and the decommissioning. So should I proceed by
> removing
> >>> the old version?
> >>>
> >>> Also, is there any JIRA ticket for this task?
> >>
> >>
> >> --
> >> databloom AI, Inc.
> >>
> >> 3401 N. MIAMI AVE. STE 230
> >> 33127 Miami, Florida
> >> United States
> >>
> >>
> >> -- Please consider the environment before printing this
> >> email --
> >>
> >>
> >>
> >> Disclaimer:
> >> The content of this message is confidential. If you
> >> have received it by mistake, please inform us by an email reply and then
> >> delete the message. It is forbidden to copy, forward, or in any way
> reveal
> >> the contents of this message to anyone. The integrity and security of
> this
> >> email cannot be guaranteed over the Internet. Therefore, the sender will
> >> not be held liable for any damage caused by the message.
>
>
> --
> databloom AI, Inc.
>
> 3401 N. MIAMI AVE. STE 230
> 33127 Miami, Florida
> United States
>
>
> -- Please consider the environment before printing this
> email --
>
>
>
> Disclaimer:
> The content of this message is confidential. If you
> have received it by mistake, please inform us by an email reply and then
> delete the message. It is forbidden to copy, forward, or in any way reveal
> the contents of this message to anyone. The integrity and security of this
> email cannot be guaranteed over the Internet. Therefore, the sender will
> not be held liable for any damage caused by the message.
>
>
Re: Code restructure
Posted by Alexander Alten <al...@databloom.ai>.
Good point, dbt has a great strategy:
https://stackoverflow.com/questions/13872226/how-to-support-multiple-scala-versions-in-a-library/13880582#13880582
Maybe we can learn from them? Not sure if it would work for the optimizer and wayangSubmit(), tbh.
—Alex
> On 2. Jun 2023, at 09:45, Dimitris Theodosakis <di...@gmail.com> wrote:
>
> Hi Alex,
>
> Thank you for your prompt reply!
>
> If we need to support multiple versions, my feeling is that it is difficult
> to do within a single branch, as code duplication may arise due to
> the modules dependencies. So maybe we need a different strategy to support
> it like 1 branch/version or something similar.
>
> Sure, I'll also add a comment in GitHub to discuss further.
>
> Regards,
> Dimitris
>
> On Fri, Jun 2, 2023 at 9:57 AM Alexander Alten <alo@databloom.ai <ma...@databloom.ai>> wrote:
>
>> Hi Dimitris,
>>
>> Great, thank you!
>>
>> For Scala - we support multiple versions, depends on Spark. Not sure
>> about if we should deprecate older Scala versions, and if so we’d need to
>> display a message that we do so in upcoming releases. And yes, I’d be in
>> favor to move /code to a clean maven structure. Just my opinion ;)
>>
>> We don’t have a Jira, we work with GutHub issues. Can you please add
>> comments to the referenced issue? Would be great!
>>
>> Thank you,
>> —Alex
>>
>> —
>> Alexander Alten
>> PPMC Apache Wayang
>> aloalt@apache.org
>>
>> @apachewayang
>>
>>> On 2. Jun 2023, at 08:47, Dimitris Theodosakis <
>> dimitris.theodosakis@gmail.com> wrote:
>>>
>>> Hi *! :)
>>>
>>> I have removed GraphChi dep from everywhere and now I am working on the
>>> code restructure.
>>>
>>> I'd just like to ask a bit about the task of moving code from */code/* to
>>> the usual maven structure.
>>>
>>> I saw there are multiple versions of scala etc and there are some
>>> duplicated packages (1 per version) in some cases that can lead to code
>>> duplication. Also I saw here:
>>>
>> https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://github.com/apache/incubator-wayang/issues/283%26source%3Dgmail-imap%26ust%3D1686293319000000%26usg%3DAOvVaw3dYGUtLkaKPXDLyup8XJ-1&source=gmail-imap&ust=1686296758000000&usg=AOvVaw2v5ZKORHQjxX9KLIZzv2AV
>> the discussion about
>>> the old version and the decommissioning. So should I proceed by removing
>>> the old version?
>>>
>>> Also, is there any JIRA ticket for this task?
>>
>>
>> --
>> databloom AI, Inc.
>>
>> 3401 N. MIAMI AVE. STE 230
>> 33127 Miami, Florida
>> United States
>>
>>
>> -- Please consider the environment before printing this
>> email --
>>
>>
>>
>> Disclaimer:
>> The content of this message is confidential. If you
>> have received it by mistake, please inform us by an email reply and then
>> delete the message. It is forbidden to copy, forward, or in any way reveal
>> the contents of this message to anyone. The integrity and security of this
>> email cannot be guaranteed over the Internet. Therefore, the sender will
>> not be held liable for any damage caused by the message.
--
databloom AI, Inc.
3401 N. MIAMI AVE. STE 230
33127 Miami, Florida
United States
-- Please consider the environment before printing this
email --
Disclaimer:
The content of this message is confidential. If you
have received it by mistake, please inform us by an email reply and then
delete the message. It is forbidden to copy, forward, or in any way reveal
the contents of this message to anyone. The integrity and security of this
email cannot be guaranteed over the Internet. Therefore, the sender will
not be held liable for any damage caused by the message.
Re: Code restructure
Posted by Dimitris Theodosakis <di...@gmail.com>.
Hi Alex,
Thank you for your prompt reply!
If we need to support multiple versions, my feeling is that it is difficult
to do within a single branch, as code duplication may arise due to
the modules dependencies. So maybe we need a different strategy to support
it like 1 branch/version or something similar.
Sure, I'll also add a comment in GitHub to discuss further.
Regards,
Dimitris
On Fri, Jun 2, 2023 at 9:57 AM Alexander Alten <al...@databloom.ai> wrote:
> Hi Dimitris,
>
> Great, thank you!
>
> For Scala - we support multiple versions, depends on Spark. Not sure
> about if we should deprecate older Scala versions, and if so we’d need to
> display a message that we do so in upcoming releases. And yes, I’d be in
> favor to move /code to a clean maven structure. Just my opinion ;)
>
> We don’t have a Jira, we work with GutHub issues. Can you please add
> comments to the referenced issue? Would be great!
>
> Thank you,
> —Alex
>
> —
> Alexander Alten
> PPMC Apache Wayang
> aloalt@apache.org
>
> @apachewayang
>
> > On 2. Jun 2023, at 08:47, Dimitris Theodosakis <
> dimitris.theodosakis@gmail.com> wrote:
> >
> > Hi *! :)
> >
> > I have removed GraphChi dep from everywhere and now I am working on the
> > code restructure.
> >
> > I'd just like to ask a bit about the task of moving code from */code/* to
> > the usual maven structure.
> >
> > I saw there are multiple versions of scala etc and there are some
> > duplicated packages (1 per version) in some cases that can lead to code
> > duplication. Also I saw here:
> >
> https://www.google.com/url?q=https://github.com/apache/incubator-wayang/issues/283&source=gmail-imap&ust=1686293319000000&usg=AOvVaw3dYGUtLkaKPXDLyup8XJ-1
> the discussion about
> > the old version and the decommissioning. So should I proceed by removing
> > the old version?
> >
> > Also, is there any JIRA ticket for this task?
>
>
> --
> databloom AI, Inc.
>
> 3401 N. MIAMI AVE. STE 230
> 33127 Miami, Florida
> United States
>
>
> -- Please consider the environment before printing this
> email --
>
>
>
> Disclaimer:
> The content of this message is confidential. If you
> have received it by mistake, please inform us by an email reply and then
> delete the message. It is forbidden to copy, forward, or in any way reveal
> the contents of this message to anyone. The integrity and security of this
> email cannot be guaranteed over the Internet. Therefore, the sender will
> not be held liable for any damage caused by the message.
>
>
Re: Code restructure
Posted by Alexander Alten <al...@databloom.ai>.
Hi Dimitris,
Great, thank you!
For Scala - we support multiple versions, depends on Spark. Not sure about if we should deprecate older Scala versions, and if so we’d need to display a message that we do so in upcoming releases. And yes, I’d be in favor to move /code to a clean maven structure. Just my opinion ;)
We don’t have a Jira, we work with GutHub issues. Can you please add comments to the referenced issue? Would be great!
Thank you,
—Alex
—
Alexander Alten
PPMC Apache Wayang
aloalt@apache.org
@apachewayang
> On 2. Jun 2023, at 08:47, Dimitris Theodosakis <di...@gmail.com> wrote:
>
> Hi *! :)
>
> I have removed GraphChi dep from everywhere and now I am working on the
> code restructure.
>
> I'd just like to ask a bit about the task of moving code from */code/* to
> the usual maven structure.
>
> I saw there are multiple versions of scala etc and there are some
> duplicated packages (1 per version) in some cases that can lead to code
> duplication. Also I saw here:
> https://www.google.com/url?q=https://github.com/apache/incubator-wayang/issues/283&source=gmail-imap&ust=1686293319000000&usg=AOvVaw3dYGUtLkaKPXDLyup8XJ-1 the discussion about
> the old version and the decommissioning. So should I proceed by removing
> the old version?
>
> Also, is there any JIRA ticket for this task?
--
databloom AI, Inc.
3401 N. MIAMI AVE. STE 230
33127 Miami, Florida
United States
-- Please consider the environment before printing this
email --
Disclaimer:
The content of this message is confidential. If you
have received it by mistake, please inform us by an email reply and then
delete the message. It is forbidden to copy, forward, or in any way reveal
the contents of this message to anyone. The integrity and security of this
email cannot be guaranteed over the Internet. Therefore, the sender will
not be held liable for any damage caused by the message.