You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by Daniel Ebi <Eb...@fzi.de> on 2021/08/10 11:20:58 UTC
Harmonization of Data Lake Query Result
Hi all,
As part of the refactoring and integration of the Data Lake REST API into the Data Explorer, it has turned out that it makes sense to harmonize the return type of Data Lake queries first. Consequently, I have already drafted two proposals for the alignment of DataResult, GroupedDataResult and PageResult classes.
I've created a new page in Confluence for these drafts, so that we can discuss further ideas (both conceptually and in terms of naming) there in the comments [1].
The page is thematically divided into two parts. First, I have put together a brief overview of the status quo of the return type of Data Lake queries. Then follows a part with the two proposals, which differ only slightly from each other.
It would be great if you leave me feedback and your ideas for adjustments there in the comments, so that we can discuss the ideas together before implementation.
Thanks and regards,
Daniel
[1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Harmonization+of+Data+Lake+Query+Result
Re: Harmonization of Data Lake Query Result
Posted by Johannes Tex <te...@apache.org>.
Hi Philipp,
that's fine for me. Git never forgets ;)
I recommend rebasing the branch with dev before merging with dev. Then you should be able to merge your branch directly with dev without merge conflicts.
Johannes
On 2021/08/12 16:42:15, Philipp Zehnder <ze...@apache.org> wrote:
> Hi Johannes,
>
> yes you are right, this is used in [1].
> Currently we mostly work in branch [2] and there class was removed because it was not referenced anymore.
> If we still need it, we can recreate the class. What do you think?
>
> In branch [2] we applied many changes, especially Dominik did a lot of refactoring.
> We have tested the code in the branch, but it is possible that there are still some problems.
> Therefore, we would like to merge it into dev as soon as possible to test everything.
>
> Are there any reasons why we should wait with the merge or can fix the resulting bugs directly in dev?
>
> Philipp
>
> [2] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-319
>
>
> > On 10. Aug 2021, at 20:08, Johannes Tex <te...@apache.org> wrote:
> >
> > Hi Daniel,
> >
> > the attributes "labels" and "pageSum", which you call obsolete in your proposals , are used in [1]. Maybe that is also obsolete ;)
> >
> > Johnnaes
> >
> > [1] https://github.com/apache/incubator-streampipes/blob/master/ui/src/app/core-ui/image/image-categorize/image-categorize.component.ts
> >
> > On 2021/08/10 11:20:58, Daniel Ebi <Eb...@fzi.de> wrote:
> >> Hi all,
> >>
> >> As part of the refactoring and integration of the Data Lake REST API into the Data Explorer, it has turned out that it makes sense to harmonize the return type of Data Lake queries first. Consequently, I have already drafted two proposals for the alignment of DataResult, GroupedDataResult and PageResult classes.
> >> I've created a new page in Confluence for these drafts, so that we can discuss further ideas (both conceptually and in terms of naming) there in the comments [1].
> >>
> >> The page is thematically divided into two parts. First, I have put together a brief overview of the status quo of the return type of Data Lake queries. Then follows a part with the two proposals, which differ only slightly from each other.
> >>
> >> It would be great if you leave me feedback and your ideas for adjustments there in the comments, so that we can discuss the ideas together before implementation.
> >>
> >> Thanks and regards,
> >> Daniel
> >>
> >>
> >> [1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Harmonization+of+Data+Lake+Query+Result
> >>
> >>
>
>
Re: Harmonization of Data Lake Query Result
Posted by Dominik Riemer <ri...@apache.org>.
Hi Philipp,
+1 for merging the branch into dev very soon!
Dominik
On 2021/08/12 16:42:15, Philipp Zehnder <ze...@apache.org> wrote:
> Hi Johannes,
>
> yes you are right, this is used in [1].
> Currently we mostly work in branch [2] and there class was removed because it was not referenced anymore.
> If we still need it, we can recreate the class. What do you think?
>
> In branch [2] we applied many changes, especially Dominik did a lot of refactoring.
> We have tested the code in the branch, but it is possible that there are still some problems.
> Therefore, we would like to merge it into dev as soon as possible to test everything.
>
> Are there any reasons why we should wait with the merge or can fix the resulting bugs directly in dev?
>
> Philipp
>
> [2] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-319
>
>
> > On 10. Aug 2021, at 20:08, Johannes Tex <te...@apache.org> wrote:
> >
> > Hi Daniel,
> >
> > the attributes "labels" and "pageSum", which you call obsolete in your proposals , are used in [1]. Maybe that is also obsolete ;)
> >
> > Johnnaes
> >
> > [1] https://github.com/apache/incubator-streampipes/blob/master/ui/src/app/core-ui/image/image-categorize/image-categorize.component.ts
> >
> > On 2021/08/10 11:20:58, Daniel Ebi <Eb...@fzi.de> wrote:
> >> Hi all,
> >>
> >> As part of the refactoring and integration of the Data Lake REST API into the Data Explorer, it has turned out that it makes sense to harmonize the return type of Data Lake queries first. Consequently, I have already drafted two proposals for the alignment of DataResult, GroupedDataResult and PageResult classes.
> >> I've created a new page in Confluence for these drafts, so that we can discuss further ideas (both conceptually and in terms of naming) there in the comments [1].
> >>
> >> The page is thematically divided into two parts. First, I have put together a brief overview of the status quo of the return type of Data Lake queries. Then follows a part with the two proposals, which differ only slightly from each other.
> >>
> >> It would be great if you leave me feedback and your ideas for adjustments there in the comments, so that we can discuss the ideas together before implementation.
> >>
> >> Thanks and regards,
> >> Daniel
> >>
> >>
> >> [1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Harmonization+of+Data+Lake+Query+Result
> >>
> >>
>
>
Re: Harmonization of Data Lake Query Result
Posted by Philipp Zehnder <ze...@apache.org>.
Hi Johannes,
yes you are right, this is used in [1].
Currently we mostly work in branch [2] and there class was removed because it was not referenced anymore.
If we still need it, we can recreate the class. What do you think?
In branch [2] we applied many changes, especially Dominik did a lot of refactoring.
We have tested the code in the branch, but it is possible that there are still some problems.
Therefore, we would like to merge it into dev as soon as possible to test everything.
Are there any reasons why we should wait with the merge or can fix the resulting bugs directly in dev?
Philipp
[2] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-319
> On 10. Aug 2021, at 20:08, Johannes Tex <te...@apache.org> wrote:
>
> Hi Daniel,
>
> the attributes "labels" and "pageSum", which you call obsolete in your proposals , are used in [1]. Maybe that is also obsolete ;)
>
> Johnnaes
>
> [1] https://github.com/apache/incubator-streampipes/blob/master/ui/src/app/core-ui/image/image-categorize/image-categorize.component.ts
>
> On 2021/08/10 11:20:58, Daniel Ebi <Eb...@fzi.de> wrote:
>> Hi all,
>>
>> As part of the refactoring and integration of the Data Lake REST API into the Data Explorer, it has turned out that it makes sense to harmonize the return type of Data Lake queries first. Consequently, I have already drafted two proposals for the alignment of DataResult, GroupedDataResult and PageResult classes.
>> I've created a new page in Confluence for these drafts, so that we can discuss further ideas (both conceptually and in terms of naming) there in the comments [1].
>>
>> The page is thematically divided into two parts. First, I have put together a brief overview of the status quo of the return type of Data Lake queries. Then follows a part with the two proposals, which differ only slightly from each other.
>>
>> It would be great if you leave me feedback and your ideas for adjustments there in the comments, so that we can discuss the ideas together before implementation.
>>
>> Thanks and regards,
>> Daniel
>>
>>
>> [1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Harmonization+of+Data+Lake+Query+Result
>>
>>
AW: Harmonization of Data Lake Query Result
Posted by Daniel Ebi <Eb...@fzi.de>.
Hi Johannes,
thank you for your hint. I simply looked in a branch that does not contain the image categorize component [1]. Sorry for this misinformation. I will correct this on the Confluence page.
I will also have a look at the component. And of course, the properties "labels" and "pageSum" are not obsolete in this context.
Regards,
Daniel
[1] https://github.com/apache/incubator-streampipes/tree/STREAMPIPES-319/ui/src/app/core-ui/image
-----Ursprüngliche Nachricht-----
Von: Johannes Tex <te...@apache.org>
Gesendet: Dienstag, 10. August 2021 20:09
An: dev@streampipes.apache.org
Betreff: Re: Harmonization of Data Lake Query Result
Hi Daniel,
the attributes "labels" and "pageSum", which you call obsolete in your proposals , are used in [1]. Maybe that is also obsolete ;)
Johnnaes
[1] https://github.com/apache/incubator-streampipes/blob/master/ui/src/app/core-ui/image/image-categorize/image-categorize.component.ts
On 2021/08/10 11:20:58, Daniel Ebi <Eb...@fzi.de> wrote:
> Hi all,
>
> As part of the refactoring and integration of the Data Lake REST API into the Data Explorer, it has turned out that it makes sense to harmonize the return type of Data Lake queries first. Consequently, I have already drafted two proposals for the alignment of DataResult, GroupedDataResult and PageResult classes.
> I've created a new page in Confluence for these drafts, so that we can discuss further ideas (both conceptually and in terms of naming) there in the comments [1].
>
> The page is thematically divided into two parts. First, I have put together a brief overview of the status quo of the return type of Data Lake queries. Then follows a part with the two proposals, which differ only slightly from each other.
>
> It would be great if you leave me feedback and your ideas for adjustments there in the comments, so that we can discuss the ideas together before implementation.
>
> Thanks and regards,
> Daniel
>
>
> [1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Harmonization+of+Data+Lake+Query+Result
>
>
Re: Harmonization of Data Lake Query Result
Posted by Johannes Tex <te...@apache.org>.
Hi Daniel,
the attributes "labels" and "pageSum", which you call obsolete in your proposals , are used in [1]. Maybe that is also obsolete ;)
Johnnaes
[1] https://github.com/apache/incubator-streampipes/blob/master/ui/src/app/core-ui/image/image-categorize/image-categorize.component.ts
On 2021/08/10 11:20:58, Daniel Ebi <Eb...@fzi.de> wrote:
> Hi all,
>
> As part of the refactoring and integration of the Data Lake REST API into the Data Explorer, it has turned out that it makes sense to harmonize the return type of Data Lake queries first. Consequently, I have already drafted two proposals for the alignment of DataResult, GroupedDataResult and PageResult classes.
> I've created a new page in Confluence for these drafts, so that we can discuss further ideas (both conceptually and in terms of naming) there in the comments [1].
>
> The page is thematically divided into two parts. First, I have put together a brief overview of the status quo of the return type of Data Lake queries. Then follows a part with the two proposals, which differ only slightly from each other.
>
> It would be great if you leave me feedback and your ideas for adjustments there in the comments, so that we can discuss the ideas together before implementation.
>
> Thanks and regards,
> Daniel
>
>
> [1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Harmonization+of+Data+Lake+Query+Result
>
>