You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by Felix John <jo...@axantu.com> on 2020/09/22 07:06:21 UTC
Datalake - REST Extension
Hey all,
I am currently working on the REST interface for our datalake. I would like to extend the existing functionality with additional REST calls that provide the user the following metadata information
- number of events
- current retention policy
- etc. ...
as well as the ability to
- change and modify the retention policy
- remove individual indices in the database
What do you think? Have you got any suggestions?
Regards,
Felix
Re: Datalake - REST Extension
Posted by Philipp Zehnder <ze...@fzi.de>.
Hi,
yes, I think a user should be able to set a default value in the data lake sink and then change this value in the configuration.
Philipp
> On 22. Sep 2020, at 12:58, Johannes Tex <te...@apache.org> wrote:
>
> Hey,
>
> IMO this is a good place.
>
> Just a suggestion: Should the retention policy also be part of the datalake sink configuration?
>
> Johannes
>
> On 2020/09/22 08:16:58, Philipp Zehnder <ze...@apache.org> wrote:
>> Hi,
>>
>> this will be very helpful, because currently the user just sees the actual data that is stored in the data explorer and has the possibility to delete everything in the configurations view.
>>
>> My suggestion would be to add this information to the configurations view, then we have one place to manage all stored data.
>>
>> How do you like that?
>>
>> Philipp
>>
>>
>>
>>> On 22. Sep 2020, at 10:07, Johannes Tex <te...@apache.org> wrote:
>>>
>>> Hi Felix,
>>>
>>> I like the idea of displaying the metadata and the ability to change configurations.
>>> Where do you want to display the metadata in the UI?
>>>
>>> Johannes
>>>
>>> On 2020/09/22 07:06:21, Felix John <jo...@axantu.com> wrote:
>>>> Hey all,
>>>>
>>>> I am currently working on the REST interface for our datalake. I would like to extend the existing functionality with additional REST calls that provide the user the following metadata information
>>>>
>>>> - number of events
>>>> - current retention policy
>>>> - etc. ...
>>>>
>>>> as well as the ability to
>>>>
>>>> - change and modify the retention policy
>>>> - remove individual indices in the database
>>>>
>>>> What do you think? Have you got any suggestions?
>>>>
>>>> Regards,
>>>> Felix
>>
>>
Re: Datalake - REST Extension
Posted by Johannes Tex <te...@apache.org>.
Hey,
IMO this is a good place.
Just a suggestion: Should the retention policy also be part of the datalake sink configuration?
Johannes
On 2020/09/22 08:16:58, Philipp Zehnder <ze...@apache.org> wrote:
> Hi,
>
> this will be very helpful, because currently the user just sees the actual data that is stored in the data explorer and has the possibility to delete everything in the configurations view.
>
> My suggestion would be to add this information to the configurations view, then we have one place to manage all stored data.
>
> How do you like that?
>
> Philipp
>
>
>
> > On 22. Sep 2020, at 10:07, Johannes Tex <te...@apache.org> wrote:
> >
> > Hi Felix,
> >
> > I like the idea of displaying the metadata and the ability to change configurations.
> > Where do you want to display the metadata in the UI?
> >
> > Johannes
> >
> > On 2020/09/22 07:06:21, Felix John <jo...@axantu.com> wrote:
> >> Hey all,
> >>
> >> I am currently working on the REST interface for our datalake. I would like to extend the existing functionality with additional REST calls that provide the user the following metadata information
> >>
> >> - number of events
> >> - current retention policy
> >> - etc. ...
> >>
> >> as well as the ability to
> >>
> >> - change and modify the retention policy
> >> - remove individual indices in the database
> >>
> >> What do you think? Have you got any suggestions?
> >>
> >> Regards,
> >> Felix
>
>
Re: Datalake - REST Extension
Posted by Philipp Zehnder <ze...@apache.org>.
Hi,
this will be very helpful, because currently the user just sees the actual data that is stored in the data explorer and has the possibility to delete everything in the configurations view.
My suggestion would be to add this information to the configurations view, then we have one place to manage all stored data.
How do you like that?
Philipp
> On 22. Sep 2020, at 10:07, Johannes Tex <te...@apache.org> wrote:
>
> Hi Felix,
>
> I like the idea of displaying the metadata and the ability to change configurations.
> Where do you want to display the metadata in the UI?
>
> Johannes
>
> On 2020/09/22 07:06:21, Felix John <jo...@axantu.com> wrote:
>> Hey all,
>>
>> I am currently working on the REST interface for our datalake. I would like to extend the existing functionality with additional REST calls that provide the user the following metadata information
>>
>> - number of events
>> - current retention policy
>> - etc. ...
>>
>> as well as the ability to
>>
>> - change and modify the retention policy
>> - remove individual indices in the database
>>
>> What do you think? Have you got any suggestions?
>>
>> Regards,
>> Felix
Re: Datalake - REST Extension
Posted by Johannes Tex <te...@apache.org>.
Hi Felix,
I like the idea of displaying the metadata and the ability to change configurations.
Where do you want to display the metadata in the UI?
Johannes
On 2020/09/22 07:06:21, Felix John <jo...@axantu.com> wrote:
> Hey all,
>
> I am currently working on the REST interface for our datalake. I would like to extend the existing functionality with additional REST calls that provide the user the following metadata information
>
> - number of events
> - current retention policy
> - etc. ...
>
> as well as the ability to
>
> - change and modify the retention policy
> - remove individual indices in the database
>
> What do you think? Have you got any suggestions?
>
> Regards,
> Felix