[jira] [Commented] (STREAMPIPES-582) Redesign the remove duplicates function for adapters

["Dominik Riemer (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STREAMPIPES-582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17639475#comment-17639475 ] 

Dominik Riemer commented on STREAMPIPES-582:
--------------------------------------------

This issue has been migrated to https://github.com/apache/streampipes/issues/761

> Redesign the remove duplicates function for adapters
> ----------------------------------------------------
>
>                 Key: STREAMPIPES-582
>                 URL: https://issues.apache.org/jira/browse/STREAMPIPES-582
>             Project: StreamPipes
>          Issue Type: Improvement
>          Components: Connect
>            Reporter: Philipp Zehnder
>            Priority: Major
>             Fix For: 1.0.0
>
>
> h2. Current functionality
> A user can select the *remove duplicate* option in the start adapter tab and select the *time interval* how long events are stored in cache (class: {{{}org.apache.streampipes.connect.adapter.preprocessing.transform.stream.DuplicateFilterPipelineElement{}}})
> h3. Problems
>  * The state is not persisted when an adapter is restarted
>  * No option to ignore the timestamp field
> h3. New functionality
>  * A user should be able to add the remove duplicate rule with the option to ignore the timestamp field.
>  * A user should be able to restart an adapter without loosing the state of the remove duplicate rule
> h3. Info
>  * To do that we need to extend the API of adapters and add the functionality to execute code when an adapter is stopped
>  * An API to load data (on startup) and persist (on shutdown) for elements is required



--
This message was sent by Atlassian Jira
(v8.20.10#820010)