[GitHub] [streampipes] tenthe edited a discussion: Designing a new Parser API for Adapters

[GitBox <gi...@apache.org>]

GitHub user tenthe edited a discussion: Designing a new Parser API for Adapters

# Design of a new parser API for adapters

In this discussion, we will explore how to design the new parser API.
Adapters that can handle different formats need a parser (e.g. FileReplay, MQTT, Kafka, ...)
In the connect refactoring branch I have implemented a first version which has the following disadvantages:

**Disadvantages of the current API:**
- The adapter has no control over the actual events
- If it is necessary to customize events inside the adapter, the developer has to change the adapter's preprocessing pipeline

## Current API Design
The current design delegates the parsing and publishing of events to the parser.

### Parser Interface
```java
public interface Parser {
...
    void parse(InputStream inputStream, IEventCollector collector) throws ParseException;
...
}
```
### Adapter
```java
public class ActualAdapter implements AdapterInterface {
...
    @Override
    public void onAdapterStarted(IAdapterParameterExtractor extractor, IEventCollector collector,
            IAdapterRuntimeContext adapterRuntimeContext) throws AdapterException {
        InputStream inputStream = // create inputStream from data
        extractor.selectedParser().parse(inputStream, collector); 
    }
...
}
```

## Design 1: Iterator Pattern
One possible improvement is to use the Iterator pattern to give the adapter more control over the actual events.

### Parser Interface
```java
public interface Parser {
...
    void init(InputStream inputStream);
    boolean hasNext();
    Map<String, Object> next();
...
}
```
### Adapter
```java
public class ActualAdapter implements AdapterInterface {
    @Override
    public void onAdapterStarted(IAdapterParameterExtractor extractor, IEventCollector collector,
            IAdapterRuntimeContext adapterRuntimeContext) throws AdapterException {
...
        InputStream inputStream = // create inputStream from data
        var parser = extractor.selectedParser();
        parser.init(inputStream);
        while(parser.hasNext()) {
            try {
                var event = parser.next();
                collector.collect(event);
            } catch (ParseException e) {
                // LOG exception
            }
        }
    }
...
}
```

## Design 2: Callback
Another possible improvement is to use a callback to publish the events.
### Parser Interface
```java
public interface Parser {
...
    void parse(InputStream inputStream, EventHandler eventHandler) throws ParseException;
...
}

public interface EventHandler {
    void handle(Map<String, Object> event);
}
```
### Adapter
```java
public class ActualAdapter implements AdapterInterface {
...
    @Override
    public void onAdapterStarted(IAdapterParameterExtractor extractor, IEventCollector collector,
            IAdapterRuntimeContext adapterRuntimeContext) throws AdapterException {
        InputStream inputStream = // create inputStream from data
        extractor.selectedParser().parse(inputStream, (Map<String, Object> event) -> {
            collector.collect(event);
        });
    }
...
}

```

## Question
- Is it possible for **Design 2** to catch `ParseExceptions`? 
- Do you have an opinion about these designs? Which one do you like best?

GitHub link: https://github.com/apache/streampipes/discussions/1563

----
This is an automatically sent email for dev@streampipes.apache.org.
To unsubscribe, please send an email to: dev-unsubscribe@streampipes.apache.org