StreamPipes API

[Dominik Riemer <ri...@apache.org>]

Hi all,

 

first of all, happy new year to everyone!

 

I've started to work on the new StreamPipes API, which we've had on our
roadmap for quite some time now. The API will expose some currently
internally managed features to developers, e.g., receiving historical and
live data, managing pipelines and deploying adapters and pipeline elements
from external applications. 

As a first step, to simplify our base concepts a little bit, I removed the
data source concept from the SDK - in the pipeline element installation and
the SDK we can now directly operate on data streams.

There is an umbrella issue in Jira to track all individual tasks at [1].

 

Feel free to add anything, and I'm also interested in any further features
that should be supported by the API.

 

Dominik

 

[1]
https://issues.apache.org/jira/projects/STREAMPIPES/issues/STREAMPIPES-272

 

 

Re: StreamPipes API

[Dominik Riemer <ri...@apache.org>]

Hi Philipp,

there will be a REST API and an additional Java client that basically wraps the REST API and provides some additional features such as accessing live data from registered streams directly from Kafka.

I'll try to set up a basic documentation in the wiki so that we a better basis for discussing required features.

On 2021/01/07 08:25:44, Philipp Zehnder <ze...@apache.org> wrote: 
> Hi Dominik,
> 
> this looks very cool!
> 
> I was wondering how someone will be able to use the API?
> Will it be a REST API or will we also provide Client Code in Java for example?
> 
> How can users get an API Token?
> 
> Philipp
> 
> > On 2. Jan 2021, at 19:55, Dominik Riemer <ri...@apache.org> wrote:
> > 
> > Hi all,
> > 
> > 
> > 
> > first of all, happy new year to everyone!
> > 
> > 
> > 
> > I've started to work on the new StreamPipes API, which we've had on our
> > roadmap for quite some time now. The API will expose some currently
> > internally managed features to developers, e.g., receiving historical and
> > live data, managing pipelines and deploying adapters and pipeline elements
> > from external applications. 
> > 
> > As a first step, to simplify our base concepts a little bit, I removed the
> > data source concept from the SDK - in the pipeline element installation and
> > the SDK we can now directly operate on data streams.
> > 
> > There is an umbrella issue in Jira to track all individual tasks at [1].
> > 
> > 
> > 
> > Feel free to add anything, and I'm also interested in any further features
> > that should be supported by the API.
> > 
> > 
> > 
> > Dominik
> > 
> > 
> > 
> > [1]
> > https://issues.apache.org/jira/projects/STREAMPIPES/issues/STREAMPIPES-272
> > 
> > 
> > 
> > 
> > 
> 
> 

Re: StreamPipes API

[Philipp Zehnder <ze...@apache.org>]

Hi Dominik,

this looks very cool!

I was wondering how someone will be able to use the API?
Will it be a REST API or will we also provide Client Code in Java for example?

How can users get an API Token?

Philipp

> On 2. Jan 2021, at 19:55, Dominik Riemer <ri...@apache.org> wrote:
> 
> Hi all,
> 
> 
> 
> first of all, happy new year to everyone!
> 
> 
> 
> I've started to work on the new StreamPipes API, which we've had on our
> roadmap for quite some time now. The API will expose some currently
> internally managed features to developers, e.g., receiving historical and
> live data, managing pipelines and deploying adapters and pipeline elements
> from external applications. 
> 
> As a first step, to simplify our base concepts a little bit, I removed the
> data source concept from the SDK - in the pipeline element installation and
> the SDK we can now directly operate on data streams.
> 
> There is an umbrella issue in Jira to track all individual tasks at [1].
> 
> 
> 
> Feel free to add anything, and I'm also interested in any further features
> that should be supported by the API.
> 
> 
> 
> Dominik
> 
> 
> 
> [1]
> https://issues.apache.org/jira/projects/STREAMPIPES/issues/STREAMPIPES-272
> 
> 
> 
> 
>