You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streams.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2014/11/21 22:20:33 UTC
[jira] [Commented] (STREAMS-222) Simple HTTP Webhook Provider
[ https://issues.apache.org/jira/browse/STREAMS-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14221451#comment-14221451 ]
ASF GitHub Bot commented on STREAMS-222:
----------------------------------------
GitHub user steveblackmon opened a pull request:
https://github.com/apache/incubator-streams/pull/141
Streams 222
I implemented this requirement with a familiar technology well-suited for API applications - creating streams-runtime-dropwizard. Also refactored DatasiftPushProvider to run using this provider, eliminating the need for any overrides or other hacks in the implementation.
The runtime contains three endpoints: /streams/webhooks/json, /streams/webhooks/json_new_line, and /streams/webhooks/json_meta which with pass any json documents posted to them to a StreamProvider named GenericWebhookResouce.
Additionally any other StreamProviders with @Path annotation will be added as jersey resources, and registered as perpetual streams which pass documents to queues named for their respective classes. They can pick whatever path they want to bind. Using the yaml file used at dropwizard start-up, any DW/Jersey parameter including port can be set, and additional configuration for the stream can be supplied. DW and typesafe don't play nice together, so this is the recommended way of supplying configuration to this runtime.
https://github.com/w2ogroup/streams-examples/tree/STREAMS-222 modules dropwizard-webhook-console and dropwizard-webhook-elasticsearch are working examples of how to use this runtime.
I think the ability to build and run streams which expose webhooks is a critical feature gap this PR addresses, in a relatively flexible manner. I look forward to your feedback.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/steveblackmon/incubator-streams STREAMS-222
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-streams/pull/141.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #141
----
commit a57c017211a499372fda1a511b6a7db063360f04
Author: sblackmon <sb...@apache.org>
Date: 2014-11-19T18:54:57Z
drop wizard runtime
commit e6112906772aa5909e09f7246005e0caef5afff5
Author: sblackmon <sb...@apache.org>
Date: 2014-11-19T22:26:18Z
Merge remote-tracking branch 'apache/master' into dropwizard
commit dbda9ed7075d0c65f0cd3c1db768dc35e63b22c0
Author: sblackmon <sb...@apache.org>
Date: 2014-11-20T21:28:31Z
/streams/webhooks/* confirmed working
commit dbb69526e7469f6b988e9ed35793616d988102f6
Author: sblackmon <sb...@apache.org>
Date: 2014-11-20T21:30:15Z
refactored DatasiftProvider to Resource
commit 1987427800b0cf90c197342ce522bab4cd72d605
Author: sblackmon <sb...@apache.org>
Date: 2014-11-20T23:18:42Z
useful to have this metadata downstream
commit 069969d7f9496fbadc18a847648eead27396518e
Author: sblackmon <sb...@apache.org>
Date: 2014-11-20T23:19:39Z
useful to have this metadata downstream
commit 6c989cb69ea0e2013d1543e987ce6b7325e30859
Author: sblackmon <sb...@apache.org>
Date: 2014-11-21T19:24:45Z
added tests
decided not to add StreamsResource to core at this time
commit 3d5f291a5f66f78d8728db0baf5ab6103ada42bf
Author: sblackmon <sb...@apache.org>
Date: 2014-11-21T19:26:43Z
more tests
javadoc headers
refactor DatasiftPushProvider for compatibility
commit 149eb23109ce4a77042c58f65ee79eee786ac9f3
Author: sblackmon <sb...@apache.org>
Date: 2014-11-21T19:30:44Z
javadoc
----
> Simple HTTP Webhook Provider
> ----------------------------
>
> Key: STREAMS-222
> URL: https://issues.apache.org/jira/browse/STREAMS-222
> Project: Streams
> Issue Type: New Feature
> Reporter: Steve Blackmon
> Assignee: Steve Blackmon
>
> Add a provider that will bind a specified socket and pass documents POSTed to it to downstream components.
> This provider should be able to pass messages between streams pipelines executing on other hosts in conjunction with STREAMS-221
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)