You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Josh <jo...@gmail.com> on 2017/05/04 16:19:42 UTC

Http sink - Does it make sense?

Hi all,

I'm computing some counts in a Beam job and want to sink the results to a
REST API via HTTP. For example, group incoming elements by a key, count by
key, and then every minute write the count to an API which supports
incremental updates.

Is this something that other people have done with Beam? I was unable to
find any examples of an Http sink online. If I write my own custom sink to
do this, is there anything to be wary of?

Thanks for any advice,
Josh

Re: Http sink - Does it make sense?

Posted by jo...@gmail.com.
Hi Seshadri,

Yes RestIO sounds like what I'm looking for. Thanks for the link! 

Josh

> On 4 May 2017, at 17:27, Seshadri Raghunathan <se...@gmail.com> wrote:
> 
> Hi Josh,
> 
> If you are looking for RestIO, I saw some discussions earlier in the group. This JIRA has more details on it -  https://issues.apache.org/jira/browse/BEAM-1946
> 
> Hope that helps !
> 
> Regards,
> Seshadri
> 
>> On Thu, May 4, 2017 at 9:19 AM, Josh <jo...@gmail.com> wrote:
>> Hi all,
>> 
>> I'm computing some counts in a Beam job and want to sink the results to a REST API via HTTP. For example, group incoming elements by a key, count by key, and then every minute write the count to an API which supports incremental updates.
>> 
>> Is this something that other people have done with Beam? I was unable to find any examples of an Http sink online. If I write my own custom sink to do this, is there anything to be wary of?
>> 
>> Thanks for any advice,
>> Josh
>

Re: Http sink - Does it make sense?

Posted by Seshadri Raghunathan <se...@gmail.com>.
Hi Josh,

If you are looking for RestIO, I saw some discussions earlier in the group.
This JIRA has more details on it -
https://issues.apache.org/jira/browse/BEAM-1946

Hope that helps !

Regards,
Seshadri

On Thu, May 4, 2017 at 9:19 AM, Josh <jo...@gmail.com> wrote:

> Hi all,
>
> I'm computing some counts in a Beam job and want to sink the results to a
> REST API via HTTP. For example, group incoming elements by a key, count by
> key, and then every minute write the count to an API which supports
> incremental updates.
>
> Is this something that other people have done with Beam? I was unable to
> find any examples of an Http sink online. If I write my own custom sink to
> do this, is there anything to be wary of?
>
> Thanks for any advice,
> Josh
>