You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Brian O'Neill <bo...@alumni.brown.edu> on 2013/12/31 17:10:58 UTC
Signature for DRPCClient.execute
I'm trying to understand the motivation for the String based interface on
the execute method of DRPCClient.
http://nathanmarz.github.io/storm/doc/backtype/storm/utils/DRPCClient.html
String execute (String, String)
I have a topology that typically receives and outputs java objects. In
order for me to call it with DRPC, do I need to layer on a marshaling layer
to translate String->POJO and back?
Would it be possible to change the signature to accept any type of object
and put it in the args field?
-brian
--
Brian ONeill
Chief Architect, Health Market Science (http://healthmarketscience.com)
mobile:215.588.6024
blog: http://brianoneill.blogspot.com/
twitter: @boneill42
Re: Signature for DRPCClient.execute
Posted by Brian O'Neill <bo...@alumni.brown.edu>.
Exactly. (the JSON-encoded list is cumbersome as well)
I¹ll open up a JIRA to see if there is enough motivation for the
enhancement.
-brian
---
Brian O'Neill
Chief Architect
Health Market Science
The Science of Better Results
2700 Horizon Drive King of Prussia, PA 19406
M: 215.588.6024 @boneill42 <http://www.twitter.com/boneill42>
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
From: Jon Logan <jm...@buffalo.edu>
Reply-To: <us...@storm.incubator.apache.org>
Date: Tuesday, December 31, 2013 at 11:42 AM
To: <us...@storm.incubator.apache.org>
Subject: Re: Signature for DRPCClient.execute
I would imagine it's because it's not supported on the DRPC server -- this
isn't just using raw Thrift at this point, it has to be funneled through the
DRPC server (as far as I know).
Also note that it'll wrap your returned object into a JSON-encoded list.
That's a bit annoying as well...
On Tue, Dec 31, 2013 at 11:39 AM, Brian O'Neill <bo...@alumni.brown.edu>
wrote:
>
> Ugh. That seems a bit awkward.
>
> Is there any reason why we wouldn¹t change the signature to accept objects?
> (and (de)serialize under the hood like we do with standard tuples?)
>
> -brian
>
> ---
> Brian O'Neill
> Chief Architect
> Health Market Science
> The Science of Better Results
> 2700 Horizon Drive King of Prussia, PA 19406
> M: 215.588.6024 <tel:215.588.6024> @boneill42
> <http://www.twitter.com/boneill42>
> healthmarketscience.com
>
>
> This information transmitted in this email message is for the intended
> recipient only and may contain confidential and/or privileged material. If you
> received this email in error and are not the intended recipient, or the person
> responsible to deliver it to the intended recipient, please contact the sender
> at the email above and delete this email and any attachments and destroy any
> copies thereof. Any review, retransmission, dissemination, copying or other
> use of, or taking any action in reliance upon, this information by persons or
> entities other than the intended recipient is strictly prohibited.
>
>
>
> From: Michael Ritsema <mi...@gmail.com>
> Reply-To: <us...@storm.incubator.apache.org>
> Date: Tuesday, December 31, 2013 at 11:35 AM
> To: <us...@storm.incubator.apache.org>
> Subject: Re: Signature for DRPCClient.execute
>
> You need to serialize with something like JSON. If you are using a binary
> serialization you'd need a codec like base64 on top of it.
>
>
>
>
> On Tue, Dec 31, 2013 at 10:10 AM, Brian O'Neill <bo...@alumni.brown.edu> wrote:
>>
>> I'm trying to understand the motivation for the String based interface on the
>> execute method of DRPCClient.
>>
>> http://nathanmarz.github.io/storm/doc/backtype/storm/utils/DRPCClient.html
>> String execute (String, String)
>>
>> I have a topology that typically receives and outputs java objects. In order
>> for me to call it with DRPC, do I need to layer on a marshaling layer to
>> translate String->POJO and back?
>>
>> Would it be possible to change the signature to accept any type of object and
>> put it in the args field?
>>
>> -brian
>>
>> --
>> Brian ONeill
>> Chief Architect, Health Market Science (http://healthmarketscience.com)
>> mobile:215.588.6024 <tel:215.588.6024>
>> blog: http://brianoneill.blogspot.com/
>> twitter: @boneill42
>
Re: Signature for DRPCClient.execute
Posted by Brian O'Neill <bo...@alumni.brown.edu>.
It looks like an issue exists already:
https://issues.apache.org/jira/browse/STORM-42
(any interested parties should vote it up)
-brian
---
Brian O'Neill
Chief Architect
Health Market Science
The Science of Better Results
2700 Horizon Drive King of Prussia, PA 19406
M: 215.588.6024 @boneill42 <http://www.twitter.com/boneill42>
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
From: Jon Logan <jm...@buffalo.edu>
Reply-To: <us...@storm.incubator.apache.org>
Date: Tuesday, December 31, 2013 at 11:42 AM
To: <us...@storm.incubator.apache.org>
Subject: Re: Signature for DRPCClient.execute
I would imagine it's because it's not supported on the DRPC server -- this
isn't just using raw Thrift at this point, it has to be funneled through the
DRPC server (as far as I know).
Also note that it'll wrap your returned object into a JSON-encoded list.
That's a bit annoying as well...
On Tue, Dec 31, 2013 at 11:39 AM, Brian O'Neill <bo...@alumni.brown.edu>
wrote:
>
> Ugh. That seems a bit awkward.
>
> Is there any reason why we wouldn¹t change the signature to accept objects?
> (and (de)serialize under the hood like we do with standard tuples?)
>
> -brian
>
> ---
> Brian O'Neill
> Chief Architect
> Health Market Science
> The Science of Better Results
> 2700 Horizon Drive King of Prussia, PA 19406
> M: 215.588.6024 <tel:215.588.6024> @boneill42
> <http://www.twitter.com/boneill42>
> healthmarketscience.com
>
>
> This information transmitted in this email message is for the intended
> recipient only and may contain confidential and/or privileged material. If you
> received this email in error and are not the intended recipient, or the person
> responsible to deliver it to the intended recipient, please contact the sender
> at the email above and delete this email and any attachments and destroy any
> copies thereof. Any review, retransmission, dissemination, copying or other
> use of, or taking any action in reliance upon, this information by persons or
> entities other than the intended recipient is strictly prohibited.
>
>
>
> From: Michael Ritsema <mi...@gmail.com>
> Reply-To: <us...@storm.incubator.apache.org>
> Date: Tuesday, December 31, 2013 at 11:35 AM
> To: <us...@storm.incubator.apache.org>
> Subject: Re: Signature for DRPCClient.execute
>
> You need to serialize with something like JSON. If you are using a binary
> serialization you'd need a codec like base64 on top of it.
>
>
>
>
> On Tue, Dec 31, 2013 at 10:10 AM, Brian O'Neill <bo...@alumni.brown.edu> wrote:
>>
>> I'm trying to understand the motivation for the String based interface on the
>> execute method of DRPCClient.
>>
>> http://nathanmarz.github.io/storm/doc/backtype/storm/utils/DRPCClient.html
>> String execute (String, String)
>>
>> I have a topology that typically receives and outputs java objects. In order
>> for me to call it with DRPC, do I need to layer on a marshaling layer to
>> translate String->POJO and back?
>>
>> Would it be possible to change the signature to accept any type of object and
>> put it in the args field?
>>
>> -brian
>>
>> --
>> Brian ONeill
>> Chief Architect, Health Market Science (http://healthmarketscience.com)
>> mobile:215.588.6024 <tel:215.588.6024>
>> blog: http://brianoneill.blogspot.com/
>> twitter: @boneill42
>
Re: Signature for DRPCClient.execute
Posted by Jon Logan <jm...@buffalo.edu>.
I would imagine it's because it's not supported on the DRPC server -- this
isn't just using raw Thrift at this point, it has to be funneled through
the DRPC server (as far as I know).
Also note that it'll wrap your returned object into a JSON-encoded list.
That's a bit annoying as well...
On Tue, Dec 31, 2013 at 11:39 AM, Brian O'Neill <bo...@alumni.brown.edu>wrote:
>
> Ugh. That seems a bit awkward.
>
> Is there any reason why we wouldn’t change the signature to accept
> objects?
> (and (de)serialize under the hood like we do with standard tuples?)
>
> -brian
>
> ---
>
> Brian O'Neill
>
> Chief Architect
>
> *Health Market Science*
>
> *The Science of Better Results*
>
> 2700 Horizon Drive • King of Prussia, PA • 19406
>
> M: 215.588.6024 • @boneill42 <http://www.twitter.com/boneill42> •
>
> healthmarketscience.com
>
>
> This information transmitted in this email message is for the intended
> recipient only and may contain confidential and/or privileged material. If
> you received this email in error and are not the intended recipient, or the
> person responsible to deliver it to the intended recipient, please contact
> the sender at the email above and delete this email and any attachments and
> destroy any copies thereof. Any review, retransmission, dissemination,
> copying or other use of, or taking any action in reliance upon, this
> information by persons or entities other than the intended recipient is
> strictly prohibited.
>
>
>
>
> From: Michael Ritsema <mi...@gmail.com>
> Reply-To: <us...@storm.incubator.apache.org>
> Date: Tuesday, December 31, 2013 at 11:35 AM
> To: <us...@storm.incubator.apache.org>
> Subject: Re: Signature for DRPCClient.execute
>
> You need to serialize with something like JSON. If you are using a binary
> serialization you'd need a codec like base64 on top of it.
>
>
>
>
> On Tue, Dec 31, 2013 at 10:10 AM, Brian O'Neill <bo...@alumni.brown.edu>wrote:
>
>>
>> I'm trying to understand the motivation for the String based interface on
>> the execute method of DRPCClient.
>>
>> http://nathanmarz.github.io/storm/doc/backtype/storm/utils/DRPCClient.html
>> String execute (String, String)
>>
>> I have a topology that typically receives and outputs java objects. In
>> order for me to call it with DRPC, do I need to layer on a marshaling layer
>> to translate String->POJO and back?
>>
>> Would it be possible to change the signature to accept any type of object
>> and put it in the args field?
>>
>> -brian
>>
>> --
>> Brian ONeill
>> Chief Architect, Health Market Science (http://healthmarketscience.com)
>> mobile:215.588.6024
>> blog: http://brianoneill.blogspot.com/
>> twitter: @boneill42
>>
>
>
Re: Signature for DRPCClient.execute
Posted by Brian O'Neill <bo...@alumni.brown.edu>.
Ugh. That seems a bit awkward.
Is there any reason why we wouldn¹t change the signature to accept objects?
(and (de)serialize under the hood like we do with standard tuples?)
-brian
---
Brian O'Neill
Chief Architect
Health Market Science
The Science of Better Results
2700 Horizon Drive King of Prussia, PA 19406
M: 215.588.6024 @boneill42 <http://www.twitter.com/boneill42>
healthmarketscience.com
This information transmitted in this email message is for the intended
recipient only and may contain confidential and/or privileged material. If
you received this email in error and are not the intended recipient, or the
person responsible to deliver it to the intended recipient, please contact
the sender at the email above and delete this email and any attachments and
destroy any copies thereof. Any review, retransmission, dissemination,
copying or other use of, or taking any action in reliance upon, this
information by persons or entities other than the intended recipient is
strictly prohibited.
From: Michael Ritsema <mi...@gmail.com>
Reply-To: <us...@storm.incubator.apache.org>
Date: Tuesday, December 31, 2013 at 11:35 AM
To: <us...@storm.incubator.apache.org>
Subject: Re: Signature for DRPCClient.execute
You need to serialize with something like JSON. If you are using a binary
serialization you'd need a codec like base64 on top of it.
On Tue, Dec 31, 2013 at 10:10 AM, Brian O'Neill <bo...@alumni.brown.edu>
wrote:
>
> I'm trying to understand the motivation for the String based interface on the
> execute method of DRPCClient.
>
> http://nathanmarz.github.io/storm/doc/backtype/storm/utils/DRPCClient.html
> String execute (String, String)
>
> I have a topology that typically receives and outputs java objects. In order
> for me to call it with DRPC, do I need to layer on a marshaling layer to
> translate String->POJO and back?
>
> Would it be possible to change the signature to accept any type of object and
> put it in the args field?
>
> -brian
>
> --
> Brian ONeill
> Chief Architect, Health Market Science (http://healthmarketscience.com)
> mobile:215.588.6024 <tel:215.588.6024>
> blog: http://brianoneill.blogspot.com/
> twitter: @boneill42
Re: Signature for DRPCClient.execute
Posted by Michael Ritsema <mi...@gmail.com>.
You need to serialize with something like JSON. If you are using a binary
serialization you'd need a codec like base64 on top of it.
On Tue, Dec 31, 2013 at 10:10 AM, Brian O'Neill <bo...@alumni.brown.edu>wrote:
>
> I'm trying to understand the motivation for the String based interface on
> the execute method of DRPCClient.
>
> http://nathanmarz.github.io/storm/doc/backtype/storm/utils/DRPCClient.html
> String execute (String, String)
>
> I have a topology that typically receives and outputs java objects. In
> order for me to call it with DRPC, do I need to layer on a marshaling layer
> to translate String->POJO and back?
>
> Would it be possible to change the signature to accept any type of object
> and put it in the args field?
>
> -brian
>
> --
> Brian ONeill
> Chief Architect, Health Market Science (http://healthmarketscience.com)
> mobile:215.588.6024
> blog: http://brianoneill.blogspot.com/
> twitter: @boneill42
>