You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomee.apache.org by David Blevins <da...@visi.com> on 2007/02/02 04:44:57 UTC
Idea: Support InputStream as a remotable type
Hey what do people think about this one?
It's always bugged me that you can't use input streams in EJBs. It's
just too useful. For the case where there is a single input stream
in the arg list, we could do something pretty simple like have the
server reorder any other args so that the input stream is last in the
data sent from the client, then the server could literally just hand
off the input stream it has open with the client over as an argument
for invocation.
For input streams as return types, we can just do the reverse of this.
If there were more than one input streams as arguments then we could
do a file upload style push to the server where the server reads in
each part to a temp file and then creates input streams to those and
passes them in as arguments for invocation.
It's kind of a crazy idea, but I think this feature would just rock.
What do you think?
-David
Re: Idea: Support InputStream as a remotable type
Posted by Manu George <ma...@gmail.com>.
Hi Dain,
Thanks for the detailed explanations. This is definitely
a good feature to have
Regards
Manu
On 2/4/07, Dain Sundstrom <da...@iq80.com> wrote:
> On Feb 3, 2007, at 1:00 AM, Manu George wrote:
>
> > Yes Excellent Explanation!! I am able to understand David's
> > mail completely now :).
> > It is a good idea. I have a few claarifications/questions
> >
> > What will be effect of creating temporary files on the server side on
> > performance? I am thinking of a large no of requests to an EJB with
> > multiple InputStreams as parameters. Will this result in a lot of disk
> > I/O slowing down the server?
>
> In that case, yes of course it will slow down the server. This a
> connivence feature that should be used with care. If you have
> thousands of connections per second all uploading large files, I
> doubt this will work for you. But if this is used in an area of an
> application that few users will access it shouldn't be a problem.
> Also, it the files are small, there is no need to write them to disk
> and in that case you will just need enough memory.
>
> As always, it comes down to expected usage patterns and how your
> application is architected.
>
> > Also for large InputStreams the method invocations may take some time
> > to complete.
> > Will we be doing something to have better performance in these cases
> > eg providing the ability to hook in a compressor/decompressor. I don't
> > know how valuable it will be because of the cpu cycles
> > compression/decompression may consume
>
> Sounds like a good option to have. Actually, you may wan to gzip the
> entire protocol. This is a simple tradeoff between cpu and
> bandwidth. One option that works well is to pass all data over HTTP
> and then use a load balancer that can compress/encrypt the stream.
> Then all the work takes place in hardware specially designed to
> handle that load.
>
> > Finally Are there any additional security considerations of passing
> > InputStreams across the wire unencrypted compared to passing objects ?
>
> No. The both have the same inherent issue. The data would be
> unencrypted. If the data needs to be encrypted you must encrypt it
> regardless of the format the data is passed over the wire.
>
> > Also what are the usecases where it is better to pass inputstreams?
>
> I want to use the ejb protocol for remote deployment of
> applications. In that case, I need to stream the archive to a the
> server. Other then that, I would guess there are a few other rare
> situations where you want to stream to the server.
>
> -dain
>
Re: Idea: Support InputStream as a remotable type
Posted by Dain Sundstrom <da...@iq80.com>.
On Feb 3, 2007, at 1:00 AM, Manu George wrote:
> Yes Excellent Explanation!! I am able to understand David's
> mail completely now :).
> It is a good idea. I have a few claarifications/questions
>
> What will be effect of creating temporary files on the server side on
> performance? I am thinking of a large no of requests to an EJB with
> multiple InputStreams as parameters. Will this result in a lot of disk
> I/O slowing down the server?
In that case, yes of course it will slow down the server. This a
connivence feature that should be used with care. If you have
thousands of connections per second all uploading large files, I
doubt this will work for you. But if this is used in an area of an
application that few users will access it shouldn't be a problem.
Also, it the files are small, there is no need to write them to disk
and in that case you will just need enough memory.
As always, it comes down to expected usage patterns and how your
application is architected.
> Also for large InputStreams the method invocations may take some time
> to complete.
> Will we be doing something to have better performance in these cases
> eg providing the ability to hook in a compressor/decompressor. I don't
> know how valuable it will be because of the cpu cycles
> compression/decompression may consume
Sounds like a good option to have. Actually, you may wan to gzip the
entire protocol. This is a simple tradeoff between cpu and
bandwidth. One option that works well is to pass all data over HTTP
and then use a load balancer that can compress/encrypt the stream.
Then all the work takes place in hardware specially designed to
handle that load.
> Finally Are there any additional security considerations of passing
> InputStreams across the wire unencrypted compared to passing objects ?
No. The both have the same inherent issue. The data would be
unencrypted. If the data needs to be encrypted you must encrypt it
regardless of the format the data is passed over the wire.
> Also what are the usecases where it is better to pass inputstreams?
I want to use the ejb protocol for remote deployment of
applications. In that case, I need to stream the archive to a the
server. Other then that, I would guess there are a few other rare
situations where you want to stream to the server.
-dain
Re: Idea: Support InputStream as a remotable type
Posted by Manu George <ma...@gmail.com>.
Yes Excellent Explanation!! I am able to understand David's
mail completely now :).
It is a good idea. I have a few claarifications/questions
What will be effect of creating temporary files on the server side on
performance? I am thinking of a large no of requests to an EJB with
multiple InputStreams as parameters. Will this result in a lot of disk
I/O slowing down the server?
Also for large InputStreams the method invocations may take some time
to complete.
Will we be doing something to have better performance in these cases
eg providing the ability to hook in a compressor/decompressor. I don't
know how valuable it will be because of the cpu cycles
compression/decompression may consume
Finally Are there any additional security considerations of passing
InputStreams across the wire unencrypted compared to passing objects ?
Also what are the usecases where it is better to pass inputstreams?
Regards
Manu
On 2/2/07, Karan Malhi <ka...@gmail.com> wrote:
> Aha!!
>
> Excellent explanation!!.
>
> Thanks
>
> On 2/2/07, Dain Sundstrom <da...@iq80.com> wrote:
> >
> > It has to do with the way input streams are encoded on the wire.
> > Assume you have the following method:
> >
> > void deploy(InputStream module, InputStream deploymentPlan);
> >
> > When a client invoked that method, they would get somthing like this
> > on the wire:
> >
> > | ejb | invoke | 42 | deploy | ${module bytes} | ${plan bytes} |
> >
> > On the server side we could read up to the first input stream, but
> > not the second since the bytes for the first input stream must be
> > read off of the socket before the bytes for the second input stream
> > can be accessed. In this case we could give a real input stream for
> > the first argument, and a proxy of the second that becomes active
> > once the first stream has been exhausted. The works until the user
> > decides to read the deploymentPlan stream first, which means the
> > module bytes must be read off the socked so we can get to the
> > deploymentPlan's bytes.
> >
> > Another problem happens when you have methods like this;
> >
> > void doSomething(InputStream module, int number)
> >
> > With normal encoding the input stream bytes will be written to the
> > socket before the number is written. Since we must have the value of
> > the integer number before we invoke the method, we must read the
> > entire contents of the input stream before we can read the number and
> > thus before the method invocation.
> >
> > A final complication is when people have input streams nested in
> > complex objects. For example:
> >
> > void doSomethingElse(Map<String,InputStream> data);
> >
> > In this case we must use special ObjectOutputStream hook on the
> > client side to replace all InputStream objects with our custom
> > RemoteInputStream:
> >
> > protected Object replaceObject(Object obj) throws IOException {
> > if (obj instanceof InputStream) {
> > return new RemoteInputStream((InputStream)obj);
> > }
> > return obj;
> > }
> >
> > -dain
> >
> > On Feb 2, 2007, at 6:57 AM, Karan Malhi wrote:
> >
> > > I think it is a good idea. I had one question though, why do we
> > > need to
> > > store data in a temp file in case of multilple input streams. If a
> > > single
> > > input stream could be handed over for invocation, what is wrong
> > > with handing
> > > over multiple input streams for invocation?
> > >
> > > On 2/1/07, David Blevins <da...@visi.com> wrote:
> > >>
> > >> Hey what do people think about this one?
> > >>
> > >> It's always bugged me that you can't use input streams in EJBs. It's
> > >> just too useful. For the case where there is a single input stream
> > >> in the arg list, we could do something pretty simple like have the
> > >> server reorder any other args so that the input stream is last in the
> > >> data sent from the client, then the server could literally just hand
> > >> off the input stream it has open with the client over as an argument
> > >> for invocation.
> > >>
> > >> For input streams as return types, we can just do the reverse of
> > >> this.
> > >>
> > >> If there were more than one input streams as arguments then we could
> > >> do a file upload style push to the server where the server reads in
> > >> each part to a temp file and then creates input streams to those and
> > >> passes them in as arguments for invocation.
> > >>
> > >> It's kind of a crazy idea, but I think this feature would just rock.
> > >> What do you think?
> > >>
> > >> -David
> > >>
> > >>
> > >
> > >
> > > --
> > > Karan Malhi
> >
> >
>
>
> --
> Karan Malhi
>
Re: Idea: Support InputStream as a remotable type
Posted by Karan Malhi <ka...@gmail.com>.
Aha!!
Excellent explanation!!.
Thanks
On 2/2/07, Dain Sundstrom <da...@iq80.com> wrote:
>
> It has to do with the way input streams are encoded on the wire.
> Assume you have the following method:
>
> void deploy(InputStream module, InputStream deploymentPlan);
>
> When a client invoked that method, they would get somthing like this
> on the wire:
>
> | ejb | invoke | 42 | deploy | ${module bytes} | ${plan bytes} |
>
> On the server side we could read up to the first input stream, but
> not the second since the bytes for the first input stream must be
> read off of the socket before the bytes for the second input stream
> can be accessed. In this case we could give a real input stream for
> the first argument, and a proxy of the second that becomes active
> once the first stream has been exhausted. The works until the user
> decides to read the deploymentPlan stream first, which means the
> module bytes must be read off the socked so we can get to the
> deploymentPlan's bytes.
>
> Another problem happens when you have methods like this;
>
> void doSomething(InputStream module, int number)
>
> With normal encoding the input stream bytes will be written to the
> socket before the number is written. Since we must have the value of
> the integer number before we invoke the method, we must read the
> entire contents of the input stream before we can read the number and
> thus before the method invocation.
>
> A final complication is when people have input streams nested in
> complex objects. For example:
>
> void doSomethingElse(Map<String,InputStream> data);
>
> In this case we must use special ObjectOutputStream hook on the
> client side to replace all InputStream objects with our custom
> RemoteInputStream:
>
> protected Object replaceObject(Object obj) throws IOException {
> if (obj instanceof InputStream) {
> return new RemoteInputStream((InputStream)obj);
> }
> return obj;
> }
>
> -dain
>
> On Feb 2, 2007, at 6:57 AM, Karan Malhi wrote:
>
> > I think it is a good idea. I had one question though, why do we
> > need to
> > store data in a temp file in case of multilple input streams. If a
> > single
> > input stream could be handed over for invocation, what is wrong
> > with handing
> > over multiple input streams for invocation?
> >
> > On 2/1/07, David Blevins <da...@visi.com> wrote:
> >>
> >> Hey what do people think about this one?
> >>
> >> It's always bugged me that you can't use input streams in EJBs. It's
> >> just too useful. For the case where there is a single input stream
> >> in the arg list, we could do something pretty simple like have the
> >> server reorder any other args so that the input stream is last in the
> >> data sent from the client, then the server could literally just hand
> >> off the input stream it has open with the client over as an argument
> >> for invocation.
> >>
> >> For input streams as return types, we can just do the reverse of
> >> this.
> >>
> >> If there were more than one input streams as arguments then we could
> >> do a file upload style push to the server where the server reads in
> >> each part to a temp file and then creates input streams to those and
> >> passes them in as arguments for invocation.
> >>
> >> It's kind of a crazy idea, but I think this feature would just rock.
> >> What do you think?
> >>
> >> -David
> >>
> >>
> >
> >
> > --
> > Karan Malhi
>
>
--
Karan Malhi
Re: Idea: Support InputStream as a remotable type
Posted by Dain Sundstrom <da...@iq80.com>.
It has to do with the way input streams are encoded on the wire.
Assume you have the following method:
void deploy(InputStream module, InputStream deploymentPlan);
When a client invoked that method, they would get somthing like this
on the wire:
| ejb | invoke | 42 | deploy | ${module bytes} | ${plan bytes} |
On the server side we could read up to the first input stream, but
not the second since the bytes for the first input stream must be
read off of the socket before the bytes for the second input stream
can be accessed. In this case we could give a real input stream for
the first argument, and a proxy of the second that becomes active
once the first stream has been exhausted. The works until the user
decides to read the deploymentPlan stream first, which means the
module bytes must be read off the socked so we can get to the
deploymentPlan's bytes.
Another problem happens when you have methods like this;
void doSomething(InputStream module, int number)
With normal encoding the input stream bytes will be written to the
socket before the number is written. Since we must have the value of
the integer number before we invoke the method, we must read the
entire contents of the input stream before we can read the number and
thus before the method invocation.
A final complication is when people have input streams nested in
complex objects. For example:
void doSomethingElse(Map<String,InputStream> data);
In this case we must use special ObjectOutputStream hook on the
client side to replace all InputStream objects with our custom
RemoteInputStream:
protected Object replaceObject(Object obj) throws IOException {
if (obj instanceof InputStream) {
return new RemoteInputStream((InputStream)obj);
}
return obj;
}
-dain
On Feb 2, 2007, at 6:57 AM, Karan Malhi wrote:
> I think it is a good idea. I had one question though, why do we
> need to
> store data in a temp file in case of multilple input streams. If a
> single
> input stream could be handed over for invocation, what is wrong
> with handing
> over multiple input streams for invocation?
>
> On 2/1/07, David Blevins <da...@visi.com> wrote:
>>
>> Hey what do people think about this one?
>>
>> It's always bugged me that you can't use input streams in EJBs. It's
>> just too useful. For the case where there is a single input stream
>> in the arg list, we could do something pretty simple like have the
>> server reorder any other args so that the input stream is last in the
>> data sent from the client, then the server could literally just hand
>> off the input stream it has open with the client over as an argument
>> for invocation.
>>
>> For input streams as return types, we can just do the reverse of
>> this.
>>
>> If there were more than one input streams as arguments then we could
>> do a file upload style push to the server where the server reads in
>> each part to a temp file and then creates input streams to those and
>> passes them in as arguments for invocation.
>>
>> It's kind of a crazy idea, but I think this feature would just rock.
>> What do you think?
>>
>> -David
>>
>>
>
>
> --
> Karan Malhi
Re: Idea: Support InputStream as a remotable type
Posted by Karan Malhi <ka...@gmail.com>.
I think it is a good idea. I had one question though, why do we need to
store data in a temp file in case of multilple input streams. If a single
input stream could be handed over for invocation, what is wrong with handing
over multiple input streams for invocation?
On 2/1/07, David Blevins <da...@visi.com> wrote:
>
> Hey what do people think about this one?
>
> It's always bugged me that you can't use input streams in EJBs. It's
> just too useful. For the case where there is a single input stream
> in the arg list, we could do something pretty simple like have the
> server reorder any other args so that the input stream is last in the
> data sent from the client, then the server could literally just hand
> off the input stream it has open with the client over as an argument
> for invocation.
>
> For input streams as return types, we can just do the reverse of this.
>
> If there were more than one input streams as arguments then we could
> do a file upload style push to the server where the server reads in
> each part to a temp file and then creates input streams to those and
> passes them in as arguments for invocation.
>
> It's kind of a crazy idea, but I think this feature would just rock.
> What do you think?
>
> -David
>
>
--
Karan Malhi