You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Robert Newson (JIRA)" <ji...@activemq.org> on 2006/03/16 19:48:25 UTC
[jira] Created: (AMQ-637) Enhance ActiveMQInput/Outputstream to use
a real TCP connection.
Enhance ActiveMQInput/Outputstream to use a real TCP connection.
----------------------------------------------------------------
Key: AMQ-637
URL: http://jira.activemq.org/jira//browse/AMQ-637
Project: ActiveMQ
Type: Improvement
Reporter: Robert Newson
Currently the ActiveMQInputStream and ActiveMQOutputStreams chunk large messages into 64k chunks.
Instead of that, could the act of calling getInputStream(Destination) negotiates a new TCP socket connection? Using ActiveMQ will still give us location transparency, but will the full streaming power of TCP.
I spoke with Hiram about this and he thought it would be a fairly simple enhancement. I'm happy to work with him on this as we have a strong need for this functionality.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.activemq.org/jira//secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (AMQ-637) Enhance ActiveMQInput/Outputstream to
use a real TCP connection.
Posted by "james strachan (JIRA)" <ji...@activemq.org>.
[ http://jira.activemq.org/jira//browse/AMQ-637?page=comments#action_35766 ]
james strachan commented on AMQ-637:
------------------------------------
Sounds fine with me :)
BTW is the idea that the socket will only be open for the duration of the input stream, so that the transfer is independent of the underlying JMS connection?
We do need to chunk the message on the broker side to avoid it having to load the whole stream into RAM; is this OK with you?
> Enhance ActiveMQInput/Outputstream to use a real TCP connection.
> ----------------------------------------------------------------
>
> Key: AMQ-637
> URL: http://jira.activemq.org/jira//browse/AMQ-637
> Project: ActiveMQ
> Type: Improvement
> Reporter: Robert Newson
>
>
> Currently the ActiveMQInputStream and ActiveMQOutputStreams chunk large messages into 64k chunks.
> Instead of that, could the act of calling getInputStream(Destination) negotiates a new TCP socket connection? Using ActiveMQ will still give us location transparency, but will the full streaming power of TCP.
> I spoke with Hiram about this and he thought it would be a fairly simple enhancement. I'm happy to work with him on this as we have a strong need for this functionality.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.activemq.org/jira//secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (AMQ-637) Enhance ActiveMQInput/Outputstream to use
a real TCP connection.
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-637?page=all ]
Hiram Chirino updated AMQ-637:
------------------------------
Fix Version: 4.2
This could take a bit of thought to implement correctly. Putting out till 4.2.
> Enhance ActiveMQInput/Outputstream to use a real TCP connection.
> ----------------------------------------------------------------
>
> Key: AMQ-637
> URL: https://issues.apache.org/activemq/browse/AMQ-637
> Project: ActiveMQ
> Type: Improvement
> Reporter: Robert Newson
> Fix For: 4.2
>
>
> Currently the ActiveMQInputStream and ActiveMQOutputStreams chunk large messages into 64k chunks.
> Instead of that, could the act of calling getInputStream(Destination) negotiates a new TCP socket connection? Using ActiveMQ will still give us location transparency, but will the full streaming power of TCP.
> I spoke with Hiram about this and he thought it would be a fairly simple enhancement. I'm happy to work with him on this as we have a strong need for this functionality.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://issues.apache.org/activemq/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (AMQ-637) Enhance ActiveMQInput/Outputstream to
use a real TCP connection.
Posted by "Robert Newson (JIRA)" <ji...@activemq.org>.
[ http://jira.activemq.org/jira//browse/AMQ-637?page=comments#action_35767 ]
Robert Newson commented on AMQ-637:
-----------------------------------
Some kind of connection pooling might be even better, but in the absence of that I'd be happy for the socket to exist only for the duration of the input stream. I'd definitely prefer it to be separate from the underlying JMS connection.
Avoiding having the whole stream in RAM is a primary goal for us. We expect to see 99% of our incoming messages at around 2k, but the 1% are likely to be enormous (potentially up to 1gb). By negotiating a tcp stream I was hoping that we could use its blocking behaviour to cap the amount of RAM consumed on either side. In our case, we're writing the data we receive to a file system.
The way I would plan to handle the two cases we have is to send an initial request message with the size of the payload and then switch to the streaming case at some threshold. For small messages, we'd include the payload in that request.
> Enhance ActiveMQInput/Outputstream to use a real TCP connection.
> ----------------------------------------------------------------
>
> Key: AMQ-637
> URL: http://jira.activemq.org/jira//browse/AMQ-637
> Project: ActiveMQ
> Type: Improvement
> Reporter: Robert Newson
>
>
> Currently the ActiveMQInputStream and ActiveMQOutputStreams chunk large messages into 64k chunks.
> Instead of that, could the act of calling getInputStream(Destination) negotiates a new TCP socket connection? Using ActiveMQ will still give us location transparency, but will the full streaming power of TCP.
> I spoke with Hiram about this and he thought it would be a fairly simple enhancement. I'm happy to work with him on this as we have a strong need for this functionality.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.activemq.org/jira//secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira