You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Chad Beaulac <ca...@gmail.com> on 2012/01/16 14:56:06 UTC
Re: [MINA 3.0] IoBuffer usage
Emmanuel, (all)
I'm working on this Camel ticket:
https://issues.apache.org/jira/browse/CAMEL-2624
I finished the initial cut of
https://issues.apache.org/jira/browse/CAMEL-3471 to create a mina2
component in Camel.
CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel.
Would it be possible to backport the IoBuffer reading and writing discussed
in this email thread from Mina3 to Mina2?
Following the depth of the stack trace through
AbstractIoSession.write(...), I'm a little concerned about the throughput.
My current code (mina-less) is supporting single TCP channels with 320+
Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol
Buffers codec I wrote. I know if this is a real problem soon when I finish
CAMEL-2624 and setup some throughput tests.
Regards,
Chad
On Tue, Dec 6, 2011 at 1:27 AM, Chad Beaulac <ca...@gmail.com> wrote:
> Looks good Emmanuel.
>
> Sent from my iPhone
>
> On Dec 5, 2011, at 10:13 AM, Emmanuel Lecharny <el...@gmail.com>
> wrote:
>
> > On 12/5/11 3:50 PM, Julien Vermillard wrote:
> >> since it's a ConcurrentLinkedQueue it could be a perf killer to do a
> .size() from the oracle javadoc : ""Beware that, unlike in most
> collections, the size method is NOT a constant-time operation. Because of
> the asynchronous nature of these queues, determining the current number of
> elements requires a traversal of the elements.""
> > Damn right...
> >
> > What about using a read/write lock instead of a synchronized block ? The
> problem with the synchornized block on queue is that we must still protect
> the queue when it's being written with another synchronized block, when if
> we use a read/write lock, we can allow parallel writes in the queue, but
> once it comes to write in the channel, we acquire a write lock and the
> queue is now protected. Something like :
> >
> > private final ReadWriteLock lock = new ReentrantReadWriteLock();
> > private final Lock queueReadLock = lock.readLock();
> > private final Lock queueWriteLock= lock.writeLock();
> > ...
> > try {
> > queueWriteLock.lock();
> >
> > do {
> > WriteRequest wreq = queue.peek();
> >
> > if (wreq == null) {
> > break;
> > }
> > ...
> > } while (!queue.isEmpty());
> > } finally {
> > queueWriteLock.unlock();
> > }
> >
> > ...
> >
> > public WriteRequest enqueueWriteRequest(Object message) {
> > DefaultWriteRequest request = new DefaultWriteRequest(message);
> >
> > try {
> > queueReadLock().lock()
> > writeQueue.add(request);
> > } finally {
> > queueReadLock.unlock();
> > }
> > }
> >
> > -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
>
Re: [MINA 3.0] IoBuffer usage
Posted by Emmanuel Lecharny <el...@gmail.com>.
On 1/18/12 2:57 AM, Chad Beaulac wrote:
> Ok. I'll test the throughput on it with the tests I have.
Let me know if you have any isues with the throughput you obtain.
Be sure to set the default read buffer size to the maximum size, AFAIR,
it's set to 2048 bytes in MINA 2 (and can't be higher than 65536),
otherwise you'll may get atrocious throughput...
>
> Thanks
> Chad
>
> Sent from my iPad
>
> On Jan 17, 2012, at 11:33 AM, Emmanuel Lecharny<el...@gmail.com> wrote:
>
>> IFAICT, on MINA 2, you should not have any issue.
>>
>> The loop where we write buffers into the channel is :
>>
>> ...
>> final int maxWrittenBytes = session.getConfig().getMaxReadBufferSize()
>> + (session.getConfig().getMaxReadBufferSize()>>> 1);
>> int writtenBytes = 0;
>> WriteRequest req = null;
>>
>> try {
>> // Clear OP_WRITE
>> setInterestedInWrite(session, false);
>>
>> do {
>> // Check for pending writes.
>> req = session.getCurrentWriteRequest();
>>
>> if (req == null) {
>> req = writeRequestQueue.poll(session);
>>
>> if (req == null) {
>> break;
>> }
>>
>> session.setCurrentWriteRequest(req);
>> }
>>
>> int localWrittenBytes = 0;
>> Object message = req.getMessage();
>>
>> if (message instanceof IoBuffer) {
>> localWrittenBytes = writeBuffer(session, req,
>> hasFragmentation, maxWrittenBytes - writtenBytes,
>> currentTime);
>>
>> if (( localWrittenBytes> 0 )
>> && ((IoBuffer) message).hasRemaining()) {
>> // the buffer isn't empty, we re-interest it in writing
>> writtenBytes += localWrittenBytes;
>> setInterestedInWrite(session, true);
>> return false;
>> }
>> } else { // Blahhh }
>>
>> if (localWrittenBytes == 0) {
>> // Kernel buffer is full.
>> setInterestedInWrite(session, true);
>> return false;
>> }
>>
>> writtenBytes += localWrittenBytes;
>>
>> if (writtenBytes>= maxWrittenBytes) {
>> // Wrote too much
>> scheduleFlush(session);
>> return false;
>> }
>> } while (writtenBytes< maxWrittenBytes);
>> ...
>>
>> with :
>>
>> private int writeBuffer(S session, WriteRequest req,
>> boolean hasFragmentation, int maxLength, long currentTime)
>> throws Exception {
>> IoBuffer buf = (IoBuffer) req.getMessage();
>> int localWrittenBytes = 0;
>>
>> if (buf.hasRemaining()) {
>> int length;
>>
>> if (hasFragmentation) {
>> length = Math.min(buf.remaining(), maxLength);
>> } else {
>> length = buf.remaining();
>> }
>>
>> localWrittenBytes = write(session, buf, length);
>> }
>>
>> and :
>>
>> protected int write(NioSession session, IoBuffer buf, int length)
>> throws Exception {
>> if (buf.remaining()<= length) {
>> return session.getChannel().write(buf.buf());
>> }
>>
>> int oldLimit = buf.limit();
>> buf.limit(buf.position() + length);
>> try {
>> return session.getChannel().write(buf.buf());
>> } finally {
>> buf.limit(oldLimit);
>> }
>> }
>>
>> So we try our best to stuff the channel with as many bytes as possible, before giving up (either because we don't have anything to write, to because the channel is full...)
>>
>> I don't see if we can do any better.
>>
>> On 1/17/12 3:57 PM, Chad Beaulac wrote:
>>> On Jan 17, 2012, at 9:32 AM, Emmanuel Lécharny<el...@apache.org> wrote:
>>>
>>>> On 1/17/12 3:02 PM, Chad Beaulac wrote:
>>>>> On Mon, Jan 16, 2012 at 1:10 PM, Emmanuel Lecharny<el...@gmail.com>wrote:
>>>>>
>>>>>> On 1/16/12 2:56 PM, Chad Beaulac wrote:
>>>>>>
>>>>>>> Emmanuel, (all)
>>>>>>>
>>>>>>> I'm working on this Camel ticket:
>>>>>>> https://issues.apache.org/**jira/browse/CAMEL-2624<https://issues.apache.org/jira/browse/CAMEL-2624>
>>>>>>>
>>>>>>> I finished the initial cut of
>>>>>>> https://issues.apache.org/**jira/browse/CAMEL-3471<https://issues.apache.org/jira/browse/CAMEL-3471>to create a mina2
>>>>>>> component in Camel.
>>>>>>>
>>>>>>> CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel.
>>>>>>>
>>>>>>> Would it be possible to backport the IoBuffer reading and writing
>>>>>>> discussed
>>>>>>> in this email thread from Mina3 to Mina2?
>>>>>>> Following the depth of the stack trace through
>>>>>>> AbstractIoSession.write(...), I'm a little concerned about the throughput.
>>>>>>> My current code (mina-less) is supporting single TCP channels with 320+
>>>>>>> Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol
>>>>>>> Buffers codec I wrote. I know if this is a real problem soon when I finish
>>>>>>> CAMEL-2624 and setup some throughput tests.
>>>>>>>
>>>>>> Another option would be to port this to MINA3, and make it work with
>>>>>> Camel. Right now, MINA 3 is pretty rough, but we have made it works well
>>>>>> with Http and LDAP. I'd rather spend some time making it a bit more solid
>>>>>> and working well in your case instead of trying to inject the code in MINA2.
>>>>>>
>>>>>> Now, it's your call. We can discuss the pros and cons of both approach if
>>>>>> you like.
>>>>>>
>>>>>>
>>>>> Hi Emmanuel,
>>>>>
>>>>> One of my pros/cons trade-offs is time-to-market. I can have a solution in
>>>>> Camel with Mina2 fairly quickly. Although I might have issues with high
>>>>> data rate streams.
>>>>> With that said, my approach would be the following:
>>>>> 1) Finish CAMEL-2624 with Mina2. This will give me the asynchronous
>>>>> endpoints I need and a quick time-to-market. I'll put off issues concerning
>>>>> high throughput.
>>>>> 2) Work on Mina3 to ensure it has low latency with small data rate streams
>>>>> and high throughput with large data pipes.
>>>>> 3) Upgrade my Google protocol buffers codec for Mina3.
>>>>> 4) When Mina3 is ready, open a new Camel ticket and create a new mina3
>>>>> Camel Component.
>>>>>
>>>>> What do you think?
>>>> I'll try to squeeze 2 hours to backport the patch to MINA 2 today or tomorrow.
>>>>
>>>> Feel free to ping me on mail or on #mina if I don't send any feedback in the next 2 days (I'm pretty busy and may slip)
>>>>
>>>>
>>>> --
>>>> Regards,
>>>> Cordialement,
>>>> Emmanuel Lécharny
>>>> www.iktek.com
>>>>
>>> Wow. That is nice! Look forward to checking it out. I'll move forward with my plan in the meantime.
>>>
>>> Chad
>>> Sent from my iPhone
>>>
>>
>> --
>> Regards,
>> Cordialement,
>> Emmanuel Lécharny
>> www.iktek.com
>>
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: [MINA 3.0] IoBuffer usage
Posted by Chad Beaulac <ca...@gmail.com>.
Ok. I'll test the throughput on it with the tests I have.
Thanks
Chad
Sent from my iPad
On Jan 17, 2012, at 11:33 AM, Emmanuel Lecharny <el...@gmail.com> wrote:
> IFAICT, on MINA 2, you should not have any issue.
>
> The loop where we write buffers into the channel is :
>
> ...
> final int maxWrittenBytes = session.getConfig().getMaxReadBufferSize()
> + (session.getConfig().getMaxReadBufferSize() >>> 1);
> int writtenBytes = 0;
> WriteRequest req = null;
>
> try {
> // Clear OP_WRITE
> setInterestedInWrite(session, false);
>
> do {
> // Check for pending writes.
> req = session.getCurrentWriteRequest();
>
> if (req == null) {
> req = writeRequestQueue.poll(session);
>
> if (req == null) {
> break;
> }
>
> session.setCurrentWriteRequest(req);
> }
>
> int localWrittenBytes = 0;
> Object message = req.getMessage();
>
> if (message instanceof IoBuffer) {
> localWrittenBytes = writeBuffer(session, req,
> hasFragmentation, maxWrittenBytes - writtenBytes,
> currentTime);
>
> if (( localWrittenBytes > 0 )
> && ((IoBuffer) message).hasRemaining()) {
> // the buffer isn't empty, we re-interest it in writing
> writtenBytes += localWrittenBytes;
> setInterestedInWrite(session, true);
> return false;
> }
> } else { // Blahhh }
>
> if (localWrittenBytes == 0) {
> // Kernel buffer is full.
> setInterestedInWrite(session, true);
> return false;
> }
>
> writtenBytes += localWrittenBytes;
>
> if (writtenBytes >= maxWrittenBytes) {
> // Wrote too much
> scheduleFlush(session);
> return false;
> }
> } while (writtenBytes < maxWrittenBytes);
> ...
>
> with :
>
> private int writeBuffer(S session, WriteRequest req,
> boolean hasFragmentation, int maxLength, long currentTime)
> throws Exception {
> IoBuffer buf = (IoBuffer) req.getMessage();
> int localWrittenBytes = 0;
>
> if (buf.hasRemaining()) {
> int length;
>
> if (hasFragmentation) {
> length = Math.min(buf.remaining(), maxLength);
> } else {
> length = buf.remaining();
> }
>
> localWrittenBytes = write(session, buf, length);
> }
>
> and :
>
> protected int write(NioSession session, IoBuffer buf, int length)
> throws Exception {
> if (buf.remaining() <= length) {
> return session.getChannel().write(buf.buf());
> }
>
> int oldLimit = buf.limit();
> buf.limit(buf.position() + length);
> try {
> return session.getChannel().write(buf.buf());
> } finally {
> buf.limit(oldLimit);
> }
> }
>
> So we try our best to stuff the channel with as many bytes as possible, before giving up (either because we don't have anything to write, to because the channel is full...)
>
> I don't see if we can do any better.
>
> On 1/17/12 3:57 PM, Chad Beaulac wrote:
>>
>> On Jan 17, 2012, at 9:32 AM, Emmanuel Lécharny<el...@apache.org> wrote:
>>
>>> On 1/17/12 3:02 PM, Chad Beaulac wrote:
>>>> On Mon, Jan 16, 2012 at 1:10 PM, Emmanuel Lecharny<el...@gmail.com>wrote:
>>>>
>>>>> On 1/16/12 2:56 PM, Chad Beaulac wrote:
>>>>>
>>>>>> Emmanuel, (all)
>>>>>>
>>>>>> I'm working on this Camel ticket:
>>>>>> https://issues.apache.org/**jira/browse/CAMEL-2624<https://issues.apache.org/jira/browse/CAMEL-2624>
>>>>>>
>>>>>> I finished the initial cut of
>>>>>> https://issues.apache.org/**jira/browse/CAMEL-3471<https://issues.apache.org/jira/browse/CAMEL-3471>to create a mina2
>>>>>> component in Camel.
>>>>>>
>>>>>> CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel.
>>>>>>
>>>>>> Would it be possible to backport the IoBuffer reading and writing
>>>>>> discussed
>>>>>> in this email thread from Mina3 to Mina2?
>>>>>> Following the depth of the stack trace through
>>>>>> AbstractIoSession.write(...), I'm a little concerned about the throughput.
>>>>>> My current code (mina-less) is supporting single TCP channels with 320+
>>>>>> Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol
>>>>>> Buffers codec I wrote. I know if this is a real problem soon when I finish
>>>>>> CAMEL-2624 and setup some throughput tests.
>>>>>>
>>>>> Another option would be to port this to MINA3, and make it work with
>>>>> Camel. Right now, MINA 3 is pretty rough, but we have made it works well
>>>>> with Http and LDAP. I'd rather spend some time making it a bit more solid
>>>>> and working well in your case instead of trying to inject the code in MINA2.
>>>>>
>>>>> Now, it's your call. We can discuss the pros and cons of both approach if
>>>>> you like.
>>>>>
>>>>>
>>>> Hi Emmanuel,
>>>>
>>>> One of my pros/cons trade-offs is time-to-market. I can have a solution in
>>>> Camel with Mina2 fairly quickly. Although I might have issues with high
>>>> data rate streams.
>>>> With that said, my approach would be the following:
>>>> 1) Finish CAMEL-2624 with Mina2. This will give me the asynchronous
>>>> endpoints I need and a quick time-to-market. I'll put off issues concerning
>>>> high throughput.
>>>> 2) Work on Mina3 to ensure it has low latency with small data rate streams
>>>> and high throughput with large data pipes.
>>>> 3) Upgrade my Google protocol buffers codec for Mina3.
>>>> 4) When Mina3 is ready, open a new Camel ticket and create a new mina3
>>>> Camel Component.
>>>>
>>>> What do you think?
>>> I'll try to squeeze 2 hours to backport the patch to MINA 2 today or tomorrow.
>>>
>>> Feel free to ping me on mail or on #mina if I don't send any feedback in the next 2 days (I'm pretty busy and may slip)
>>>
>>>
>>> --
>>> Regards,
>>> Cordialement,
>>> Emmanuel Lécharny
>>> www.iktek.com
>>>
>> Wow. That is nice! Look forward to checking it out. I'll move forward with my plan in the meantime.
>>
>> Chad
>> Sent from my iPhone
>>
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
Re: [MINA 3.0] IoBuffer usage
Posted by Emmanuel Lecharny <el...@gmail.com>.
IFAICT, on MINA 2, you should not have any issue.
The loop where we write buffers into the channel is :
...
final int maxWrittenBytes =
session.getConfig().getMaxReadBufferSize()
+ (session.getConfig().getMaxReadBufferSize() >>> 1);
int writtenBytes = 0;
WriteRequest req = null;
try {
// Clear OP_WRITE
setInterestedInWrite(session, false);
do {
// Check for pending writes.
req = session.getCurrentWriteRequest();
if (req == null) {
req = writeRequestQueue.poll(session);
if (req == null) {
break;
}
session.setCurrentWriteRequest(req);
}
int localWrittenBytes = 0;
Object message = req.getMessage();
if (message instanceof IoBuffer) {
localWrittenBytes = writeBuffer(session, req,
hasFragmentation, maxWrittenBytes -
writtenBytes,
currentTime);
if (( localWrittenBytes > 0 )
&& ((IoBuffer) message).hasRemaining()) {
// the buffer isn't empty, we re-interest it in
writing
writtenBytes += localWrittenBytes;
setInterestedInWrite(session, true);
return false;
}
} else { // Blahhh }
if (localWrittenBytes == 0) {
// Kernel buffer is full.
setInterestedInWrite(session, true);
return false;
}
writtenBytes += localWrittenBytes;
if (writtenBytes >= maxWrittenBytes) {
// Wrote too much
scheduleFlush(session);
return false;
}
} while (writtenBytes < maxWrittenBytes);
...
with :
private int writeBuffer(S session, WriteRequest req,
boolean hasFragmentation, int maxLength, long currentTime)
throws Exception {
IoBuffer buf = (IoBuffer) req.getMessage();
int localWrittenBytes = 0;
if (buf.hasRemaining()) {
int length;
if (hasFragmentation) {
length = Math.min(buf.remaining(), maxLength);
} else {
length = buf.remaining();
}
localWrittenBytes = write(session, buf, length);
}
and :
protected int write(NioSession session, IoBuffer buf, int length)
throws Exception {
if (buf.remaining() <= length) {
return session.getChannel().write(buf.buf());
}
int oldLimit = buf.limit();
buf.limit(buf.position() + length);
try {
return session.getChannel().write(buf.buf());
} finally {
buf.limit(oldLimit);
}
}
So we try our best to stuff the channel with as many bytes as possible,
before giving up (either because we don't have anything to write, to
because the channel is full...)
I don't see if we can do any better.
On 1/17/12 3:57 PM, Chad Beaulac wrote:
>
> On Jan 17, 2012, at 9:32 AM, Emmanuel Lécharny<el...@apache.org> wrote:
>
>> On 1/17/12 3:02 PM, Chad Beaulac wrote:
>>> On Mon, Jan 16, 2012 at 1:10 PM, Emmanuel Lecharny<el...@gmail.com>wrote:
>>>
>>>> On 1/16/12 2:56 PM, Chad Beaulac wrote:
>>>>
>>>>> Emmanuel, (all)
>>>>>
>>>>> I'm working on this Camel ticket:
>>>>> https://issues.apache.org/**jira/browse/CAMEL-2624<https://issues.apache.org/jira/browse/CAMEL-2624>
>>>>>
>>>>> I finished the initial cut of
>>>>> https://issues.apache.org/**jira/browse/CAMEL-3471<https://issues.apache.org/jira/browse/CAMEL-3471>to create a mina2
>>>>> component in Camel.
>>>>>
>>>>> CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel.
>>>>>
>>>>> Would it be possible to backport the IoBuffer reading and writing
>>>>> discussed
>>>>> in this email thread from Mina3 to Mina2?
>>>>> Following the depth of the stack trace through
>>>>> AbstractIoSession.write(...), I'm a little concerned about the throughput.
>>>>> My current code (mina-less) is supporting single TCP channels with 320+
>>>>> Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol
>>>>> Buffers codec I wrote. I know if this is a real problem soon when I finish
>>>>> CAMEL-2624 and setup some throughput tests.
>>>>>
>>>> Another option would be to port this to MINA3, and make it work with
>>>> Camel. Right now, MINA 3 is pretty rough, but we have made it works well
>>>> with Http and LDAP. I'd rather spend some time making it a bit more solid
>>>> and working well in your case instead of trying to inject the code in MINA2.
>>>>
>>>> Now, it's your call. We can discuss the pros and cons of both approach if
>>>> you like.
>>>>
>>>>
>>> Hi Emmanuel,
>>>
>>> One of my pros/cons trade-offs is time-to-market. I can have a solution in
>>> Camel with Mina2 fairly quickly. Although I might have issues with high
>>> data rate streams.
>>> With that said, my approach would be the following:
>>> 1) Finish CAMEL-2624 with Mina2. This will give me the asynchronous
>>> endpoints I need and a quick time-to-market. I'll put off issues concerning
>>> high throughput.
>>> 2) Work on Mina3 to ensure it has low latency with small data rate streams
>>> and high throughput with large data pipes.
>>> 3) Upgrade my Google protocol buffers codec for Mina3.
>>> 4) When Mina3 is ready, open a new Camel ticket and create a new mina3
>>> Camel Component.
>>>
>>> What do you think?
>> I'll try to squeeze 2 hours to backport the patch to MINA 2 today or tomorrow.
>>
>> Feel free to ping me on mail or on #mina if I don't send any feedback in the next 2 days (I'm pretty busy and may slip)
>>
>>
>> --
>> Regards,
>> Cordialement,
>> Emmanuel Lécharny
>> www.iktek.com
>>
> Wow. That is nice! Look forward to checking it out. I'll move forward with my plan in the meantime.
>
> Chad
> Sent from my iPhone
>
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: [MINA 3.0] IoBuffer usage
Posted by Chad Beaulac <ca...@gmail.com>.
On Jan 17, 2012, at 9:32 AM, Emmanuel Lécharny <el...@apache.org> wrote:
> On 1/17/12 3:02 PM, Chad Beaulac wrote:
>> On Mon, Jan 16, 2012 at 1:10 PM, Emmanuel Lecharny<el...@gmail.com>wrote:
>>
>>> On 1/16/12 2:56 PM, Chad Beaulac wrote:
>>>
>>>> Emmanuel, (all)
>>>>
>>>> I'm working on this Camel ticket:
>>>> https://issues.apache.org/**jira/browse/CAMEL-2624<https://issues.apache.org/jira/browse/CAMEL-2624>
>>>>
>>>> I finished the initial cut of
>>>> https://issues.apache.org/**jira/browse/CAMEL-3471<https://issues.apache.org/jira/browse/CAMEL-3471>to create a mina2
>>>> component in Camel.
>>>>
>>>> CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel.
>>>>
>>>> Would it be possible to backport the IoBuffer reading and writing
>>>> discussed
>>>> in this email thread from Mina3 to Mina2?
>>>> Following the depth of the stack trace through
>>>> AbstractIoSession.write(...), I'm a little concerned about the throughput.
>>>> My current code (mina-less) is supporting single TCP channels with 320+
>>>> Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol
>>>> Buffers codec I wrote. I know if this is a real problem soon when I finish
>>>> CAMEL-2624 and setup some throughput tests.
>>>>
>>> Another option would be to port this to MINA3, and make it work with
>>> Camel. Right now, MINA 3 is pretty rough, but we have made it works well
>>> with Http and LDAP. I'd rather spend some time making it a bit more solid
>>> and working well in your case instead of trying to inject the code in MINA2.
>>>
>>> Now, it's your call. We can discuss the pros and cons of both approach if
>>> you like.
>>>
>>>
>> Hi Emmanuel,
>>
>> One of my pros/cons trade-offs is time-to-market. I can have a solution in
>> Camel with Mina2 fairly quickly. Although I might have issues with high
>> data rate streams.
>> With that said, my approach would be the following:
>> 1) Finish CAMEL-2624 with Mina2. This will give me the asynchronous
>> endpoints I need and a quick time-to-market. I'll put off issues concerning
>> high throughput.
>> 2) Work on Mina3 to ensure it has low latency with small data rate streams
>> and high throughput with large data pipes.
>> 3) Upgrade my Google protocol buffers codec for Mina3.
>> 4) When Mina3 is ready, open a new Camel ticket and create a new mina3
>> Camel Component.
>>
>> What do you think?
>
> I'll try to squeeze 2 hours to backport the patch to MINA 2 today or tomorrow.
>
> Feel free to ping me on mail or on #mina if I don't send any feedback in the next 2 days (I'm pretty busy and may slip)
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
Wow. That is nice! Look forward to checking it out. I'll move forward with my plan in the meantime.
Chad
Sent from my iPhone
Re: [MINA 3.0] IoBuffer usage
Posted by Emmanuel Lécharny <el...@apache.org>.
On 1/17/12 3:02 PM, Chad Beaulac wrote:
> On Mon, Jan 16, 2012 at 1:10 PM, Emmanuel Lecharny<el...@gmail.com>wrote:
>
>> On 1/16/12 2:56 PM, Chad Beaulac wrote:
>>
>>> Emmanuel, (all)
>>>
>>> I'm working on this Camel ticket:
>>> https://issues.apache.org/**jira/browse/CAMEL-2624<https://issues.apache.org/jira/browse/CAMEL-2624>
>>>
>>> I finished the initial cut of
>>> https://issues.apache.org/**jira/browse/CAMEL-3471<https://issues.apache.org/jira/browse/CAMEL-3471>to create a mina2
>>> component in Camel.
>>>
>>> CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel.
>>>
>>> Would it be possible to backport the IoBuffer reading and writing
>>> discussed
>>> in this email thread from Mina3 to Mina2?
>>> Following the depth of the stack trace through
>>> AbstractIoSession.write(...), I'm a little concerned about the throughput.
>>> My current code (mina-less) is supporting single TCP channels with 320+
>>> Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol
>>> Buffers codec I wrote. I know if this is a real problem soon when I finish
>>> CAMEL-2624 and setup some throughput tests.
>>>
>> Another option would be to port this to MINA3, and make it work with
>> Camel. Right now, MINA 3 is pretty rough, but we have made it works well
>> with Http and LDAP. I'd rather spend some time making it a bit more solid
>> and working well in your case instead of trying to inject the code in MINA2.
>>
>> Now, it's your call. We can discuss the pros and cons of both approach if
>> you like.
>>
>>
> Hi Emmanuel,
>
> One of my pros/cons trade-offs is time-to-market. I can have a solution in
> Camel with Mina2 fairly quickly. Although I might have issues with high
> data rate streams.
> With that said, my approach would be the following:
> 1) Finish CAMEL-2624 with Mina2. This will give me the asynchronous
> endpoints I need and a quick time-to-market. I'll put off issues concerning
> high throughput.
> 2) Work on Mina3 to ensure it has low latency with small data rate streams
> and high throughput with large data pipes.
> 3) Upgrade my Google protocol buffers codec for Mina3.
> 4) When Mina3 is ready, open a new Camel ticket and create a new mina3
> Camel Component.
>
> What do you think?
I'll try to squeeze 2 hours to backport the patch to MINA 2 today or
tomorrow.
Feel free to ping me on mail or on #mina if I don't send any feedback in
the next 2 days (I'm pretty busy and may slip)
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: [MINA 3.0] IoBuffer usage
Posted by Chad Beaulac <ca...@gmail.com>.
On Mon, Jan 16, 2012 at 1:10 PM, Emmanuel Lecharny <el...@gmail.com>wrote:
> On 1/16/12 2:56 PM, Chad Beaulac wrote:
>
>> Emmanuel, (all)
>>
>> I'm working on this Camel ticket:
>> https://issues.apache.org/**jira/browse/CAMEL-2624<https://issues.apache.org/jira/browse/CAMEL-2624>
>>
>> I finished the initial cut of
>> https://issues.apache.org/**jira/browse/CAMEL-3471<https://issues.apache.org/jira/browse/CAMEL-3471>to create a mina2
>> component in Camel.
>>
>> CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel.
>>
>> Would it be possible to backport the IoBuffer reading and writing
>> discussed
>> in this email thread from Mina3 to Mina2?
>> Following the depth of the stack trace through
>> AbstractIoSession.write(...), I'm a little concerned about the throughput.
>> My current code (mina-less) is supporting single TCP channels with 320+
>> Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol
>> Buffers codec I wrote. I know if this is a real problem soon when I finish
>> CAMEL-2624 and setup some throughput tests.
>>
>
> Another option would be to port this to MINA3, and make it work with
> Camel. Right now, MINA 3 is pretty rough, but we have made it works well
> with Http and LDAP. I'd rather spend some time making it a bit more solid
> and working well in your case instead of trying to inject the code in MINA2.
>
> Now, it's your call. We can discuss the pros and cons of both approach if
> you like.
>
>
Hi Emmanuel,
One of my pros/cons trade-offs is time-to-market. I can have a solution in
Camel with Mina2 fairly quickly. Although I might have issues with high
data rate streams.
With that said, my approach would be the following:
1) Finish CAMEL-2624 with Mina2. This will give me the asynchronous
endpoints I need and a quick time-to-market. I'll put off issues concerning
high throughput.
2) Work on Mina3 to ensure it has low latency with small data rate streams
and high throughput with large data pipes.
3) Upgrade my Google protocol buffers codec for Mina3.
4) When Mina3 is ready, open a new Camel ticket and create a new mina3
Camel Component.
What do you think?
Regards,
Chad
www.objectivesolutions.com
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>
Re: [MINA 3.0] IoBuffer usage
Posted by Emmanuel Lecharny <el...@gmail.com>.
On 1/16/12 2:56 PM, Chad Beaulac wrote:
> Emmanuel, (all)
>
> I'm working on this Camel ticket:
> https://issues.apache.org/jira/browse/CAMEL-2624
>
> I finished the initial cut of
> https://issues.apache.org/jira/browse/CAMEL-3471 to create a mina2
> component in Camel.
>
> CAMEL-2624 adds the full async behavior for Mina2 endpoints in Camel.
>
> Would it be possible to backport the IoBuffer reading and writing discussed
> in this email thread from Mina3 to Mina2?
> Following the depth of the stack trace through
> AbstractIoSession.write(...), I'm a little concerned about the throughput.
> My current code (mina-less) is supporting single TCP channels with 320+
> Mb/s rates. I'm porting my code to use Mina with a Mina Google Protocol
> Buffers codec I wrote. I know if this is a real problem soon when I finish
> CAMEL-2624 and setup some throughput tests.
Another option would be to port this to MINA3, and make it work with
Camel. Right now, MINA 3 is pretty rough, but we have made it works well
with Http and LDAP. I'd rather spend some time making it a bit more
solid and working well in your case instead of trying to inject the code
in MINA2.
Now, it's your call. We can discuss the pros and cons of both approach
if you like.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com