You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by Rupert Smith <ru...@googlemail.com> on 2007/02/23 18:24:36 UTC
Re: ByteBuffer in .NET Client
I've been trying to get the .Net and Java clients passing field tables
to each other and ran into a NotImplementedException in the
Qpid.Buffer.FixedByteBuffer class in the slice method. This has lead
me to start taking a more in depth look at some of this buffer stuff,
which as you rightly point out Tomas is a bit of a mess. I'm wondering
why a FixedByteBuffer wraps a HeapByteBuffer and why we have all these
different buffer implementations. It looks to me like this stuff has
been ported over as a quick and dirty hack from Mina code.
Tomas, I'm just wondering if you've done anything with any of this yet
or just left it well alone (I certianly don't blame you)? I'm planning
to go through it all , document it so that I can comprehend it,
hopefully just make a few small changes to the field table stuff
working asap and then we can decide how to refactor it into something
sensible beyond that.
Rupert
On 1/24/07, Tomas Restrepo <to...@devdeo.com> wrote:
> Hi Robert,
>
>
> > I think the java byte buffer is a very useful abstraction when writing
> > network apps - certainly much better than dealing with byte[] directly
> > - which was one of the reasons for taking it over to .NET. I had a
> > look but could not see anything directly comparable to it in .NET (but
> > that could just be my ignorance since I am not terribly familiar with
> > .NET).
>
> Well, in .NET usually most network apps I've been involved in basically deal
> with Streams and some form of stream readers and writers (either
> TextReader/Writer derived classes or something like a BinaryReader/Writer).
> I agree the model in the Java code seems interesting, and I agree
> manipulating a bunch of byte[] arrays ain't fun, so you'll get no discussion
> from me on that.
>
> That's why I'm proposing understanding the existing code better so we can
> clean it up a .NETify it a bit, I'm certainly not proposing a complete
> replacement with a different paradigm.
>
>
> Tomas Restrepo
> tomas.restrepo@devdeo.com
> http://www.winterdom.com/weblog/
>
>
>
>
>
RE: ByteBuffer in .NET Client
Posted by Tomas Restrepo <to...@devdeo.com>.
Hi Rupert,
> I've been trying to get the .Net and Java clients passing field tables
> to each other and ran into a NotImplementedException in the
> Qpid.Buffer.FixedByteBuffer class in the slice method.
I fixed a few of those when doing my AMQTypes work, but I'm sure they were
not all.
> This has lead
> me to start taking a more in depth look at some of this buffer stuff,
> which as you rightly point out Tomas is a bit of a mess. I'm wondering
> why a FixedByteBuffer wraps a HeapByteBuffer and why we have all these
> different buffer implementations. It looks to me like this stuff has
> been ported over as a quick and dirty hack from Mina code.
Yep. That's my thought as well.
> Tomas, I'm just wondering if you've done anything with any of this yet
> or just left it well alone (I certianly don't blame you)?
There's one reason I didn't touch much of it, except the bare minimum I had
to do to get some of the native type serialization (typed and untyped
integers) to work. Some reasons why I haven't done significant work here was
that a) I'm pretty sure I didn't understand it all, b) it has 0 unit tests,
making it hard to touch without screwing things up and c) there are some
dependencies on that stuff from the generated code, which I didn't want to
break.
I do think it's a fantastic idea to get rid of any complexity there and get
something simpler and fully tested + implemented.
> I'm planning
> to go through it all , document it so that I can comprehend it,
> hopefully just make a few small changes to the field table stuff
> working asap and then we can decide how to refactor it into something
> sensible beyond that.
That sounds good. Since I did part of the field table rewrite work, I'm
fairly familiar with that part, so if you tell me more exactly what's the
problem you're having with, I'd be happy to help out in clearing it up. Ping
me via the list or my email and we can work on it together if you want (if
you want to IM or something let me know as well).
Tomas Restrepo
tomas.restrepo@devdeo.com
http://www.winterdom.com/weblog/
RE: ByteBuffer in .NET Client
Posted by Tomas Restrepo <to...@devdeo.com>.
Hi Rupert,
> Problem 1: Qpid.Client.Message.QpidHeaders is equivalent to
> org.apache.qpid.client.message.JMSHeaderAdapter in the Java code. It
> prepends the header field types onto the property names which the Java
> code no longer does. Sending a message with a field table from .Net to
> Java confuses the Java because it gets the property names wrong. I'm
> bringing QpidHeaders inline with the Java code at the moment so I'll
> patch that one soon enough.
Yep, noticed that one. But there's much more wrong there.
>
> Problem 2: Sending a header field from Java to .Net causes
> NotImplementedException at Qpid.Buffer.FixedByteBuffer, line 178
> (slice method). Not sure of the cause of this yet but will be
> investigating soon.
I know it. Implementing it with the code as is, is a lot of work.
> Yes, I see what you mean about lack of tests and dependencies. I'll
> follow your approach of changing as little as possible, and
> documenting as I go for the moment.
Rupert, if you want, I spent some time working this weekend on this and made
a lot of progress here. If you're interested, I can send you my code.
Basically, I've got an experimental patch that:
- Reimplements Qpid.Buffer almost from scratch with only what we need for
now. It's got a new set of unit tests giving close to 80% coverage for now.
It probably has bugs, but it works pretty well so far. It also implements
the necessary Slice(), fixes the coding convention names (i.e gets rid of
the java naming)
- Fixed several bugs I ran into in FieldTable while testing
- Alreary rewrites the QpidHeaders class to get rid of the type code issue
and fixes a couple more things there
- Fixes some issues in the EncodingUtils class I ran into
- Fixes some issues in how the message expiration header was getting parsed
(it was handled incorrectly which sometimes caused a NullReferenceException
to get thrown, plus, the field was of the wrong type).
With this changes, I've been able to successfully send a message from a Java
client to a .NET client and back through the Java broker and back, including
having a custom int header property. So that much works on my current
implementation.
If you're interested in it, it might be a place to start.
Tomas Restrepo
tomas.restrepo@devdeo.com
http://www.winterdom.com/weblog/
RE: ByteBuffer in .NET Client
Posted by Tomas Restrepo <to...@devdeo.com>.
Hi Rupert,
> > Tomas, when I built this using the build-dotnet11 script it complained
> > (with an error not just a warning), that exposing an unsigned byte in
> > a public API would make the library non-CLS compliant. So I commented
> > out the Get/SetSByte method declarations in IHeaders for the moment.
> > Do you think that we should be trying to keep CLS compliance?
>
> That's a good question. I think it's important, but we don't necessarily
> need to force it. We should be able to explicitly mark any problem members
> with a [CLSCompliant(false)] attribute so they don't cause trouble. I'll
> see
> about it and update the patch accordingly (and probably run it through
> FxCop
> and see what comes up).
Tool a new look at this and indeed this is going to be a bit of a problem.
The issue here is that IHeaders is an interface, so you can't selectively
mark members as "Non CLS Compliant", like you can with a class. The reason
for this is that the compiler assumes you might want to implement said
interface in a language only supporting the CLS. This is obviously not our
case, but marking the entire interface Non-CLS compliant might have other
consequences for users of the interface (i.e. client applications that
reference the message headers), so I'm not too keen on that.
There might be a way to get around it, I think, by breaking up the interface
in two (one base CLS compliant, like we have now, and another one that
inherits it with all the Non-CLS compliant members, like GetSByte() and the
like). I'll check it out tonight. I'll also fix a few FxCop warnings (can't
get rid of all unfortunately) in Qpid.Buffers.
Tomas Restrepo
tomas.restrepo@devdeo.com
http://www.winterdom.com/weblog/
RE: ByteBuffer in .NET Client
Posted by Tomas Restrepo <to...@devdeo.com>.
Hi Rupert,
> Tomas, when I built this using the build-dotnet11 script it complained
> (with an error not just a warning), that exposing an unsigned byte in
> a public API would make the library non-CLS compliant. So I commented
> out the Get/SetSByte method declarations in IHeaders for the moment.
> Do you think that we should be trying to keep CLS compliance?
That's a good question. I think it's important, but we don't necessarily
need to force it. We should be able to explicitly mark any problem members
with a [CLSCompliant(false)] attribute so they don't cause trouble. I'll see
about it and update the patch accordingly (and probably run it through FxCop
and see what comes up).
That should still allow a language only supporting CLSCompliance to access
any CLSCompliant members in the interface and ignore the rest.
Have you found anything else in the patch that needs addressing?
Tomas Restrepo
tomas.restrepo@devdeo.com
http://www.winterdom.com/weblog/
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
ByteBuffers and other patches for field tables is working nicely. Able
to pass messages with field tables between java and .net now. Thanks
Tomas.
Rupert
On 2/27/07, Tomas Restrepo <to...@devdeo.com> wrote:
> Hi Rupert,
>
> > Seems a bit of a shame to me that they didn't include unsigned types
> > in the CLS. Could be just pass unsinged bytes as shorts and so on,
> > like we will have to do in the Java code? Not a very elegant solution,
> > I know.
>
> That seems worse. That said, I think it might be a bit over the top.
>
> I think a better option would be to just have CLS-compliant methods
> available to only CLS-Compliant languages. At least for now. They can always
> use a bit of casting to get around the rest.
>
> On the other hand, at least for headers, which are externally visible, we
> could always make GetByte()/SetByte() set signed bytes underneath to make it
> compatible (by force) with the java implementation. Then again, I personally
> don't plan on using custom byte headers all that much myself :)
>
>
> Tomas Restrepo
> tomas.restrepo@devdeo.com
> http://www.winterdom.com/weblog/
>
>
>
>
>
RE: ByteBuffer in .NET Client
Posted by Tomas Restrepo <to...@devdeo.com>.
Hi Rupert,
> Seems a bit of a shame to me that they didn't include unsigned types
> in the CLS. Could be just pass unsinged bytes as shorts and so on,
> like we will have to do in the Java code? Not a very elegant solution,
> I know.
That seems worse. That said, I think it might be a bit over the top.
I think a better option would be to just have CLS-compliant methods
available to only CLS-Compliant languages. At least for now. They can always
use a bit of casting to get around the rest.
On the other hand, at least for headers, which are externally visible, we
could always make GetByte()/SetByte() set signed bytes underneath to make it
compatible (by force) with the java implementation. Then again, I personally
don't plan on using custom byte headers all that much myself :)
Tomas Restrepo
tomas.restrepo@devdeo.com
http://www.winterdom.com/weblog/
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
Seems a bit of a shame to me that they didn't include unsigned types
in the CLS. Could be just pass unsinged bytes as shorts and so on,
like we will have to do in the Java code? Not a very elegant solution,
I know.
On 2/27/07, Tomas Restrepo <to...@devdeo.com> wrote:
> Hi Rupert,
>
> > > Tomas, when I built this using the build-dotnet11 script it complained
> > > (with an error not just a warning), that exposing an unsigned byte in
> > > a public API would make the library non-CLS compliant. So I commented
> > > out the Get/SetSByte method declarations in IHeaders for the moment.
> > > Do you think that we should be trying to keep CLS compliance?
> >
> > That's a good question. I think it's important, but we don't necessarily
> > need to force it. We should be able to explicitly mark any problem members
> > with a [CLSCompliant(false)] attribute so they don't cause trouble. I'll
> > see
> > about it and update the patch accordingly (and probably run it through
> > FxCop
> > and see what comes up).
>
> Tool a new look at this and indeed this is going to be a bit of a problem.
> The issue here is that IHeaders is an interface, so you can't selectively
> mark members as "Non CLS Compliant", like you can with a class. The reason
> for this is that the compiler assumes you might want to implement said
> interface in a language only supporting the CLS. This is obviously not our
> case, but marking the entire interface Non-CLS compliant might have other
> consequences for users of the interface (i.e. client applications that
> reference the message headers), so I'm not too keen on that.
>
> There might be a way to get around it, I think, by breaking up the interface
> in two (one base CLS compliant, like we have now, and another one that
> inherits it with all the Non-CLS compliant members, like GetSByte() and the
> like). I'll check it out tonight. I'll also fix a few FxCop warnings (can't
> get rid of all unfortunately) in Qpid.Buffers.
>
>
> Tomas Restrepo
> tomas.restrepo@devdeo.com
> http://www.winterdom.com/weblog/
>
>
>
>
>
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
Also, your patch has been applied, so you can do an update now.
On 2/27/07, Rupert Smith <ru...@googlemail.com> wrote:
> Not yet, but I'm just about to test it ;-)
>
> On 2/27/07, Tomas Restrepo <to...@devdeo.com> wrote:
> > Hi Rupert,
> >
> > > Tomas, when I built this using the build-dotnet11 script it complained
> > > (with an error not just a warning), that exposing an unsigned byte in
> > > a public API would make the library non-CLS compliant. So I commented
> > > out the Get/SetSByte method declarations in IHeaders for the moment.
> > > Do you think that we should be trying to keep CLS compliance?
> >
> > That's a good question. I think it's important, but we don't necessarily
> > need to force it. We should be able to explicitly mark any problem members
> > with a [CLSCompliant(false)] attribute so they don't cause trouble. I'll see
> > about it and update the patch accordingly (and probably run it through FxCop
> > and see what comes up).
> >
> > That should still allow a language only supporting CLSCompliance to access
> > any CLSCompliant members in the interface and ignore the rest.
> >
> > Have you found anything else in the patch that needs addressing?
> >
> > Tomas Restrepo
> > tomas.restrepo@devdeo.com
> > http://www.winterdom.com/weblog/
> >
> >
> >
> >
> >
>
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
Not yet, but I'm just about to test it ;-)
On 2/27/07, Tomas Restrepo <to...@devdeo.com> wrote:
> Hi Rupert,
>
> > Tomas, when I built this using the build-dotnet11 script it complained
> > (with an error not just a warning), that exposing an unsigned byte in
> > a public API would make the library non-CLS compliant. So I commented
> > out the Get/SetSByte method declarations in IHeaders for the moment.
> > Do you think that we should be trying to keep CLS compliance?
>
> That's a good question. I think it's important, but we don't necessarily
> need to force it. We should be able to explicitly mark any problem members
> with a [CLSCompliant(false)] attribute so they don't cause trouble. I'll see
> about it and update the patch accordingly (and probably run it through FxCop
> and see what comes up).
>
> That should still allow a language only supporting CLSCompliance to access
> any CLSCompliant members in the interface and ignore the rest.
>
> Have you found anything else in the patch that needs addressing?
>
> Tomas Restrepo
> tomas.restrepo@devdeo.com
> http://www.winterdom.com/weblog/
>
>
>
>
>
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
Tomas, when I built this using the build-dotnet11 script it complained
(with an error not just a warning), that exposing an unsigned byte in
a public API would make the library non-CLS compliant. So I commented
out the Get/SetSByte method declarations in IHeaders for the moment.
Do you think that we should be trying to keep CLS compliance? If its
not important, I guess its possible to change the build-dotnet11
script to reduce this to a warning and ignore it. If retaining CLS
compliance is important, maybe we'll have to do something like we will
have to do in Java and pass the unsigned byte as a short?
Rupert
On 2/26/07, Rupert Smith <ru...@googlemail.com> wrote:
> Nice, good to see that useless stack of buffer implementations dissappear.
>
> Yeah, I was just thinking about the different byte conventions in java
> and .net. I suppose at some point the Qpid API is going to have to be
> normalized over all clients. I can imagine there being explicit
> set/getSignedByte and set/getUnsignedByte or similar because there
> isn't a default convention that agrees between the different
> languages. The slightly annoying thing with java is that an unsigned
> byte will need to be a short and so on. And then what to do with an
> unsigned long? BigInteger, I suppose.
>
> Good work with the patch, I haven't applied it yet but I'm feeling a
> lot more confident this is going to fly now.
>
> Rupert
>
> On 2/26/07, Tomas Restrepo <to...@devdeo.com> wrote:
> > Also, you'll need to run the scripts again to rebuild the generated framing
> > code form the protocol spec before the solution will compile.
> >
> >
> > > -----Original Message-----
> > > From: Tomas Restrepo [mailto:tomas.restrepo@devdeo.com]
> > > Sent: Monday, February 26, 2007 9:29 AM
> > > To: qpid-dev@incubator.apache.org
> > > Subject: RE: ByteBuffer in .NET Client
> > >
> > > Hi Rupert,
> > >
> > > > Tomas, would you be able to send your patch to this list? I can't get
> > > > to the ASF JIRA site at all at the moment.
> > >
> > > Attaching it, as I can't get to JIRA either.
> > >
> > > Some comments:
> > > - I tested and was able to interop custom message headers for Boolean,
> > > byte, short, int, long, float, double and string. One thing to note here
> > is
> > > that on the .NET side you'll need to use the new overload for GetSByte()
> > > and
> > > SetSByte() instead of GetByte/SetByte in QpidHeaders because the java code
> > > uses signed bytes. I'm not sure yet if we want to work around this issue
> > in
> > > another way.
> > >
> > > - If the patch doesn't do it, you'll have to manually delete the following
> > > files from Qpid.Buffer which are no longer needed:
> > > D Qpid.Buffer\FixedByteBuffer.cs
> > > D Qpid.Buffer\BufferDataException.cs
> > > D Qpid.Buffer\BaseByteBuffer.cs
> > > D Qpid.Buffer\HeapByteBuffer.cs
> > > D Qpid.Buffer\ByteBufferProxy.cs
> > >
> > > If you run into any issues, let me know, I'll help you out!
> > >
> > >
> > > Tomas Restrepo
> > > tomas.restrepo@devdeo.com
> > > http://www.winterdom.com/weblog/
> > >
> > >
> >
> >
> >
>
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
Nice, good to see that useless stack of buffer implementations dissappear.
Yeah, I was just thinking about the different byte conventions in java
and .net. I suppose at some point the Qpid API is going to have to be
normalized over all clients. I can imagine there being explicit
set/getSignedByte and set/getUnsignedByte or similar because there
isn't a default convention that agrees between the different
languages. The slightly annoying thing with java is that an unsigned
byte will need to be a short and so on. And then what to do with an
unsigned long? BigInteger, I suppose.
Good work with the patch, I haven't applied it yet but I'm feeling a
lot more confident this is going to fly now.
Rupert
On 2/26/07, Tomas Restrepo <to...@devdeo.com> wrote:
> Also, you'll need to run the scripts again to rebuild the generated framing
> code form the protocol spec before the solution will compile.
>
>
> > -----Original Message-----
> > From: Tomas Restrepo [mailto:tomas.restrepo@devdeo.com]
> > Sent: Monday, February 26, 2007 9:29 AM
> > To: qpid-dev@incubator.apache.org
> > Subject: RE: ByteBuffer in .NET Client
> >
> > Hi Rupert,
> >
> > > Tomas, would you be able to send your patch to this list? I can't get
> > > to the ASF JIRA site at all at the moment.
> >
> > Attaching it, as I can't get to JIRA either.
> >
> > Some comments:
> > - I tested and was able to interop custom message headers for Boolean,
> > byte, short, int, long, float, double and string. One thing to note here
> is
> > that on the .NET side you'll need to use the new overload for GetSByte()
> > and
> > SetSByte() instead of GetByte/SetByte in QpidHeaders because the java code
> > uses signed bytes. I'm not sure yet if we want to work around this issue
> in
> > another way.
> >
> > - If the patch doesn't do it, you'll have to manually delete the following
> > files from Qpid.Buffer which are no longer needed:
> > D Qpid.Buffer\FixedByteBuffer.cs
> > D Qpid.Buffer\BufferDataException.cs
> > D Qpid.Buffer\BaseByteBuffer.cs
> > D Qpid.Buffer\HeapByteBuffer.cs
> > D Qpid.Buffer\ByteBufferProxy.cs
> >
> > If you run into any issues, let me know, I'll help you out!
> >
> >
> > Tomas Restrepo
> > tomas.restrepo@devdeo.com
> > http://www.winterdom.com/weblog/
> >
> >
>
>
>
RE: ByteBuffer in .NET Client
Posted by Tomas Restrepo <to...@devdeo.com>.
Also, you'll need to run the scripts again to rebuild the generated framing
code form the protocol spec before the solution will compile.
> -----Original Message-----
> From: Tomas Restrepo [mailto:tomas.restrepo@devdeo.com]
> Sent: Monday, February 26, 2007 9:29 AM
> To: qpid-dev@incubator.apache.org
> Subject: RE: ByteBuffer in .NET Client
>
> Hi Rupert,
>
> > Tomas, would you be able to send your patch to this list? I can't get
> > to the ASF JIRA site at all at the moment.
>
> Attaching it, as I can't get to JIRA either.
>
> Some comments:
> - I tested and was able to interop custom message headers for Boolean,
> byte, short, int, long, float, double and string. One thing to note here
is
> that on the .NET side you'll need to use the new overload for GetSByte()
> and
> SetSByte() instead of GetByte/SetByte in QpidHeaders because the java code
> uses signed bytes. I'm not sure yet if we want to work around this issue
in
> another way.
>
> - If the patch doesn't do it, you'll have to manually delete the following
> files from Qpid.Buffer which are no longer needed:
> D Qpid.Buffer\FixedByteBuffer.cs
> D Qpid.Buffer\BufferDataException.cs
> D Qpid.Buffer\BaseByteBuffer.cs
> D Qpid.Buffer\HeapByteBuffer.cs
> D Qpid.Buffer\ByteBufferProxy.cs
>
> If you run into any issues, let me know, I'll help you out!
>
>
> Tomas Restrepo
> tomas.restrepo@devdeo.com
> http://www.winterdom.com/weblog/
>
>
RE: ByteBuffer in .NET Client
Posted by Tomas Restrepo <to...@devdeo.com>.
Hi Rupert,
> Tomas, would you be able to send your patch to this list? I can't get
> to the ASF JIRA site at all at the moment.
Attaching it, as I can't get to JIRA either.
Some comments:
- I tested and was able to interop custom message headers for Boolean, byte,
short, int, long, float, double and string. One thing to note here is that
on the .NET side you'll need to use the new overload for GetSByte() and
SetSByte() instead of GetByte/SetByte in QpidHeaders because the java code
uses signed bytes. I'm not sure yet if we want to work around this issue in
another way.
- If the patch doesn't do it, you'll have to manually delete the following
files from Qpid.Buffer which are no longer needed:
D Qpid.Buffer\FixedByteBuffer.cs
D Qpid.Buffer\BufferDataException.cs
D Qpid.Buffer\BaseByteBuffer.cs
D Qpid.Buffer\HeapByteBuffer.cs
D Qpid.Buffer\ByteBufferProxy.cs
If you run into any issues, let me know, I'll help you out!
Tomas Restrepo
tomas.restrepo@devdeo.com
http://www.winterdom.com/weblog/
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
Tomas, would you be able to send your patch to this list? I can't get
to the ASF JIRA site at all at the moment.
On 2/26/07, Rupert Smith <ru...@googlemail.com> wrote:
> Good stuff. I'm trying to look on Jira to see if you have any other
> outstanding patches that need to be applied, but the site seems to be
> very slow at the moment. Please remind me if there are any I have
> missed.
>
> On 2/26/07, Tomas Restrepo <to...@devdeo.com> wrote:
> > Hi Rupert,
> >
> > >
> > > I have a problem, in that I'm currently trying to make a build of this
> > > for an internal client. As I haven't really looked at this code in
> > > great depth I had to take it on faith that it works and will interop
> > > with the Java (note that I'm not blaming you for this, quite the
> > > opposite as you are doing a lot to fix and improve it, I mean that I'm
> > > relying on the code left behind by Steve Shaw being in working order -
> > > with no docs and little tests and with the Java code having evolved
> > > substantially since he stopped working on the .Net this doesn't seem
> > > likely...). No interop tests means that I had to write some on friday
> > > and then find out that it doesn't work.
> > >
> > > An 'experimental patch' at the last moment before putting out a build
> > > seems a bit hairy!
> >
> > I completely understand your dilemma and were I in your shoes, I'd say the
> > same thing :)
> >
> > We might as well be brutally honest here, and say that the .NET client isn't
> > really in a very good spot right now. It has fallen way behind the
> > development of the corresponding java code and there is a lot unimplemented
> > (or half implemented) in there. In fact, I'll be as bold to say that a lot
> > there doesn't even work even if both ends were the .NET client! It also
> > lacks a lot in unit and integration tests (I once ran the basic tests under
> > coverage to discover just about 10% of the code was getting exercised by the
> > tests... not a good place to be!).
> >
> > It doesn't quite help the fact that there is a lot more people contributing
> > to the Java code than to the .NET code for now...
> >
> > I'm fairly confident in the code I wrote this weekend in the sense that as I
> > said it has fairly good coverage and that once I plugged it into the main
> > code (which took a bit of time because of the naming changes), a lot worked
> > "right away". One of the reasons I said it was experimental is not only that
> > I just wrote the code this weekend, but also the fact that I haven't tested
> > all possible types yet in the message headers. I was kind of planning to do
> > that today. But, I can say for sure that right now at least custom integer
> > headers are going through between the Java and .NET clients which I think
> > it's a lot to say given the state of things.
> >
> > And I might as well add that indeed there is a *lot* more changes needed
> > besides the Buffer stuff. I agree I *should've* caught some of these when I
> > did the base field-table changes patch, and it was pretty unfortunate that I
> > didn't :(
> >
> > Another reason I say it is experimental is that I haven't done yet a few of
> > the "niceties", like getting the monodevelop project files up to date with
> > all the changes.
> >
> > I'm going to create a new JIRA for the patch and link it to the original
> > fieldtable type jira and upload the current patch. At least give it a quick
> > try and let me know what you think.
> >
> >
> > Tomas Restrepo
> > tomas.restrepo@devdeo.com
> > http://www.winterdom.com/weblog/
> >
> >
> >
> >
> >
>
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
Good stuff. I'm trying to look on Jira to see if you have any other
outstanding patches that need to be applied, but the site seems to be
very slow at the moment. Please remind me if there are any I have
missed.
On 2/26/07, Tomas Restrepo <to...@devdeo.com> wrote:
> Hi Rupert,
>
> >
> > I have a problem, in that I'm currently trying to make a build of this
> > for an internal client. As I haven't really looked at this code in
> > great depth I had to take it on faith that it works and will interop
> > with the Java (note that I'm not blaming you for this, quite the
> > opposite as you are doing a lot to fix and improve it, I mean that I'm
> > relying on the code left behind by Steve Shaw being in working order -
> > with no docs and little tests and with the Java code having evolved
> > substantially since he stopped working on the .Net this doesn't seem
> > likely...). No interop tests means that I had to write some on friday
> > and then find out that it doesn't work.
> >
> > An 'experimental patch' at the last moment before putting out a build
> > seems a bit hairy!
>
> I completely understand your dilemma and were I in your shoes, I'd say the
> same thing :)
>
> We might as well be brutally honest here, and say that the .NET client isn't
> really in a very good spot right now. It has fallen way behind the
> development of the corresponding java code and there is a lot unimplemented
> (or half implemented) in there. In fact, I'll be as bold to say that a lot
> there doesn't even work even if both ends were the .NET client! It also
> lacks a lot in unit and integration tests (I once ran the basic tests under
> coverage to discover just about 10% of the code was getting exercised by the
> tests... not a good place to be!).
>
> It doesn't quite help the fact that there is a lot more people contributing
> to the Java code than to the .NET code for now...
>
> I'm fairly confident in the code I wrote this weekend in the sense that as I
> said it has fairly good coverage and that once I plugged it into the main
> code (which took a bit of time because of the naming changes), a lot worked
> "right away". One of the reasons I said it was experimental is not only that
> I just wrote the code this weekend, but also the fact that I haven't tested
> all possible types yet in the message headers. I was kind of planning to do
> that today. But, I can say for sure that right now at least custom integer
> headers are going through between the Java and .NET clients which I think
> it's a lot to say given the state of things.
>
> And I might as well add that indeed there is a *lot* more changes needed
> besides the Buffer stuff. I agree I *should've* caught some of these when I
> did the base field-table changes patch, and it was pretty unfortunate that I
> didn't :(
>
> Another reason I say it is experimental is that I haven't done yet a few of
> the "niceties", like getting the monodevelop project files up to date with
> all the changes.
>
> I'm going to create a new JIRA for the patch and link it to the original
> fieldtable type jira and upload the current patch. At least give it a quick
> try and let me know what you think.
>
>
> Tomas Restrepo
> tomas.restrepo@devdeo.com
> http://www.winterdom.com/weblog/
>
>
>
>
>
RE: ByteBuffer in .NET Client
Posted by Tomas Restrepo <to...@devdeo.com>.
Hi Rupert,
>
> I have a problem, in that I'm currently trying to make a build of this
> for an internal client. As I haven't really looked at this code in
> great depth I had to take it on faith that it works and will interop
> with the Java (note that I'm not blaming you for this, quite the
> opposite as you are doing a lot to fix and improve it, I mean that I'm
> relying on the code left behind by Steve Shaw being in working order -
> with no docs and little tests and with the Java code having evolved
> substantially since he stopped working on the .Net this doesn't seem
> likely...). No interop tests means that I had to write some on friday
> and then find out that it doesn't work.
>
> An 'experimental patch' at the last moment before putting out a build
> seems a bit hairy!
I completely understand your dilemma and were I in your shoes, I'd say the
same thing :)
We might as well be brutally honest here, and say that the .NET client isn't
really in a very good spot right now. It has fallen way behind the
development of the corresponding java code and there is a lot unimplemented
(or half implemented) in there. In fact, I'll be as bold to say that a lot
there doesn't even work even if both ends were the .NET client! It also
lacks a lot in unit and integration tests (I once ran the basic tests under
coverage to discover just about 10% of the code was getting exercised by the
tests... not a good place to be!).
It doesn't quite help the fact that there is a lot more people contributing
to the Java code than to the .NET code for now...
I'm fairly confident in the code I wrote this weekend in the sense that as I
said it has fairly good coverage and that once I plugged it into the main
code (which took a bit of time because of the naming changes), a lot worked
"right away". One of the reasons I said it was experimental is not only that
I just wrote the code this weekend, but also the fact that I haven't tested
all possible types yet in the message headers. I was kind of planning to do
that today. But, I can say for sure that right now at least custom integer
headers are going through between the Java and .NET clients which I think
it's a lot to say given the state of things.
And I might as well add that indeed there is a *lot* more changes needed
besides the Buffer stuff. I agree I *should've* caught some of these when I
did the base field-table changes patch, and it was pretty unfortunate that I
didn't :(
Another reason I say it is experimental is that I haven't done yet a few of
the "niceties", like getting the monodevelop project files up to date with
all the changes.
I'm going to create a new JIRA for the patch and link it to the original
fieldtable type jira and upload the current patch. At least give it a quick
try and let me know what you think.
Tomas Restrepo
tomas.restrepo@devdeo.com
http://www.winterdom.com/weblog/
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
I have a problem, in that I'm currently trying to make a build of this
for an internal client. As I haven't really looked at this code in
great depth I had to take it on faith that it works and will interop
with the Java (note that I'm not blaming you for this, quite the
opposite as you are doing a lot to fix and improve it, I mean that I'm
relying on the code left behind by Steve Shaw being in working order -
with no docs and little tests and with the Java code having evolved
substantially since he stopped working on the .Net this doesn't seem
likely...). No interop tests means that I had to write some on friday
and then find out that it doesn't work.
An 'experimental patch' at the last moment before putting out a build
seems a bit hairy! But I'm also looking at the existing code and
begining to realise that there are a lot of things needed to get this
working and that it just isn't in a fit state for giving out a build
other than for experimental purposes. I think I may as well take your
patch and go from there, as I'm going to have to fix those things
anyway. I have an implementation of the topic listener and publisher
(which are the current interop tests capable of talking to the c++ and
java clients) so I can try those with your patch. I'll get the tests
and patch submitted too, so we have a common code base to work from.
Suggest you attach your patch to the most appropriate Jira or create a
new one for it.
On 2/26/07, Tomas Restrepo <to...@devdeo.com> wrote:
> Hi Rupert,
>
> > Problem 1: Qpid.Client.Message.QpidHeaders is equivalent to
> > org.apache.qpid.client.message.JMSHeaderAdapter in the Java code. It
> > prepends the header field types onto the property names which the Java
> > code no longer does. Sending a message with a field table from .Net to
> > Java confuses the Java because it gets the property names wrong. I'm
> > bringing QpidHeaders inline with the Java code at the moment so I'll
> > patch that one soon enough.
>
> Yep, noticed that one. But there's much more wrong there.
>
> >
> > Problem 2: Sending a header field from Java to .Net causes
> > NotImplementedException at Qpid.Buffer.FixedByteBuffer, line 178
> > (slice method). Not sure of the cause of this yet but will be
> > investigating soon.
>
> I know it. Implementing it with the code as is, is a lot of work.
>
> > Yes, I see what you mean about lack of tests and dependencies. I'll
> > follow your approach of changing as little as possible, and
> > documenting as I go for the moment.
>
> Rupert, if you want, I spent some time working this weekend on this and made
> a lot of progress here. If you're interested, I can send you my code.
> Basically, I've got an experimental patch that:
>
> - Reimplements Qpid.Buffer almost from scratch with only what we need for
> now. It's got a new set of unit tests giving close to 80% coverage for now.
> It probably has bugs, but it works pretty well so far. It also implements
> the necessary Slice(), fixes the coding convention names (i.e gets rid of
> the java naming)
>
> - Fixed several bugs I ran into in FieldTable while testing
> - Alreary rewrites the QpidHeaders class to get rid of the type code issue
> and fixes a couple more things there
> - Fixes some issues in the EncodingUtils class I ran into
> - Fixes some issues in how the message expiration header was getting parsed
> (it was handled incorrectly which sometimes caused a NullReferenceException
> to get thrown, plus, the field was of the wrong type).
>
> With this changes, I've been able to successfully send a message from a Java
> client to a .NET client and back through the Java broker and back, including
> having a custom int header property. So that much works on my current
> implementation.
>
> If you're interested in it, it might be a place to start.
>
>
> Tomas Restrepo
> tomas.restrepo@devdeo.com
> http://www.winterdom.com/weblog/
>
>
>
>
Re: ByteBuffer in .NET Client
Posted by Rupert Smith <ru...@googlemail.com>.
Problem 1: Qpid.Client.Message.QpidHeaders is equivalent to
org.apache.qpid.client.message.JMSHeaderAdapter in the Java code. It
prepends the header field types onto the property names which the Java
code no longer does. Sending a message with a field table from .Net to
Java confuses the Java because it gets the property names wrong. I'm
bringing QpidHeaders inline with the Java code at the moment so I'll
patch that one soon enough.
Problem 2: Sending a header field from Java to .Net causes
NotImplementedException at Qpid.Buffer.FixedByteBuffer, line 178
(slice method). Not sure of the cause of this yet but will be
investigating soon.
Yes, I see what you mean about lack of tests and dependencies. I'll
follow your approach of changing as little as possible, and
documenting as I go for the moment.
Rupert
On 2/23/07, Tomas Restrepo <to...@devdeo.com> wrote:
> Hi Rupert,
>
> > I've been trying to get the .Net and Java clients passing field tables
> > to each other and ran into a NotImplementedException in the
> > Qpid.Buffer.FixedByteBuffer class in the slice method.
>
> I fixed a few of those when doing my AMQTypes work, but I'm sure they were
> not all.
>
> > This has lead
> > me to start taking a more in depth look at some of this buffer stuff,
> > which as you rightly point out Tomas is a bit of a mess. I'm wondering
> > why a FixedByteBuffer wraps a HeapByteBuffer and why we have all these
> > different buffer implementations. It looks to me like this stuff has
> > been ported over as a quick and dirty hack from Mina code.
>
> Yep. That's my thought as well.
>
> > Tomas, I'm just wondering if you've done anything with any of this yet
> > or just left it well alone (I certianly don't blame you)?
>
> There's one reason I didn't touch much of it, except the bare minimum I had
> to do to get some of the native type serialization (typed and untyped
> integers) to work. Some reasons why I haven't done significant work here was
> that a) I'm pretty sure I didn't understand it all, b) it has 0 unit tests,
> making it hard to touch without screwing things up and c) there are some
> dependencies on that stuff from the generated code, which I didn't want to
> break.
>
> I do think it's a fantastic idea to get rid of any complexity there and get
> something simpler and fully tested + implemented.
>
> > I'm planning
> > to go through it all , document it so that I can comprehend it,
> > hopefully just make a few small changes to the field table stuff
> > working asap and then we can decide how to refactor it into something
> > sensible beyond that.
>
> That sounds good. Since I did part of the field table rewrite work, I'm
> fairly familiar with that part, so if you tell me more exactly what's the
> problem you're having with, I'd be happy to help out in clearing it up. Ping
> me via the list or my email and we can work on it together if you want (if
> you want to IM or something let me know as well).
>
>
> Tomas Restrepo
> tomas.restrepo@devdeo.com
> http://www.winterdom.com/weblog/
>
>
>
>
>