You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomee.apache.org by Jean-Louis MONTEIRO <je...@gmail.com> on 2013/12/09 13:17:22 UTC
OpenEJB Client incompatibility between 3.1.x and 4.x
Guys,
Remember Romain doing some changes in the client part in order to make the
serialization pluggeable.
He broke somehow the compatibility and Andy did some fixes by introducing
the property *openejb.client.protocol.version* (
http://tomee.apache.org/properties-listing.html) .
It looks like it does not work as expected or specified.
Any inputs guys before I dig into?
Thanks,
Jean-Louis
--
Jean-Louis
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by AndyG <an...@orprovision.com>.
Check for usage of org.apache.openejb.client.ProtocolMetaData#isAtLeast.
The chances of getting 3.x to 4.x working are slim (to none), and will
require action on the byte 'version', and this is even more useless with a
.eg 1.6 to 1.7 runtime communication due to the missing serial id's - You'll
just get a bunch of serialization errors.
I added legacy tests to itest to try and prevent this in the future.
Andy.
--
View this message in context: http://openejb.979440.n4.nabble.com/OpenEJB-Client-incompatibility-between-3-1-x-and-4-x-tp4666640p4666648.html
Sent from the OpenEJB Dev mailing list archive at Nabble.com.
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by Romain Manni-Bucau <rm...@gmail.com>.
well if both doesnt run with same java version it woudln't have worked
before too...so not a new issue ;)
Romain Manni-Bucau
Twitter: @rmannibucau
Blog: http://rmannibucau.wordpress.com/
LinkedIn: http://fr.linkedin.com/in/rmannibucau
Github: https://github.com/rmannibucau
2013/12/18 AndyG <an...@orprovision.com>:
> Shading is not not going to resolve the situation, as serialization will
> break and 3.1 - early 4.6-SNAPSHOT had virtually no serialVersionUID's at
> all. Serialization between runtime 1.6- and 1.7+ was/is completely broken at
> many levels.
>
> The only hope we had was the byte 'version', but that needed to be used
> properly from the start. How are you going to tell an older client that the
> server is going write something it knows nothing about? In this case the
> server must not write the new data, but how is the server supposed to know
> what *not* to write when it has no idea what the client version is? Chicken
> and the egg question.
>
> The only way I could see to get this working was to extract the whole
> communication layer as a versioned plugin - I toyed with that, but soon
> discovered it would be a huge amount of work. Even then I wasn't sure how I
> could swap this layer out when I had no idea what version the calling client
> was? It was my own assumption when I was weighing this up that anyone using
> 3.1 was likely to be using JRE 1.6 or older, and 4.6 was going to be 1.6 or
> newer. So this effort was going to be a wast of time with no cross JRE
> results.
>
> The way it is done now may/will eventually lead to some 'if this do that
> else do this' code, but it is hardly likely to change much so this was a
> choice. Now every request is versioned and every response is too. Everything
> that goes over the wire has a serialVersionUID, regardless of
> write/read-Object methods. With hindsight Externalizable should have been
> used *everywhere* with strong versioning.
>
> If we add protocol changes then it is up to the writer of that change to
> ensure the protocol version is updated and that the new information is only
> written to or read from clients that understand the same ProtocolMetaData
> version.
>
> Andy.
>
>
> Andy.
>
>
>
>
>
> --
> View this message in context: http://openejb.979440.n4.nabble.com/OpenEJB-Client-incompatibility-between-3-1-x-and-4-x-tp4666640p4666798.html
> Sent from the OpenEJB Dev mailing list archive at Nabble.com.
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by AndyG <an...@orprovision.com>.
Shading is not not going to resolve the situation, as serialization will
break and 3.1 - early 4.6-SNAPSHOT had virtually no serialVersionUID's at
all. Serialization between runtime 1.6- and 1.7+ was/is completely broken at
many levels.
The only hope we had was the byte 'version', but that needed to be used
properly from the start. How are you going to tell an older client that the
server is going write something it knows nothing about? In this case the
server must not write the new data, but how is the server supposed to know
what *not* to write when it has no idea what the client version is? Chicken
and the egg question.
The only way I could see to get this working was to extract the whole
communication layer as a versioned plugin - I toyed with that, but soon
discovered it would be a huge amount of work. Even then I wasn't sure how I
could swap this layer out when I had no idea what version the calling client
was? It was my own assumption when I was weighing this up that anyone using
3.1 was likely to be using JRE 1.6 or older, and 4.6 was going to be 1.6 or
newer. So this effort was going to be a wast of time with no cross JRE
results.
The way it is done now may/will eventually lead to some 'if this do that
else do this' code, but it is hardly likely to change much so this was a
choice. Now every request is versioned and every response is too. Everything
that goes over the wire has a serialVersionUID, regardless of
write/read-Object methods. With hindsight Externalizable should have been
used *everywhere* with strong versioning.
If we add protocol changes then it is up to the writer of that change to
ensure the protocol version is updated and that the new information is only
written to or read from clients that understand the same ProtocolMetaData
version.
Andy.
Andy.
--
View this message in context: http://openejb.979440.n4.nabble.com/OpenEJB-Client-incompatibility-between-3-1-x-and-4-x-tp4666640p4666798.html
Sent from the OpenEJB Dev mailing list archive at Nabble.com.
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by Jean-Louis MONTEIRO <je...@gmail.com>.
What about Java Serialization Romain. Something is not clear in my mind.
Shading ends with renaming packages basically. So when creating the intial
context, the factory must be adapted, not a big deal.
But a.MyClass does not serialize to b.MyClass, isn't it?
So it will allow to have 2 different versions of the same openejb-client in
the same JVM, but it won't help in solving serialization IMHO.
Jean-Louis
2013/12/18 Romain Manni-Bucau <rm...@gmail.com>
> if you shade the 3.1 you'll be able to use it to contact 3.1 servers
> and there would be no conflicts with the version 4...but not an
> elegant solution
> Romain Manni-Bucau
> Twitter: @rmannibucau
> Blog: http://rmannibucau.wordpress.com/
> LinkedIn: http://fr.linkedin.com/in/rmannibucau
> Github: https://github.com/rmannibucau
>
>
>
> 2013/12/18 Jean-Louis MONTEIRO <je...@gmail.com>:
> > Thanks a lot Andy for summarizing the whole story :)
> > That really helps.
> >
> > Ok so the conclusion, is that we can not have any backward compatibility
> > from 4.x client to 3.1.x servers.
> > @Romain, I don't see how a shaded version of the 3.1 client could help,
> btw
> >
> > So the only way is just to do like you did, I mean changing classes in
> > 3.1.x side to solve java serialization issues.
> >
> > Jean-Louis
> >
> >
> >
> >
> > 2013/12/10 AndyG <an...@orprovision.com>
> >
> >> So to sum up both:
> >>
> >> org.apache.openejb.client.Client#processRequest
> >>
> >> and...
> >>
> >> org.apache.openejb.server.ejbd.RequestHandler#processRequest (4
> >> Implementations all added various levels of complexity)
> >>
> >> Introduced calls that broke backwards compatablity between 3.x / 4.5.x
> and
> >> early 4.6.x-SNAPSHOT versions.
> >>
> >> org.apache.openejb.client.EJBResponse#VERSION, which was effectively
> never
> >> used correctly, could potentially be used to try and get some level of
> >> backwards compatibility, but I doubt it as the server has no idea if the
> >> calling client is 3.x / 4.5.x or early 4.6.x-SNAPSHOT versions (and
> >> serialization is also likely to fail).
> >>
> >> With 4.6.0+ we now have concrete means to ensure this does not happen
> >> again:
> >>
> >> Providing the de-serialized client ProtocolMetaData to each server
> object
> >> that goes over the wire now allows us to know on the server which client
> >> version is calling and how to respond, and serialVersionUID's are now
> also
> >> defined - The legacy tests will fail if someone forgets to follow the
> >> pattern when making changes at protocol level.
> >>
> >> And finally to respond to the initial topic - The system property
> >> 'openejb.client.protocol.version' is documented correctly (Allows new
> >> clients to potentially communicate with older servers), but it needs to
> be
> >> made more clear that this does NOT allow old clients to communicate with
> >> new
> >> servers prior to 4.6.0.
> >>
> >> Andy.
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://openejb.979440.n4.nabble.com/OpenEJB-Client-incompatibility-between-3-1-x-and-4-x-tp4666640p4666650.html
> >> Sent from the OpenEJB Dev mailing list archive at Nabble.com.
> >>
> >
> >
> >
> > --
> > Jean-Louis
>
--
Jean-Louis
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by Romain Manni-Bucau <rm...@gmail.com>.
if you shade the 3.1 you'll be able to use it to contact 3.1 servers
and there would be no conflicts with the version 4...but not an
elegant solution
Romain Manni-Bucau
Twitter: @rmannibucau
Blog: http://rmannibucau.wordpress.com/
LinkedIn: http://fr.linkedin.com/in/rmannibucau
Github: https://github.com/rmannibucau
2013/12/18 Jean-Louis MONTEIRO <je...@gmail.com>:
> Thanks a lot Andy for summarizing the whole story :)
> That really helps.
>
> Ok so the conclusion, is that we can not have any backward compatibility
> from 4.x client to 3.1.x servers.
> @Romain, I don't see how a shaded version of the 3.1 client could help, btw
>
> So the only way is just to do like you did, I mean changing classes in
> 3.1.x side to solve java serialization issues.
>
> Jean-Louis
>
>
>
>
> 2013/12/10 AndyG <an...@orprovision.com>
>
>> So to sum up both:
>>
>> org.apache.openejb.client.Client#processRequest
>>
>> and...
>>
>> org.apache.openejb.server.ejbd.RequestHandler#processRequest (4
>> Implementations all added various levels of complexity)
>>
>> Introduced calls that broke backwards compatablity between 3.x / 4.5.x and
>> early 4.6.x-SNAPSHOT versions.
>>
>> org.apache.openejb.client.EJBResponse#VERSION, which was effectively never
>> used correctly, could potentially be used to try and get some level of
>> backwards compatibility, but I doubt it as the server has no idea if the
>> calling client is 3.x / 4.5.x or early 4.6.x-SNAPSHOT versions (and
>> serialization is also likely to fail).
>>
>> With 4.6.0+ we now have concrete means to ensure this does not happen
>> again:
>>
>> Providing the de-serialized client ProtocolMetaData to each server object
>> that goes over the wire now allows us to know on the server which client
>> version is calling and how to respond, and serialVersionUID's are now also
>> defined - The legacy tests will fail if someone forgets to follow the
>> pattern when making changes at protocol level.
>>
>> And finally to respond to the initial topic - The system property
>> 'openejb.client.protocol.version' is documented correctly (Allows new
>> clients to potentially communicate with older servers), but it needs to be
>> made more clear that this does NOT allow old clients to communicate with
>> new
>> servers prior to 4.6.0.
>>
>> Andy.
>>
>>
>>
>> --
>> View this message in context:
>> http://openejb.979440.n4.nabble.com/OpenEJB-Client-incompatibility-between-3-1-x-and-4-x-tp4666640p4666650.html
>> Sent from the OpenEJB Dev mailing list archive at Nabble.com.
>>
>
>
>
> --
> Jean-Louis
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by Jean-Louis MONTEIRO <je...@gmail.com>.
Thanks a lot Andy for summarizing the whole story :)
That really helps.
Ok so the conclusion, is that we can not have any backward compatibility
from 4.x client to 3.1.x servers.
@Romain, I don't see how a shaded version of the 3.1 client could help, btw
So the only way is just to do like you did, I mean changing classes in
3.1.x side to solve java serialization issues.
Jean-Louis
2013/12/10 AndyG <an...@orprovision.com>
> So to sum up both:
>
> org.apache.openejb.client.Client#processRequest
>
> and...
>
> org.apache.openejb.server.ejbd.RequestHandler#processRequest (4
> Implementations all added various levels of complexity)
>
> Introduced calls that broke backwards compatablity between 3.x / 4.5.x and
> early 4.6.x-SNAPSHOT versions.
>
> org.apache.openejb.client.EJBResponse#VERSION, which was effectively never
> used correctly, could potentially be used to try and get some level of
> backwards compatibility, but I doubt it as the server has no idea if the
> calling client is 3.x / 4.5.x or early 4.6.x-SNAPSHOT versions (and
> serialization is also likely to fail).
>
> With 4.6.0+ we now have concrete means to ensure this does not happen
> again:
>
> Providing the de-serialized client ProtocolMetaData to each server object
> that goes over the wire now allows us to know on the server which client
> version is calling and how to respond, and serialVersionUID's are now also
> defined - The legacy tests will fail if someone forgets to follow the
> pattern when making changes at protocol level.
>
> And finally to respond to the initial topic - The system property
> 'openejb.client.protocol.version' is documented correctly (Allows new
> clients to potentially communicate with older servers), but it needs to be
> made more clear that this does NOT allow old clients to communicate with
> new
> servers prior to 4.6.0.
>
> Andy.
>
>
>
> --
> View this message in context:
> http://openejb.979440.n4.nabble.com/OpenEJB-Client-incompatibility-between-3-1-x-and-4-x-tp4666640p4666650.html
> Sent from the OpenEJB Dev mailing list archive at Nabble.com.
>
--
Jean-Louis
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by David Blevins <da...@gmail.com>.
Adding some notes as the person who made this mess :)
What we had there was very much the case of "someday we can use this", but someday never came till Andy finished it. We've historically told users the exact server and client version needed to match.
Didn't say it at the time, but *huge* thank you Andy for completing this. More than anything, thank you for adding *tests* for it! That's really the critical part. Code without tests is "temporary in order" :)
-David
On Dec 10, 2013, at 3:50 AM, AndyG <an...@orprovision.com> wrote:
> So to sum up both:
>
> org.apache.openejb.client.Client#processRequest
>
> and...
>
> org.apache.openejb.server.ejbd.RequestHandler#processRequest (4
> Implementations all added various levels of complexity)
>
> Introduced calls that broke backwards compatablity between 3.x / 4.5.x and
> early 4.6.x-SNAPSHOT versions.
>
> org.apache.openejb.client.EJBResponse#VERSION, which was effectively never
> used correctly, could potentially be used to try and get some level of
> backwards compatibility, but I doubt it as the server has no idea if the
> calling client is 3.x / 4.5.x or early 4.6.x-SNAPSHOT versions (and
> serialization is also likely to fail).
>
> With 4.6.0+ we now have concrete means to ensure this does not happen again:
>
> Providing the de-serialized client ProtocolMetaData to each server object
> that goes over the wire now allows us to know on the server which client
> version is calling and how to respond, and serialVersionUID's are now also
> defined - The legacy tests will fail if someone forgets to follow the
> pattern when making changes at protocol level.
>
> And finally to respond to the initial topic - The system property
> 'openejb.client.protocol.version' is documented correctly (Allows new
> clients to potentially communicate with older servers), but it needs to be
> made more clear that this does NOT allow old clients to communicate with new
> servers prior to 4.6.0.
>
> Andy.
>
>
>
> --
> View this message in context: http://openejb.979440.n4.nabble.com/OpenEJB-Client-incompatibility-between-3-1-x-and-4-x-tp4666640p4666650.html
> Sent from the OpenEJB Dev mailing list archive at Nabble.com.
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by AndyG <an...@orprovision.com>.
So to sum up both:
org.apache.openejb.client.Client#processRequest
and...
org.apache.openejb.server.ejbd.RequestHandler#processRequest (4
Implementations all added various levels of complexity)
Introduced calls that broke backwards compatablity between 3.x / 4.5.x and
early 4.6.x-SNAPSHOT versions.
org.apache.openejb.client.EJBResponse#VERSION, which was effectively never
used correctly, could potentially be used to try and get some level of
backwards compatibility, but I doubt it as the server has no idea if the
calling client is 3.x / 4.5.x or early 4.6.x-SNAPSHOT versions (and
serialization is also likely to fail).
With 4.6.0+ we now have concrete means to ensure this does not happen again:
Providing the de-serialized client ProtocolMetaData to each server object
that goes over the wire now allows us to know on the server which client
version is calling and how to respond, and serialVersionUID's are now also
defined - The legacy tests will fail if someone forgets to follow the
pattern when making changes at protocol level.
And finally to respond to the initial topic - The system property
'openejb.client.protocol.version' is documented correctly (Allows new
clients to potentially communicate with older servers), but it needs to be
made more clear that this does NOT allow old clients to communicate with new
servers prior to 4.6.0.
Andy.
--
View this message in context: http://openejb.979440.n4.nabble.com/OpenEJB-Client-incompatibility-between-3-1-x-and-4-x-tp4666640p4666650.html
Sent from the OpenEJB Dev mailing list archive at Nabble.com.
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by AndyG <an...@orprovision.com>.
Woa - been away over the weekend, so missed all this. Give me a moment.
I'll rush in without checking code so you get an idea, - I'll polish my
comments in minute if I have it wrong.
Newer clients can talk to older servers by setting the property, but not the
other way round.
The protocol initially had several flaws, most critical was the lack of
defined serialVersionUID which can even break between runtime minor
versions. It left us with broken serialization of many critical objects
between runtimes. The other major issue was the misuse of a 'version' byte
in all readExternal/writeExternal calls without thought given to interop
between versions.
I went back an generated serial id's on a 3.x checkout (using dk 1.5) for
everything I could find that goes over the wire, then imported all those
into corresponding objects in the 4.x - This solves serialization of objects
from a 1.7 client to a 1.5/6/7 server (and any future runtime).
However, this does not (and cannot) solve old clients (3.x, and snapshot
4.x) talking to a new server (4+). This is due to new calls and
serialization and deserialization methods introduced to the new server that
an older client has no knowledge of.
New clients can still talk to older servers by setting the protocol via
'openejb.client.protocol.version' - this was nore important to get right, as
this is the usual scenario - A client update, rather than a server update.
If a server is updated to 4.6.0, then all clients will require an update -
period. However, we now have a future safe 'ProtocolMetaData' parameter that
is passed to all 'over the wire' objects, and this can now be used for
backwards compatibility in 'both' directions.
Andy.
--
View this message in context: http://openejb.979440.n4.nabble.com/OpenEJB-Client-incompatibility-between-3-1-x-and-4-x-tp4666640p4666647.html
Sent from the OpenEJB Dev mailing list archive at Nabble.com.
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by Romain Manni-Bucau <rm...@gmail.com>.
We only support the opposite in degraded mode. The easier way to do the
other is to shade old client jar.
We loose key features if we want to support full bckwd compatibility since
the protocol was not design to be updated.
Now that's done and we can enhance it keeping compatibility for next
versions.
Last note: that s the risk with proprietary protocols ;). Thats even worse
since java version can break client/server with ejbd
Le 9 déc. 2013 22:22, "Jean-Louis MONTEIRO" <je...@gmail.com> a écrit :
> I've found a discussion between you and Andy to add backward compatibility
> between 4.5.x and 4.6.x using a property.
> But that is currently not possible to call a 3.1.x server with a 4.6.x
> client.
>
> Is there any way to restore backward compatibility?
>
> Jean-Louis
>
>
> 2013/12/9 Romain Manni-Bucau <rm...@gmail.com>
>
> > Hi
> >
> > Was tested (and is used) by at least 2 users + we have a test for it
> > (itests -> LegacyTest iirc)....so would need more details
> > Le 9 déc. 2013 13:17, "Jean-Louis MONTEIRO" <je...@gmail.com> a
> écrit :
> >
> > > Guys,
> > >
> > >
> > > Remember Romain doing some changes in the client part in order to make
> > the
> > > serialization pluggeable.
> > >
> > > He broke somehow the compatibility and Andy did some fixes by
> introducing
> > > the property *openejb.client.protocol.version* (
> > > http://tomee.apache.org/properties-listing.html) .
> > >
> > > It looks like it does not work as expected or specified.
> > >
> > > Any inputs guys before I dig into?
> > > Thanks,
> > > Jean-Louis
> > >
> > > --
> > > Jean-Louis
> > >
> >
>
>
>
> --
> Jean-Louis
>
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by Jean-Louis MONTEIRO <je...@gmail.com>.
I've found a discussion between you and Andy to add backward compatibility
between 4.5.x and 4.6.x using a property.
But that is currently not possible to call a 3.1.x server with a 4.6.x
client.
Is there any way to restore backward compatibility?
Jean-Louis
2013/12/9 Romain Manni-Bucau <rm...@gmail.com>
> Hi
>
> Was tested (and is used) by at least 2 users + we have a test for it
> (itests -> LegacyTest iirc)....so would need more details
> Le 9 déc. 2013 13:17, "Jean-Louis MONTEIRO" <je...@gmail.com> a écrit :
>
> > Guys,
> >
> >
> > Remember Romain doing some changes in the client part in order to make
> the
> > serialization pluggeable.
> >
> > He broke somehow the compatibility and Andy did some fixes by introducing
> > the property *openejb.client.protocol.version* (
> > http://tomee.apache.org/properties-listing.html) .
> >
> > It looks like it does not work as expected or specified.
> >
> > Any inputs guys before I dig into?
> > Thanks,
> > Jean-Louis
> >
> > --
> > Jean-Louis
> >
>
--
Jean-Louis
Re: OpenEJB Client incompatibility between 3.1.x and 4.x
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi
Was tested (and is used) by at least 2 users + we have a test for it
(itests -> LegacyTest iirc)....so would need more details
Le 9 déc. 2013 13:17, "Jean-Louis MONTEIRO" <je...@gmail.com> a écrit :
> Guys,
>
>
> Remember Romain doing some changes in the client part in order to make the
> serialization pluggeable.
>
> He broke somehow the compatibility and Andy did some fixes by introducing
> the property *openejb.client.protocol.version* (
> http://tomee.apache.org/properties-listing.html) .
>
> It looks like it does not work as expected or specified.
>
> Any inputs guys before I dig into?
> Thanks,
> Jean-Louis
>
> --
> Jean-Louis
>