You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ofbiz.apache.org by Adam Heath <do...@brainfood.com> on 2007/10/22 20:36:07 UTC

Re: svn commit: r587109 - /ofbiz/trunk/framework/base/src/base/org/ofbiz/base/util/UtilMisc.java

jaz@apache.org wrote:
> Author: jaz
> Date: Mon Oct 22 07:23:48 2007
> New Revision: 587109
> 
> URL: http://svn.apache.org/viewvc?rev=587109&view=rev
> Log:
> small change to allow ofbiz to compile again
> 
> Modified:
>     ofbiz/trunk/framework/base/src/base/org/ofbiz/base/util/UtilMisc.java
> 
> Modified: ofbiz/trunk/framework/base/src/base/org/ofbiz/base/util/UtilMisc.java
> URL: http://svn.apache.org/viewvc/ofbiz/trunk/framework/base/src/base/org/ofbiz/base/util/UtilMisc.java?rev=587109&r1=587108&r2=587109&view=diff
> ==============================================================================
> --- ofbiz/trunk/framework/base/src/base/org/ofbiz/base/util/UtilMisc.java (original)
> +++ ofbiz/trunk/framework/base/src/base/org/ofbiz/base/util/UtilMisc.java Mon Oct 22 07:23:48 2007
> @@ -146,20 +146,20 @@
>       * Create a map from passed nameX, valueX parameters
>       * @return The resulting Map
>       */
> -    public static <K, V> Map<K, V> toMap(Object... data) {
> +    public static <K, V> Map<String, V> toMap(Object... data) {
>          if (data == null) {
>              return null;
>          }
>          if (data.length == 1 && data[0] instanceof Map) {
>              // Fix for javac's broken type inferring
> -            return UtilGenerics.<K, V>checkMap(data[0]);
> +            return UtilGenerics.<String, V>checkMap(data[0]);
>          }
>          if (data.length % 2 == 1) {
>              throw new IllegalArgumentException("You must pass an even sized array to the toMap method");
>          }
> -        Map<K, V> map = FastMap.newInstance();
> +        Map<String, V> map = FastMap.newInstance();
>          for (int i = 0; i < data.length; ) {
> -            map.put((K) data[i++], (V) data[i++]);
> +            map.put((String) data[i++], (V) data[i++]);
>          }
>          return map;
>      }

This is *not* the right way to fix whatever problem you are having.
Absolutely *not* right.

What error did you get?  Everything compiles for me.


Re: svn commit: r587109 - /ofbiz/trunk/framework/base/src/base/org/ofbiz/base/util/UtilMisc.java

Posted by Adrian Crum <ad...@hlmksw.com>.
Yeah, that's starting to bug me too.

Scott Gray wrote:
> Also, is there a reason why Andrew almost never replies to this type of
> email?  I think it's important that all committers respond to concerns
> especially when they are coming from other committers.
> 
> Scott
> 
> On 23/10/2007, Adam Heath <do...@brainfood.com> wrote:
> 
>>jaz@apache.org wrote:
>>
>>>Author: jaz
>>>Date: Mon Oct 22 07:23:48 2007
>>>New Revision: 587109
>>>
>>>URL: http://svn.apache.org/viewvc?rev=587109&view=rev
>>>Log:
>>>small change to allow ofbiz to compile again
>>>
>>
>>This is *not* the right way to fix whatever problem you are having.
>>Absolutely *not* right.
>>
>>What error did you get?  Everything compiles for me.
>>
>>
> 
> 


Re: svn commit: r587109 - /ofbiz/trunk/framework/base/src/base/org/ofbiz/base/util/UtilMisc.java

Posted by Scott Gray <le...@gmail.com>.
Also, is there a reason why Andrew almost never replies to this type of
email?  I think it's important that all committers respond to concerns
especially when they are coming from other committers.

Scott

On 23/10/2007, Adam Heath <do...@brainfood.com> wrote:
>
> jaz@apache.org wrote:
> > Author: jaz
> > Date: Mon Oct 22 07:23:48 2007
> > New Revision: 587109
> >
> > URL: http://svn.apache.org/viewvc?rev=587109&view=rev
> > Log:
> > small change to allow ofbiz to compile again
> >
>
> This is *not* the right way to fix whatever problem you are having.
> Absolutely *not* right.
>
> What error did you get?  Everything compiles for me.
>
>

Re: Issues with pulling BLOG data from the database

Posted by David E Jones <jo...@hotwaxmedia.com>.
Okay, this is finally done... :)

The changes are in revision 587433, and more notes are in the commit  
comments.

This is a pretty big hole to plug, and I put a lot of effort into  
backward compatibility, so hopefully the update service  
(unwrapByteWrappers) will allow those with existing data sets to be  
able to update. Also in the interim reading old serialized  
ByteWrapper data should also work fine.

All new and updated data will be persisted as a raw byte[] data  
instead of wrapped in a serialized Java object, which makes it so  
other applications can't read it, and makes it very sensitive to Java  
serialization which isn't very reliable or consistent (which brought  
up the whole problem...).

All those little details made this implementation a bit of a run  
around, so please let me know if anyone has any issues with this.

Someone I'm working with will be testing this soon on a client  
database that has been around for a while, but if anyone runs into  
anything sooner, again please let me know.

-David


On Oct 22, 2007, at 4:21 PM, David E Jones wrote:

>
> Would be interested in feedback on this, here is what I propose:
>
> 1. revert Adam's changes to the ByteWrapper object so that data  
> stored with it can be migrated to be plain bytes instead of a  
> serialized ByteWrapper
> 2. deprecate the ByteWrapper object
> 3. change all code that uses the ByteWrapper object to just use a  
> byte[] (I think this should work fine in Java 1.5)
> 4. create a utility that unwraps all existing serialized  
> ByteWrapper objects and just drops the byte[] into the db; this  
> would have to be run for all existing blob data in any existing  
> database
>
> -David
>
>
> On Oct 22, 2007, at 3:57 PM, Tim Ruppert wrote:
>
>> I do have the zip file that I've sent along - and yes rolling back  
>> worked fine - but there is a much bigger issue 'round this  
>> one . . . :(
>>
>> Cheers,
>> Tim
>> --
>> Tim Ruppert
>> HotWax Media
>> http://www.hotwaxmedia.com
>>
>> o:801.649.6594
>> f:801.649.6595
>>
>>
>> On Oct 22, 2007, at 3:42 PM, David E Jones wrote:
>>
>>>
>>> Interesting... this is worrisome as it means something isn't  
>>> using the ByteWrapper properly. Only the bytes in the ByteWrapper  
>>> should be sent to the database, not a serialized version of the  
>>> object itself.
>>>
>>> Because this issue exists we have a problem though... updating  
>>> the ByteWrapper object will make existing database data stored  
>>> with it useless unless the ByteWrapper object NEVER changes.
>>>
>>> Tim, do you have a full stack trace on that? I'd like to trace  
>>> through the code in the content stuff as well as the database  
>>> that does both the saves and loads.
>>>
>>> As an interim fix, change the ByteWrapper.java file back to its  
>>> original version (before the enhancements that Adam did  
>>> recently). I don't know if we want to do this in the trunk, but  
>>> I'll look at that as part of the above mentioned review.
>>>
>>> -David
>>>
>>>
>>> On Oct 22, 2007, at 3:35 PM, Tim Ruppert wrote:
>>>
>>>> This was working last week before the supreme amount of  
>>>> changes.  Anybody got any ideas why this would be the case now:
>>>>
>>>> org.ofbiz.webapp.view.ViewHandlerException: Error in  
>>>> streamDataResource (Unable to read BLOB data from input stream  
>>>> while getting value : imageData [IMAGE_DATA] (1):  
>>>> java.io.InvalidClassException:  
>>>> org.ofbiz.entity.util.ByteWrapper; local class incompatible:  
>>>> stream classdesc serialVersionUID = -2019470610749810950, local  
>>>> class serialVersionUID = -7811889960144925895  
>>>> (org.ofbiz.entity.util.ByteWrapper; local class incompatible:  
>>>> stream classdesc serialVersionUID = -2019470610749810950, local  
>>>> class serialVersionUID = -7811889960144925895))
>>>>
>>>> This is on a fresh checkout, so there are no old classes, etc.
>>>>
>>>> Cheers,
>>>> Tim
>>>> --
>>>> Tim Ruppert
>>>> HotWax Media
>>>> http://www.hotwaxmedia.com
>>>>
>>>> o:801.649.6594
>>>> f:801.649.6595
>>>>
>>>>
>>>
>>
>


Re: Issues with pulling BLOG data from the database

Posted by David E Jones <jo...@hotwaxmedia.com>.
Would be interested in feedback on this, here is what I propose:

1. revert Adam's changes to the ByteWrapper object so that data  
stored with it can be migrated to be plain bytes instead of a  
serialized ByteWrapper
2. deprecate the ByteWrapper object
3. change all code that uses the ByteWrapper object to just use a byte 
[] (I think this should work fine in Java 1.5)
4. create a utility that unwraps all existing serialized ByteWrapper  
objects and just drops the byte[] into the db; this would have to be  
run for all existing blob data in any existing database

-David


On Oct 22, 2007, at 3:57 PM, Tim Ruppert wrote:

> I do have the zip file that I've sent along - and yes rolling back  
> worked fine - but there is a much bigger issue 'round this  
> one . . . :(
>
> Cheers,
> Tim
> --
> Tim Ruppert
> HotWax Media
> http://www.hotwaxmedia.com
>
> o:801.649.6594
> f:801.649.6595
>
>
> On Oct 22, 2007, at 3:42 PM, David E Jones wrote:
>
>>
>> Interesting... this is worrisome as it means something isn't using  
>> the ByteWrapper properly. Only the bytes in the ByteWrapper should  
>> be sent to the database, not a serialized version of the object  
>> itself.
>>
>> Because this issue exists we have a problem though... updating the  
>> ByteWrapper object will make existing database data stored with it  
>> useless unless the ByteWrapper object NEVER changes.
>>
>> Tim, do you have a full stack trace on that? I'd like to trace  
>> through the code in the content stuff as well as the database that  
>> does both the saves and loads.
>>
>> As an interim fix, change the ByteWrapper.java file back to its  
>> original version (before the enhancements that Adam did recently).  
>> I don't know if we want to do this in the trunk, but I'll look at  
>> that as part of the above mentioned review.
>>
>> -David
>>
>>
>> On Oct 22, 2007, at 3:35 PM, Tim Ruppert wrote:
>>
>>> This was working last week before the supreme amount of changes.   
>>> Anybody got any ideas why this would be the case now:
>>>
>>> org.ofbiz.webapp.view.ViewHandlerException: Error in  
>>> streamDataResource (Unable to read BLOB data from input stream  
>>> while getting value : imageData [IMAGE_DATA] (1):  
>>> java.io.InvalidClassException: org.ofbiz.entity.util.ByteWrapper;  
>>> local class incompatible: stream classdesc serialVersionUID =  
>>> -2019470610749810950, local class serialVersionUID =  
>>> -7811889960144925895 (org.ofbiz.entity.util.ByteWrapper; local  
>>> class incompatible: stream classdesc serialVersionUID =  
>>> -2019470610749810950, local class serialVersionUID =  
>>> -7811889960144925895))
>>>
>>> This is on a fresh checkout, so there are no old classes, etc.
>>>
>>> Cheers,
>>> Tim
>>> --
>>> Tim Ruppert
>>> HotWax Media
>>> http://www.hotwaxmedia.com
>>>
>>> o:801.649.6594
>>> f:801.649.6595
>>>
>>>
>>
>


Re: Issues with pulling BLOG data from the database

Posted by Tim Ruppert <ti...@hotwaxmedia.com>.
I do have the zip file that I've sent along - and yes rolling back  
worked fine - but there is a much bigger issue 'round this one . . . :(

Cheers,
Tim
--
Tim Ruppert
HotWax Media
http://www.hotwaxmedia.com

o:801.649.6594
f:801.649.6595


On Oct 22, 2007, at 3:42 PM, David E Jones wrote:

>
> Interesting... this is worrisome as it means something isn't using  
> the ByteWrapper properly. Only the bytes in the ByteWrapper should  
> be sent to the database, not a serialized version of the object  
> itself.
>
> Because this issue exists we have a problem though... updating the  
> ByteWrapper object will make existing database data stored with it  
> useless unless the ByteWrapper object NEVER changes.
>
> Tim, do you have a full stack trace on that? I'd like to trace  
> through the code in the content stuff as well as the database that  
> does both the saves and loads.
>
> As an interim fix, change the ByteWrapper.java file back to its  
> original version (before the enhancements that Adam did recently).  
> I don't know if we want to do this in the trunk, but I'll look at  
> that as part of the above mentioned review.
>
> -David
>
>
> On Oct 22, 2007, at 3:35 PM, Tim Ruppert wrote:
>
>> This was working last week before the supreme amount of changes.   
>> Anybody got any ideas why this would be the case now:
>>
>> org.ofbiz.webapp.view.ViewHandlerException: Error in  
>> streamDataResource (Unable to read BLOB data from input stream  
>> while getting value : imageData [IMAGE_DATA] (1):  
>> java.io.InvalidClassException: org.ofbiz.entity.util.ByteWrapper;  
>> local class incompatible: stream classdesc serialVersionUID =  
>> -2019470610749810950, local class serialVersionUID =  
>> -7811889960144925895 (org.ofbiz.entity.util.ByteWrapper; local  
>> class incompatible: stream classdesc serialVersionUID =  
>> -2019470610749810950, local class serialVersionUID =  
>> -7811889960144925895))
>>
>> This is on a fresh checkout, so there are no old classes, etc.
>>
>> Cheers,
>> Tim
>> --
>> Tim Ruppert
>> HotWax Media
>> http://www.hotwaxmedia.com
>>
>> o:801.649.6594
>> f:801.649.6595
>>
>>
>


Re: Issues with pulling BLOG data from the database

Posted by David E Jones <jo...@hotwaxmedia.com>.
On Oct 23, 2007, at 1:23 AM, Adam Heath wrote:

> David E Jones wrote:
>>
>> The REAL problem, as I see it, is that what makes it into the  
>> database
>> is not the binary content, it is the binary content in a  
>> serialized Java
>> object, which is really weird. So, I'm working on code changes to get
>> away from that and just send the bytes over.
>
> Well, we may need to look at all implementations of Serializable in  
> the
> code base for the same kind of problems.  Data gets put into the
> session, and saved between restarts.
>
> But I agree, serialized java objects in a database suck.

I'm not so worried about serialized session stuff. That isn't a great  
loss if the object is out of date as the stuff is meant to be  
temporary anyway.

For binary data in the database, my opinion is that it isn't  
acceptable...

-David


Re: Issues with pulling BLOG data from the database

Posted by Adam Heath <do...@brainfood.com>.
David E Jones wrote:
> 
> The REAL problem, as I see it, is that what makes it into the database
> is not the binary content, it is the binary content in a serialized Java
> object, which is really weird. So, I'm working on code changes to get
> away from that and just send the bytes over.

Well, we may need to look at all implementations of Serializable in the
code base for the same kind of problems.  Data gets put into the
session, and saved between restarts.

But I agree, serialized java objects in a database suck.

Re: Issues with pulling BLOG data from the database

Posted by David E Jones <jo...@hotwaxmedia.com>.
On Oct 23, 2007, at 12:32 AM, Adam Heath wrote:

> David E Jones wrote:
>>
>> Interesting... this is worrisome as it means something isn't using  
>> the
>> ByteWrapper properly. Only the bytes in the ByteWrapper should be  
>> sent
>> to the database, not a serialized version of the object itself.
>>
>> Because this issue exists we have a problem though... updating the
>> ByteWrapper object will make existing database data stored with it
>> useless unless the ByteWrapper object NEVER changes.
>>
>> Tim, do you have a full stack trace on that? I'd like to trace  
>> through
>> the code in the content stuff as well as the database that does  
>> both the
>> saves and loads.
>>
>> As an interim fix, change the ByteWrapper.java file back to its  
>> original
>> version (before the enhancements that Adam did recently). I don't  
>> know
>> if we want to do this in the trunk, but I'll look at that as part  
>> of the
>> above mentioned review.
>
> Oh bother.
>
> I added 2 new write methods.  Since ByteWrapper implements  
> Serializable,
> and does *not* set it's own serialVersionID, one is auto-generated  
> based
> on the defined method signatures, and a hash.
>
> This can be fixed, but requires implementing the hidden private  
> methods.
>    That'll fix it going forward.
>
> Since there is no set serialVersionID, each compiler is free to
> implement it's own algorithm for setting it; it's really poor to not
> have one.  I'm not sure this can be fixed in a backwards compatible  
> manner.
>
> (this goes for other serializable classes too)

The REAL problem, as I see it, is that what makes it into the  
database is not the binary content, it is the binary content in a  
serialized Java object, which is really weird. So, I'm working on  
code changes to get away from that and just send the bytes over.

To make migration possible I'm reverting the ByteWrapper object to  
it's previous state, and deprecating it.

Where an object to wrap the byte[] is needed I'm changing everything  
from using the ByteWrapper to use the java.nio.ByteBuffer (hopefully  
a little more standardized and flexible...).

-David


Re: Issues with pulling BLOG data from the database

Posted by Adam Heath <do...@brainfood.com>.
David E Jones wrote:
> 
> Interesting... this is worrisome as it means something isn't using the
> ByteWrapper properly. Only the bytes in the ByteWrapper should be sent
> to the database, not a serialized version of the object itself.
> 
> Because this issue exists we have a problem though... updating the
> ByteWrapper object will make existing database data stored with it
> useless unless the ByteWrapper object NEVER changes.
> 
> Tim, do you have a full stack trace on that? I'd like to trace through
> the code in the content stuff as well as the database that does both the
> saves and loads.
> 
> As an interim fix, change the ByteWrapper.java file back to its original
> version (before the enhancements that Adam did recently). I don't know
> if we want to do this in the trunk, but I'll look at that as part of the
> above mentioned review.

Oh bother.

I added 2 new write methods.  Since ByteWrapper implements Serializable,
and does *not* set it's own serialVersionID, one is auto-generated based
on the defined method signatures, and a hash.

This can be fixed, but requires implementing the hidden private methods.
   That'll fix it going forward.

Since there is no set serialVersionID, each compiler is free to
implement it's own algorithm for setting it; it's really poor to not
have one.  I'm not sure this can be fixed in a backwards compatible manner.

(this goes for other serializable classes too)



> 
> -David
> 
> 
> On Oct 22, 2007, at 3:35 PM, Tim Ruppert wrote:
> 
>> This was working last week before the supreme amount of changes. 
>> Anybody got any ideas why this would be the case now:
>>
>> org.ofbiz.webapp.view.ViewHandlerException: Error in
>> streamDataResource (Unable to read BLOB data from input stream while
>> getting value : imageData [IMAGE_DATA] (1):
>> java.io.InvalidClassException: org.ofbiz.entity.util.ByteWrapper;
>> local class incompatible: stream classdesc serialVersionUID =
>> -2019470610749810950, local class serialVersionUID =
>> -7811889960144925895 (org.ofbiz.entity.util.ByteWrapper; local class
>> incompatible: stream classdesc serialVersionUID =
>> -2019470610749810950, local class serialVersionUID =
>> -7811889960144925895))
>>
>> This is on a fresh checkout, so there are no old classes, etc.
>>
>> Cheers,
>> Tim
>> -- 
>> Tim Ruppert
>> HotWax Media
>> http://www.hotwaxmedia.com
>>
>> o:801.649.6594
>> f:801.649.6595
>>
>>
> 


Re: Issues with pulling BLOG data from the database

Posted by David E Jones <jo...@hotwaxmedia.com>.
Interesting... this is worrisome as it means something isn't using  
the ByteWrapper properly. Only the bytes in the ByteWrapper should be  
sent to the database, not a serialized version of the object itself.

Because this issue exists we have a problem though... updating the  
ByteWrapper object will make existing database data stored with it  
useless unless the ByteWrapper object NEVER changes.

Tim, do you have a full stack trace on that? I'd like to trace  
through the code in the content stuff as well as the database that  
does both the saves and loads.

As an interim fix, change the ByteWrapper.java file back to its  
original version (before the enhancements that Adam did recently). I  
don't know if we want to do this in the trunk, but I'll look at that  
as part of the above mentioned review.

-David


On Oct 22, 2007, at 3:35 PM, Tim Ruppert wrote:

> This was working last week before the supreme amount of changes.   
> Anybody got any ideas why this would be the case now:
>
> org.ofbiz.webapp.view.ViewHandlerException: Error in  
> streamDataResource (Unable to read BLOB data from input stream  
> while getting value : imageData [IMAGE_DATA] (1):  
> java.io.InvalidClassException: org.ofbiz.entity.util.ByteWrapper;  
> local class incompatible: stream classdesc serialVersionUID =  
> -2019470610749810950, local class serialVersionUID =  
> -7811889960144925895 (org.ofbiz.entity.util.ByteWrapper; local  
> class incompatible: stream classdesc serialVersionUID =  
> -2019470610749810950, local class serialVersionUID =  
> -7811889960144925895))
>
> This is on a fresh checkout, so there are no old classes, etc.
>
> Cheers,
> Tim
> --
> Tim Ruppert
> HotWax Media
> http://www.hotwaxmedia.com
>
> o:801.649.6594
> f:801.649.6595
>
>


Issues with pulling BLOG data from the database

Posted by Tim Ruppert <ti...@hotwaxmedia.com>.
This was working last week before the supreme amount of changes.   
Anybody got any ideas why this would be the case now:

org.ofbiz.webapp.view.ViewHandlerException: Error in  
streamDataResource (Unable to read BLOB data from input stream while  
getting value : imageData [IMAGE_DATA] (1):  
java.io.InvalidClassException: org.ofbiz.entity.util.ByteWrapper;  
local class incompatible: stream classdesc serialVersionUID =  
-2019470610749810950, local class serialVersionUID =  
-7811889960144925895 (org.ofbiz.entity.util.ByteWrapper; local class  
incompatible: stream classdesc serialVersionUID =  
-2019470610749810950, local class serialVersionUID =  
-7811889960144925895))

This is on a fresh checkout, so there are no old classes, etc.

Cheers,
Tim
--
Tim Ruppert
HotWax Media
http://www.hotwaxmedia.com

o:801.649.6594
f:801.649.6595