You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ofbiz.apache.org by Adam Heath <do...@brainfood.com> on 2012/04/19 17:19:38 UTC
recent HashCrypt changes, and using salt-based password hashing
As some may have noticed, I recently changed the way ofbiz creates
password hashes when it stores them in the database. Each time a new
password is created, a bit of randomness is used, to create a
random-length, random-content salt. This is placed at the beginning
of the hashed password, stored along with it in the database.
Then, when it comes time to compare passwords, the salt is extracted,
and used to re-hash.
If someone continously changes their password to the same value,
you'll end up getting different raw hashed values in the database. It
also increases the workload for crackers, and makes rainbow tables
much less fruitful.
I wrote this change over 2 years ago, in direct response to the jira
intrusion that happened.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Adam Heath" <do...@brainfood.com>
> On 04/19/2012 04:28 PM, Jacques Le Roux wrote:
>> Looking forward for
>> https://issues.apache.org/jira/browse/OFBIZ-1151
>> https://issues.apache.org/jira/browse/OFBIZ-2729
>> https://issues.apache.org/jira/browse/OFBIZ-3006
>
> 2729 doesn't apply for what I am doing. Here's the list of things
> that this solves:
Oops indeed sorry, I too hastily picked them up from https://issues.apache.org/jira/browse/OFBIZ-1525
2729 is "only" about permission. For that we want to introduce Shiro but that's not for a near future: we agreed on slim down
1st...
Jacques
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adrian Crum <ad...@sandglass-software.com>.
On 4/20/2012 12:23 AM, Adam Heath wrote:
> On 04/19/2012 06:13 PM, Scott Gray wrote:
>> On 20/04/2012, at 9:49 AM, Adam Heath wrote:
>>
>>> On 04/19/2012 04:28 PM, Jacques Le Roux wrote:
>>>> Looking forward for
>>>> https://issues.apache.org/jira/browse/OFBIZ-1151
>>>> https://issues.apache.org/jira/browse/OFBIZ-2729
>>>> https://issues.apache.org/jira/browse/OFBIZ-3006
>>> 2729 doesn't apply for what I am doing. Here's the list of things
>>> that this solves:
>>>
>>> 1: salt-based UserLogin.currentPassword
>>> (done, I have a fix for the recent issue locally)
>>> 2: salt-based CreditCard.cardNumber encrypted value
>>> (s/b done tonight)
>>> 3: salt-based EntityKeyStore.keyText
>>> (s/b done tonight)
>>> 4: key-encrypting-key for EntityKeyStore.keyText. The
>>> key-encrypting-key will be available somewhere in ${ofbiz.home.dir}.
>>> (need to read java security doco)
>>>
>>> I've currently rewritten a bunch of EntityCrypto already to fix the
>>> issues I posted into the jira issue. I'm now at the point I can start
>>> adding new features.
>>>
>>> This set of changes I currently have are rather straight forward, just
>>> moving code around. When I finally get around to adding the new
>>> features, then there is a very definate chance of breaking stuff.
>> Please just do your best to ensure backwards compatibility is maintained. I can't imagine anything worse than doing an upgrade and discovering that all my user passwords are no longer valid.
> Sure, I've got things done as an array of handlers now(2 current, plus
> 1 new).
>
> How do we feel about moving EntityKeyStore into a separate database?
> It'd improve the security a little bit, and is rather simple to do
> with ofbiz.
+1
I believe credit card companies split security-sensitive data across
databases in a similar way.
-Adrian
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacques Le Roux <ja...@les7arts.com>.
Initially I suggested 5 as min
Jacques
From: "Paul Foxworthy" <pa...@cohsoft.com.au>
> Hi Jacques and Adam,
>
> Yes, I understand why a salt is a good idea. As well as making a
> dictionary attack much harder, a salt ensures that if two users happen
> to choose the same password, they will have different encrypted
> passwords.
>
> Since the salt should be unique for each user, shouldn't the length be
> sufficient to ensure that's so? As it stands, a random length between
> 1 and 15 means about one-fifteenth of our users will have one
> character salts. The salt characters are drawn from a list of 64, so
> if we have more than 960 (15 * 64) users, probably two of them will
> have the same salt character. Or to put it another way, if there's
> more than 480 users, there's a better than even chance two of them
> have the same salt character. So we reduce the chance of discovering
> that two users have the same password by a factor of hundreds. Make
> the salt always four characters, and the chances are one in 16
> million.
>
> I don't see any point in a shorter salt than that.
>
> There's a discussion on the issues at
> http://stackoverflow.com/questions/184112/what-is-the-optimal-length-for-user-password-salt
>
> Cheers
>
> Paul Foxworthy
>
> On 27 April 2012 16:46, Jacques Le Roux [via OFBiz]
> <ml...@n4.nabble.com> wrote:
>> If you believe 1 is enough then +1 for me
>> Actually as it's only salt, I think it's ok
>>
>> Jacques
>>
>> From: "Adam Heath" <[hidden email]>
>>
>>> On 04/24/2012 07:49 AM, Paul Foxworthy wrote:
>>>> Hi Adam,
>>>>
>>>> Maybe I'm missing something, but if the salt is a random length and might
>>>> be
>>>> 0 characters, doesn't that mean that some passwords, randomly, won't get
>>>> the
>>>> benefit of a salt? Why not make the salt a fixed length, or a random
>>>> length
>>>> with a reasonable minimum?
>>>
>>> The key is that if you continously set the *same* password value, you
>>> will get *different* crypted output. With no salt, the same output
>>> will happen. Maybe I could have a minimum length of 1. The main
>>> thing, however, is to make it more complex for crackers to use a
>>> dictionary attack.
>>>
>>> As for having a fixed or random length, it allows for having a few
>>> more bits of randomness as part of the salt.
>>
>>
>> ________________________________
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://ofbiz.135035.n4.nabble.com/recent-HashCrypt-changes-and-using-salt-based-password-hashing-tp4571241p4591890.html
>> To unsubscribe from recent HashCrypt changes, and using salt-based password
>> hashing, click here.
>> NAML
>
>
>
> --
> Coherent Software Australia Pty Ltd
> PO Box 2773
> Cheltenham Vic 3192
> Phone: (03) 9585 6788
> Fax: (03) 9585 1086
> Web: http://www.cohsoft.com.au/
> Email: sales@cohsoft.com.au
>
> Bonsai ERP, the all-inclusive ERP system
> http://www.bonsaierp.com.au/
>
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Paul Foxworthy <pa...@cohsoft.com.au>.
Hi Jacques and Adam,
Yes, I understand why a salt is a good idea. As well as making a
dictionary attack much harder, a salt ensures that if two users happen
to choose the same password, they will have different encrypted
passwords.
Since the salt should be unique for each user, shouldn't the length be
sufficient to ensure that's so? As it stands, a random length between
1 and 15 means about one-fifteenth of our users will have one
character salts. The salt characters are drawn from a list of 64, so
if we have more than 960 (15 * 64) users, probably two of them will
have the same salt character. Or to put it another way, if there's
more than 480 users, there's a better than even chance two of them
have the same salt character. So we reduce the chance of discovering
that two users have the same password by a factor of hundreds. Make
the salt always four characters, and the chances are one in 16
million.
I don't see any point in a shorter salt than that.
There's a discussion on the issues at
http://stackoverflow.com/questions/184112/what-is-the-optimal-length-for-user-password-salt
Cheers
Paul Foxworthy
On 27 April 2012 16:46, Jacques Le Roux [via OFBiz]
<ml...@n4.nabble.com> wrote:
> If you believe 1 is enough then +1 for me
> Actually as it's only salt, I think it's ok
>
> Jacques
>
> From: "Adam Heath" <[hidden email]>
>
>> On 04/24/2012 07:49 AM, Paul Foxworthy wrote:
>>> Hi Adam,
>>>
>>> Maybe I'm missing something, but if the salt is a random length and might
>>> be
>>> 0 characters, doesn't that mean that some passwords, randomly, won't get
>>> the
>>> benefit of a salt? Why not make the salt a fixed length, or a random
>>> length
>>> with a reasonable minimum?
>>
>> The key is that if you continously set the *same* password value, you
>> will get *different* crypted output. With no salt, the same output
>> will happen. Maybe I could have a minimum length of 1. The main
>> thing, however, is to make it more complex for crackers to use a
>> dictionary attack.
>>
>> As for having a fixed or random length, it allows for having a few
>> more bits of randomness as part of the salt.
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion
> below:
> http://ofbiz.135035.n4.nabble.com/recent-HashCrypt-changes-and-using-salt-based-password-hashing-tp4571241p4591890.html
> To unsubscribe from recent HashCrypt changes, and using salt-based password
> hashing, click here.
> NAML
--
Coherent Software Australia Pty Ltd
PO Box 2773
Cheltenham Vic 3192
Phone: (03) 9585 6788
Fax: (03) 9585 1086
Web: http://www.cohsoft.com.au/
Email: sales@cohsoft.com.au
Bonsai ERP, the all-inclusive ERP system
http://www.bonsaierp.com.au/
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacques Le Roux <ja...@les7arts.com>.
If you believe 1 is enough then +1 for me
Actually as it's only salt, I think it's ok
Jacques
From: "Adam Heath" <do...@brainfood.com>
> On 04/24/2012 07:49 AM, Paul Foxworthy wrote:
>> Hi Adam,
>>
>> Maybe I'm missing something, but if the salt is a random length and might be
>> 0 characters, doesn't that mean that some passwords, randomly, won't get the
>> benefit of a salt? Why not make the salt a fixed length, or a random length
>> with a reasonable minimum?
>
> The key is that if you continously set the *same* password value, you
> will get *different* crypted output. With no salt, the same output
> will happen. Maybe I could have a minimum length of 1. The main
> thing, however, is to make it more complex for crackers to use a
> dictionary attack.
>
> As for having a fixed or random length, it allows for having a few
> more bits of randomness as part of the salt.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adam Heath <do...@brainfood.com>.
On 04/24/2012 07:49 AM, Paul Foxworthy wrote:
> Hi Adam,
>
> Maybe I'm missing something, but if the salt is a random length and might be
> 0 characters, doesn't that mean that some passwords, randomly, won't get the
> benefit of a salt? Why not make the salt a fixed length, or a random length
> with a reasonable minimum?
The key is that if you continously set the *same* password value, you
will get *different* crypted output. With no salt, the same output
will happen. Maybe I could have a minimum length of 1. The main
thing, however, is to make it more complex for crackers to use a
dictionary attack.
As for having a fixed or random length, it allows for having a few
more bits of randomness as part of the salt.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacques Le Roux <ja...@les7arts.com>.
This makes sense indeed, thanks Paul
I'd suggest 5 rather than 0? (still random length)
Jacques
From: "Paul Foxworthy" <pa...@cohsoft.com.au>
> Hi Adam,
>
> Maybe I'm missing something, but if the salt is a random length and might be
> 0 characters, doesn't that mean that some passwords, randomly, won't get the
> benefit of a salt? Why not make the salt a fixed length, or a random length
> with a reasonable minimum?
>
> Cheers
>
> Paul Foxworthy
>
>
> Adam Heath-2 wrote
>>
>> On 04/20/2012 12:53 AM, Pierre Smits wrote:
>>> Hi Adam,
>>>
>>> How would that be? That would be one per tenant in a multi-tenant setup?
>>> I
>>> can imagine in a multi-tenant setup with the db backend not on derby (as
>>> we
>>> all recommend) the upgrade/migration aspect can be enormous. Even more so
>>> in a HAFO-setup.
>>
>> Moving EntityKeyStore to a separate database would not be hard, no code
>> changes at all. Just a new entitygroup mapping, and updating
>> entityengine.xml(or TenantDataSource) to point it at a different
>> database. This would then mean running pg_dump(or whatever) would not
>> see the keys.
>>
>> I currently have the new crypto storage done. It uses base64 to store
>> the hashed keyname, the key value, and the encrypted column values
>> scattered around the database. A random-length(0-15) random-value salt
>> is pre-pended to each value during encryption, so if you continually set
>> the same value, you'll get different encrypted values.
>>
>> I do not yet have key-encrypting-key(KEK) support working. I'm
>> currently thinking there would be one 'master' KEK. This is what
>> EntityCrypto would use by default. In sub-tenant delegators, the sub
>> EntityCrypto would fetch a key from it's parent delegator. The parent
>> delegator would be using the master KEK to encode it's keys. The
>> sub-delegator would be using a unique KEK stored in the base delegator.
>> The base delegator has it's own EntityCrypto.
>>
>> So, the master KEK could be stored in entityengine.xml(base64 encoded, I
>> can provide a cmdline tool to generate it), or some other file.
>>
>
>
> -----
> --
> Coherent Software Australia Pty Ltd
> http://www.cohsoft.com.au/
>
> Bonsai ERP, the all-inclusive ERP system
> http://www.bonsaierp.com.au/
>
> --
> View this message in context:
> http://ofbiz.135035.n4.nabble.com/recent-HashCrypt-changes-and-using-salt-based-password-hashing-tp4571241p4583331.html
> Sent from the OFBiz - Dev mailing list archive at Nabble.com.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Paul Foxworthy <pa...@cohsoft.com.au>.
Hi Adam,
Maybe I'm missing something, but if the salt is a random length and might be
0 characters, doesn't that mean that some passwords, randomly, won't get the
benefit of a salt? Why not make the salt a fixed length, or a random length
with a reasonable minimum?
Cheers
Paul Foxworthy
Adam Heath-2 wrote
>
> On 04/20/2012 12:53 AM, Pierre Smits wrote:
>> Hi Adam,
>>
>> How would that be? That would be one per tenant in a multi-tenant setup?
>> I
>> can imagine in a multi-tenant setup with the db backend not on derby (as
>> we
>> all recommend) the upgrade/migration aspect can be enormous. Even more so
>> in a HAFO-setup.
>
> Moving EntityKeyStore to a separate database would not be hard, no code
> changes at all. Just a new entitygroup mapping, and updating
> entityengine.xml(or TenantDataSource) to point it at a different
> database. This would then mean running pg_dump(or whatever) would not
> see the keys.
>
> I currently have the new crypto storage done. It uses base64 to store
> the hashed keyname, the key value, and the encrypted column values
> scattered around the database. A random-length(0-15) random-value salt
> is pre-pended to each value during encryption, so if you continually set
> the same value, you'll get different encrypted values.
>
> I do not yet have key-encrypting-key(KEK) support working. I'm
> currently thinking there would be one 'master' KEK. This is what
> EntityCrypto would use by default. In sub-tenant delegators, the sub
> EntityCrypto would fetch a key from it's parent delegator. The parent
> delegator would be using the master KEK to encode it's keys. The
> sub-delegator would be using a unique KEK stored in the base delegator.
> The base delegator has it's own EntityCrypto.
>
> So, the master KEK could be stored in entityengine.xml(base64 encoded, I
> can provide a cmdline tool to generate it), or some other file.
>
-----
--
Coherent Software Australia Pty Ltd
http://www.cohsoft.com.au/
Bonsai ERP, the all-inclusive ERP system
http://www.bonsaierp.com.au/
--
View this message in context: http://ofbiz.135035.n4.nabble.com/recent-HashCrypt-changes-and-using-salt-based-password-hashing-tp4571241p4583331.html
Sent from the OFBiz - Dev mailing list archive at Nabble.com.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adam Heath <do...@brainfood.com>.
On 04/23/2012 08:40 AM, Jacopo Cappellato wrote:
> Adrian,
>
> the issue you are experiencing is interesting and I had a look at it; I was only able to recreate the same failures by changing the timeouts in UtilCacheTests.java at line 302 and 308; instead of:
>
> cache.setExpireTime(100); // line 302
> Thread.sleep(200); // line 308
>
> I set:
>
> cache.setExpireTime(100); // line 302
> Thread.sleep(99); // line 308
>
> and (of course) I got the error.
> It is kind of weird that you are getting the error with the current settings (100 and 200): is this consistent? I mean, are you able to recreate the issue frequently? did you try to increase the time for sleep:
>
> Thread.sleep(300); // line 308
Ah, brilliant. Different platforms have different minimum
time-slices. If my guess is correct, probably fixing this to run in
*all* cases is really rather tricky. I'll definiately have a look at
this, but it will take even longer to come up with a proper correct fix.
Re: framework/base tests failing
Posted by Adam Heath <do...@brainfood.com>.
On 04/24/2012 03:06 AM, Jacopo Cappellato wrote:
> This is really weird... I am wondering if there are issues caused by concurrent threads accessing the cache but the JUnit tests should be all executed in the same thread... so this should not be the case.
The tests run in the foreground, yes, but soft-references are removed
by a singleton background thread that is reading from a
ReferenceQueue, and timed entries are also removed *immediately* by a
singleton background thread.
Re: framework/base tests failing (was: recent HashCrypt changes, and using salt-based password hashing)
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
This is really weird... I am wondering if there are issues caused by concurrent threads accessing the cache but the JUnit tests should be all executed in the same thread... so this should not be the case.
Jacopo
On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
> The tests failed intermittently in the OFBiz buildbot (back when it was running). They always fail on my development machine.
>
> I tried experimenting with the sleep timing and I also replaced the Thread.sleep call with a safer version, but the tests still failed.
>
> -Adrian
>
> On 4/23/2012 2:40 PM, Jacopo Cappellato wrote:
>> Adrian,
>>
>> the issue you are experiencing is interesting and I had a look at it; I was only able to recreate the same failures by changing the timeouts in UtilCacheTests.java at line 302 and 308; instead of:
>>
>> cache.setExpireTime(100); // line 302
>> Thread.sleep(200); // line 308
>>
>> I set:
>>
>> cache.setExpireTime(100); // line 302
>> Thread.sleep(99); // line 308
>>
>> and (of course) I got the error.
>> It is kind of weird that you are getting the error with the current settings (100 and 200): is this consistent? I mean, are you able to recreate the issue frequently? did you try to increase the time for sleep:
>>
>> Thread.sleep(300); // line 308
>>
>> ?
>>
>> Regards,
>>
>> Jacopo
>>
>>
>> On Apr 22, 2012, at 10:53 AM, Adrian Crum wrote:
>>
>>> On 4/20/2012 9:00 PM, Adam Heath wrote:
>>>> On 04/20/2012 12:13 PM, Jacques Le Roux wrote:
>>>>> Not sure you noticed but Builbot is no longer running
>>>>> http://ci.apache.org/waterfall?show_events=false&branch=&builder=ofbiz-trunk&reload=none
>>>> Yeah, noticed that buildbot wasn't emailing. Looking at that link, it
>>>> appears that isis_ubuntu is offline.
>>>>
>>>> I always run all my changes thru a full clean/test run before
>>>> committing(even when I commit> 10 at a time).
>>> Adam,
>>>
>>> If it's not too much trouble, could you look at
>>>
>>> https://issues.apache.org/jira/browse/OFBIZ-4239 ?
>>>
>>> Buildbot was failing the framework/base tests intermittently. I think I tracked the problem down to a timing issue in the UtilCache tests - see the attached test report and my comments for details.
>>>
>>> -Adrian
>>>
Re: framework/base tests failing
Posted by Adam Heath <do...@brainfood.com>.
On 05/03/2012 04:44 AM, Adrian Crum wrote:
> I agree that it is a workaround and not a solution. The problem is,
> the tests fail 100% of the time on my development machine, and they
> fail intermittently on the ASF CI machine and our (1Tech) CI machine.
> So, the workaround is needed to get various CI systems working again.
>
> Increasing the timeout fixed two of the three failing tests.
All the times in that file are finely tuned tho, to not be too large,
and the test takes a long time, but still be the logical values, based
on ttls in the code being tested.
I'll queue this up to look at, after the kek stuff is done(so very
close on this), and before backporting the view-condition new-feature.
Re: framework/base tests failing
Posted by Adrian Crum <ad...@sandglass-software.com>.
I agree that it is a workaround and not a solution. The problem is, the
tests fail 100% of the time on my development machine, and they fail
intermittently on the ASF CI machine and our (1Tech) CI machine. So, the
workaround is needed to get various CI systems working again.
Increasing the timeout fixed two of the three failing tests.
-Adrian
On 5/2/2012 10:32 PM, Adam Heath wrote:
> [trying to stop this thread a bit]
>
> If the root cause *is* a time-slice issue, the fix is to *not*
> increase the timeout. Dead stop. Changing the timeout will just
> cause it to again fail at some point in the future when computers get
> faster yet again.
>
> The correct fix is to make the timeout *not matter*. However, writing
> such test cases is *extremely* hard; that's why the current tests use
> timeouts, because they are much simpler to write, and understand by
> mortals.
>
> I can fix the tests, but it will take a *long* time; and sometimes it
> requires adding correct non-polling/event-dispatch to tons of other
> classes.
>
> On 04/30/2012 07:48 AM, Jacopo Cappellato wrote:
>> Pierre,
>>
>> please also consider that the dev list should be used by OFBiz committers to discuss about development and project related tasks; we are happy if non committers follow the discussions and participate to votes (non binding votes) but they should limit the number of posts in the dev list and most of all avoid to argue with committers (to avoid confusion and waste of time of expert resources).
>>
>> Kind regards,
>>
>> Jacopo
>>
>> On Apr 30, 2012, at 2:36 PM, Adrian Crum wrote:
>>
>>> "whereby end-users can tweak this in there own environment (by e.g. a configuration setting)"
>>>
>>> There has been plenty of discussion on this already. Please read the previous replies, and the Jira issue mentioned in the replies.
>>>
>>> -Adrian
>>>
>>> On 4/30/2012 1:33 PM, Pierre Smits wrote:
>>>> Is it so difficult to answer the questions?
>>>>
>>>> I did not state that it should be a configuration setting. I was just
>>>> asking a few civilized questions in order to understand it more.
>>>>
>>>> Regards,
>>>>
>>>> Pierre
>>>>
>>>>
>>>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>>>>
>>>>> This is NOT a configuration issue. Please stop trying to turn it into one.
>>>>>
>>>>> -Adrian
>>>>>
>>>>>
>>>>> On 4/30/2012 1:23 PM, Pierre Smits wrote:
>>>>>
>>>>>> Adrian,
>>>>>>
>>>>>> I accept that there is a difference, but using vastly is an exaggeration.
>>>>>>
>>>>>> Are we going to provide a fix for this issue, whereby end-users can tweak
>>>>>> this in there own environment (by e.g. a configuration setting), or are we
>>>>>> just trying to find an optimal number so that these test don't fail
>>>>>> anymore?
>>>>>>
>>>>>> How dependent on the environment is OFBiz regarding these unit test?
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Pierre
>>>>>>
>>>>>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>>>>>> The two are vastly different. Configuring ports is something the end user
>>>>>>> is responsible for. Cache unit tests that are failing need to be fixed.
>>>>>>> Configuration != failed unit tests.
>>>>>>>
>>>>>>> -Adrian
>>>>>>>
>>>>>>>
>>>>>>> On 4/30/2012 12:58 PM, Pierre Smits wrote:
>>>>>>>
>>>>>>> This issue seems to be a same kind of problem as the change of test
>>>>>>>> ports
>>>>>>>> in trunk.
>>>>>>>>
>>>>>>>> Why are we so adament that end users should and must apply patches in
>>>>>>>> their
>>>>>>>> own test environment regarding test ports, while we - on the other hand
>>>>>>>> -
>>>>>>>> are trying to fix something in trunk that is along the same line?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Pierre
>>>>>>>>
>>>>>>>> 2012/4/30 Adrian Crum<adrian.crum@sandglass-**s**oftware.com<http://software.com>
>>>>>>>> <ad...@sandglass-software.com>
>>>>>>>> I will give it a try, but it will have to wait until tomorrow.
>>>>>>>>
>>>>>>>>> -Adrian
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>>>>>>>>>
>>>>>>>>> If, as Adam mentioned, it is an issue caused by the time-slice in your
>>>>>>>>>
>>>>>>>>>> box, then setting a greater timeout may fix the issue... if you will
>>>>>>>>>> be
>>>>>>>>>> able to make it work with, let's say 600 ms (or even 1s) then I would
>>>>>>>>>> like
>>>>>>>>>> to commit the change to make the test a bit more robust (even if it
>>>>>>>>>> will be
>>>>>>>>>> slower).
>>>>>>>>>>
>>>>>>>>>> Jacopo
>>>>>>>>>>
>>>>>>>>>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>>>>>>>>>
>>>>>>>>>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>>>>>>>>
>>>>>>>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>>>>>>>>> I tried experimenting with the sleep timing and I also replaced the
>>>>>>>>>>>>
>>>>>>>>>>>> Thread.sleep call with a safer version, but the tests still failed.
>>>>>>>>>>>>> interesting... but if you change the Thread.sleep timeout from 200
>>>>>>>>>>>>>
>>>>>>>>>>>> to
>>>>>>>>>>>> 2000 it works, right?
>>>>>>>>>>>>
>>>>>>>>>>>> I changed it to 300. By the way, the test finally passed for the
>>>>>>>>>>>>
>>>>>>>>>>> first
>>>>>>>>>>> time when I had another non-OFBiz process running at the same time
>>>>>>>>>>> that was
>>>>>>>>>>> making heavy use of the hard disk.
>>>>>>>>>>>
>>>>>>>>>>> -Adrian
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
Re: framework/base tests failing
Posted by Adam Heath <do...@brainfood.com>.
On 05/02/2012 05:26 PM, Jacopo Cappellato wrote:
>
> On May 2, 2012, at 11:32 PM, Adam Heath wrote:
>
>> If the root cause *is* a time-slice issue, the fix is to *not*
>> increase the timeout. Dead stop. Changing the timeout will just
>> cause it to again fail at some point in the future when computers get
>> faster yet again.
>
> I would instead increase the timeout and then revisit the issue when we will run OFBiz in the computer of the future... but if you are willing to fix this without this tweak then you are most welcome.
Increasing timeouts to fix a race condition just shows there might
actually be a very rare race condition that happen in production cases.
Who here likes fixing race conditions that only happen when OOM is
occuring? It's not that there is a mem-leak, or that the problem is
OOM, but that only when an object is garbage-collected just before OOM
is thrown, and other nearby-code doesn't like the object going away,
and the race then occurs.
Re: framework/base tests failing
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
On May 2, 2012, at 11:32 PM, Adam Heath wrote:
> If the root cause *is* a time-slice issue, the fix is to *not*
> increase the timeout. Dead stop. Changing the timeout will just
> cause it to again fail at some point in the future when computers get
> faster yet again.
I would instead increase the timeout and then revisit the issue when we will run OFBiz in the computer of the future... but if you are willing to fix this without this tweak then you are most welcome.
Jacopo
Re: framework/base tests failing
Posted by Adam Heath <do...@brainfood.com>.
[trying to stop this thread a bit]
If the root cause *is* a time-slice issue, the fix is to *not*
increase the timeout. Dead stop. Changing the timeout will just
cause it to again fail at some point in the future when computers get
faster yet again.
The correct fix is to make the timeout *not matter*. However, writing
such test cases is *extremely* hard; that's why the current tests use
timeouts, because they are much simpler to write, and understand by
mortals.
I can fix the tests, but it will take a *long* time; and sometimes it
requires adding correct non-polling/event-dispatch to tons of other
classes.
On 04/30/2012 07:48 AM, Jacopo Cappellato wrote:
> Pierre,
>
> please also consider that the dev list should be used by OFBiz committers to discuss about development and project related tasks; we are happy if non committers follow the discussions and participate to votes (non binding votes) but they should limit the number of posts in the dev list and most of all avoid to argue with committers (to avoid confusion and waste of time of expert resources).
>
> Kind regards,
>
> Jacopo
>
> On Apr 30, 2012, at 2:36 PM, Adrian Crum wrote:
>
>> "whereby end-users can tweak this in there own environment (by e.g. a configuration setting)"
>>
>> There has been plenty of discussion on this already. Please read the previous replies, and the Jira issue mentioned in the replies.
>>
>> -Adrian
>>
>> On 4/30/2012 1:33 PM, Pierre Smits wrote:
>>> Is it so difficult to answer the questions?
>>>
>>> I did not state that it should be a configuration setting. I was just
>>> asking a few civilized questions in order to understand it more.
>>>
>>> Regards,
>>>
>>> Pierre
>>>
>>>
>>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>>>
>>>> This is NOT a configuration issue. Please stop trying to turn it into one.
>>>>
>>>> -Adrian
>>>>
>>>>
>>>> On 4/30/2012 1:23 PM, Pierre Smits wrote:
>>>>
>>>>> Adrian,
>>>>>
>>>>> I accept that there is a difference, but using vastly is an exaggeration.
>>>>>
>>>>> Are we going to provide a fix for this issue, whereby end-users can tweak
>>>>> this in there own environment (by e.g. a configuration setting), or are we
>>>>> just trying to find an optimal number so that these test don't fail
>>>>> anymore?
>>>>>
>>>>> How dependent on the environment is OFBiz regarding these unit test?
>>>>>
>>>>> Regards,
>>>>>
>>>>> Pierre
>>>>>
>>>>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>>>>> The two are vastly different. Configuring ports is something the end user
>>>>>> is responsible for. Cache unit tests that are failing need to be fixed.
>>>>>> Configuration != failed unit tests.
>>>>>>
>>>>>> -Adrian
>>>>>>
>>>>>>
>>>>>> On 4/30/2012 12:58 PM, Pierre Smits wrote:
>>>>>>
>>>>>> This issue seems to be a same kind of problem as the change of test
>>>>>>> ports
>>>>>>> in trunk.
>>>>>>>
>>>>>>> Why are we so adament that end users should and must apply patches in
>>>>>>> their
>>>>>>> own test environment regarding test ports, while we - on the other hand
>>>>>>> -
>>>>>>> are trying to fix something in trunk that is along the same line?
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Pierre
>>>>>>>
>>>>>>> 2012/4/30 Adrian Crum<adrian.crum@sandglass-**s**oftware.com<http://software.com>
>>>>>>> <ad...@sandglass-software.com>
>>>>>>> I will give it a try, but it will have to wait until tomorrow.
>>>>>>>
>>>>>>>> -Adrian
>>>>>>>>
>>>>>>>>
>>>>>>>> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>>>>>>>>
>>>>>>>> If, as Adam mentioned, it is an issue caused by the time-slice in your
>>>>>>>>
>>>>>>>>> box, then setting a greater timeout may fix the issue... if you will
>>>>>>>>> be
>>>>>>>>> able to make it work with, let's say 600 ms (or even 1s) then I would
>>>>>>>>> like
>>>>>>>>> to commit the change to make the test a bit more robust (even if it
>>>>>>>>> will be
>>>>>>>>> slower).
>>>>>>>>>
>>>>>>>>> Jacopo
>>>>>>>>>
>>>>>>>>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>>>>>>>>
>>>>>>>>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>>>>>>>
>>>>>>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>>>>>>>> I tried experimenting with the sleep timing and I also replaced the
>>>>>>>>>>>
>>>>>>>>>>> Thread.sleep call with a safer version, but the tests still failed.
>>>>>>>>>>>> interesting... but if you change the Thread.sleep timeout from 200
>>>>>>>>>>>>
>>>>>>>>>>> to
>>>>>>>>>>> 2000 it works, right?
>>>>>>>>>>>
>>>>>>>>>>> I changed it to 300. By the way, the test finally passed for the
>>>>>>>>>>>
>>>>>>>>>> first
>>>>>>>>>> time when I had another non-OFBiz process running at the same time
>>>>>>>>>> that was
>>>>>>>>>> making heavy use of the hard disk.
>>>>>>>>>>
>>>>>>>>>> -Adrian
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>
Re: framework/base tests failing
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
Pierre,
please also consider that the dev list should be used by OFBiz committers to discuss about development and project related tasks; we are happy if non committers follow the discussions and participate to votes (non binding votes) but they should limit the number of posts in the dev list and most of all avoid to argue with committers (to avoid confusion and waste of time of expert resources).
Kind regards,
Jacopo
On Apr 30, 2012, at 2:36 PM, Adrian Crum wrote:
> "whereby end-users can tweak this in there own environment (by e.g. a configuration setting)"
>
> There has been plenty of discussion on this already. Please read the previous replies, and the Jira issue mentioned in the replies.
>
> -Adrian
>
> On 4/30/2012 1:33 PM, Pierre Smits wrote:
>> Is it so difficult to answer the questions?
>>
>> I did not state that it should be a configuration setting. I was just
>> asking a few civilized questions in order to understand it more.
>>
>> Regards,
>>
>> Pierre
>>
>>
>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>>
>>> This is NOT a configuration issue. Please stop trying to turn it into one.
>>>
>>> -Adrian
>>>
>>>
>>> On 4/30/2012 1:23 PM, Pierre Smits wrote:
>>>
>>>> Adrian,
>>>>
>>>> I accept that there is a difference, but using vastly is an exaggeration.
>>>>
>>>> Are we going to provide a fix for this issue, whereby end-users can tweak
>>>> this in there own environment (by e.g. a configuration setting), or are we
>>>> just trying to find an optimal number so that these test don't fail
>>>> anymore?
>>>>
>>>> How dependent on the environment is OFBiz regarding these unit test?
>>>>
>>>> Regards,
>>>>
>>>> Pierre
>>>>
>>>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>>>> The two are vastly different. Configuring ports is something the end user
>>>>> is responsible for. Cache unit tests that are failing need to be fixed.
>>>>> Configuration != failed unit tests.
>>>>>
>>>>> -Adrian
>>>>>
>>>>>
>>>>> On 4/30/2012 12:58 PM, Pierre Smits wrote:
>>>>>
>>>>> This issue seems to be a same kind of problem as the change of test
>>>>>> ports
>>>>>> in trunk.
>>>>>>
>>>>>> Why are we so adament that end users should and must apply patches in
>>>>>> their
>>>>>> own test environment regarding test ports, while we - on the other hand
>>>>>> -
>>>>>> are trying to fix something in trunk that is along the same line?
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Pierre
>>>>>>
>>>>>> 2012/4/30 Adrian Crum<adrian.crum@sandglass-**s**oftware.com<http://software.com>
>>>>>> <ad...@sandglass-software.com>
>>>>>> I will give it a try, but it will have to wait until tomorrow.
>>>>>>
>>>>>>> -Adrian
>>>>>>>
>>>>>>>
>>>>>>> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>>>>>>>
>>>>>>> If, as Adam mentioned, it is an issue caused by the time-slice in your
>>>>>>>
>>>>>>>> box, then setting a greater timeout may fix the issue... if you will
>>>>>>>> be
>>>>>>>> able to make it work with, let's say 600 ms (or even 1s) then I would
>>>>>>>> like
>>>>>>>> to commit the change to make the test a bit more robust (even if it
>>>>>>>> will be
>>>>>>>> slower).
>>>>>>>>
>>>>>>>> Jacopo
>>>>>>>>
>>>>>>>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>>>>>>>
>>>>>>>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>>>>>>
>>>>>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>>>>>>> I tried experimenting with the sleep timing and I also replaced the
>>>>>>>>>>
>>>>>>>>>> Thread.sleep call with a safer version, but the tests still failed.
>>>>>>>>>>> interesting... but if you change the Thread.sleep timeout from 200
>>>>>>>>>>>
>>>>>>>>>> to
>>>>>>>>>> 2000 it works, right?
>>>>>>>>>>
>>>>>>>>>> I changed it to 300. By the way, the test finally passed for the
>>>>>>>>>>
>>>>>>>>> first
>>>>>>>>> time when I had another non-OFBiz process running at the same time
>>>>>>>>> that was
>>>>>>>>> making heavy use of the hard disk.
>>>>>>>>>
>>>>>>>>> -Adrian
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
Re: framework/base tests failing
Posted by Adrian Crum <ad...@sandglass-software.com>.
"whereby end-users can tweak this in there own environment (by e.g. a
configuration setting)"
There has been plenty of discussion on this already. Please read the
previous replies, and the Jira issue mentioned in the replies.
-Adrian
On 4/30/2012 1:33 PM, Pierre Smits wrote:
> Is it so difficult to answer the questions?
>
> I did not state that it should be a configuration setting. I was just
> asking a few civilized questions in order to understand it more.
>
> Regards,
>
> Pierre
>
>
> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>
>> This is NOT a configuration issue. Please stop trying to turn it into one.
>>
>> -Adrian
>>
>>
>> On 4/30/2012 1:23 PM, Pierre Smits wrote:
>>
>>> Adrian,
>>>
>>> I accept that there is a difference, but using vastly is an exaggeration.
>>>
>>> Are we going to provide a fix for this issue, whereby end-users can tweak
>>> this in there own environment (by e.g. a configuration setting), or are we
>>> just trying to find an optimal number so that these test don't fail
>>> anymore?
>>>
>>> How dependent on the environment is OFBiz regarding these unit test?
>>>
>>> Regards,
>>>
>>> Pierre
>>>
>>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>>> The two are vastly different. Configuring ports is something the end user
>>>> is responsible for. Cache unit tests that are failing need to be fixed.
>>>> Configuration != failed unit tests.
>>>>
>>>> -Adrian
>>>>
>>>>
>>>> On 4/30/2012 12:58 PM, Pierre Smits wrote:
>>>>
>>>> This issue seems to be a same kind of problem as the change of test
>>>>> ports
>>>>> in trunk.
>>>>>
>>>>> Why are we so adament that end users should and must apply patches in
>>>>> their
>>>>> own test environment regarding test ports, while we - on the other hand
>>>>> -
>>>>> are trying to fix something in trunk that is along the same line?
>>>>>
>>>>> Regards,
>>>>>
>>>>> Pierre
>>>>>
>>>>> 2012/4/30 Adrian Crum<adrian.crum@sandglass-**s**oftware.com<http://software.com>
>>>>> <ad...@sandglass-software.com>
>>>>> I will give it a try, but it will have to wait until tomorrow.
>>>>>
>>>>>> -Adrian
>>>>>>
>>>>>>
>>>>>> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>>>>>>
>>>>>> If, as Adam mentioned, it is an issue caused by the time-slice in your
>>>>>>
>>>>>>> box, then setting a greater timeout may fix the issue... if you will
>>>>>>> be
>>>>>>> able to make it work with, let's say 600 ms (or even 1s) then I would
>>>>>>> like
>>>>>>> to commit the change to make the test a bit more robust (even if it
>>>>>>> will be
>>>>>>> slower).
>>>>>>>
>>>>>>> Jacopo
>>>>>>>
>>>>>>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>>>>>>
>>>>>>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>>>>>
>>>>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>>>>>> I tried experimenting with the sleep timing and I also replaced the
>>>>>>>>>
>>>>>>>>> Thread.sleep call with a safer version, but the tests still failed.
>>>>>>>>>> interesting... but if you change the Thread.sleep timeout from 200
>>>>>>>>>>
>>>>>>>>> to
>>>>>>>>> 2000 it works, right?
>>>>>>>>>
>>>>>>>>> I changed it to 300. By the way, the test finally passed for the
>>>>>>>>>
>>>>>>>> first
>>>>>>>> time when I had another non-OFBiz process running at the same time
>>>>>>>> that was
>>>>>>>> making heavy use of the hard disk.
>>>>>>>>
>>>>>>>> -Adrian
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
Re: framework/base tests failing
Posted by Pierre Smits <pi...@gmail.com>.
Is it so difficult to answer the questions?
I did not state that it should be a configuration setting. I was just
asking a few civilized questions in order to understand it more.
Regards,
Pierre
2012/4/30 Adrian Crum <ad...@sandglass-software.com>
> This is NOT a configuration issue. Please stop trying to turn it into one.
>
> -Adrian
>
>
> On 4/30/2012 1:23 PM, Pierre Smits wrote:
>
>> Adrian,
>>
>> I accept that there is a difference, but using vastly is an exaggeration.
>>
>> Are we going to provide a fix for this issue, whereby end-users can tweak
>> this in there own environment (by e.g. a configuration setting), or are we
>> just trying to find an optimal number so that these test don't fail
>> anymore?
>>
>> How dependent on the environment is OFBiz regarding these unit test?
>>
>> Regards,
>>
>> Pierre
>>
>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>> >
>>
>> The two are vastly different. Configuring ports is something the end user
>>> is responsible for. Cache unit tests that are failing need to be fixed.
>>> Configuration != failed unit tests.
>>>
>>> -Adrian
>>>
>>>
>>> On 4/30/2012 12:58 PM, Pierre Smits wrote:
>>>
>>> This issue seems to be a same kind of problem as the change of test
>>>> ports
>>>> in trunk.
>>>>
>>>> Why are we so adament that end users should and must apply patches in
>>>> their
>>>> own test environment regarding test ports, while we - on the other hand
>>>> -
>>>> are trying to fix something in trunk that is along the same line?
>>>>
>>>> Regards,
>>>>
>>>> Pierre
>>>>
>>>> 2012/4/30 Adrian Crum<adrian.crum@sandglass-**s**oftware.com<http://software.com>
>>>> <ad...@sandglass-software.com>
>>>> >
>>>>
>>>> I will give it a try, but it will have to wait until tomorrow.
>>>>
>>>>> -Adrian
>>>>>
>>>>>
>>>>> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>>>>>
>>>>> If, as Adam mentioned, it is an issue caused by the time-slice in your
>>>>>
>>>>>> box, then setting a greater timeout may fix the issue... if you will
>>>>>> be
>>>>>> able to make it work with, let's say 600 ms (or even 1s) then I would
>>>>>> like
>>>>>> to commit the change to make the test a bit more robust (even if it
>>>>>> will be
>>>>>> slower).
>>>>>>
>>>>>> Jacopo
>>>>>>
>>>>>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>>>>>
>>>>>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>>>>
>>>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>>>>
>>>>>>>> I tried experimenting with the sleep timing and I also replaced the
>>>>>>>>
>>>>>>>> Thread.sleep call with a safer version, but the tests still failed.
>>>>>>>>>
>>>>>>>>> interesting... but if you change the Thread.sleep timeout from 200
>>>>>>>>>
>>>>>>>> to
>>>>>>>> 2000 it works, right?
>>>>>>>>
>>>>>>>> I changed it to 300. By the way, the test finally passed for the
>>>>>>>>
>>>>>>> first
>>>>>>> time when I had another non-OFBiz process running at the same time
>>>>>>> that was
>>>>>>> making heavy use of the hard disk.
>>>>>>>
>>>>>>> -Adrian
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
Re: framework/base tests failing
Posted by Adrian Crum <ad...@sandglass-software.com>.
This is NOT a configuration issue. Please stop trying to turn it into one.
-Adrian
On 4/30/2012 1:23 PM, Pierre Smits wrote:
> Adrian,
>
> I accept that there is a difference, but using vastly is an exaggeration.
>
> Are we going to provide a fix for this issue, whereby end-users can tweak
> this in there own environment (by e.g. a configuration setting), or are we
> just trying to find an optimal number so that these test don't fail anymore?
>
> How dependent on the environment is OFBiz regarding these unit test?
>
> Regards,
>
> Pierre
>
> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>
>> The two are vastly different. Configuring ports is something the end user
>> is responsible for. Cache unit tests that are failing need to be fixed.
>> Configuration != failed unit tests.
>>
>> -Adrian
>>
>>
>> On 4/30/2012 12:58 PM, Pierre Smits wrote:
>>
>>> This issue seems to be a same kind of problem as the change of test ports
>>> in trunk.
>>>
>>> Why are we so adament that end users should and must apply patches in
>>> their
>>> own test environment regarding test ports, while we - on the other hand -
>>> are trying to fix something in trunk that is along the same line?
>>>
>>> Regards,
>>>
>>> Pierre
>>>
>>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>>> I will give it a try, but it will have to wait until tomorrow.
>>>> -Adrian
>>>>
>>>>
>>>> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>>>>
>>>> If, as Adam mentioned, it is an issue caused by the time-slice in your
>>>>> box, then setting a greater timeout may fix the issue... if you will be
>>>>> able to make it work with, let's say 600 ms (or even 1s) then I would
>>>>> like
>>>>> to commit the change to make the test a bit more robust (even if it
>>>>> will be
>>>>> slower).
>>>>>
>>>>> Jacopo
>>>>>
>>>>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>>>>
>>>>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>>>
>>>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>>>> I tried experimenting with the sleep timing and I also replaced the
>>>>>>>
>>>>>>>> Thread.sleep call with a safer version, but the tests still failed.
>>>>>>>>
>>>>>>>> interesting... but if you change the Thread.sleep timeout from 200
>>>>>>> to
>>>>>>> 2000 it works, right?
>>>>>>>
>>>>>>> I changed it to 300. By the way, the test finally passed for the
>>>>>> first
>>>>>> time when I had another non-OFBiz process running at the same time
>>>>>> that was
>>>>>> making heavy use of the hard disk.
>>>>>>
>>>>>> -Adrian
>>>>>>
>>>>>>
>>>>>>
Re: framework/base tests failing
Posted by Pierre Smits <pi...@gmail.com>.
Adrian,
I accept that there is a difference, but using vastly is an exaggeration.
Are we going to provide a fix for this issue, whereby end-users can tweak
this in there own environment (by e.g. a configuration setting), or are we
just trying to find an optimal number so that these test don't fail anymore?
How dependent on the environment is OFBiz regarding these unit test?
Regards,
Pierre
2012/4/30 Adrian Crum <ad...@sandglass-software.com>
> The two are vastly different. Configuring ports is something the end user
> is responsible for. Cache unit tests that are failing need to be fixed.
> Configuration != failed unit tests.
>
> -Adrian
>
>
> On 4/30/2012 12:58 PM, Pierre Smits wrote:
>
>> This issue seems to be a same kind of problem as the change of test ports
>> in trunk.
>>
>> Why are we so adament that end users should and must apply patches in
>> their
>> own test environment regarding test ports, while we - on the other hand -
>> are trying to fix something in trunk that is along the same line?
>>
>> Regards,
>>
>> Pierre
>>
>> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>> >
>>
>> I will give it a try, but it will have to wait until tomorrow.
>>>
>>> -Adrian
>>>
>>>
>>> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>>>
>>> If, as Adam mentioned, it is an issue caused by the time-slice in your
>>>> box, then setting a greater timeout may fix the issue... if you will be
>>>> able to make it work with, let's say 600 ms (or even 1s) then I would
>>>> like
>>>> to commit the change to make the test a bit more robust (even if it
>>>> will be
>>>> slower).
>>>>
>>>> Jacopo
>>>>
>>>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>>>
>>>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>>
>>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>>>
>>>>>> I tried experimenting with the sleep timing and I also replaced the
>>>>>>
>>>>>>> Thread.sleep call with a safer version, but the tests still failed.
>>>>>>>
>>>>>>> interesting... but if you change the Thread.sleep timeout from 200
>>>>>> to
>>>>>> 2000 it works, right?
>>>>>>
>>>>>> I changed it to 300. By the way, the test finally passed for the
>>>>> first
>>>>> time when I had another non-OFBiz process running at the same time
>>>>> that was
>>>>> making heavy use of the hard disk.
>>>>>
>>>>> -Adrian
>>>>>
>>>>>
>>>>>
Re: framework/base tests failing
Posted by Adrian Crum <ad...@sandglass-software.com>.
The two are vastly different. Configuring ports is something the end
user is responsible for. Cache unit tests that are failing need to be
fixed. Configuration != failed unit tests.
-Adrian
On 4/30/2012 12:58 PM, Pierre Smits wrote:
> This issue seems to be a same kind of problem as the change of test ports
> in trunk.
>
> Why are we so adament that end users should and must apply patches in their
> own test environment regarding test ports, while we - on the other hand -
> are trying to fix something in trunk that is along the same line?
>
> Regards,
>
> Pierre
>
> 2012/4/30 Adrian Crum<ad...@sandglass-software.com>
>
>> I will give it a try, but it will have to wait until tomorrow.
>>
>> -Adrian
>>
>>
>> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>>
>>> If, as Adam mentioned, it is an issue caused by the time-slice in your
>>> box, then setting a greater timeout may fix the issue... if you will be
>>> able to make it work with, let's say 600 ms (or even 1s) then I would like
>>> to commit the change to make the test a bit more robust (even if it will be
>>> slower).
>>>
>>> Jacopo
>>>
>>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>>
>>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>>
>>>>> I tried experimenting with the sleep timing and I also replaced the
>>>>>> Thread.sleep call with a safer version, but the tests still failed.
>>>>>>
>>>>> interesting... but if you change the Thread.sleep timeout from 200 to
>>>>> 2000 it works, right?
>>>>>
>>>> I changed it to 300. By the way, the test finally passed for the first
>>>> time when I had another non-OFBiz process running at the same time that was
>>>> making heavy use of the hard disk.
>>>>
>>>> -Adrian
>>>>
>>>>
Re: framework/base tests failing
Posted by Pierre Smits <pi...@gmail.com>.
This issue seems to be a same kind of problem as the change of test ports
in trunk.
Why are we so adament that end users should and must apply patches in their
own test environment regarding test ports, while we - on the other hand -
are trying to fix something in trunk that is along the same line?
Regards,
Pierre
2012/4/30 Adrian Crum <ad...@sandglass-software.com>
> I will give it a try, but it will have to wait until tomorrow.
>
> -Adrian
>
>
> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>
>> If, as Adam mentioned, it is an issue caused by the time-slice in your
>> box, then setting a greater timeout may fix the issue... if you will be
>> able to make it work with, let's say 600 ms (or even 1s) then I would like
>> to commit the change to make the test a bit more robust (even if it will be
>> slower).
>>
>> Jacopo
>>
>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>
>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>
>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>
>>>> I tried experimenting with the sleep timing and I also replaced the
>>>>> Thread.sleep call with a safer version, but the tests still failed.
>>>>>
>>>> interesting... but if you change the Thread.sleep timeout from 200 to
>>>> 2000 it works, right?
>>>>
>>> I changed it to 300. By the way, the test finally passed for the first
>>> time when I had another non-OFBiz process running at the same time that was
>>> making heavy use of the hard disk.
>>>
>>> -Adrian
>>>
>>>
Re: framework/base tests failing
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
Thank you Adrian, and of course there is no rush at all.
Jacopo
On Apr 30, 2012, at 1:45 PM, Adrian Crum wrote:
> I will give it a try, but it will have to wait until tomorrow.
>
> -Adrian
>
> On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
>> If, as Adam mentioned, it is an issue caused by the time-slice in your box, then setting a greater timeout may fix the issue... if you will be able to make it work with, let's say 600 ms (or even 1s) then I would like to commit the change to make the test a bit more robust (even if it will be slower).
>>
>> Jacopo
>>
>> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>>
>>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>>
>>>>> I tried experimenting with the sleep timing and I also replaced the Thread.sleep call with a safer version, but the tests still failed.
>>>> interesting... but if you change the Thread.sleep timeout from 200 to 2000 it works, right?
>>> I changed it to 300. By the way, the test finally passed for the first time when I had another non-OFBiz process running at the same time that was making heavy use of the hard disk.
>>>
>>> -Adrian
>>>
Re: framework/base tests failing
Posted by Adrian Crum <ad...@sandglass-software.com>.
I will give it a try, but it will have to wait until tomorrow.
-Adrian
On 4/30/2012 12:42 PM, Jacopo Cappellato wrote:
> If, as Adam mentioned, it is an issue caused by the time-slice in your box, then setting a greater timeout may fix the issue... if you will be able to make it work with, let's say 600 ms (or even 1s) then I would like to commit the change to make the test a bit more robust (even if it will be slower).
>
> Jacopo
>
> On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
>
>> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>>
>>>> I tried experimenting with the sleep timing and I also replaced the Thread.sleep call with a safer version, but the tests still failed.
>>> interesting... but if you change the Thread.sleep timeout from 200 to 2000 it works, right?
>> I changed it to 300. By the way, the test finally passed for the first time when I had another non-OFBiz process running at the same time that was making heavy use of the hard disk.
>>
>> -Adrian
>>
Re: framework/base tests failing
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
If, as Adam mentioned, it is an issue caused by the time-slice in your box, then setting a greater timeout may fix the issue... if you will be able to make it work with, let's say 600 ms (or even 1s) then I would like to commit the change to make the test a bit more robust (even if it will be slower).
Jacopo
On Apr 30, 2012, at 12:17 PM, Adrian Crum wrote:
> On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
>> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>>
>>> I tried experimenting with the sleep timing and I also replaced the Thread.sleep call with a safer version, but the tests still failed.
>> interesting... but if you change the Thread.sleep timeout from 200 to 2000 it works, right?
>
> I changed it to 300. By the way, the test finally passed for the first time when I had another non-OFBiz process running at the same time that was making heavy use of the hard disk.
>
> -Adrian
>
Re: framework/base tests failing
Posted by Adrian Crum <ad...@sandglass-software.com>.
On 4/30/2012 10:27 AM, Jacopo Cappellato wrote:
> On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
>
>> I tried experimenting with the sleep timing and I also replaced the Thread.sleep call with a safer version, but the tests still failed.
> interesting... but if you change the Thread.sleep timeout from 200 to 2000 it works, right?
I changed it to 300. By the way, the test finally passed for the first
time when I had another non-OFBiz process running at the same time that
was making heavy use of the hard disk.
-Adrian
Re: framework/base tests failing (was: recent HashCrypt changes, and using salt-based password hashing)
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
On Apr 23, 2012, at 3:47 PM, Adrian Crum wrote:
> I tried experimenting with the sleep timing and I also replaced the Thread.sleep call with a safer version, but the tests still failed.
interesting... but if you change the Thread.sleep timeout from 200 to 2000 it works, right?
Jacopo
framework/base tests failing (was: recent HashCrypt changes, and
using salt-based password hashing)
Posted by Adrian Crum <ad...@sandglass-software.com>.
The tests failed intermittently in the OFBiz buildbot (back when it was
running). They always fail on my development machine.
I tried experimenting with the sleep timing and I also replaced the
Thread.sleep call with a safer version, but the tests still failed.
-Adrian
On 4/23/2012 2:40 PM, Jacopo Cappellato wrote:
> Adrian,
>
> the issue you are experiencing is interesting and I had a look at it; I was only able to recreate the same failures by changing the timeouts in UtilCacheTests.java at line 302 and 308; instead of:
>
> cache.setExpireTime(100); // line 302
> Thread.sleep(200); // line 308
>
> I set:
>
> cache.setExpireTime(100); // line 302
> Thread.sleep(99); // line 308
>
> and (of course) I got the error.
> It is kind of weird that you are getting the error with the current settings (100 and 200): is this consistent? I mean, are you able to recreate the issue frequently? did you try to increase the time for sleep:
>
> Thread.sleep(300); // line 308
>
> ?
>
> Regards,
>
> Jacopo
>
>
> On Apr 22, 2012, at 10:53 AM, Adrian Crum wrote:
>
>> On 4/20/2012 9:00 PM, Adam Heath wrote:
>>> On 04/20/2012 12:13 PM, Jacques Le Roux wrote:
>>>> Not sure you noticed but Builbot is no longer running
>>>> http://ci.apache.org/waterfall?show_events=false&branch=&builder=ofbiz-trunk&reload=none
>>> Yeah, noticed that buildbot wasn't emailing. Looking at that link, it
>>> appears that isis_ubuntu is offline.
>>>
>>> I always run all my changes thru a full clean/test run before
>>> committing(even when I commit> 10 at a time).
>> Adam,
>>
>> If it's not too much trouble, could you look at
>>
>> https://issues.apache.org/jira/browse/OFBIZ-4239 ?
>>
>> Buildbot was failing the framework/base tests intermittently. I think I tracked the problem down to a timing issue in the UtilCache tests - see the attached test report and my comments for details.
>>
>> -Adrian
>>
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacopo Cappellato <ja...@hotwaxmedia.com>.
Adrian,
the issue you are experiencing is interesting and I had a look at it; I was only able to recreate the same failures by changing the timeouts in UtilCacheTests.java at line 302 and 308; instead of:
cache.setExpireTime(100); // line 302
Thread.sleep(200); // line 308
I set:
cache.setExpireTime(100); // line 302
Thread.sleep(99); // line 308
and (of course) I got the error.
It is kind of weird that you are getting the error with the current settings (100 and 200): is this consistent? I mean, are you able to recreate the issue frequently? did you try to increase the time for sleep:
Thread.sleep(300); // line 308
?
Regards,
Jacopo
On Apr 22, 2012, at 10:53 AM, Adrian Crum wrote:
> On 4/20/2012 9:00 PM, Adam Heath wrote:
>> On 04/20/2012 12:13 PM, Jacques Le Roux wrote:
>>> Not sure you noticed but Builbot is no longer running
>>> http://ci.apache.org/waterfall?show_events=false&branch=&builder=ofbiz-trunk&reload=none
>> Yeah, noticed that buildbot wasn't emailing. Looking at that link, it
>> appears that isis_ubuntu is offline.
>>
>> I always run all my changes thru a full clean/test run before
>> committing(even when I commit> 10 at a time).
>
> Adam,
>
> If it's not too much trouble, could you look at
>
> https://issues.apache.org/jira/browse/OFBIZ-4239 ?
>
> Buildbot was failing the framework/base tests intermittently. I think I tracked the problem down to a timing issue in the UtilCache tests - see the attached test report and my comments for details.
>
> -Adrian
>
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adam Heath <do...@brainfood.com>.
On 04/22/2012 03:53 AM, Adrian Crum wrote:
> On 4/20/2012 9:00 PM, Adam Heath wrote:
>> On 04/20/2012 12:13 PM, Jacques Le Roux wrote:
>>> Not sure you noticed but Builbot is no longer running
>>> http://ci.apache.org/waterfall?show_events=false&branch=&builder=ofbiz-trunk&reload=none
>>>
>> Yeah, noticed that buildbot wasn't emailing. Looking at that link, it
>> appears that isis_ubuntu is offline.
>>
>> I always run all my changes thru a full clean/test run before
>> committing(even when I commit> 10 at a time).
>
> Adam,
>
> If it's not too much trouble, could you look at
>
> https://issues.apache.org/jira/browse/OFBIZ-4239 ?
>
> Buildbot was failing the framework/base tests intermittently. I think
> I tracked the problem down to a timing issue in the UtilCache tests -
> see the attached test report and my comments for details.
Eventually, yes, I'm still pushing hard on key-encrypting-key support
in EntityCrypto(a continuation of the hashcrypt changes).
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adrian Crum <ad...@sandglass-software.com>.
On 4/20/2012 9:00 PM, Adam Heath wrote:
> On 04/20/2012 12:13 PM, Jacques Le Roux wrote:
>> Not sure you noticed but Builbot is no longer running
>> http://ci.apache.org/waterfall?show_events=false&branch=&builder=ofbiz-trunk&reload=none
> Yeah, noticed that buildbot wasn't emailing. Looking at that link, it
> appears that isis_ubuntu is offline.
>
> I always run all my changes thru a full clean/test run before
> committing(even when I commit> 10 at a time).
Adam,
If it's not too much trouble, could you look at
https://issues.apache.org/jira/browse/OFBIZ-4239 ?
Buildbot was failing the framework/base tests intermittently. I think I
tracked the problem down to a timing issue in the UtilCache tests - see
the attached test report and my comments for details.
-Adrian
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Adam Heath" <do...@brainfood.com>
> On 04/20/2012 12:13 PM, Jacques Le Roux wrote:
>> Not sure you noticed but Builbot is no longer running
>> http://ci.apache.org/waterfall?show_events=false&branch=&builder=ofbiz-trunk&reload=none
>
> Yeah, noticed that buildbot wasn't emailing. Looking at that link, it
> appears that isis_ubuntu is offline.
Yes: https://issues.apache.org/jira/browse/INFRA-4562, please push...
> I always run all my changes thru a full clean/test run before
> committing(even when I commit > 10 at a time).
>
> I am willing to post some of my new stuff somewhere before committing,
> 'cuz changing EntityCrypto is not something you would do lightly.
>
>> That's maybe why Pierre and Hans want an easy way to check before
>> integrating?
>
> Verifying that the existing test cases run(because buildbot is broken)
> is completely separate than wanting to verify an implementation before
> committing to trunk.
I meant we can't rely on a shared (fast) CI...
Jacques
>
>>
>> Jacques
>
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adam Heath <do...@brainfood.com>.
On 04/20/2012 12:13 PM, Jacques Le Roux wrote:
> Not sure you noticed but Builbot is no longer running
> http://ci.apache.org/waterfall?show_events=false&branch=&builder=ofbiz-trunk&reload=none
Yeah, noticed that buildbot wasn't emailing. Looking at that link, it
appears that isis_ubuntu is offline.
I always run all my changes thru a full clean/test run before
committing(even when I commit > 10 at a time).
I am willing to post some of my new stuff somewhere before committing,
'cuz changing EntityCrypto is not something you would do lightly.
> That's maybe why Pierre and Hans want an easy way to check before
> integrating?
Verifying that the existing test cases run(because buildbot is broken)
is completely separate than wanting to verify an implementation before
committing to trunk.
>
> Jacques
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Adam Heath" <do...@brainfood.com>
> On 04/20/2012 09:56 AM, Pierre Smits wrote:
>> Hi Adam,
>>
>> I can follow your reasoning. And it seems reasonable. But I would like to
>> suggest to you to create a JIRA for this where patches are uploaded and can
>> be tested in a separate ofbiz instantiation. This would then help with the
>> creation of implementation procedures and documentation prior to uploading
>> it to trunk.
>
> Well, I won't be sending any patches into JIRA. That would be a worse
> workflow than I currently have. I use git to do everything now. I'd
> publish a git link somewhere in JIRA.
>
> However, the nature of this changes is tending to keep all the
> alterations in a small location. Under those circumstances, it can be
> quite difficult to have multiple people work on it at once. I do
> acknowledge the suggestion about documentation. Implementation will
> be simple, you'll upgrade, generate the master KEK, and everything
> else is automatic.
Adam,
Not sure you noticed but Builbot is no longer running
http://ci.apache.org/waterfall?show_events=false&branch=&builder=ofbiz-trunk&reload=none
That's maybe why Pierre and Hans want an easy way to check before integrating?
Jacques
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adam Heath <do...@brainfood.com>.
On 04/20/2012 09:56 AM, Pierre Smits wrote:
> Hi Adam,
>
> I can follow your reasoning. And it seems reasonable. But I would like to
> suggest to you to create a JIRA for this where patches are uploaded and can
> be tested in a separate ofbiz instantiation. This would then help with the
> creation of implementation procedures and documentation prior to uploading
> it to trunk.
Well, I won't be sending any patches into JIRA. That would be a worse
workflow than I currently have. I use git to do everything now. I'd
publish a git link somewhere in JIRA.
However, the nature of this changes is tending to keep all the
alterations in a small location. Under those circumstances, it can be
quite difficult to have multiple people work on it at once. I do
acknowledge the suggestion about documentation. Implementation will
be simple, you'll upgrade, generate the master KEK, and everything
else is automatic.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Pierre Smits <pi...@gmail.com>.
Hi Adam,
I can follow your reasoning. And it seems reasonable. But I would like to
suggest to you to create a JIRA for this where patches are uploaded and can
be tested in a separate ofbiz instantiation. This would then help with the
creation of implementation procedures and documentation prior to uploading
it to trunk.
Regards,
Pierre
Op 20 april 2012 16:41 schreef Adam Heath <do...@brainfood.com> het
volgende:
> On 04/20/2012 12:53 AM, Pierre Smits wrote:
>
>> Hi Adam,
>>
>> How would that be? That would be one per tenant in a multi-tenant setup? I
>> can imagine in a multi-tenant setup with the db backend not on derby (as
>> we
>> all recommend) the upgrade/migration aspect can be enormous. Even more so
>> in a HAFO-setup.
>>
>
> Moving EntityKeyStore to a separate database would not be hard, no code
> changes at all. Just a new entitygroup mapping, and updating
> entityengine.xml(or TenantDataSource) to point it at a different database.
> This would then mean running pg_dump(or whatever) would not see the keys.
>
> I currently have the new crypto storage done. It uses base64 to store the
> hashed keyname, the key value, and the encrypted column values scattered
> around the database. A random-length(0-15) random-value salt is pre-pended
> to each value during encryption, so if you continually set the same value,
> you'll get different encrypted values.
>
> I do not yet have key-encrypting-key(KEK) support working. I'm currently
> thinking there would be one 'master' KEK. This is what EntityCrypto would
> use by default. In sub-tenant delegators, the sub EntityCrypto would fetch
> a key from it's parent delegator. The parent delegator would be using the
> master KEK to encode it's keys. The sub-delegator would be using a unique
> KEK stored in the base delegator. The base delegator has it's own
> EntityCrypto.
>
> So, the master KEK could be stored in entityengine.xml(base64 encoded, I
> can provide a cmdline tool to generate it), or some other file.
>
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adam Heath <do...@brainfood.com>.
On 04/20/2012 12:53 AM, Pierre Smits wrote:
> Hi Adam,
>
> How would that be? That would be one per tenant in a multi-tenant setup? I
> can imagine in a multi-tenant setup with the db backend not on derby (as we
> all recommend) the upgrade/migration aspect can be enormous. Even more so
> in a HAFO-setup.
Moving EntityKeyStore to a separate database would not be hard, no code
changes at all. Just a new entitygroup mapping, and updating
entityengine.xml(or TenantDataSource) to point it at a different
database. This would then mean running pg_dump(or whatever) would not
see the keys.
I currently have the new crypto storage done. It uses base64 to store
the hashed keyname, the key value, and the encrypted column values
scattered around the database. A random-length(0-15) random-value salt
is pre-pended to each value during encryption, so if you continually set
the same value, you'll get different encrypted values.
I do not yet have key-encrypting-key(KEK) support working. I'm
currently thinking there would be one 'master' KEK. This is what
EntityCrypto would use by default. In sub-tenant delegators, the sub
EntityCrypto would fetch a key from it's parent delegator. The parent
delegator would be using the master KEK to encode it's keys. The
sub-delegator would be using a unique KEK stored in the base delegator.
The base delegator has it's own EntityCrypto.
So, the master KEK could be stored in entityengine.xml(base64 encoded, I
can provide a cmdline tool to generate it), or some other file.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacques Le Roux <ja...@les7arts.com>.
Yes I'm not too inclined too. This could be an option though for more security concerned projects...
Jacques
From: "Pierre Smits" <pi...@gmail.com>
> Hi Adam,
>
> How would that be? That would be one per tenant in a multi-tenant setup? I
> can imagine in a multi-tenant setup with the db backend not on derby (as we
> all recommend) the upgrade/migration aspect can be enormous. Even more so
> in a HAFO-setup.
>
> Regards,
>
> Pierre
>
> Op 20 april 2012 01:23 schreef Adam Heath <do...@brainfood.com> het
> volgende:
>
>> On 04/19/2012 06:13 PM, Scott Gray wrote:
>> >
>> > On 20/04/2012, at 9:49 AM, Adam Heath wrote:
>> >
>> >> On 04/19/2012 04:28 PM, Jacques Le Roux wrote:
>> >>> Looking forward for
>> >>> https://issues.apache.org/jira/browse/OFBIZ-1151
>> >>> https://issues.apache.org/jira/browse/OFBIZ-2729
>> >>> https://issues.apache.org/jira/browse/OFBIZ-3006
>> >>
>> >> 2729 doesn't apply for what I am doing. Here's the list of things
>> >> that this solves:
>> >>
>> >> 1: salt-based UserLogin.currentPassword
>> >> (done, I have a fix for the recent issue locally)
>> >> 2: salt-based CreditCard.cardNumber encrypted value
>> >> (s/b done tonight)
>> >> 3: salt-based EntityKeyStore.keyText
>> >> (s/b done tonight)
>> >> 4: key-encrypting-key for EntityKeyStore.keyText. The
>> >> key-encrypting-key will be available somewhere in ${ofbiz.home.dir}.
>> >> (need to read java security doco)
>> >>
>> >> I've currently rewritten a bunch of EntityCrypto already to fix the
>> >> issues I posted into the jira issue. I'm now at the point I can start
>> >> adding new features.
>> >>
>> >> This set of changes I currently have are rather straight forward, just
>> >> moving code around. When I finally get around to adding the new
>> >> features, then there is a very definate chance of breaking stuff.
>> >
>> > Please just do your best to ensure backwards compatibility is
>> maintained. I can't imagine anything worse than doing an upgrade and
>> discovering that all my user passwords are no longer valid.
>>
>> Sure, I've got things done as an array of handlers now(2 current, plus
>> 1 new).
>>
>> How do we feel about moving EntityKeyStore into a separate database?
>> It'd improve the security a little bit, and is rather simple to do
>> with ofbiz.
>>
>>
>
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Pierre Smits <pi...@gmail.com>.
Hi Adam,
How would that be? That would be one per tenant in a multi-tenant setup? I
can imagine in a multi-tenant setup with the db backend not on derby (as we
all recommend) the upgrade/migration aspect can be enormous. Even more so
in a HAFO-setup.
Regards,
Pierre
Op 20 april 2012 01:23 schreef Adam Heath <do...@brainfood.com> het
volgende:
> On 04/19/2012 06:13 PM, Scott Gray wrote:
> >
> > On 20/04/2012, at 9:49 AM, Adam Heath wrote:
> >
> >> On 04/19/2012 04:28 PM, Jacques Le Roux wrote:
> >>> Looking forward for
> >>> https://issues.apache.org/jira/browse/OFBIZ-1151
> >>> https://issues.apache.org/jira/browse/OFBIZ-2729
> >>> https://issues.apache.org/jira/browse/OFBIZ-3006
> >>
> >> 2729 doesn't apply for what I am doing. Here's the list of things
> >> that this solves:
> >>
> >> 1: salt-based UserLogin.currentPassword
> >> (done, I have a fix for the recent issue locally)
> >> 2: salt-based CreditCard.cardNumber encrypted value
> >> (s/b done tonight)
> >> 3: salt-based EntityKeyStore.keyText
> >> (s/b done tonight)
> >> 4: key-encrypting-key for EntityKeyStore.keyText. The
> >> key-encrypting-key will be available somewhere in ${ofbiz.home.dir}.
> >> (need to read java security doco)
> >>
> >> I've currently rewritten a bunch of EntityCrypto already to fix the
> >> issues I posted into the jira issue. I'm now at the point I can start
> >> adding new features.
> >>
> >> This set of changes I currently have are rather straight forward, just
> >> moving code around. When I finally get around to adding the new
> >> features, then there is a very definate chance of breaking stuff.
> >
> > Please just do your best to ensure backwards compatibility is
> maintained. I can't imagine anything worse than doing an upgrade and
> discovering that all my user passwords are no longer valid.
>
> Sure, I've got things done as an array of handlers now(2 current, plus
> 1 new).
>
> How do we feel about moving EntityKeyStore into a separate database?
> It'd improve the security a little bit, and is rather simple to do
> with ofbiz.
>
>
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adam Heath <do...@brainfood.com>.
On 04/19/2012 06:13 PM, Scott Gray wrote:
>
> On 20/04/2012, at 9:49 AM, Adam Heath wrote:
>
>> On 04/19/2012 04:28 PM, Jacques Le Roux wrote:
>>> Looking forward for
>>> https://issues.apache.org/jira/browse/OFBIZ-1151
>>> https://issues.apache.org/jira/browse/OFBIZ-2729
>>> https://issues.apache.org/jira/browse/OFBIZ-3006
>>
>> 2729 doesn't apply for what I am doing. Here's the list of things
>> that this solves:
>>
>> 1: salt-based UserLogin.currentPassword
>> (done, I have a fix for the recent issue locally)
>> 2: salt-based CreditCard.cardNumber encrypted value
>> (s/b done tonight)
>> 3: salt-based EntityKeyStore.keyText
>> (s/b done tonight)
>> 4: key-encrypting-key for EntityKeyStore.keyText. The
>> key-encrypting-key will be available somewhere in ${ofbiz.home.dir}.
>> (need to read java security doco)
>>
>> I've currently rewritten a bunch of EntityCrypto already to fix the
>> issues I posted into the jira issue. I'm now at the point I can start
>> adding new features.
>>
>> This set of changes I currently have are rather straight forward, just
>> moving code around. When I finally get around to adding the new
>> features, then there is a very definate chance of breaking stuff.
>
> Please just do your best to ensure backwards compatibility is maintained. I can't imagine anything worse than doing an upgrade and discovering that all my user passwords are no longer valid.
Sure, I've got things done as an array of handlers now(2 current, plus
1 new).
How do we feel about moving EntityKeyStore into a separate database?
It'd improve the security a little bit, and is rather simple to do
with ofbiz.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Scott Gray <sc...@hotwaxmedia.com>.
On 20/04/2012, at 9:49 AM, Adam Heath wrote:
> On 04/19/2012 04:28 PM, Jacques Le Roux wrote:
>> Looking forward for
>> https://issues.apache.org/jira/browse/OFBIZ-1151
>> https://issues.apache.org/jira/browse/OFBIZ-2729
>> https://issues.apache.org/jira/browse/OFBIZ-3006
>
> 2729 doesn't apply for what I am doing. Here's the list of things
> that this solves:
>
> 1: salt-based UserLogin.currentPassword
> (done, I have a fix for the recent issue locally)
> 2: salt-based CreditCard.cardNumber encrypted value
> (s/b done tonight)
> 3: salt-based EntityKeyStore.keyText
> (s/b done tonight)
> 4: key-encrypting-key for EntityKeyStore.keyText. The
> key-encrypting-key will be available somewhere in ${ofbiz.home.dir}.
> (need to read java security doco)
>
> I've currently rewritten a bunch of EntityCrypto already to fix the
> issues I posted into the jira issue. I'm now at the point I can start
> adding new features.
>
> This set of changes I currently have are rather straight forward, just
> moving code around. When I finally get around to adding the new
> features, then there is a very definate chance of breaking stuff.
Please just do your best to ensure backwards compatibility is maintained. I can't imagine anything worse than doing an upgrade and discovering that all my user passwords are no longer valid.
>> Jacques
>>
>> From: "Adam Heath" <do...@brainfood.com>
>>> As some may have noticed, I recently changed the way ofbiz creates
>>> password hashes when it stores them in the database. Each time a new
>>> password is created, a bit of randomness is used, to create a
>>> random-length, random-content salt. This is placed at the beginning
>>> of the hashed password, stored along with it in the database.
>>>
>>> Then, when it comes time to compare passwords, the salt is extracted,
>>> and used to re-hash.
>>>
>>> If someone continously changes their password to the same value,
>>> you'll end up getting different raw hashed values in the database. It
>>> also increases the workload for crackers, and makes rainbow tables
>>> much less fruitful.
>>>
>>> I wrote this change over 2 years ago, in direct response to the jira
>>> intrusion that happened.
>
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Adam Heath" <do...@brainfood.com>
> On 04/19/2012 04:49 PM, Adam Heath wrote:
>> On 04/19/2012 04:28 PM, Jacques Le Roux wrote:
>>> Looking forward for
>>> https://issues.apache.org/jira/browse/OFBIZ-1151
>>> https://issues.apache.org/jira/browse/OFBIZ-2729
>>> https://issues.apache.org/jira/browse/OFBIZ-3006
>>
>> 2729 doesn't apply for what I am doing. Here's the list of things
>> that this solves:
>>
>> 1: salt-based UserLogin.currentPassword
>> (done, I have a fix for the recent issue locally)
>> 2: salt-based CreditCard.cardNumber encrypted value
>> (s/b done tonight)
>> 3: salt-based EntityKeyStore.keyText
>> (s/b done tonight)
>> 4: key-encrypting-key for EntityKeyStore.keyText. The
>> key-encrypting-key will be available somewhere in ${ofbiz.home.dir}.
>> (need to read java security doco)
>
> Also, no longer using hex-encoding for the values, but base64, as it
> produces a smaller encoded value. The new salt-based password stuff
> I've committed already does that.
Sounds good, long awaited "feature"...
Jacques
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adam Heath <do...@brainfood.com>.
On 04/19/2012 04:49 PM, Adam Heath wrote:
> On 04/19/2012 04:28 PM, Jacques Le Roux wrote:
>> Looking forward for
>> https://issues.apache.org/jira/browse/OFBIZ-1151
>> https://issues.apache.org/jira/browse/OFBIZ-2729
>> https://issues.apache.org/jira/browse/OFBIZ-3006
>
> 2729 doesn't apply for what I am doing. Here's the list of things
> that this solves:
>
> 1: salt-based UserLogin.currentPassword
> (done, I have a fix for the recent issue locally)
> 2: salt-based CreditCard.cardNumber encrypted value
> (s/b done tonight)
> 3: salt-based EntityKeyStore.keyText
> (s/b done tonight)
> 4: key-encrypting-key for EntityKeyStore.keyText. The
> key-encrypting-key will be available somewhere in ${ofbiz.home.dir}.
> (need to read java security doco)
Also, no longer using hex-encoding for the values, but base64, as it
produces a smaller encoded value. The new salt-based password stuff
I've committed already does that.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Adam Heath <do...@brainfood.com>.
On 04/19/2012 04:28 PM, Jacques Le Roux wrote:
> Looking forward for
> https://issues.apache.org/jira/browse/OFBIZ-1151
> https://issues.apache.org/jira/browse/OFBIZ-2729
> https://issues.apache.org/jira/browse/OFBIZ-3006
2729 doesn't apply for what I am doing. Here's the list of things
that this solves:
1: salt-based UserLogin.currentPassword
(done, I have a fix for the recent issue locally)
2: salt-based CreditCard.cardNumber encrypted value
(s/b done tonight)
3: salt-based EntityKeyStore.keyText
(s/b done tonight)
4: key-encrypting-key for EntityKeyStore.keyText. The
key-encrypting-key will be available somewhere in ${ofbiz.home.dir}.
(need to read java security doco)
I've currently rewritten a bunch of EntityCrypto already to fix the
issues I posted into the jira issue. I'm now at the point I can start
adding new features.
This set of changes I currently have are rather straight forward, just
moving code around. When I finally get around to adding the new
features, then there is a very definate chance of breaking stuff.
>
> Jacques
>
> From: "Adam Heath" <do...@brainfood.com>
>> As some may have noticed, I recently changed the way ofbiz creates
>> password hashes when it stores them in the database. Each time a new
>> password is created, a bit of randomness is used, to create a
>> random-length, random-content salt. This is placed at the beginning
>> of the hashed password, stored along with it in the database.
>>
>> Then, when it comes time to compare passwords, the salt is extracted,
>> and used to re-hash.
>>
>> If someone continously changes their password to the same value,
>> you'll end up getting different raw hashed values in the database. It
>> also increases the workload for crackers, and makes rainbow tables
>> much less fruitful.
>>
>> I wrote this change over 2 years ago, in direct response to the jira
>> intrusion that happened.
Re: recent HashCrypt changes, and using salt-based password hashing
Posted by Jacques Le Roux <ja...@les7arts.com>.
Looking forward for
https://issues.apache.org/jira/browse/OFBIZ-1151
https://issues.apache.org/jira/browse/OFBIZ-2729
https://issues.apache.org/jira/browse/OFBIZ-3006
Jacques
From: "Adam Heath" <do...@brainfood.com>
> As some may have noticed, I recently changed the way ofbiz creates
> password hashes when it stores them in the database. Each time a new
> password is created, a bit of randomness is used, to create a
> random-length, random-content salt. This is placed at the beginning
> of the hashed password, stored along with it in the database.
>
> Then, when it comes time to compare passwords, the salt is extracted,
> and used to re-hash.
>
> If someone continously changes their password to the same value,
> you'll end up getting different raw hashed values in the database. It
> also increases the workload for crackers, and makes rainbow tables
> much less fruitful.
>
> I wrote this change over 2 years ago, in direct response to the jira
> intrusion that happened.