Re: codec - thread safe

[Henri Yandell <fl...@gmail.com>]

On 9/29/07, Qingtian Wang <qi...@gmail.com> wrote:
> I apologize for this question as it must have been asked a million
> times: I was unable to search the mailing list archive for this.
>
> Are all the "encode/decode" methods in commons-codec intended to be thread safe?
>
> I peeked into the source code for a couple of those methods. They both
> seem to be thread safe. But I'd rather ask here to make sure that's
> the general intention when the code is written.

I think you'll find it varies. The various algorithms come from
different sources, and though a common interface was put on them all,
I doubt that time was spent to make sure they were all threadsafe.

Looking at the objects, I see a few with private attributes that are
set at more than just construction time - so very likely that one or
another is thread unsafe.

Hen

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

RE: codec - thread safe

[Jörg Schaible <Jo...@Elsag-Solutions.com>]

simon wrote on Monday, October 08, 2007 8:19 AM:

> On Sun, 2007-10-07 at 15:23 -0500, Qingtian Wang wrote:
>> Ok, I got the point.
>> 
>> So let's say I wanted to work on this. What's the most effective way
>> to do it? 
>> 
>> Search the entire code base line by line trying to ID all the thread
>> unsafe points by myself? I guess that's very ineffective compared to
>> have an issue open, and have the individual developers who write the
>> code to address it - They know what needs to be tweaked without
>> having to even spend any time since it's their own code. Or at
>> least, the dev team as a whole can come up a list of points that
>> need to be worked on. I think that'd be much more effective. Any
>> established channel where that can happen?
> 
> I would suggest that you just check the classes/methods that you
> yourself want to be threadsafe, and make any necessary fixes.
> 
> There's no obligation to fix anything else; if somebody needs
> thread-safety improvements in a different part, then *they* can do it.

[snip]

Well, I don't think that this solution is really good. Either codec classes are supposed to be thread-safe or not. A  solution where some are and other are not is probably not what is expected. Why not simply go with the JDK Collections approach and provide a CodecUtil with static methods for synchronizedEncoder(), synchronizedStringEncoder(), synchronizedBinaryEncoder and the corresponding methods for the decoders? So everyone can decide on its own whether he need synchronization or takes a pool approach in case of concurrency.

- Jörg

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[simon <si...@chello.at>]

On Sun, 2007-10-07 at 15:23 -0500, Qingtian Wang wrote:
> Ok, I got the point.
> 
> So let's say I wanted to work on this. What's the most effective way to do it?
> 
> Search the entire code base line by line trying to ID all the thread
> unsafe points by myself? I guess that's very ineffective compared to
> have an issue open, and have the individual developers who write the
> code to address it - They know what needs to be tweaked without having
> to even spend any time since it's their own code. Or at least, the dev
> team as a whole can come up a list of points that need to be worked
> on. I think that'd be much more effective. Any established channel
> where that can happen?

I would suggest that you just check the classes/methods that you
yourself want to be threadsafe, and make any necessary fixes.

There's no obligation to fix anything else; if somebody needs
thread-safety improvements in a different part, then *they* can do it.

And then piece by piece the software gets better for everyone, using a
process that is both cooperative and fair.

Agreed, this doesn't work for major infrastructure type work, but that
is not the case here; just a method or a class needs to be checked for
safety and maybe a few internal synchronization commands added. Codec
really is pretty simple stuff; it's not necessary to have written the
code in order to understand it.


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Qingtian Wang <qi...@gmail.com>]

On 10/8/07, sebb <se...@gmail.com> wrote:
> On 08/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> > On 10/7/07, sebb <se...@gmail.com> wrote:
> > > On 07/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> > > > Ok, I got the point.
> > > >
> > > > So let's say I wanted to work on this. What's the most effective way to do it?
> > >
> > > You could try running some of the code checking utilities on the library.
> > >
> > > For example Findbugs and PMD.
> > >
> > > I've done a quick check with FIndbugs, but that has not found any
> > > synchronisation issues. This is because there are no synchronised
> > > methods, so it cannot find any inconsistencies.
> > >
> > > But it should be possible to configure PMD to find any instance or
> > > static variables that are modified after construction.
> > >
> > > Or indeed you could perhaps try making all such variables final, and
> > > see whether it's possible to fix the compiler errors - but that might
> > > be a lot of work.
> > >
> > > > Search the entire code base line by line trying to ID all the thread
> > > > unsafe points by myself? I guess that's very ineffective compared to
> > > > have an issue open, and have the individual developers who write the
> > > > code to address it - They know what needs to be tweaked without having
> > > > to even spend any time since it's their own code. Or at least, the dev
> > > > team as a whole can come up a list of points that need to be worked
> > >
> > > Not necessarily - the code may have been developed over a long period,
> > > and the original authors may not longer be around. Unless the code has
> > > been worked on recently, it's possible that no-one has much detailed
> > > knowledge of the code.
> >
> > Wow, if that is really the case with commons codec that no one has
> > detailed knowledge of the code, it would suggest to me that no one
> > should use it as a off the shell usable product since no one really
> > knows what it would do.
>
> I did not write specifically about Commons Codec - I was just pointing
> out that any software that has been developed over a long period is
> likely to have areas that are unfamiliar to some  or all. Indeed, even
> if you wrote the software yourself, would you still be familiar with
> it several years later? Or even several months?

No, admitted I wouldn't. But I do believe, if I am still physically
around, it's easier for me to get back the code I wrote, and ask
myself "what the heck was I thinking?" than someone tries to do the
same.

>
> This is not just a software problem. Any large or long-lived project
> will have areas that are not immediately familiar to anyone.

Agreed. But at the same time, trying to fully utilize the available
knowledge base to start with might be beneficial. That's all I am
trying to say.


>
> > I mean, it's one thing to say "Here's what our intention what this
> > product behaves. But use it at your own risk since there is no
> > guarantee of any kind.", but it's entirely different thing to say
> > "Here's a bunch of code, but we don't know how it behaves, you're
> > welcome to find out for yourself and play around with it."
>
> That's not the case here.
> Does Commons Codec offer any thread-safe guarantees?
> If not, then yes, one cannot say how it may behave in multi-threaded mode.
> But that does not mean the behaviour is unpredictable in single-threaded usage.

That's exactly what I was trying to figure out: Is thread-safe an
intention in general for the code base?

And after I found out that the answer is "No", and worse, it seems
some methods are thread-safe and others are not, and nobody knows
what's what. That's where I made an issue out of it: Since no body
even knows for sure which is and which not, can we at least try to be
able to say for sure on the existing code base in terms of
thread-safety.

And I'd say before we can be sure of that, tell the users up front: No
thread-safety is considered. To me, that's much better than saying
nothing.


>
> For example, Java's SimpleDateFormat is not thread-safe.
> Can one predict exactly what happens in multi-threaded mode?
> Probably not.

That is actually a very good example: In the Javadoc of the that
class, Sun says it loud and clear: "This not thread-safe." My wish is
simply that codec is able to give those kind of statements, and then
make the main business methods such as encode/decode thread safe,
rather than saying nothing and leave it for the users to find out.

>
>
> > To me those two things put a product at two rather different level of
> > maturity. If codec falls in the latter, a normal user that needs a
> > codec library for their work might as well look some where else or
> > roll their own stuff to hand their codec needs, since it'd take same
> > or even more time to dig in commons-codec and try to find out what
> > exactly it's trying to do.
>
> Not so with Commons Codec, since it is quite a small library.

Well big or small is rather subjective: For a slow developer like
myself in a rush to find a codec library for my work, it's big enough.
:)


>
> > -Q
> >
> >
> >
> >
> > >
> > > > on. I think that'd be much more effective. Any established channel
> > > > where that can happen?
> > >
> > > If you find some bugs, then report them via JIRA.
> > > It helps if you can provide test cases; obviously patches as well is
> > > even better.
> > >
> > > > Thanks,
> > > > -Q
> > > >
> > > >
> > > >
> > > > On 10/7/07, Torsten Curdt <tc...@apache.org> wrote:
> > > > > >>> Can the dev team make that happen? - a humble request from a user.
> > > > > >>
> > > > > >> The think about open source is that there is no distinction between
> > > > > >> developer and user.
> > > > > >
> > > > > > That's interesting. Why then almost all open source projects have
> > > > > > "users doc vs. developer doc", "users mailing list vs. developers
> > > > > > mailing list"....?
> > > > >
> > > > > Point taken. But this is more about the presentation of information.
> > > > > Still the distinction is blurry as many developers and contributors
> > > > > come initially from the user background. They felt the itch to
> > > > > improve the code and ended up providing the fix/feature
> > > > > themselves ...I guess we are trying to get you to do the same :) You
> > > > > should see that more as an invitation to help.
> > > > >
> > > > > >> The developers develop because they want to use the
> > > > > >> code. And when somebody wants to use a feature that doesn't exist,
> > > > > >> then
> > > > > >> they can develop it.
> > > > > >>
> > > > > >> In short, if you want this feature, you can do this yourself and
> > > > > >> post a
> > > > > >> patch to this list so it gets included in the next release. There
> > > > > >> is no
> > > > > >> paid "dev team" for any of the commons code.
> > > > > >
> > > > > > OK i see. My intention was just to bring up an issue/request. There is
> > > > > > no saying the developers are obligated to work on it. And contributing
> > > > > > code is certainly not the only way of involving with an open source
> > > > > > project.
> > > > >
> > > > > That is of course true. But I guess what Simon was wanting to point
> > > > > out is that "requesting a feature" rarely means it gets done unless a
> > > > > developer needs that feature too. The personal TODO lists are usually
> > > > > very long. So usually you are better of contributing a patch
> > > > > yourself ...or pay someone to do it for you. That's how it works.
> > > > >
> > > > > > Nevertheless, if the dev team (unpaid as I understand it)
> > > > > > wants the project to be more popular, it might help listening to what
> > > > > > other users have to say other than themselves; otherwise just forget
> > > > > > about it.
> > > > >
> > > > > Well, sure ...listening is great - but someone gotta do it.
> > > > > There is no us vs you, no dev vs user ....there is just us.
> > > > >
> > > > > Does that make sense?
> > > > >
> > > > > cheers
> > > > > --
> > > > > Torsten
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: user-help@commons.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: user-help@commons.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[sebb <se...@gmail.com>]

On 08/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> On 10/7/07, sebb <se...@gmail.com> wrote:
> > On 07/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> > > Ok, I got the point.
> > >
> > > So let's say I wanted to work on this. What's the most effective way to do it?
> >
> > You could try running some of the code checking utilities on the library.
> >
> > For example Findbugs and PMD.
> >
> > I've done a quick check with FIndbugs, but that has not found any
> > synchronisation issues. This is because there are no synchronised
> > methods, so it cannot find any inconsistencies.
> >
> > But it should be possible to configure PMD to find any instance or
> > static variables that are modified after construction.
> >
> > Or indeed you could perhaps try making all such variables final, and
> > see whether it's possible to fix the compiler errors - but that might
> > be a lot of work.
> >
> > > Search the entire code base line by line trying to ID all the thread
> > > unsafe points by myself? I guess that's very ineffective compared to
> > > have an issue open, and have the individual developers who write the
> > > code to address it - They know what needs to be tweaked without having
> > > to even spend any time since it's their own code. Or at least, the dev
> > > team as a whole can come up a list of points that need to be worked
> >
> > Not necessarily - the code may have been developed over a long period,
> > and the original authors may not longer be around. Unless the code has
> > been worked on recently, it's possible that no-one has much detailed
> > knowledge of the code.
>
> Wow, if that is really the case with commons codec that no one has
> detailed knowledge of the code, it would suggest to me that no one
> should use it as a off the shell usable product since no one really
> knows what it would do.

I did not write specifically about Commons Codec - I was just pointing
out that any software that has been developed over a long period is
likely to have areas that are unfamiliar to some  or all. Indeed, even
if you wrote the software yourself, would you still be familiar with
it several years later? Or even several months?

This is not just a software problem. Any large or long-lived project
will have areas that are not immediately familiar to anyone.

> I mean, it's one thing to say "Here's what our intention what this
> product behaves. But use it at your own risk since there is no
> guarantee of any kind.", but it's entirely different thing to say
> "Here's a bunch of code, but we don't know how it behaves, you're
> welcome to find out for yourself and play around with it."

That's not the case here.
Does Commons Codec offer any thread-safe guarantees?
If not, then yes, one cannot say how it may behave in multi-threaded mode.
But that does not mean the behaviour is unpredictable in single-threaded usage.

For example, Java's SimpleDateFormat is not thread-safe.
Can one predict exactly what happens in multi-threaded mode?
Probably not.


> To me those two things put a product at two rather different level of
> maturity. If codec falls in the latter, a normal user that needs a
> codec library for their work might as well look some where else or
> roll their own stuff to hand their codec needs, since it'd take same
> or even more time to dig in commons-codec and try to find out what
> exactly it's trying to do.

Not so with Commons Codec, since it is quite a small library.

> -Q
>
>
>
>
> >
> > > on. I think that'd be much more effective. Any established channel
> > > where that can happen?
> >
> > If you find some bugs, then report them via JIRA.
> > It helps if you can provide test cases; obviously patches as well is
> > even better.
> >
> > > Thanks,
> > > -Q
> > >
> > >
> > >
> > > On 10/7/07, Torsten Curdt <tc...@apache.org> wrote:
> > > > >>> Can the dev team make that happen? - a humble request from a user.
> > > > >>
> > > > >> The think about open source is that there is no distinction between
> > > > >> developer and user.
> > > > >
> > > > > That's interesting. Why then almost all open source projects have
> > > > > "users doc vs. developer doc", "users mailing list vs. developers
> > > > > mailing list"....?
> > > >
> > > > Point taken. But this is more about the presentation of information.
> > > > Still the distinction is blurry as many developers and contributors
> > > > come initially from the user background. They felt the itch to
> > > > improve the code and ended up providing the fix/feature
> > > > themselves ...I guess we are trying to get you to do the same :) You
> > > > should see that more as an invitation to help.
> > > >
> > > > >> The developers develop because they want to use the
> > > > >> code. And when somebody wants to use a feature that doesn't exist,
> > > > >> then
> > > > >> they can develop it.
> > > > >>
> > > > >> In short, if you want this feature, you can do this yourself and
> > > > >> post a
> > > > >> patch to this list so it gets included in the next release. There
> > > > >> is no
> > > > >> paid "dev team" for any of the commons code.
> > > > >
> > > > > OK i see. My intention was just to bring up an issue/request. There is
> > > > > no saying the developers are obligated to work on it. And contributing
> > > > > code is certainly not the only way of involving with an open source
> > > > > project.
> > > >
> > > > That is of course true. But I guess what Simon was wanting to point
> > > > out is that "requesting a feature" rarely means it gets done unless a
> > > > developer needs that feature too. The personal TODO lists are usually
> > > > very long. So usually you are better of contributing a patch
> > > > yourself ...or pay someone to do it for you. That's how it works.
> > > >
> > > > > Nevertheless, if the dev team (unpaid as I understand it)
> > > > > wants the project to be more popular, it might help listening to what
> > > > > other users have to say other than themselves; otherwise just forget
> > > > > about it.
> > > >
> > > > Well, sure ...listening is great - but someone gotta do it.
> > > > There is no us vs you, no dev vs user ....there is just us.
> > > >
> > > > Does that make sense?
> > > >
> > > > cheers
> > > > --
> > > > Torsten
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: user-help@commons.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Qingtian Wang <qi...@gmail.com>]

On 10/7/07, sebb <se...@gmail.com> wrote:
> On 07/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> > Ok, I got the point.
> >
> > So let's say I wanted to work on this. What's the most effective way to do it?
>
> You could try running some of the code checking utilities on the library.
>
> For example Findbugs and PMD.
>
> I've done a quick check with FIndbugs, but that has not found any
> synchronisation issues. This is because there are no synchronised
> methods, so it cannot find any inconsistencies.
>
> But it should be possible to configure PMD to find any instance or
> static variables that are modified after construction.
>
> Or indeed you could perhaps try making all such variables final, and
> see whether it's possible to fix the compiler errors - but that might
> be a lot of work.
>
> > Search the entire code base line by line trying to ID all the thread
> > unsafe points by myself? I guess that's very ineffective compared to
> > have an issue open, and have the individual developers who write the
> > code to address it - They know what needs to be tweaked without having
> > to even spend any time since it's their own code. Or at least, the dev
> > team as a whole can come up a list of points that need to be worked
>
> Not necessarily - the code may have been developed over a long period,
> and the original authors may not longer be around. Unless the code has
> been worked on recently, it's possible that no-one has much detailed
> knowledge of the code.

Wow, if that is really the case with commons codec that no one has
detailed knowledge of the code, it would suggest to me that no one
should use it as a off the shell usable product since no one really
knows what it would do.

I mean, it's one thing to say "Here's what our intention what this
product behaves. But use it at your own risk since there is no
guarantee of any kind.", but it's entirely different thing to say
"Here's a bunch of code, but we don't know how it behaves, you're
welcome to find out for yourself and play around with it."

To me those two things put a product at two rather different level of
maturity. If codec falls in the latter, a normal user that needs a
codec library for their work might as well look some where else or
roll their own stuff to hand their codec needs, since it'd take same
or even more time to dig in commons-codec and try to find out what
exactly it's trying to do.

-Q




>
> > on. I think that'd be much more effective. Any established channel
> > where that can happen?
>
> If you find some bugs, then report them via JIRA.
> It helps if you can provide test cases; obviously patches as well is
> even better.
>
> > Thanks,
> > -Q
> >
> >
> >
> > On 10/7/07, Torsten Curdt <tc...@apache.org> wrote:
> > > >>> Can the dev team make that happen? - a humble request from a user.
> > > >>
> > > >> The think about open source is that there is no distinction between
> > > >> developer and user.
> > > >
> > > > That's interesting. Why then almost all open source projects have
> > > > "users doc vs. developer doc", "users mailing list vs. developers
> > > > mailing list"....?
> > >
> > > Point taken. But this is more about the presentation of information.
> > > Still the distinction is blurry as many developers and contributors
> > > come initially from the user background. They felt the itch to
> > > improve the code and ended up providing the fix/feature
> > > themselves ...I guess we are trying to get you to do the same :) You
> > > should see that more as an invitation to help.
> > >
> > > >> The developers develop because they want to use the
> > > >> code. And when somebody wants to use a feature that doesn't exist,
> > > >> then
> > > >> they can develop it.
> > > >>
> > > >> In short, if you want this feature, you can do this yourself and
> > > >> post a
> > > >> patch to this list so it gets included in the next release. There
> > > >> is no
> > > >> paid "dev team" for any of the commons code.
> > > >
> > > > OK i see. My intention was just to bring up an issue/request. There is
> > > > no saying the developers are obligated to work on it. And contributing
> > > > code is certainly not the only way of involving with an open source
> > > > project.
> > >
> > > That is of course true. But I guess what Simon was wanting to point
> > > out is that "requesting a feature" rarely means it gets done unless a
> > > developer needs that feature too. The personal TODO lists are usually
> > > very long. So usually you are better of contributing a patch
> > > yourself ...or pay someone to do it for you. That's how it works.
> > >
> > > > Nevertheless, if the dev team (unpaid as I understand it)
> > > > wants the project to be more popular, it might help listening to what
> > > > other users have to say other than themselves; otherwise just forget
> > > > about it.
> > >
> > > Well, sure ...listening is great - but someone gotta do it.
> > > There is no us vs you, no dev vs user ....there is just us.
> > >
> > > Does that make sense?
> > >
> > > cheers
> > > --
> > > Torsten
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[sebb <se...@gmail.com>]

On 07/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> Ok, I got the point.
>
> So let's say I wanted to work on this. What's the most effective way to do it?

You could try running some of the code checking utilities on the library.

For example Findbugs and PMD.

I've done a quick check with FIndbugs, but that has not found any
synchronisation issues. This is because there are no synchronised
methods, so it cannot find any inconsistencies.

But it should be possible to configure PMD to find any instance or
static variables that are modified after construction.

Or indeed you could perhaps try making all such variables final, and
see whether it's possible to fix the compiler errors - but that might
be a lot of work.

> Search the entire code base line by line trying to ID all the thread
> unsafe points by myself? I guess that's very ineffective compared to
> have an issue open, and have the individual developers who write the
> code to address it - They know what needs to be tweaked without having
> to even spend any time since it's their own code. Or at least, the dev
> team as a whole can come up a list of points that need to be worked

Not necessarily - the code may have been developed over a long period,
and the original authors may not longer be around. Unless the code has
been worked on recently, it's possible that no-one has much detailed
knowledge of the code.

> on. I think that'd be much more effective. Any established channel
> where that can happen?

If you find some bugs, then report them via JIRA.
It helps if you can provide test cases; obviously patches as well is
even better.

> Thanks,
> -Q
>
>
>
> On 10/7/07, Torsten Curdt <tc...@apache.org> wrote:
> > >>> Can the dev team make that happen? - a humble request from a user.
> > >>
> > >> The think about open source is that there is no distinction between
> > >> developer and user.
> > >
> > > That's interesting. Why then almost all open source projects have
> > > "users doc vs. developer doc", "users mailing list vs. developers
> > > mailing list"....?
> >
> > Point taken. But this is more about the presentation of information.
> > Still the distinction is blurry as many developers and contributors
> > come initially from the user background. They felt the itch to
> > improve the code and ended up providing the fix/feature
> > themselves ...I guess we are trying to get you to do the same :) You
> > should see that more as an invitation to help.
> >
> > >> The developers develop because they want to use the
> > >> code. And when somebody wants to use a feature that doesn't exist,
> > >> then
> > >> they can develop it.
> > >>
> > >> In short, if you want this feature, you can do this yourself and
> > >> post a
> > >> patch to this list so it gets included in the next release. There
> > >> is no
> > >> paid "dev team" for any of the commons code.
> > >
> > > OK i see. My intention was just to bring up an issue/request. There is
> > > no saying the developers are obligated to work on it. And contributing
> > > code is certainly not the only way of involving with an open source
> > > project.
> >
> > That is of course true. But I guess what Simon was wanting to point
> > out is that "requesting a feature" rarely means it gets done unless a
> > developer needs that feature too. The personal TODO lists are usually
> > very long. So usually you are better of contributing a patch
> > yourself ...or pay someone to do it for you. That's how it works.
> >
> > > Nevertheless, if the dev team (unpaid as I understand it)
> > > wants the project to be more popular, it might help listening to what
> > > other users have to say other than themselves; otherwise just forget
> > > about it.
> >
> > Well, sure ...listening is great - but someone gotta do it.
> > There is no us vs you, no dev vs user ....there is just us.
> >
> > Does that make sense?
> >
> > cheers
> > --
> > Torsten
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Qingtian Wang <qi...@gmail.com>]

Ok, I got the point.

So let's say I wanted to work on this. What's the most effective way to do it?

Search the entire code base line by line trying to ID all the thread
unsafe points by myself? I guess that's very ineffective compared to
have an issue open, and have the individual developers who write the
code to address it - They know what needs to be tweaked without having
to even spend any time since it's their own code. Or at least, the dev
team as a whole can come up a list of points that need to be worked
on. I think that'd be much more effective. Any established channel
where that can happen?

Thanks,
-Q



On 10/7/07, Torsten Curdt <tc...@apache.org> wrote:
> >>> Can the dev team make that happen? - a humble request from a user.
> >>
> >> The think about open source is that there is no distinction between
> >> developer and user.
> >
> > That's interesting. Why then almost all open source projects have
> > "users doc vs. developer doc", "users mailing list vs. developers
> > mailing list"....?
>
> Point taken. But this is more about the presentation of information.
> Still the distinction is blurry as many developers and contributors
> come initially from the user background. They felt the itch to
> improve the code and ended up providing the fix/feature
> themselves ...I guess we are trying to get you to do the same :) You
> should see that more as an invitation to help.
>
> >> The developers develop because they want to use the
> >> code. And when somebody wants to use a feature that doesn't exist,
> >> then
> >> they can develop it.
> >>
> >> In short, if you want this feature, you can do this yourself and
> >> post a
> >> patch to this list so it gets included in the next release. There
> >> is no
> >> paid "dev team" for any of the commons code.
> >
> > OK i see. My intention was just to bring up an issue/request. There is
> > no saying the developers are obligated to work on it. And contributing
> > code is certainly not the only way of involving with an open source
> > project.
>
> That is of course true. But I guess what Simon was wanting to point
> out is that "requesting a feature" rarely means it gets done unless a
> developer needs that feature too. The personal TODO lists are usually
> very long. So usually you are better of contributing a patch
> yourself ...or pay someone to do it for you. That's how it works.
>
> > Nevertheless, if the dev team (unpaid as I understand it)
> > wants the project to be more popular, it might help listening to what
> > other users have to say other than themselves; otherwise just forget
> > about it.
>
> Well, sure ...listening is great - but someone gotta do it.
> There is no us vs you, no dev vs user ....there is just us.
>
> Does that make sense?
>
> cheers
> --
> Torsten
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Torsten Curdt <tc...@apache.org>]

>>> Can the dev team make that happen? - a humble request from a user.
>>
>> The think about open source is that there is no distinction between
>> developer and user.
>
> That's interesting. Why then almost all open source projects have
> "users doc vs. developer doc", "users mailing list vs. developers
> mailing list"....?

Point taken. But this is more about the presentation of information.  
Still the distinction is blurry as many developers and contributors  
come initially from the user background. They felt the itch to  
improve the code and ended up providing the fix/feature  
themselves ...I guess we are trying to get you to do the same :) You  
should see that more as an invitation to help.

>> The developers develop because they want to use the
>> code. And when somebody wants to use a feature that doesn't exist,  
>> then
>> they can develop it.
>>
>> In short, if you want this feature, you can do this yourself and  
>> post a
>> patch to this list so it gets included in the next release. There  
>> is no
>> paid "dev team" for any of the commons code.
>
> OK i see. My intention was just to bring up an issue/request. There is
> no saying the developers are obligated to work on it. And contributing
> code is certainly not the only way of involving with an open source
> project.

That is of course true. But I guess what Simon was wanting to point  
out is that "requesting a feature" rarely means it gets done unless a  
developer needs that feature too. The personal TODO lists are usually  
very long. So usually you are better of contributing a patch  
yourself ...or pay someone to do it for you. That's how it works.

> Nevertheless, if the dev team (unpaid as I understand it)
> wants the project to be more popular, it might help listening to what
> other users have to say other than themselves; otherwise just forget
> about it.

Well, sure ...listening is great - but someone gotta do it.
There is no us vs you, no dev vs user ....there is just us.

Does that make sense?

cheers
--
Torsten

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Qingtian Wang <qi...@gmail.com>]

On 10/7/07, simon <si...@chello.at> wrote:
> On Sat, 2007-10-06 at 23:31 -0500, Qingtian Wang wrote:
> > Well, it's pick-your-poison kind of a deal. Either block on one
> > instance and take a performance hit, or burn up the memory with lots
> > of instances.....
> >
> > But in the case of BCodec, I think encode/decode is thread safe.
> > Unfortunately per Henri, that's not generally true for others.
> >
> > Well, let me make it clear that I am a total layman on codec. But it
> > seems to me it's not that difficult to implement all the codec methods
> > in a thread safe manner, without sync blocks.
> >
> > Can the dev team make that happen? - a humble request from a user.
>
> The think about open source is that there is no distinction between
> developer and user.

That's interesting. Why then almost all open source projects have
"users doc vs. developer doc", "users mailing list vs. developers
mailing list"....?

> The developers develop because they want to use the
> code. And when somebody wants to use a feature that doesn't exist, then
> they can develop it.
>
> In short, if you want this feature, you can do this yourself and post a
> patch to this list so it gets included in the next release. There is no
> paid "dev team" for any of the commons code.

OK i see. My intention was just to bring up an issue/request. There is
no saying the developers are obligated to work on it. And contributing
code is certainly not the only way of involving with an open source
project. Nevertheless, if the dev team (unpaid as I understand it)
wants the project to be more popular, it might help listening to what
other users have to say other than themselves; otherwise just forget
about it.

-Q






>
> Regards,
>
> Simon
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[sebb <se...@gmail.com>]

On 07/10/2007, simon <si...@chello.at> wrote:
> On Sat, 2007-10-06 at 23:31 -0500, Qingtian Wang wrote:
> > Well, it's pick-your-poison kind of a deal. Either block on one
> > instance and take a performance hit, or burn up the memory with lots
> > of instances.....
> >
> > But in the case of BCodec, I think encode/decode is thread safe.
> > Unfortunately per Henri, that's not generally true for others.
> >
> > Well, let me make it clear that I am a total layman on codec. But it
> > seems to me it's not that difficult to implement all the codec methods
> > in a thread safe manner, without sync blocks.
> >
> > Can the dev team make that happen? - a humble request from a user.
>
> The think about open source is that there is no distinction between
> developer and user. The developers develop because they want to use the
> code. And when somebody wants to use a feature that doesn't exist, then
> they can develop it.
>
> In short, if you want this feature, you can do this yourself and post a
> patch to this list so it gets included in the next release. There is no
> paid "dev team" for any of the commons code.

Might be better to create a JIRA issue and attach the patch to that:
* others can easily see that the issue has been reported
* patches can get mangled when posted to lists, and are more difficult
to keep track of.

For Codec, start here:
http://commons.apache.org/codec/issue-tracking.html

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[simon <si...@chello.at>]

On Sat, 2007-10-06 at 23:31 -0500, Qingtian Wang wrote:
> Well, it's pick-your-poison kind of a deal. Either block on one
> instance and take a performance hit, or burn up the memory with lots
> of instances.....
> 
> But in the case of BCodec, I think encode/decode is thread safe.
> Unfortunately per Henri, that's not generally true for others.
> 
> Well, let me make it clear that I am a total layman on codec. But it
> seems to me it's not that difficult to implement all the codec methods
> in a thread safe manner, without sync blocks.
> 
> Can the dev team make that happen? - a humble request from a user.

The think about open source is that there is no distinction between
developer and user. The developers develop because they want to use the
code. And when somebody wants to use a feature that doesn't exist, then
they can develop it.

In short, if you want this feature, you can do this yourself and post a
patch to this list so it gets included in the next release. There is no
paid "dev team" for any of the commons code.

Regards,

Simon


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[sebb <se...@gmail.com>]

Or take a copy of QCodec and fix it to remove the offending code...

The change would be more difficult to make in the codec library,
because removing the offending method would not be backwards
compatible.

On 09/10/2007, James Carman <ja...@carmanconsulting.com> wrote:
> And, a simple map would suffice in that case (lazy-initialized of
> course).  If you need QCodec, then you'd have in incorporate the
> encodeBlanks and the charset into the map's key (if you really have
> cases where you do/do not want to encode blanks).
>
> On 10/8/07, sebb <se...@gmail.com> wrote:
> > Which methods do you actually need?
> >
> > If you only need BCodec, then that (and Base64 which it calls) look to
> > be thread-safe, so you only need to instantiate it once for each
> > different charset.
> >
> > On 08/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> > > The SLA of the project I am working on is 2000 transactions per
> > > second. And I need to decode a 1K string on each request.....
> > >
> > > -Q
> > >
> > >
> > >
> > > On 10/8/07, Will Pugh <wi...@sourcelabs.com> wrote:
> > > > Ya know.  Just to put this in a little perspective.
> > > >
> > > > These objects are pretty small (QCodec has only two members in it), and
> > > > have practically no instantiation cost.  Instantiating these and quickly
> > > > throwing them out is probably not such a bad idea, and may give you
> > > > better performance that doing anything with ThreadLocal.  Java VMs are
> > > > fairly well tuned for dealing with short-lived, small objects (which
> > > > these are).
> > > >
> > > > I can't imagine that many people are using thousands of these per
> > > > request, probably, more on the order of one or two a request?
> > > >
> > > >     --Will
> > > >
> > > > James Carman wrote:
> > > > > Try using a ThreadLocal variable to store your Encoder/Decoder that
> > > > > you need to be thread safe.
> > > > >
> > > > > On 10/8/07, sebb <se...@gmail.com> wrote:
> > > > >
> > > > >> On 08/10/2007, Will Pugh <wi...@sourcelabs.com> wrote:
> > > > >>
> > > > >>> David J. Biesack wrote:
> > > > >>>
> > > > >>>>> Date: Mon, 8 Oct 2007 17:01:02 +0200
> > > > >>>>> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>
> > > > >>>>>> The java.util.concurrent backport
> > > > >>>>>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> > > > >>>>>> kind of use.
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
> > > > >>>>>
> > > > >>>>>
> > > > >>>> Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>> Might seem like a silly question, but has anyone found anything that
> > > > >>> looks like a threading issue in Codec?
> > > > >>>
> > > > >> Yes -  QCodec has a public method that sets an instance variable (see
> > > > >> my comments on CODEC-55).
> > > > >>
> > > > >>
> > > > >>> The code is pretty simple.  I did a quick look through the code, and it
> > > > >>> seemed like everything was pretty thread safe.  It looks like you should
> > > > >>> be O.K. as long as:
> > > > >>>    1)  You don't change the parameters to a codec while it's running,
> > > > >>> e.g. changing charsets, etc.
> > > > >>>
> > > > >> See above.
> > > > >>
> > > > >> Having said that, I think it would not be too difficult to fix the
> > > > >> classes so that they are thread-safe. Most of the ones I looked at
> > > > >> could even be made invariant.
> > > > >>
> > > > >> So their thread-safety would only depend on any external calls they
> > > > >> made; this could presumably be fixed by providing synchronised
> > > > >> versions as suggested earlier.
> > > > >>
> > > > >>
> > > > >>>    2)  You should not use a MessageDigest returned by DigestUtils by
> > > > >>> multiple threads.  This should be clear given the API (it's a SUN api)
> > > > >>>
> > > > >>> It also seems to me that the difference between code that you can trust
> > > > >>> as being thread safe, and code you cannot trust as being thread safe has
> > > > >>> a lot to do with whether it is tested for thread safety.  Seems to me
> > > > >>> like another way of helping out here would be to build concurrency tests
> > > > >>> for whichever APIs you are interested using on multiple threads.  Not
> > > > >>> sure where we would put them yet, but seems like the only way to assure
> > > > >>> thread-safety (especially if we could get someone to volunteer running
> > > > >>> them on a big beefy multi-proc machine :).
> > > > >>>
> > > > >> Indeed.
> > > > >>
> > > > >>
> > > > >>>>> - Jörg
> > > > >>>>>
> > > > >>>>>
> > > > >>>>
> > > > >>> ---------------------------------------------------------------------
> > > > >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > > >>> For additional commands, e-mail: user-help@commons.apache.org
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >> ---------------------------------------------------------------------
> > > > >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > > >> For additional commands, e-mail: user-help@commons.apache.org
> > > > >>
> > > > >>
> > > > >>
> > > > >
> > > > > ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > > > For additional commands, e-mail: user-help@commons.apache.org
> > > > >
> > > > >
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: user-help@commons.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Will Pugh <wi...@sourcelabs.com>]

sebb wrote:
> On 09/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
>   
>> On 10/8/07, sebb <se...@gmail.com> wrote:
>>     
>>> Which methods do you actually need?
>>>
>>> If you only need BCodec, then that (and Base64 which it calls) look to
>>> be thread-safe, so you only need to instantiate it once for each
>>> different charset.
>>>       
>> Yes, I sort of figured that out for myself already when I first
>> started the conversation with Henri. Now thanks for your help, I got
>> another pair of eyes confirming this. So I am good in my case.
>>
>> But all the discussion has been more about the "in general" case. I
>> just feel that,
>>
>> 1. Codec as a commons library, it should not be this hard to find out
>> about information like this. It should be either this or that, "find
>> out for yourself" is no good situation. As someone else pointed out
>> earlier, we could use a better documentation.
>>     
>
> Agreed. It should be stated clearly whether or not the code is thread-safe.
>   
I attached a patch to this bug.

My solution here was to add static methods called
    createThreadSafeCodec

on all the classes that are thread safe.  For a few, this simply returns 
an instance of the object.  For others, it returns an instance of the 
object that overrides the dangerous set* methods.  That way, if anyone 
tried to use it in an unsafe manner, they will deterministically get an 
error.

My patch also adds a blurb in package.html basically saying this.
>   
>> 2. It'd be nice for the biz method implementations to be thread safe
>> (Ideally in a high performance manner as a value add-on of using a
>> commons brand library such that user doesn't have to be too creative
>> as some of the suggestions given in this discussion to achieve
>> performance). Most of them may already be thread safe. And as it seems
>> agreed by all that it's not hard to make them that way if not.
>>     
>
> For a library such as codec, I agree that it should be thread-safe by
> design. Where this is not possible, the unsafe classes should be
> clearly identified.
>
> I don't think it makes sense to separate methods into "biz" and the rest.
>   
I agree.
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>   

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[sebb <se...@gmail.com>]

On 09/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> On 10/8/07, sebb <se...@gmail.com> wrote:
> > Which methods do you actually need?
> >
> > If you only need BCodec, then that (and Base64 which it calls) look to
> > be thread-safe, so you only need to instantiate it once for each
> > different charset.
>
> Yes, I sort of figured that out for myself already when I first
> started the conversation with Henri. Now thanks for your help, I got
> another pair of eyes confirming this. So I am good in my case.
>
> But all the discussion has been more about the "in general" case. I
> just feel that,
>
> 1. Codec as a commons library, it should not be this hard to find out
> about information like this. It should be either this or that, "find
> out for yourself" is no good situation. As someone else pointed out
> earlier, we could use a better documentation.

Agreed. It should be stated clearly whether or not the code is thread-safe.

> 2. It'd be nice for the biz method implementations to be thread safe
> (Ideally in a high performance manner as a value add-on of using a
> commons brand library such that user doesn't have to be too creative
> as some of the suggestions given in this discussion to achieve
> performance). Most of them may already be thread safe. And as it seems
> agreed by all that it's not hard to make them that way if not.

For a library such as codec, I agree that it should be thread-safe by
design. Where this is not possible, the unsafe classes should be
clearly identified.

I don't think it makes sense to separate methods into "biz" and the rest.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Qingtian Wang <qi...@gmail.com>]

On 10/8/07, sebb <se...@gmail.com> wrote:
> Which methods do you actually need?
>
> If you only need BCodec, then that (and Base64 which it calls) look to
> be thread-safe, so you only need to instantiate it once for each
> different charset.

Yes, I sort of figured that out for myself already when I first
started the conversation with Henri. Now thanks for your help, I got
another pair of eyes confirming this. So I am good in my case.

But all the discussion has been more about the "in general" case. I
just feel that,

1. Codec as a commons library, it should not be this hard to find out
about information like this. It should be either this or that, "find
out for yourself" is no good situation. As someone else pointed out
earlier, we could use a better documentation.

2. It'd be nice for the biz method implementations to be thread safe
(Ideally in a high performance manner as a value add-on of using a
commons brand library such that user doesn't have to be too creative
as some of the suggestions given in this discussion to achieve
performance). Most of them may already be thread safe. And as it seems
agreed by all that it's not hard to make them that way if not.

Thanks to all for your help!
-Q


>
> On 08/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> > The SLA of the project I am working on is 2000 transactions per
> > second. And I need to decode a 1K string on each request.....
> >
> > -Q
> >
> >
> >
> > On 10/8/07, Will Pugh <wi...@sourcelabs.com> wrote:
> > > Ya know.  Just to put this in a little perspective.
> > >
> > > These objects are pretty small (QCodec has only two members in it), and
> > > have practically no instantiation cost.  Instantiating these and quickly
> > > throwing them out is probably not such a bad idea, and may give you
> > > better performance that doing anything with ThreadLocal.  Java VMs are
> > > fairly well tuned for dealing with short-lived, small objects (which
> > > these are).
> > >
> > > I can't imagine that many people are using thousands of these per
> > > request, probably, more on the order of one or two a request?
> > >
> > >     --Will
> > >
> > > James Carman wrote:
> > > > Try using a ThreadLocal variable to store your Encoder/Decoder that
> > > > you need to be thread safe.
> > > >
> > > > On 10/8/07, sebb <se...@gmail.com> wrote:
> > > >
> > > >> On 08/10/2007, Will Pugh <wi...@sourcelabs.com> wrote:
> > > >>
> > > >>> David J. Biesack wrote:
> > > >>>
> > > >>>>> Date: Mon, 8 Oct 2007 17:01:02 +0200
> > > >>>>> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>>> The java.util.concurrent backport
> > > >>>>>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> > > >>>>>> kind of use.
> > > >>>>>>
> > > >>>>>>
> > > >>>>> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
> > > >>>>>
> > > >>>>>
> > > >>>> Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>> Might seem like a silly question, but has anyone found anything that
> > > >>> looks like a threading issue in Codec?
> > > >>>
> > > >> Yes -  QCodec has a public method that sets an instance variable (see
> > > >> my comments on CODEC-55).
> > > >>
> > > >>
> > > >>> The code is pretty simple.  I did a quick look through the code, and it
> > > >>> seemed like everything was pretty thread safe.  It looks like you should
> > > >>> be O.K. as long as:
> > > >>>    1)  You don't change the parameters to a codec while it's running,
> > > >>> e.g. changing charsets, etc.
> > > >>>
> > > >> See above.
> > > >>
> > > >> Having said that, I think it would not be too difficult to fix the
> > > >> classes so that they are thread-safe. Most of the ones I looked at
> > > >> could even be made invariant.
> > > >>
> > > >> So their thread-safety would only depend on any external calls they
> > > >> made; this could presumably be fixed by providing synchronised
> > > >> versions as suggested earlier.
> > > >>
> > > >>
> > > >>>    2)  You should not use a MessageDigest returned by DigestUtils by
> > > >>> multiple threads.  This should be clear given the API (it's a SUN api)
> > > >>>
> > > >>> It also seems to me that the difference between code that you can trust
> > > >>> as being thread safe, and code you cannot trust as being thread safe has
> > > >>> a lot to do with whether it is tested for thread safety.  Seems to me
> > > >>> like another way of helping out here would be to build concurrency tests
> > > >>> for whichever APIs you are interested using on multiple threads.  Not
> > > >>> sure where we would put them yet, but seems like the only way to assure
> > > >>> thread-safety (especially if we could get someone to volunteer running
> > > >>> them on a big beefy multi-proc machine :).
> > > >>>
> > > >> Indeed.
> > > >>
> > > >>
> > > >>>>> - Jörg
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>> ---------------------------------------------------------------------
> > > >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > >>> For additional commands, e-mail: user-help@commons.apache.org
> > > >>>
> > > >>>
> > > >>>
> > > >> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > >> For additional commands, e-mail: user-help@commons.apache.org
> > > >>
> > > >>
> > > >>
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: user-help@commons.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[James Carman <ja...@carmanconsulting.com>]

And, a simple map would suffice in that case (lazy-initialized of
course).  If you need QCodec, then you'd have in incorporate the
encodeBlanks and the charset into the map's key (if you really have
cases where you do/do not want to encode blanks).

On 10/8/07, sebb <se...@gmail.com> wrote:
> Which methods do you actually need?
>
> If you only need BCodec, then that (and Base64 which it calls) look to
> be thread-safe, so you only need to instantiate it once for each
> different charset.
>
> On 08/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> > The SLA of the project I am working on is 2000 transactions per
> > second. And I need to decode a 1K string on each request.....
> >
> > -Q
> >
> >
> >
> > On 10/8/07, Will Pugh <wi...@sourcelabs.com> wrote:
> > > Ya know.  Just to put this in a little perspective.
> > >
> > > These objects are pretty small (QCodec has only two members in it), and
> > > have practically no instantiation cost.  Instantiating these and quickly
> > > throwing them out is probably not such a bad idea, and may give you
> > > better performance that doing anything with ThreadLocal.  Java VMs are
> > > fairly well tuned for dealing with short-lived, small objects (which
> > > these are).
> > >
> > > I can't imagine that many people are using thousands of these per
> > > request, probably, more on the order of one or two a request?
> > >
> > >     --Will
> > >
> > > James Carman wrote:
> > > > Try using a ThreadLocal variable to store your Encoder/Decoder that
> > > > you need to be thread safe.
> > > >
> > > > On 10/8/07, sebb <se...@gmail.com> wrote:
> > > >
> > > >> On 08/10/2007, Will Pugh <wi...@sourcelabs.com> wrote:
> > > >>
> > > >>> David J. Biesack wrote:
> > > >>>
> > > >>>>> Date: Mon, 8 Oct 2007 17:01:02 +0200
> > > >>>>> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> > > >>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>>> The java.util.concurrent backport
> > > >>>>>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> > > >>>>>> kind of use.
> > > >>>>>>
> > > >>>>>>
> > > >>>>> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
> > > >>>>>
> > > >>>>>
> > > >>>> Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>> Might seem like a silly question, but has anyone found anything that
> > > >>> looks like a threading issue in Codec?
> > > >>>
> > > >> Yes -  QCodec has a public method that sets an instance variable (see
> > > >> my comments on CODEC-55).
> > > >>
> > > >>
> > > >>> The code is pretty simple.  I did a quick look through the code, and it
> > > >>> seemed like everything was pretty thread safe.  It looks like you should
> > > >>> be O.K. as long as:
> > > >>>    1)  You don't change the parameters to a codec while it's running,
> > > >>> e.g. changing charsets, etc.
> > > >>>
> > > >> See above.
> > > >>
> > > >> Having said that, I think it would not be too difficult to fix the
> > > >> classes so that they are thread-safe. Most of the ones I looked at
> > > >> could even be made invariant.
> > > >>
> > > >> So their thread-safety would only depend on any external calls they
> > > >> made; this could presumably be fixed by providing synchronised
> > > >> versions as suggested earlier.
> > > >>
> > > >>
> > > >>>    2)  You should not use a MessageDigest returned by DigestUtils by
> > > >>> multiple threads.  This should be clear given the API (it's a SUN api)
> > > >>>
> > > >>> It also seems to me that the difference between code that you can trust
> > > >>> as being thread safe, and code you cannot trust as being thread safe has
> > > >>> a lot to do with whether it is tested for thread safety.  Seems to me
> > > >>> like another way of helping out here would be to build concurrency tests
> > > >>> for whichever APIs you are interested using on multiple threads.  Not
> > > >>> sure where we would put them yet, but seems like the only way to assure
> > > >>> thread-safety (especially if we could get someone to volunteer running
> > > >>> them on a big beefy multi-proc machine :).
> > > >>>
> > > >> Indeed.
> > > >>
> > > >>
> > > >>>>> - Jörg
> > > >>>>>
> > > >>>>>
> > > >>>>
> > > >>> ---------------------------------------------------------------------
> > > >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > >>> For additional commands, e-mail: user-help@commons.apache.org
> > > >>>
> > > >>>
> > > >>>
> > > >> ---------------------------------------------------------------------
> > > >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > >> For additional commands, e-mail: user-help@commons.apache.org
> > > >>
> > > >>
> > > >>
> > > >
> > > > ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > > For additional commands, e-mail: user-help@commons.apache.org
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[sebb <se...@gmail.com>]

Which methods do you actually need?

If you only need BCodec, then that (and Base64 which it calls) look to
be thread-safe, so you only need to instantiate it once for each
different charset.

On 08/10/2007, Qingtian Wang <qi...@gmail.com> wrote:
> The SLA of the project I am working on is 2000 transactions per
> second. And I need to decode a 1K string on each request.....
>
> -Q
>
>
>
> On 10/8/07, Will Pugh <wi...@sourcelabs.com> wrote:
> > Ya know.  Just to put this in a little perspective.
> >
> > These objects are pretty small (QCodec has only two members in it), and
> > have practically no instantiation cost.  Instantiating these and quickly
> > throwing them out is probably not such a bad idea, and may give you
> > better performance that doing anything with ThreadLocal.  Java VMs are
> > fairly well tuned for dealing with short-lived, small objects (which
> > these are).
> >
> > I can't imagine that many people are using thousands of these per
> > request, probably, more on the order of one or two a request?
> >
> >     --Will
> >
> > James Carman wrote:
> > > Try using a ThreadLocal variable to store your Encoder/Decoder that
> > > you need to be thread safe.
> > >
> > > On 10/8/07, sebb <se...@gmail.com> wrote:
> > >
> > >> On 08/10/2007, Will Pugh <wi...@sourcelabs.com> wrote:
> > >>
> > >>> David J. Biesack wrote:
> > >>>
> > >>>>> Date: Mon, 8 Oct 2007 17:01:02 +0200
> > >>>>> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>> The java.util.concurrent backport
> > >>>>>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> > >>>>>> kind of use.
> > >>>>>>
> > >>>>>>
> > >>>>> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
> > >>>>>
> > >>>>>
> > >>>> Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
> > >>>>
> > >>>>
> > >>>>
> > >>> Might seem like a silly question, but has anyone found anything that
> > >>> looks like a threading issue in Codec?
> > >>>
> > >> Yes -  QCodec has a public method that sets an instance variable (see
> > >> my comments on CODEC-55).
> > >>
> > >>
> > >>> The code is pretty simple.  I did a quick look through the code, and it
> > >>> seemed like everything was pretty thread safe.  It looks like you should
> > >>> be O.K. as long as:
> > >>>    1)  You don't change the parameters to a codec while it's running,
> > >>> e.g. changing charsets, etc.
> > >>>
> > >> See above.
> > >>
> > >> Having said that, I think it would not be too difficult to fix the
> > >> classes so that they are thread-safe. Most of the ones I looked at
> > >> could even be made invariant.
> > >>
> > >> So their thread-safety would only depend on any external calls they
> > >> made; this could presumably be fixed by providing synchronised
> > >> versions as suggested earlier.
> > >>
> > >>
> > >>>    2)  You should not use a MessageDigest returned by DigestUtils by
> > >>> multiple threads.  This should be clear given the API (it's a SUN api)
> > >>>
> > >>> It also seems to me that the difference between code that you can trust
> > >>> as being thread safe, and code you cannot trust as being thread safe has
> > >>> a lot to do with whether it is tested for thread safety.  Seems to me
> > >>> like another way of helping out here would be to build concurrency tests
> > >>> for whichever APIs you are interested using on multiple threads.  Not
> > >>> sure where we would put them yet, but seems like the only way to assure
> > >>> thread-safety (especially if we could get someone to volunteer running
> > >>> them on a big beefy multi-proc machine :).
> > >>>
> > >> Indeed.
> > >>
> > >>
> > >>>>> - Jörg
> > >>>>>
> > >>>>>
> > >>>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > >>> For additional commands, e-mail: user-help@commons.apache.org
> > >>>
> > >>>
> > >>>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > >> For additional commands, e-mail: user-help@commons.apache.org
> > >>
> > >>
> > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[James Carman <ja...@carmanconsulting.com>]

That's kind of what I said on the JIRA issue, too.

On 10/8/07, Will Pugh <wi...@sourcelabs.com> wrote:
> Ya know.  Just to put this in a little perspective.
>
> These objects are pretty small (QCodec has only two members in it), and
> have practically no instantiation cost.  Instantiating these and quickly
> throwing them out is probably not such a bad idea, and may give you
> better performance that doing anything with ThreadLocal.  Java VMs are
> fairly well tuned for dealing with short-lived, small objects (which
> these are).
>
> I can't imagine that many people are using thousands of these per
> request, probably, more on the order of one or two a request?
>
>     --Will
>
> James Carman wrote:
> > Try using a ThreadLocal variable to store your Encoder/Decoder that
> > you need to be thread safe.
> >
> > On 10/8/07, sebb <se...@gmail.com> wrote:
> >
> >> On 08/10/2007, Will Pugh <wi...@sourcelabs.com> wrote:
> >>
> >>> David J. Biesack wrote:
> >>>
> >>>>> Date: Mon, 8 Oct 2007 17:01:02 +0200
> >>>>> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> The java.util.concurrent backport
> >>>>>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> >>>>>> kind of use.
> >>>>>>
> >>>>>>
> >>>>> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
> >>>>>
> >>>>>
> >>>> Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
> >>>>
> >>>>
> >>>>
> >>> Might seem like a silly question, but has anyone found anything that
> >>> looks like a threading issue in Codec?
> >>>
> >> Yes -  QCodec has a public method that sets an instance variable (see
> >> my comments on CODEC-55).
> >>
> >>
> >>> The code is pretty simple.  I did a quick look through the code, and it
> >>> seemed like everything was pretty thread safe.  It looks like you should
> >>> be O.K. as long as:
> >>>    1)  You don't change the parameters to a codec while it's running,
> >>> e.g. changing charsets, etc.
> >>>
> >> See above.
> >>
> >> Having said that, I think it would not be too difficult to fix the
> >> classes so that they are thread-safe. Most of the ones I looked at
> >> could even be made invariant.
> >>
> >> So their thread-safety would only depend on any external calls they
> >> made; this could presumably be fixed by providing synchronised
> >> versions as suggested earlier.
> >>
> >>
> >>>    2)  You should not use a MessageDigest returned by DigestUtils by
> >>> multiple threads.  This should be clear given the API (it's a SUN api)
> >>>
> >>> It also seems to me that the difference between code that you can trust
> >>> as being thread safe, and code you cannot trust as being thread safe has
> >>> a lot to do with whether it is tested for thread safety.  Seems to me
> >>> like another way of helping out here would be to build concurrency tests
> >>> for whichever APIs you are interested using on multiple threads.  Not
> >>> sure where we would put them yet, but seems like the only way to assure
> >>> thread-safety (especially if we could get someone to volunteer running
> >>> them on a big beefy multi-proc machine :).
> >>>
> >> Indeed.
> >>
> >>
> >>>>> - Jörg
> >>>>>
> >>>>>
> >>>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >>> For additional commands, e-mail: user-help@commons.apache.org
> >>>
> >>>
> >>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: user-help@commons.apache.org
> >>
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Qingtian Wang <qi...@gmail.com>]

The SLA of the project I am working on is 2000 transactions per
second. And I need to decode a 1K string on each request.....

-Q



On 10/8/07, Will Pugh <wi...@sourcelabs.com> wrote:
> Ya know.  Just to put this in a little perspective.
>
> These objects are pretty small (QCodec has only two members in it), and
> have practically no instantiation cost.  Instantiating these and quickly
> throwing them out is probably not such a bad idea, and may give you
> better performance that doing anything with ThreadLocal.  Java VMs are
> fairly well tuned for dealing with short-lived, small objects (which
> these are).
>
> I can't imagine that many people are using thousands of these per
> request, probably, more on the order of one or two a request?
>
>     --Will
>
> James Carman wrote:
> > Try using a ThreadLocal variable to store your Encoder/Decoder that
> > you need to be thread safe.
> >
> > On 10/8/07, sebb <se...@gmail.com> wrote:
> >
> >> On 08/10/2007, Will Pugh <wi...@sourcelabs.com> wrote:
> >>
> >>> David J. Biesack wrote:
> >>>
> >>>>> Date: Mon, 8 Oct 2007 17:01:02 +0200
> >>>>> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> The java.util.concurrent backport
> >>>>>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> >>>>>> kind of use.
> >>>>>>
> >>>>>>
> >>>>> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
> >>>>>
> >>>>>
> >>>> Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
> >>>>
> >>>>
> >>>>
> >>> Might seem like a silly question, but has anyone found anything that
> >>> looks like a threading issue in Codec?
> >>>
> >> Yes -  QCodec has a public method that sets an instance variable (see
> >> my comments on CODEC-55).
> >>
> >>
> >>> The code is pretty simple.  I did a quick look through the code, and it
> >>> seemed like everything was pretty thread safe.  It looks like you should
> >>> be O.K. as long as:
> >>>    1)  You don't change the parameters to a codec while it's running,
> >>> e.g. changing charsets, etc.
> >>>
> >> See above.
> >>
> >> Having said that, I think it would not be too difficult to fix the
> >> classes so that they are thread-safe. Most of the ones I looked at
> >> could even be made invariant.
> >>
> >> So their thread-safety would only depend on any external calls they
> >> made; this could presumably be fixed by providing synchronised
> >> versions as suggested earlier.
> >>
> >>
> >>>    2)  You should not use a MessageDigest returned by DigestUtils by
> >>> multiple threads.  This should be clear given the API (it's a SUN api)
> >>>
> >>> It also seems to me that the difference between code that you can trust
> >>> as being thread safe, and code you cannot trust as being thread safe has
> >>> a lot to do with whether it is tested for thread safety.  Seems to me
> >>> like another way of helping out here would be to build concurrency tests
> >>> for whichever APIs you are interested using on multiple threads.  Not
> >>> sure where we would put them yet, but seems like the only way to assure
> >>> thread-safety (especially if we could get someone to volunteer running
> >>> them on a big beefy multi-proc machine :).
> >>>
> >> Indeed.
> >>
> >>
> >>>>> - Jörg
> >>>>>
> >>>>>
> >>>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >>> For additional commands, e-mail: user-help@commons.apache.org
> >>>
> >>>
> >>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: user-help@commons.apache.org
> >>
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Will Pugh <wi...@sourcelabs.com>]

Ya know.  Just to put this in a little perspective. 

These objects are pretty small (QCodec has only two members in it), and 
have practically no instantiation cost.  Instantiating these and quickly 
throwing them out is probably not such a bad idea, and may give you 
better performance that doing anything with ThreadLocal.  Java VMs are 
fairly well tuned for dealing with short-lived, small objects (which 
these are).

I can't imagine that many people are using thousands of these per 
request, probably, more on the order of one or two a request?

    --Will

James Carman wrote:
> Try using a ThreadLocal variable to store your Encoder/Decoder that
> you need to be thread safe.
>
> On 10/8/07, sebb <se...@gmail.com> wrote:
>   
>> On 08/10/2007, Will Pugh <wi...@sourcelabs.com> wrote:
>>     
>>> David J. Biesack wrote:
>>>       
>>>>> Date: Mon, 8 Oct 2007 17:01:02 +0200
>>>>> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
>>>>>
>>>>>
>>>>>           
>>>>>> The java.util.concurrent backport
>>>>>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
>>>>>> kind of use.
>>>>>>
>>>>>>             
>>>>> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
>>>>>
>>>>>           
>>>> Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
>>>>
>>>>
>>>>         
>>> Might seem like a silly question, but has anyone found anything that
>>> looks like a threading issue in Codec?
>>>       
>> Yes -  QCodec has a public method that sets an instance variable (see
>> my comments on CODEC-55).
>>
>>     
>>> The code is pretty simple.  I did a quick look through the code, and it
>>> seemed like everything was pretty thread safe.  It looks like you should
>>> be O.K. as long as:
>>>    1)  You don't change the parameters to a codec while it's running,
>>> e.g. changing charsets, etc.
>>>       
>> See above.
>>
>> Having said that, I think it would not be too difficult to fix the
>> classes so that they are thread-safe. Most of the ones I looked at
>> could even be made invariant.
>>
>> So their thread-safety would only depend on any external calls they
>> made; this could presumably be fixed by providing synchronised
>> versions as suggested earlier.
>>
>>     
>>>    2)  You should not use a MessageDigest returned by DigestUtils by
>>> multiple threads.  This should be clear given the API (it's a SUN api)
>>>
>>> It also seems to me that the difference between code that you can trust
>>> as being thread safe, and code you cannot trust as being thread safe has
>>> a lot to do with whether it is tested for thread safety.  Seems to me
>>> like another way of helping out here would be to build concurrency tests
>>> for whichever APIs you are interested using on multiple threads.  Not
>>> sure where we would put them yet, but seems like the only way to assure
>>> thread-safety (especially if we could get someone to volunteer running
>>> them on a big beefy multi-proc machine :).
>>>       
>> Indeed.
>>
>>     
>>>>> - Jörg
>>>>>
>>>>>           
>>>>         
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: user-help@commons.apache.org
>>>
>>>
>>>       
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>>
>>
>>     
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>   

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[James Carman <ja...@carmanconsulting.com>]

Try using a ThreadLocal variable to store your Encoder/Decoder that
you need to be thread safe.

On 10/8/07, sebb <se...@gmail.com> wrote:
> On 08/10/2007, Will Pugh <wi...@sourcelabs.com> wrote:
> > David J. Biesack wrote:
> > >> Date: Mon, 8 Oct 2007 17:01:02 +0200
> > >> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> > >>
> > >>
> > >>> The java.util.concurrent backport
> > >>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> > >>> kind of use.
> > >>>
> > >> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
> > >>
> > >
> > > Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
> > >
> > >
> > Might seem like a silly question, but has anyone found anything that
> > looks like a threading issue in Codec?
>
> Yes -  QCodec has a public method that sets an instance variable (see
> my comments on CODEC-55).
>
> > The code is pretty simple.  I did a quick look through the code, and it
> > seemed like everything was pretty thread safe.  It looks like you should
> > be O.K. as long as:
> >    1)  You don't change the parameters to a codec while it's running,
> > e.g. changing charsets, etc.
>
> See above.
>
> Having said that, I think it would not be too difficult to fix the
> classes so that they are thread-safe. Most of the ones I looked at
> could even be made invariant.
>
> So their thread-safety would only depend on any external calls they
> made; this could presumably be fixed by providing synchronised
> versions as suggested earlier.
>
> >    2)  You should not use a MessageDigest returned by DigestUtils by
> > multiple threads.  This should be clear given the API (it's a SUN api)
> >
> > It also seems to me that the difference between code that you can trust
> > as being thread safe, and code you cannot trust as being thread safe has
> > a lot to do with whether it is tested for thread safety.  Seems to me
> > like another way of helping out here would be to build concurrency tests
> > for whichever APIs you are interested using on multiple threads.  Not
> > sure where we would put them yet, but seems like the only way to assure
> > thread-safety (especially if we could get someone to volunteer running
> > them on a big beefy multi-proc machine :).
>
> Indeed.
>
> > >> - Jörg
> > >>
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[sebb <se...@gmail.com>]

On 08/10/2007, Will Pugh <wi...@sourcelabs.com> wrote:
> David J. Biesack wrote:
> >> Date: Mon, 8 Oct 2007 17:01:02 +0200
> >> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> >>
> >>
> >>> The java.util.concurrent backport
> >>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> >>> kind of use.
> >>>
> >> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
> >>
> >
> > Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
> >
> >
> Might seem like a silly question, but has anyone found anything that
> looks like a threading issue in Codec?

Yes -  QCodec has a public method that sets an instance variable (see
my comments on CODEC-55).

> The code is pretty simple.  I did a quick look through the code, and it
> seemed like everything was pretty thread safe.  It looks like you should
> be O.K. as long as:
>    1)  You don't change the parameters to a codec while it's running,
> e.g. changing charsets, etc.

See above.

Having said that, I think it would not be too difficult to fix the
classes so that they are thread-safe. Most of the ones I looked at
could even be made invariant.

So their thread-safety would only depend on any external calls they
made; this could presumably be fixed by providing synchronised
versions as suggested earlier.

>    2)  You should not use a MessageDigest returned by DigestUtils by
> multiple threads.  This should be clear given the API (it's a SUN api)
>
> It also seems to me that the difference between code that you can trust
> as being thread safe, and code you cannot trust as being thread safe has
> a lot to do with whether it is tested for thread safety.  Seems to me
> like another way of helping out here would be to build concurrency tests
> for whichever APIs you are interested using on multiple threads.  Not
> sure where we would put them yet, but seems like the only way to assure
> thread-safety (especially if we could get someone to volunteer running
> them on a big beefy multi-proc machine :).

Indeed.

> >> - Jörg
> >>
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Will Pugh <wi...@sourcelabs.com>]

David J. Biesack wrote:
>> Date: Mon, 8 Oct 2007 17:01:02 +0200
>> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
>>
>>     
>>> The java.util.concurrent backport
>>> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
>>> kind of use. 
>>>       
>> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)
>>     
>
> Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.
>
>   
Might seem like a silly question, but has anyone found anything that 
looks like a threading issue in Codec? 

The code is pretty simple.  I did a quick look through the code, and it 
seemed like everything was pretty thread safe.  It looks like you should 
be O.K. as long as:
    1)  You don't change the parameters to a codec while it's running, 
e.g. changing charsets, etc.
    2)  You should not use a MessageDigest returned by DigestUtils by 
multiple threads.  This should be clear given the API (it's a SUN api)

It also seems to me that the difference between code that you can trust 
as being thread safe, and code you cannot trust as being thread safe has 
a lot to do with whether it is tested for thread safety.  Seems to me 
like another way of helping out here would be to build concurrency tests 
for whichever APIs you are interested using on multiple threads.  Not 
sure where we would put them yet, but seems like the only way to assure 
thread-safety (especially if we could get someone to volunteer running 
them on a big beefy multi-proc machine :).
>> - Jörg
>>     
>
>   

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

["David J. Biesack" <Da...@sas.com>]

> Date: Mon, 8 Oct 2007 17:01:02 +0200
> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> 
> > The java.util.concurrent backport
> > http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> > kind of use. 
> 
> I know this, but I doubt that we wanna start to depend Apache common components on it ;-)

Agreed; I, like you, was merely suggesting to what Commons consumers might do to deal with non-threadsafe codecs.

> - J�rg

-- 
David J. Biesack     SAS Institute Inc.
(919) 531-7771       SAS Campus Drive
http://www.sas.com   Cary, NC 27513


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

RE: codec - thread safe

[Jörg Schaible <Jo...@Elsag-Solutions.com>]

David J. Biesack wrote on Monday, October 08, 2007 4:40 PM:

>> Date: Mon, 8 Oct 2007 16:23:59 +0200
>> From: =?iso-8859-1?Q?J=F6rg_Schaible?=
> <Jo...@Elsag-Solutions.com>
>> 
>> David J. Biesack wrote on Monday, October 08, 2007 3:02 PM:
>> 
>>>> Date: Sat, 6 Oct 2007 23:31:19 -0500
>>>> From: "Qingtian Wang" <qi...@gmail.com>
>>>> 
>>>> Well, it's pick-your-poison kind of a deal. Either block on one
>>>> instance and take a performance hit, or burn up the memory with
>>>> lots of instances.....
>>> 
>>> Why not compromise? Create a ThreadPoolExecutor ...
>> 
>> Because it runs on JDK 1.3?
> 
> The java.util.concurrent backport
> http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this
> kind of use. 

I know this, but I doubt that we wanna start to depend Apache common components on it ;-)

- Jörg

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

["David J. Biesack" <Da...@sas.com>]

> Date: Mon, 8 Oct 2007 16:23:59 +0200
> From: =?iso-8859-1?Q?J=F6rg_Schaible?= <Jo...@Elsag-Solutions.com>
> 
> David J. Biesack wrote on Monday, October 08, 2007 3:02 PM:
> 
> >> Date: Sat, 6 Oct 2007 23:31:19 -0500
> >> From: "Qingtian Wang" <qi...@gmail.com>
> >> 
> >> Well, it's pick-your-poison kind of a deal. Either block on one
> >> instance and take a performance hit, or burn up the memory with lots
> >> of instances.....
> > 
> > Why not compromise? Create a ThreadPoolExecutor ...
> 
> Because it runs on JDK 1.3?

The java.util.concurrent backport http://backport-jsr166.sourceforge.net/ runs on 1.3, for just this kind of use.

> However, that's the reason why I argumented not to promise thread-safety for any codec and provide synchronization wrappers or a user might take the pool approach ... which is quite easy with JDK 5 as you've provided here :)

I think it makes a lot of sense to document the thread safety attributes of Commons libraries.

> - J�rg

-- 
David J. Biesack     SAS Institute Inc.
(919) 531-7771       SAS Campus Drive
http://www.sas.com   Cary, NC 27513


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

RE: codec - thread safe

[Jörg Schaible <Jo...@Elsag-Solutions.com>]

David J. Biesack wrote on Monday, October 08, 2007 3:02 PM:

>> Date: Sat, 6 Oct 2007 23:31:19 -0500
>> From: "Qingtian Wang" <qi...@gmail.com>
>> 
>> Well, it's pick-your-poison kind of a deal. Either block on one
>> instance and take a performance hit, or burn up the memory with lots
>> of instances.....
> 
> Why not compromise? Create a ThreadPoolExecutor of some
> reasonable size and submit a FutureTask
> to run the encode or decode, and then do a future.get() when
> the value is needed. You might
> even get some concurrency if you can start the encode, then continue
> doing other work until you need the result.

Because it runs on JDK 1.3? However, that's the reason why I argumented not to promise thread-safety for any codec and provide synchronization wrappers or a user might take the pool approach ... which is quite easy with JDK 5 as you've provided here :)

- Jörg

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

["David J. Biesack" <Da...@sas.com>]

> Date: Sat, 6 Oct 2007 23:31:19 -0500
> From: "Qingtian Wang" <qi...@gmail.com>
> 
> Well, it's pick-your-poison kind of a deal. Either block on one
> instance and take a performance hit, or burn up the memory with lots
> of instances.....

Why not compromise? Create a ThreadPoolExecutor of some reasonable size and submit a FutureTask
to run the encode or decode, and then do a future.get() when the value is needed. You might
even get some concurrency if you can start the encode, then continue doing other work
until you need the result.

-- 
David J. Biesack     SAS Institute Inc.
(919) 531-7771       SAS Campus Drive
http://www.sas.com   Cary, NC 27513


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Qingtian Wang <qi...@gmail.com>]

Well, it's pick-your-poison kind of a deal. Either block on one
instance and take a performance hit, or burn up the memory with lots
of instances.....

But in the case of BCodec, I think encode/decode is thread safe.
Unfortunately per Henri, that's not generally true for others.

Well, let me make it clear that I am a total layman on codec. But it
seems to me it's not that difficult to implement all the codec methods
in a thread safe manner, without sync blocks.

Can the dev team make that happen? - a humble request from a user.

Thanks,
-Q




On 10/6/07, ben short <ja...@gmail.com> wrote:
> How about...
>
> MyStringCodec {
>    BCodec delegate = new BCodec();
>
>    String encode(String in) {
>
>        String result = null;
>
>        synchronized(delegate)
>            {
>            result = delegate.encode(in);
>            }
>
>        return result;
>
>    }
> }
>
>
> On 10/6/07, Qingtian Wang <qi...@gmail.com> wrote:
> > Henri,
> >
> > That was an unpleasant surprise.
> >
> > So what would be the general suggested program pattern to use the API
> > if one wants to be thread safe?
> >
> > Is it 1:
> >
> > MyStringCodec {
> >     String encode(String in) {
> >         return new BCodec().encode(in);
> >     }
> > }
> >
> > or 2:
> >
> > MyStringCodec {
> >     BCodec delegate = new BCodec();
> >
> >     String encode(String in) {
> >         return delegate.encode(in);
> >     }
> > }
> >
> > Option 2 apparently won't work in a multi-thread scenario if BCodec is
> > thread unsafe. But option 1 really creates A LOT of BCodec objects in
> > memory....
> >
> > -Q
> >
> >
> >
> >
> > On 10/5/07, Henri Yandell <fl...@gmail.com> wrote:
> > > On 9/29/07, Qingtian Wang <qi...@gmail.com> wrote:
> > > > I apologize for this question as it must have been asked a million
> > > > times: I was unable to search the mailing list archive for this.
> > > >
> > > > Are all the "encode/decode" methods in commons-codec intended to be thread safe?
> > > >
> > > > I peeked into the source code for a couple of those methods. They both
> > > > seem to be thread safe. But I'd rather ask here to make sure that's
> > > > the general intention when the code is written.
> > >
> > > I think you'll find it varies. The various algorithms come from
> > > different sources, and though a common interface was put on them all,
> > > I doubt that time was spent to make sure they were all threadsafe.
> > >
> > > Looking at the objects, I see a few with private attributes that are
> > > set at more than just construction time - so very likely that one or
> > > another is thread unsafe.
> > >
> > > Hen
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: user-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[ben short <ja...@gmail.com>]

How about...

MyStringCodec {
   BCodec delegate = new BCodec();

   String encode(String in) {

       String result = null;

       synchronized(delegate)
           {
           result = delegate.encode(in);
           }

       return result;

   }
}


On 10/6/07, Qingtian Wang <qi...@gmail.com> wrote:
> Henri,
>
> That was an unpleasant surprise.
>
> So what would be the general suggested program pattern to use the API
> if one wants to be thread safe?
>
> Is it 1:
>
> MyStringCodec {
>     String encode(String in) {
>         return new BCodec().encode(in);
>     }
> }
>
> or 2:
>
> MyStringCodec {
>     BCodec delegate = new BCodec();
>
>     String encode(String in) {
>         return delegate.encode(in);
>     }
> }
>
> Option 2 apparently won't work in a multi-thread scenario if BCodec is
> thread unsafe. But option 1 really creates A LOT of BCodec objects in
> memory....
>
> -Q
>
>
>
>
> On 10/5/07, Henri Yandell <fl...@gmail.com> wrote:
> > On 9/29/07, Qingtian Wang <qi...@gmail.com> wrote:
> > > I apologize for this question as it must have been asked a million
> > > times: I was unable to search the mailing list archive for this.
> > >
> > > Are all the "encode/decode" methods in commons-codec intended to be thread safe?
> > >
> > > I peeked into the source code for a couple of those methods. They both
> > > seem to be thread safe. But I'd rather ask here to make sure that's
> > > the general intention when the code is written.
> >
> > I think you'll find it varies. The various algorithms come from
> > different sources, and though a common interface was put on them all,
> > I doubt that time was spent to make sure they were all threadsafe.
> >
> > Looking at the objects, I see a few with private attributes that are
> > set at more than just construction time - so very likely that one or
> > another is thread unsafe.
> >
> > Hen
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: codec - thread safe

[Qingtian Wang <qi...@gmail.com>]

Henri,

That was an unpleasant surprise.

So what would be the general suggested program pattern to use the API
if one wants to be thread safe?

Is it 1:

MyStringCodec {
    String encode(String in) {
        return new BCodec().encode(in);
    }
}

or 2:

MyStringCodec {
    BCodec delegate = new BCodec();

    String encode(String in) {
        return delegate.encode(in);
    }
}

Option 2 apparently won't work in a multi-thread scenario if BCodec is
thread unsafe. But option 1 really creates A LOT of BCodec objects in
memory....

-Q




On 10/5/07, Henri Yandell <fl...@gmail.com> wrote:
> On 9/29/07, Qingtian Wang <qi...@gmail.com> wrote:
> > I apologize for this question as it must have been asked a million
> > times: I was unable to search the mailing list archive for this.
> >
> > Are all the "encode/decode" methods in commons-codec intended to be thread safe?
> >
> > I peeked into the source code for a couple of those methods. They both
> > seem to be thread safe. But I'd rather ask here to make sure that's
> > the general intention when the code is written.
>
> I think you'll find it varies. The various algorithms come from
> different sources, and though a common interface was put on them all,
> I doubt that time was spent to make sure they were all threadsafe.
>
> Looking at the objects, I see a few with private attributes that are
> set at more than just construction time - so very likely that one or
> another is thread unsafe.
>
> Hen
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org