You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by rherbert <na...@ryan.xar.us> on 2021/06/09 13:32:10 UTC
Bad random number seed for link names
We spent a while tracking down a problem when we start up many services in
Docker containers simultaneously - when a client sent a request, we would
get responses from multiple services, even though each service should have
been bound to its own exchange.
We tracked the problem down to here:
https://github.com/apache/qpid-proton/blob/8ddf5399013b02f1fd13bda3fee7886bd48a98be/cpp/src/uuid.cpp#L45
When you're starting services in Docker containers, they all get the same
PID - so if you're starting a bunch of Docker containers simultaneously, the
odds are good that the seed will be the same across several of the services.
We were even seeing this problem across multiple hosts when the deployment
would push out a new build.
We've worked around the issue by adding service identifiers and our own
unique identifier using std::mt19937_64. I imagine our multi-container
situation is the only use case where this error would be common, but with a
lot of users, I would think that there would be intermittent issues.
Is this something I should open a ticket for? I don't have a patch for
proton - I don't know if using std::mt19937_64 is acceptable for the proton
code base.
--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Bad random number seed for link names
Posted by Jiri Daněk <jd...@redhat.com>.
There are two separate aspects in this, IMO. First, the random seed, and
then second, the pseudorandom algorithm.
The problem to me seems to be the seed. Even Mersenne Twisters will produce
duplicate UUIDs if they get seeded with the same initial seed data.
Therefore, what seems to be needed is to incorporate use
of std::random_device when seeding the UUID generator. Afaik it is a good
idea to also use Mersenne Twister instead of std::rand while making these
changes, but that is just an extra improvement to me.
If I understand correctly, in your application you are now using the
two-parameter version of container::container(messaging_handler& h, const
std::string& id) and you are providing your own id, generated using
std::mt19937_64. Is that correct?
On Wed, Jun 9, 2021 at 8:10 PM Robbie Gemmell <ro...@gmail.com>
wrote:
> I have no idea what approach might be taken if any, but this certainly
> seems worth a JIRAt o me to track the general issue at least.
>
> On Wed, 9 Jun 2021 at 14:32, rherbert <na...@ryan.xar.us> wrote:
> >
> > We spent a while tracking down a problem when we start up many services
> in
> > Docker containers simultaneously - when a client sent a request, we would
> > get responses from multiple services, even though each service should
> have
> > been bound to its own exchange.
> >
> > We tracked the problem down to here:
> >
> https://github.com/apache/qpid-proton/blob/8ddf5399013b02f1fd13bda3fee7886bd48a98be/cpp/src/uuid.cpp#L45
> >
> > When you're starting services in Docker containers, they all get the same
> > PID - so if you're starting a bunch of Docker containers simultaneously,
> the
> > odds are good that the seed will be the same across several of the
> services.
> > We were even seeing this problem across multiple hosts when the
> deployment
> > would push out a new build.
> >
> > We've worked around the issue by adding service identifiers and our own
> > unique identifier using std::mt19937_64. I imagine our multi-container
> > situation is the only use case where this error would be common, but
> with a
> > lot of users, I would think that there would be intermittent issues.
> >
> > Is this something I should open a ticket for? I don't have a patch for
> > proton - I don't know if using std::mt19937_64 is acceptable for the
> proton
> > code base.
> >
> >
> >
> > --
> > Sent from:
> http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> > For additional commands, e-mail: users-help@qpid.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
Mit freundlichen Grüßen / Kind regards
Jiri Daněk
Re: Bad random number seed for link names
Posted by Robbie Gemmell <ro...@gmail.com>.
I have no idea what approach might be taken if any, but this certainly
seems worth a JIRAt o me to track the general issue at least.
On Wed, 9 Jun 2021 at 14:32, rherbert <na...@ryan.xar.us> wrote:
>
> We spent a while tracking down a problem when we start up many services in
> Docker containers simultaneously - when a client sent a request, we would
> get responses from multiple services, even though each service should have
> been bound to its own exchange.
>
> We tracked the problem down to here:
> https://github.com/apache/qpid-proton/blob/8ddf5399013b02f1fd13bda3fee7886bd48a98be/cpp/src/uuid.cpp#L45
>
> When you're starting services in Docker containers, they all get the same
> PID - so if you're starting a bunch of Docker containers simultaneously, the
> odds are good that the seed will be the same across several of the services.
> We were even seeing this problem across multiple hosts when the deployment
> would push out a new build.
>
> We've worked around the issue by adding service identifiers and our own
> unique identifier using std::mt19937_64. I imagine our multi-container
> situation is the only use case where this error would be common, but with a
> lot of users, I would think that there would be intermittent issues.
>
> Is this something I should open a ticket for? I don't have a patch for
> proton - I don't know if using std::mt19937_64 is acceptable for the proton
> code base.
>
>
>
> --
> Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Bad random number seed for link names
Posted by rherbert <na...@ryan.xar.us>.
Thank you for the inputs, I will try to get std::random_device added in on
our side. Currently we are seeding with
std::chrono::system_clock::now().time_since_epoch().now(), but that COULD
run into the same issue (although it is much, much less likely).
I've created PROTON-2396 with all the relevant info here:
https://issues.apache.org/jira/browse/PROTON-2396
--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Bad random number seed for link names
Posted by Andrew Stitcher <as...@apache.org>.
On Wed, 2021-06-09 at 06:32 -0700, rherbert wrote:
> ...
> Is this something I should open a ticket for? I don't have a patch
> for
> proton - I don't know if using std::mt19937_64 is acceptable for the
> proton
> code base.
Thank you for isolating the cause of you problem - this is really
useful for us.
Please do open an JIRA for this - it looks like an important issue for
a number of use cases.
At this point the C++ binding is open to any C++11 standard code. I
haven't checked whether what you used is in 11 or a later standard but
we're trying to hew to C++11 with no later standards used and as few
non-portable pieces of code as possible (as typeically we have to
duplicate these several times for the various platforms we support).
Thanks
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org