You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Ken Giusti (JIRA)" <qp...@incubator.apache.org> on 2009/11/04 17:27:32 UTC
[jira] Created: (QPID-2187) Allow clients to make
secure/authenticated connections to a cluster.
Allow clients to make secure/authenticated connections to a cluster.
--------------------------------------------------------------------
Key: QPID-2187
URL: https://issues.apache.org/jira/browse/QPID-2187
Project: Qpid
Issue Type: Improvement
Environment: all
Reporter: Ken Giusti
The current implementation of clustering does not correctly handle authentication correctly. From the trunk build:
[kgiusti@localhost src]$ ./qpidd --auth yes --realm KGIUSTI.COM --log-enable info+ --load-module ./.libs/cluster.so --cluster-name ken
2009-11-02 10:30:58 info Loaded Module: ./.libs/cluster.so
2009-11-02 10:30:58 info Management enabled
2009-11-02 10:30:58 notice Initializing CPG
2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) membership change: 127.0.0.1:14581 (joined: 127.0.0.1:14581(joined) )
2009-11-02 10:30:58 info No message store configured, persistence is disabled.
2009-11-02 10:30:58 info SASL enabled
2009-11-02 10:30:58 notice Listening on TCP port 5672
2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) joining cluster ken with url=amqp:tcp:10.16.19.19:5672,tcp:10.16.14.69:5672,tcp:192.168.122.1:5672
2009-11-02 10:30:58 notice Broker running
2009-11-02 10:30:58 info cluster(127.0.0.1:14581 READY) member update: 127.0.0.1:14581(member)
2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 READY) first in cluster
2009-11-02 10:31:05 info SASL: Mechanism list: ANONYMOUS PLAIN DIGEST-MD5 LOGIN GSSAPI CRAM-MD5
2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) new local connection 127.0.0.1:14581-1
2009-11-02 10:31:05 info SASL: Starting authentication with mechanism: GSSAPI
2009-11-02 10:31:05 info SASL: Authentication succeeded for: testuser@KGIUSTI.COM
2009-11-02 10:31:05 error cluster(127.0.0.1:14581 READY) aborting connection 127.0.0.1:14581-1: framing-error: Reserved bits not zero (qpid/framing/AMQFrame.cpp:132)
2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) connection closed 127.0.0.1:14581-1
The above error occurs when running perftest against the cluster in the following manner:
[kgiusti@localhost tests]$ /usr/kerberos/bin/kinit testuser@KGIUSTI.COM
[kgiusti@localhost tests]$ ./perftest -b localhost.localdomain --mechanism GSSAPI --username testuser --tx 1 --count 1 --summary --log-enable info+
2009-11-02 10:31:05 info Connecting to tcp:localhost.localdomain:5672
2009-11-02 10:31:05 info Installing security layer, SSF: 56
2009-11-02 10:31:05 warning Connection closed
Running the same test, but turning off clustering, authentication succeeds.
Alan has determined that the problem is due to the way the clustered broker constructs the codec chain. The chain is built without the codec for a secure connection.
The correct solution would implement a mechanism that allows more generic chaining of the codecs. It should be possible to allow codecs to be built that support both clustering and security/authentication.
In this case, the fix would secure the client/broker connection, and mirror the unencrypted data across the cluster.
Does this make sense? Opinions welcome.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Assigned: (QPID-2187) Allow clients to make
secure/authenticated connections to a cluster.
Posted by "Gordon Sim (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gordon Sim reassigned QPID-2187:
--------------------------------
Assignee: michael j. goulish
> Allow clients to make secure/authenticated connections to a cluster.
> --------------------------------------------------------------------
>
> Key: QPID-2187
> URL: https://issues.apache.org/jira/browse/QPID-2187
> Project: Qpid
> Issue Type: Improvement
> Environment: all
> Reporter: Ken Giusti
> Assignee: michael j. goulish
>
> The current implementation of clustering does not correctly handle authentication correctly. From the trunk build:
> [kgiusti@localhost src]$ ./qpidd --auth yes --realm KGIUSTI.COM --log-enable info+ --load-module ./.libs/cluster.so --cluster-name ken
> 2009-11-02 10:30:58 info Loaded Module: ./.libs/cluster.so
> 2009-11-02 10:30:58 info Management enabled
> 2009-11-02 10:30:58 notice Initializing CPG
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) membership change: 127.0.0.1:14581 (joined: 127.0.0.1:14581(joined) )
> 2009-11-02 10:30:58 info No message store configured, persistence is disabled.
> 2009-11-02 10:30:58 info SASL enabled
> 2009-11-02 10:30:58 notice Listening on TCP port 5672
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) joining cluster ken with url=amqp:tcp:10.16.19.19:5672,tcp:10.16.14.69:5672,tcp:192.168.122.1:5672
> 2009-11-02 10:30:58 notice Broker running
> 2009-11-02 10:30:58 info cluster(127.0.0.1:14581 READY) member update: 127.0.0.1:14581(member)
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 READY) first in cluster
> 2009-11-02 10:31:05 info SASL: Mechanism list: ANONYMOUS PLAIN DIGEST-MD5 LOGIN GSSAPI CRAM-MD5
> 2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) new local connection 127.0.0.1:14581-1
> 2009-11-02 10:31:05 info SASL: Starting authentication with mechanism: GSSAPI
> 2009-11-02 10:31:05 info SASL: Authentication succeeded for: testuser@KGIUSTI.COM
> 2009-11-02 10:31:05 error cluster(127.0.0.1:14581 READY) aborting connection 127.0.0.1:14581-1: framing-error: Reserved bits not zero (qpid/framing/AMQFrame.cpp:132)
> 2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) connection closed 127.0.0.1:14581-1
>
> The above error occurs when running perftest against the cluster in the following manner:
> [kgiusti@localhost tests]$ /usr/kerberos/bin/kinit testuser@KGIUSTI.COM
> [kgiusti@localhost tests]$ ./perftest -b localhost.localdomain --mechanism GSSAPI --username testuser --tx 1 --count 1 --summary --log-enable info+
> 2009-11-02 10:31:05 info Connecting to tcp:localhost.localdomain:5672
> 2009-11-02 10:31:05 info Installing security layer, SSF: 56
> 2009-11-02 10:31:05 warning Connection closed
> Running the same test, but turning off clustering, authentication succeeds.
> Alan has determined that the problem is due to the way the clustered broker constructs the codec chain. The chain is built without the codec for a secure connection.
> The correct solution would implement a mechanism that allows more generic chaining of the codecs. It should be possible to allow codecs to be built that support both clustering and security/authentication.
> In this case, the fix would secure the client/broker connection, and mirror the unencrypted data across the cluster.
> Does this make sense? Opinions welcome.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-2187) Allow clients to make
secure/authenticated connections to a cluster.
Posted by "michael j. goulish (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
michael j. goulish updated QPID-2187:
-------------------------------------
Attachment: 944158.diff
Cluster + Security
-----------------------------------
* initial observation of a problem was a 2% failure rate in perftests
of 20,000 messages against a cluster with security enabled.
Problem was occasional receit of encrypted frames before the
security codec had been enabled. This is fixed with locking in
cluster code (no new locks in broker code) and a callback that is
fired by broker::ConnectionHandler::Handler to tell the cluster
code when the opening handshake has finished.
This was never a problem in the non-clustered broker before because
everything happened in a single thread.
* the brokers that "shadow" the connection must not have null
authenticators rather than real ones, so that they go through all
the motions but don't do anythig. Only the directly-connected
broker can perform the security handshake.
* once the directly-connected broker receives the real user ID
from its callback, it mcasts that ID to all other brokers.
Otherwise the shadowing brokers will al think that the user ID
is "anonymous".
Check this by doing a substantial perftest, and using
qpid-stat -c localhost:PORT
to confirm that the brokers all have the same userID for the
same connection.
* the user ID, negotiated during the Sasl security startup, is
communicated from the directly connected broker to all other
cluster brokers.
* If security is *not* being used, then this code should *not* tell
the brokers anything about the userID -- or it will step on the value
that is being set by other code pathways.
* test program at cpp/src/tests/cluster_authentication_soak is not yet
fully automated -- run it with something like
"sudo ./cluster_authentication_soak 500"
> Allow clients to make secure/authenticated connections to a cluster.
> --------------------------------------------------------------------
>
> Key: QPID-2187
> URL: https://issues.apache.org/jira/browse/QPID-2187
> Project: Qpid
> Issue Type: Improvement
> Environment: all
> Reporter: Ken Giusti
> Assignee: michael j. goulish
> Attachments: 944158.diff
>
>
> The current implementation of clustering does not correctly handle authentication correctly. From the trunk build:
> [kgiusti@localhost src]$ ./qpidd --auth yes --realm KGIUSTI.COM --log-enable info+ --load-module ./.libs/cluster.so --cluster-name ken
> 2009-11-02 10:30:58 info Loaded Module: ./.libs/cluster.so
> 2009-11-02 10:30:58 info Management enabled
> 2009-11-02 10:30:58 notice Initializing CPG
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) membership change: 127.0.0.1:14581 (joined: 127.0.0.1:14581(joined) )
> 2009-11-02 10:30:58 info No message store configured, persistence is disabled.
> 2009-11-02 10:30:58 info SASL enabled
> 2009-11-02 10:30:58 notice Listening on TCP port 5672
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) joining cluster ken with url=amqp:tcp:10.16.19.19:5672,tcp:10.16.14.69:5672,tcp:192.168.122.1:5672
> 2009-11-02 10:30:58 notice Broker running
> 2009-11-02 10:30:58 info cluster(127.0.0.1:14581 READY) member update: 127.0.0.1:14581(member)
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 READY) first in cluster
> 2009-11-02 10:31:05 info SASL: Mechanism list: ANONYMOUS PLAIN DIGEST-MD5 LOGIN GSSAPI CRAM-MD5
> 2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) new local connection 127.0.0.1:14581-1
> 2009-11-02 10:31:05 info SASL: Starting authentication with mechanism: GSSAPI
> 2009-11-02 10:31:05 info SASL: Authentication succeeded for: testuser@KGIUSTI.COM
> 2009-11-02 10:31:05 error cluster(127.0.0.1:14581 READY) aborting connection 127.0.0.1:14581-1: framing-error: Reserved bits not zero (qpid/framing/AMQFrame.cpp:132)
> 2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) connection closed 127.0.0.1:14581-1
>
> The above error occurs when running perftest against the cluster in the following manner:
> [kgiusti@localhost tests]$ /usr/kerberos/bin/kinit testuser@KGIUSTI.COM
> [kgiusti@localhost tests]$ ./perftest -b localhost.localdomain --mechanism GSSAPI --username testuser --tx 1 --count 1 --summary --log-enable info+
> 2009-11-02 10:31:05 info Connecting to tcp:localhost.localdomain:5672
> 2009-11-02 10:31:05 info Installing security layer, SSF: 56
> 2009-11-02 10:31:05 warning Connection closed
> Running the same test, but turning off clustering, authentication succeeds.
> Alan has determined that the problem is due to the way the clustered broker constructs the codec chain. The chain is built without the codec for a secure connection.
> The correct solution would implement a mechanism that allows more generic chaining of the codecs. It should be possible to allow codecs to be built that support both clustering and security/authentication.
> In this case, the fix would secure the client/broker connection, and mirror the unencrypted data across the cluster.
> Does this make sense? Opinions welcome.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Closed: (QPID-2187) Allow clients to make
secure/authenticated connections to a cluster.
Posted by "michael j. goulish (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
michael j. goulish closed QPID-2187.
------------------------------------
Resolution: Fixed
Cluster + Security
-----------------------------------
* initial observation of a problem was a 2% failure rate in perftests
of 20,000 messages against a cluster with security enabled.
Problem was occasional receit of encrypted frames before the
security codec had been enabled. This is fixed with locking in
cluster code (no new locks in broker code) and a callback that is
fired by broker::ConnectionHandler::Handler to tell the cluster
code when the opening handshake has finished.
This was never a problem in the non-clustered broker before because
everything happened in a single thread.
* the brokers that "shadow" the connection must not have null
authenticators rather than real ones, so that they go through all
the motions but don't do anythig. Only the directly-connected
broker can perform the security handshake.
* once the directly-connected broker receives the real user ID
from its callback, it mcasts that ID to all other brokers.
Otherwise the shadowing brokers will al think that the user ID
is "anonymous".
Check this by doing a substantial perftest, and using
qpid-stat -c localhost:PORT
to confirm that the brokers all have the same userID for the
same connection.
* the user ID, negotiated during the Sasl security startup, is
communicated from the directly connected broker to all other
cluster brokers.
* If security is *not* being used, then this code should *not* tell
the brokers anything about the userID -- or it will step on the value
that is being set by other code pathways.
* test program at cpp/src/tests/cluster_authentication_soak is not yet
fully automated -- run it with something like
"sudo ./cluster_authentication_soak 500"
> Allow clients to make secure/authenticated connections to a cluster.
> --------------------------------------------------------------------
>
> Key: QPID-2187
> URL: https://issues.apache.org/jira/browse/QPID-2187
> Project: Qpid
> Issue Type: Improvement
> Environment: all
> Reporter: Ken Giusti
> Assignee: michael j. goulish
> Attachments: 944158.diff
>
>
> The current implementation of clustering does not correctly handle authentication correctly. From the trunk build:
> [kgiusti@localhost src]$ ./qpidd --auth yes --realm KGIUSTI.COM --log-enable info+ --load-module ./.libs/cluster.so --cluster-name ken
> 2009-11-02 10:30:58 info Loaded Module: ./.libs/cluster.so
> 2009-11-02 10:30:58 info Management enabled
> 2009-11-02 10:30:58 notice Initializing CPG
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) membership change: 127.0.0.1:14581 (joined: 127.0.0.1:14581(joined) )
> 2009-11-02 10:30:58 info No message store configured, persistence is disabled.
> 2009-11-02 10:30:58 info SASL enabled
> 2009-11-02 10:30:58 notice Listening on TCP port 5672
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) joining cluster ken with url=amqp:tcp:10.16.19.19:5672,tcp:10.16.14.69:5672,tcp:192.168.122.1:5672
> 2009-11-02 10:30:58 notice Broker running
> 2009-11-02 10:30:58 info cluster(127.0.0.1:14581 READY) member update: 127.0.0.1:14581(member)
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 READY) first in cluster
> 2009-11-02 10:31:05 info SASL: Mechanism list: ANONYMOUS PLAIN DIGEST-MD5 LOGIN GSSAPI CRAM-MD5
> 2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) new local connection 127.0.0.1:14581-1
> 2009-11-02 10:31:05 info SASL: Starting authentication with mechanism: GSSAPI
> 2009-11-02 10:31:05 info SASL: Authentication succeeded for: testuser@KGIUSTI.COM
> 2009-11-02 10:31:05 error cluster(127.0.0.1:14581 READY) aborting connection 127.0.0.1:14581-1: framing-error: Reserved bits not zero (qpid/framing/AMQFrame.cpp:132)
> 2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) connection closed 127.0.0.1:14581-1
>
> The above error occurs when running perftest against the cluster in the following manner:
> [kgiusti@localhost tests]$ /usr/kerberos/bin/kinit testuser@KGIUSTI.COM
> [kgiusti@localhost tests]$ ./perftest -b localhost.localdomain --mechanism GSSAPI --username testuser --tx 1 --count 1 --summary --log-enable info+
> 2009-11-02 10:31:05 info Connecting to tcp:localhost.localdomain:5672
> 2009-11-02 10:31:05 info Installing security layer, SSF: 56
> 2009-11-02 10:31:05 warning Connection closed
> Running the same test, but turning off clustering, authentication succeeds.
> Alan has determined that the problem is due to the way the clustered broker constructs the codec chain. The chain is built without the codec for a secure connection.
> The correct solution would implement a mechanism that allows more generic chaining of the codecs. It should be possible to allow codecs to be built that support both clustering and security/authentication.
> In this case, the fix would secure the client/broker connection, and mirror the unencrypted data across the cluster.
> Does this make sense? Opinions welcome.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-2187) Allow clients to make
secure/authenticated connections to a cluster.
Posted by "Alan Conway (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12874628#action_12874628 ]
Alan Conway commented on QPID-2187:
-----------------------------------
In revision 950608 fixed a remaining race condition that caused clients to exit with "reserved bits not 0" exception.
> Allow clients to make secure/authenticated connections to a cluster.
> --------------------------------------------------------------------
>
> Key: QPID-2187
> URL: https://issues.apache.org/jira/browse/QPID-2187
> Project: Qpid
> Issue Type: Improvement
> Environment: all
> Reporter: Ken Giusti
> Assignee: michael j. goulish
> Attachments: 944158.diff
>
>
> The current implementation of clustering does not correctly handle authentication correctly. From the trunk build:
> [kgiusti@localhost src]$ ./qpidd --auth yes --realm KGIUSTI.COM --log-enable info+ --load-module ./.libs/cluster.so --cluster-name ken
> 2009-11-02 10:30:58 info Loaded Module: ./.libs/cluster.so
> 2009-11-02 10:30:58 info Management enabled
> 2009-11-02 10:30:58 notice Initializing CPG
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) membership change: 127.0.0.1:14581 (joined: 127.0.0.1:14581(joined) )
> 2009-11-02 10:30:58 info No message store configured, persistence is disabled.
> 2009-11-02 10:30:58 info SASL enabled
> 2009-11-02 10:30:58 notice Listening on TCP port 5672
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 INIT) joining cluster ken with url=amqp:tcp:10.16.19.19:5672,tcp:10.16.14.69:5672,tcp:192.168.122.1:5672
> 2009-11-02 10:30:58 notice Broker running
> 2009-11-02 10:30:58 info cluster(127.0.0.1:14581 READY) member update: 127.0.0.1:14581(member)
> 2009-11-02 10:30:58 notice cluster(127.0.0.1:14581 READY) first in cluster
> 2009-11-02 10:31:05 info SASL: Mechanism list: ANONYMOUS PLAIN DIGEST-MD5 LOGIN GSSAPI CRAM-MD5
> 2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) new local connection 127.0.0.1:14581-1
> 2009-11-02 10:31:05 info SASL: Starting authentication with mechanism: GSSAPI
> 2009-11-02 10:31:05 info SASL: Authentication succeeded for: testuser@KGIUSTI.COM
> 2009-11-02 10:31:05 error cluster(127.0.0.1:14581 READY) aborting connection 127.0.0.1:14581-1: framing-error: Reserved bits not zero (qpid/framing/AMQFrame.cpp:132)
> 2009-11-02 10:31:05 info cluster(127.0.0.1:14581 READY) connection closed 127.0.0.1:14581-1
>
> The above error occurs when running perftest against the cluster in the following manner:
> [kgiusti@localhost tests]$ /usr/kerberos/bin/kinit testuser@KGIUSTI.COM
> [kgiusti@localhost tests]$ ./perftest -b localhost.localdomain --mechanism GSSAPI --username testuser --tx 1 --count 1 --summary --log-enable info+
> 2009-11-02 10:31:05 info Connecting to tcp:localhost.localdomain:5672
> 2009-11-02 10:31:05 info Installing security layer, SSF: 56
> 2009-11-02 10:31:05 warning Connection closed
> Running the same test, but turning off clustering, authentication succeeds.
> Alan has determined that the problem is due to the way the clustered broker constructs the codec chain. The chain is built without the codec for a secure connection.
> The correct solution would implement a mechanism that allows more generic chaining of the codecs. It should be possible to allow codecs to be built that support both clustering and security/authentication.
> In this case, the fix would secure the client/broker connection, and mirror the unencrypted data across the cluster.
> Does this make sense? Opinions welcome.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org