You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Benoy Antony <ba...@gmail.com> on 2015/10/29 22:15:27 UTC
Re: why UserGroupInformation equals method depend on the subject object
Hi wenli,
I think, this is a bug. Could you please file a jira on the Hadoop project
so that we can try fix it if its really a bug. Please let me know the
jiraand I'll review it.
thanks ,
benoy
On Sat, Oct 17, 2015 at 7:06 AM, Wangwenli <wa...@huawei.com> wrote:
>
> Hi all,
>
> my question is why UserGroupInformation equals method depend on
> the subject object?
>
>
> try below code which is extract from HiveMetaStore:
>
> UserGroupInformation clientUgi = null;
> UserGroupInformation clientUgi2 = null;
> try {
>
> clientUgi = UserGroupInformation.createProxyUser("*user2*",
> UserGroupInformation.getLoginUser());
>
> clientUgi2 = UserGroupInformation.createProxyUser("*user2*",
> UserGroupInformation.getLoginUser());
>
> if (clientUgi.equals(clientUgi2)) {
>
> System.out.println("==");
>
> } else {
>
> System.out.println("!="); // strangely this will be hit
>
> }
> } catch (IOException e1) {
>
> e1.printStackTrace();
>
> }
>
>
> i found that it is because the equal method from
> UserGroupInformation is compare on subject object ref : *subject ==
> ((UserGroupInformation) o).subject*; .
>
> as you know, ipc.Client connect to namenode, connections.get(
> ConnectionId) this code will try to reuse the same socket to namenode,
> but because of ConnectionId's equal depend on ugi equal, which will cause
> connections.get(ConnectionId) cann't get the same socket, suppose many
> connect to HiveMetaStore, then many connection to Namenode will established.
>
> so my doubts is why UserGroupInformation is compare on subject
> object ref : *subject == ((UserGroupInformation) o).subject, it should
> compare on subject's principal, am i right?*
>
>
>
>
>
> ------------------------------
> wenli
> Regards
>
Re: Re: why UserGroupInformation equals method depend on the
subject object
Posted by Wangwenli <wa...@huawei.com>.
thanks benoy and chris's reply, i raised here: HADOOP-12529<https://issues.apache.org/jira/browse/HADOOP-12529>
________________________________
wenli
Regards
From: Chris Nauroth<ma...@hortonworks.com>
Date: 2015-10-30 05:31
To: user<ma...@hadoop.apache.org>; user@hive.apache.org<ma...@hive.apache.org>
Subject: Re: why UserGroupInformation equals method depend on the subject object
This sounds like potentially a dangerous change in the general case. If there are 2 different Subjects/UGIs with the same principal, but different credentials attached, then an equality check on principal alone could cause a get from the FileSystem cache or IPC client connection cache to return an instance with incorrect credentials.
This might be just a theoretical concern on my part. For the case of proxy users, perhaps an optimization is appropriate, since the proxied user's credentials are not in play anyway.
Regardless, a JIRA is a good idea, and we can take further discussion there.
--Chris Nauroth
From: Benoy Antony <ba...@gmail.com>>
Reply-To: user <us...@hadoop.apache.org>>
Date: Thursday, October 29, 2015 at 2:15 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Cc: user <us...@hadoop.apache.org>>
Subject: Re: why UserGroupInformation equals method depend on the subject object
Hi wenli,
I think, this is a bug. Could you please file a jira on the Hadoop project so that we can try fix it if its really a bug. Please let me know the jiraand I'll review it.
thanks ,
benoy
On Sat, Oct 17, 2015 at 7:06 AM, Wangwenli <wa...@huawei.com>> wrote:
Hi all,
my question is why UserGroupInformation equals method depend on the subject object?
try below code which is extract from HiveMetaStore:
UserGroupInformation clientUgi = null;
UserGroupInformation clientUgi2 = null;
try {
clientUgi = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
clientUgi2 = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
if (clientUgi.equals(clientUgi2)) {
System.out.println("==");
} else {
System.out.println("!="); // strangely this will be hit
}
} catch (IOException e1) {
e1.printStackTrace();
}
i found that it is because the equal method from UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject; .
as you know, ipc.Client connect to namenode, connections.get(ConnectionId) this code will try to reuse the same socket to namenode, but because of ConnectionId's equal depend on ugi equal, which will cause connections.get(ConnectionId) cann't get the same socket, suppose many connect to HiveMetaStore, then many connection to Namenode will established.
so my doubts is why UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject, it should compare on subject's principal, am i right?
________________________________
wenli
Regards
Re: why UserGroupInformation equals method depend on the subject
object
Posted by Chris Nauroth <cn...@hortonworks.com>.
This sounds like potentially a dangerous change in the general case. If there are 2 different Subjects/UGIs with the same principal, but different credentials attached, then an equality check on principal alone could cause a get from the FileSystem cache or IPC client connection cache to return an instance with incorrect credentials.
This might be just a theoretical concern on my part. For the case of proxy users, perhaps an optimization is appropriate, since the proxied user's credentials are not in play anyway.
Regardless, a JIRA is a good idea, and we can take further discussion there.
--Chris Nauroth
From: Benoy Antony <ba...@gmail.com>>
Reply-To: user <us...@hadoop.apache.org>>
Date: Thursday, October 29, 2015 at 2:15 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Cc: user <us...@hadoop.apache.org>>
Subject: Re: why UserGroupInformation equals method depend on the subject object
Hi wenli,
I think, this is a bug. Could you please file a jira on the Hadoop project so that we can try fix it if its really a bug. Please let me know the jiraand I'll review it.
thanks ,
benoy
On Sat, Oct 17, 2015 at 7:06 AM, Wangwenli <wa...@huawei.com>> wrote:
Hi all,
my question is why UserGroupInformation equals method depend on the subject object?
try below code which is extract from HiveMetaStore:
UserGroupInformation clientUgi = null;
UserGroupInformation clientUgi2 = null;
try {
clientUgi = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
clientUgi2 = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
if (clientUgi.equals(clientUgi2)) {
System.out.println("==");
} else {
System.out.println("!="); // strangely this will be hit
}
} catch (IOException e1) {
e1.printStackTrace();
}
i found that it is because the equal method from UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject; .
as you know, ipc.Client connect to namenode, connections.get(ConnectionId) this code will try to reuse the same socket to namenode, but because of ConnectionId's equal depend on ugi equal, which will cause connections.get(ConnectionId) cann't get the same socket, suppose many connect to HiveMetaStore, then many connection to Namenode will established.
so my doubts is why UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject, it should compare on subject's principal, am i right?
________________________________
wenli
Regards
Re: why UserGroupInformation equals method depend on the subject
object
Posted by Chris Nauroth <cn...@hortonworks.com>.
This sounds like potentially a dangerous change in the general case. If there are 2 different Subjects/UGIs with the same principal, but different credentials attached, then an equality check on principal alone could cause a get from the FileSystem cache or IPC client connection cache to return an instance with incorrect credentials.
This might be just a theoretical concern on my part. For the case of proxy users, perhaps an optimization is appropriate, since the proxied user's credentials are not in play anyway.
Regardless, a JIRA is a good idea, and we can take further discussion there.
--Chris Nauroth
From: Benoy Antony <ba...@gmail.com>>
Reply-To: user <us...@hadoop.apache.org>>
Date: Thursday, October 29, 2015 at 2:15 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Cc: user <us...@hadoop.apache.org>>
Subject: Re: why UserGroupInformation equals method depend on the subject object
Hi wenli,
I think, this is a bug. Could you please file a jira on the Hadoop project so that we can try fix it if its really a bug. Please let me know the jiraand I'll review it.
thanks ,
benoy
On Sat, Oct 17, 2015 at 7:06 AM, Wangwenli <wa...@huawei.com>> wrote:
Hi all,
my question is why UserGroupInformation equals method depend on the subject object?
try below code which is extract from HiveMetaStore:
UserGroupInformation clientUgi = null;
UserGroupInformation clientUgi2 = null;
try {
clientUgi = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
clientUgi2 = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
if (clientUgi.equals(clientUgi2)) {
System.out.println("==");
} else {
System.out.println("!="); // strangely this will be hit
}
} catch (IOException e1) {
e1.printStackTrace();
}
i found that it is because the equal method from UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject; .
as you know, ipc.Client connect to namenode, connections.get(ConnectionId) this code will try to reuse the same socket to namenode, but because of ConnectionId's equal depend on ugi equal, which will cause connections.get(ConnectionId) cann't get the same socket, suppose many connect to HiveMetaStore, then many connection to Namenode will established.
so my doubts is why UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject, it should compare on subject's principal, am i right?
________________________________
wenli
Regards
Re: why UserGroupInformation equals method depend on the subject
object
Posted by Chris Nauroth <cn...@hortonworks.com>.
This sounds like potentially a dangerous change in the general case. If there are 2 different Subjects/UGIs with the same principal, but different credentials attached, then an equality check on principal alone could cause a get from the FileSystem cache or IPC client connection cache to return an instance with incorrect credentials.
This might be just a theoretical concern on my part. For the case of proxy users, perhaps an optimization is appropriate, since the proxied user's credentials are not in play anyway.
Regardless, a JIRA is a good idea, and we can take further discussion there.
--Chris Nauroth
From: Benoy Antony <ba...@gmail.com>>
Reply-To: user <us...@hadoop.apache.org>>
Date: Thursday, October 29, 2015 at 2:15 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Cc: user <us...@hadoop.apache.org>>
Subject: Re: why UserGroupInformation equals method depend on the subject object
Hi wenli,
I think, this is a bug. Could you please file a jira on the Hadoop project so that we can try fix it if its really a bug. Please let me know the jiraand I'll review it.
thanks ,
benoy
On Sat, Oct 17, 2015 at 7:06 AM, Wangwenli <wa...@huawei.com>> wrote:
Hi all,
my question is why UserGroupInformation equals method depend on the subject object?
try below code which is extract from HiveMetaStore:
UserGroupInformation clientUgi = null;
UserGroupInformation clientUgi2 = null;
try {
clientUgi = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
clientUgi2 = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
if (clientUgi.equals(clientUgi2)) {
System.out.println("==");
} else {
System.out.println("!="); // strangely this will be hit
}
} catch (IOException e1) {
e1.printStackTrace();
}
i found that it is because the equal method from UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject; .
as you know, ipc.Client connect to namenode, connections.get(ConnectionId) this code will try to reuse the same socket to namenode, but because of ConnectionId's equal depend on ugi equal, which will cause connections.get(ConnectionId) cann't get the same socket, suppose many connect to HiveMetaStore, then many connection to Namenode will established.
so my doubts is why UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject, it should compare on subject's principal, am i right?
________________________________
wenli
Regards
Re: why UserGroupInformation equals method depend on the subject
object
Posted by Chris Nauroth <cn...@hortonworks.com>.
This sounds like potentially a dangerous change in the general case. If there are 2 different Subjects/UGIs with the same principal, but different credentials attached, then an equality check on principal alone could cause a get from the FileSystem cache or IPC client connection cache to return an instance with incorrect credentials.
This might be just a theoretical concern on my part. For the case of proxy users, perhaps an optimization is appropriate, since the proxied user's credentials are not in play anyway.
Regardless, a JIRA is a good idea, and we can take further discussion there.
--Chris Nauroth
From: Benoy Antony <ba...@gmail.com>>
Reply-To: user <us...@hadoop.apache.org>>
Date: Thursday, October 29, 2015 at 2:15 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Cc: user <us...@hadoop.apache.org>>
Subject: Re: why UserGroupInformation equals method depend on the subject object
Hi wenli,
I think, this is a bug. Could you please file a jira on the Hadoop project so that we can try fix it if its really a bug. Please let me know the jiraand I'll review it.
thanks ,
benoy
On Sat, Oct 17, 2015 at 7:06 AM, Wangwenli <wa...@huawei.com>> wrote:
Hi all,
my question is why UserGroupInformation equals method depend on the subject object?
try below code which is extract from HiveMetaStore:
UserGroupInformation clientUgi = null;
UserGroupInformation clientUgi2 = null;
try {
clientUgi = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
clientUgi2 = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
if (clientUgi.equals(clientUgi2)) {
System.out.println("==");
} else {
System.out.println("!="); // strangely this will be hit
}
} catch (IOException e1) {
e1.printStackTrace();
}
i found that it is because the equal method from UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject; .
as you know, ipc.Client connect to namenode, connections.get(ConnectionId) this code will try to reuse the same socket to namenode, but because of ConnectionId's equal depend on ugi equal, which will cause connections.get(ConnectionId) cann't get the same socket, suppose many connect to HiveMetaStore, then many connection to Namenode will established.
so my doubts is why UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject, it should compare on subject's principal, am i right?
________________________________
wenli
Regards
Re: why UserGroupInformation equals method depend on the subject
object
Posted by Chris Nauroth <cn...@hortonworks.com>.
This sounds like potentially a dangerous change in the general case. If there are 2 different Subjects/UGIs with the same principal, but different credentials attached, then an equality check on principal alone could cause a get from the FileSystem cache or IPC client connection cache to return an instance with incorrect credentials.
This might be just a theoretical concern on my part. For the case of proxy users, perhaps an optimization is appropriate, since the proxied user's credentials are not in play anyway.
Regardless, a JIRA is a good idea, and we can take further discussion there.
--Chris Nauroth
From: Benoy Antony <ba...@gmail.com>>
Reply-To: user <us...@hadoop.apache.org>>
Date: Thursday, October 29, 2015 at 2:15 PM
To: "user@hive.apache.org<ma...@hive.apache.org>" <us...@hive.apache.org>>
Cc: user <us...@hadoop.apache.org>>
Subject: Re: why UserGroupInformation equals method depend on the subject object
Hi wenli,
I think, this is a bug. Could you please file a jira on the Hadoop project so that we can try fix it if its really a bug. Please let me know the jiraand I'll review it.
thanks ,
benoy
On Sat, Oct 17, 2015 at 7:06 AM, Wangwenli <wa...@huawei.com>> wrote:
Hi all,
my question is why UserGroupInformation equals method depend on the subject object?
try below code which is extract from HiveMetaStore:
UserGroupInformation clientUgi = null;
UserGroupInformation clientUgi2 = null;
try {
clientUgi = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
clientUgi2 = UserGroupInformation.createProxyUser("user2", UserGroupInformation.getLoginUser());
if (clientUgi.equals(clientUgi2)) {
System.out.println("==");
} else {
System.out.println("!="); // strangely this will be hit
}
} catch (IOException e1) {
e1.printStackTrace();
}
i found that it is because the equal method from UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject; .
as you know, ipc.Client connect to namenode, connections.get(ConnectionId) this code will try to reuse the same socket to namenode, but because of ConnectionId's equal depend on ugi equal, which will cause connections.get(ConnectionId) cann't get the same socket, suppose many connect to HiveMetaStore, then many connection to Namenode will established.
so my doubts is why UserGroupInformation is compare on subject object ref : subject == ((UserGroupInformation) o).subject, it should compare on subject's principal, am i right?
________________________________
wenli
Regards