You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Eric Newton <er...@gmail.com> on 2015/01/21 06:29:36 UTC
Re: accumulo query delete
I'm having some difficulty understanding the question.
Can you provide an example using java or just the accumulo shell to
demonstrate the question?
-Eric
On Tue, Jan 20, 2015 at 9:58 PM, panqingcui@163.com <pa...@163.com>
wrote:
> HI
> Recently the use of accumulo to do the user information storage, a
> tenant contains a plurality of user, the user can only query the current
> tenants of the data, set the Visbility for the tenants, then the query ID,
> according to each tenant ID query user information, but for every need to
> query modify permissions (setauths), when the tenants under different user,
> query, delete all need to modify permissions, how to ensure the
> synchronous? For example,:a tenant Join， b tenant Jack , and jack
> operation at the same time, the join Jack query to delete user information,
> user information, delete the former setauths = a ,query before setauts=b
> when the concurrent situation should be how to deal with?
>
>
>
> panqingcui@163.com
>
Re: Re: accumulo query delete
Posted by Eric Newton <er...@gmail.com>.
Yes, it will most likely cause confusion, even if it is consistent. Just
like creating and deleting a table from different threads would cause a
confusing situation, even if it works correctly.
Example:
Thread A adds "write permission for User1 to table X"
Thread B removes "write permission for User1 to table X"
Now, what should the write permission be for User1 on table X? It cannot
be determined, because the order the permissions were run cannot be known.
However, it should be safe, in that they are executed atomically, and the
updates are sent in the same order to all the servers, resulting in
eventual agreement (in a few seconds).
-Eric
On Fri, Jan 23, 2015 at 1:49 AM, panqingcui@163.com <pa...@163.com>
wrote:
> HI ,thank you for this eamil, My question is if multiple threads
> modifying user permissions at the same time, will lead to a problem? For
> example, a users will need to admin, B users will need to root, assuming
> the two at the same time, will lead to a problem.
>
>
>
> panqingcui@163.com
>
> From: Eric Newton [via Apache Accumulo]
> Date: 2015-01-22 00:32
> To: panqingcui@163.com
> Subject: Re: accumulo query delete
> Ah. OK.
>
> All users' permissions and Authorizations are stored in zookeeper and
> update servers asynchronously. The assumption is that these operations are
> not needed to be fast and atomic: they are performed infrequently. That is,
> a few seconds update delay is acceptable, caching the data is good.
>
> If you need row-level atomic updates, please look into ConditionalWriter.
> In some cases, an IsolatedScanner may work better than the
> WholeRowIterator, though for small records, the WholeRowIterator works
> fine.
>
> You probably want the priority of the WholeRowIterator to be higher than
> the VersioningIterator, so that you do not get multiple versions for a
> particular cell of the table.
>
> -Eric
>
> On Wed, Jan 21, 2015 at 3:12 AM, [hidden email] <[hidden email]>
> wrote:
>
> > Thank your reply this question.
> > 1、 public void deleteUser(String licenseKey, String userId) throws
> > AccumuloException {
> > Connector conn = AccumuloClientSingleton.INSTANCE.getConnector();
> > BatchDeleter bd = null;
> > try {
> >
> >
> >
> conn.securityOperations().changeUserAuthorizations(AccumuloConstants.USER_NAME,
> > new Authorizations(licenseKey));
> > bd = conn.createBatchDeleter(AccumuloConstants.XT_USER, new
> > Authorizations(licenseKey), 10,
> > new BatchWriterConfig());
> > Set<Range> ranges = new HashSet<Range>();
> > ranges.add(new Range(userId));
> > bd.setRanges(ranges);
> > bd.delete();
> > } catch (Exception e) {
> > throw new AccumuloException("删除用户信息时异常", e);
> > } finally {
> > if (bd != null)
> > bd.close();
> > }
> > }
> > 2、 public XtUser getUser(String licenseKey, String userAccount) throws
> > AccumuloException {
> > Connector conn = AccumuloClientSingleton.INSTANCE.getConnector();
> > Map<String, XtUser> map = new HashMap<String, XtUser>();
> > try {
> >
> >
> >
> conn.securityOperations().changeUserAuthorizations(AccumuloConstants.USER_NAME,
> > new Authorizations(licenseKey));
> > Scanner scanner =
> > conn.createScanner(AccumuloConstants.USER_NAME, new
> > Authorizations(licenseKey));
> > // 行迭代器
> > IteratorSetting it = new IteratorSetting(1,
> "WholeRowIterator",
> > WholeRowIterator.class);
> > scanner.addScanIterator(it);
> > for (Entry<Key, Value> entry : scanner) {
> > XtUser xtUser = new XtUser();
> > for (Entry<Key, Value> actualEntry :
> > WholeRowIterator.decodeRow(entry.getKey(), entry.getValue())
> > .entrySet()) {
> > String qualifier =
> > actualEntry.getKey().getColumnFamily().toString();
> > String value = actualEntry.getValue().toString();
> > if (qualifier.equals("role_id")) {
> > xtUser.setRoleId(value);
> > } else if (qualifier.equals("role_name")) {
> > xtUser.setRoleName("role_name");
> > } else if (qualifier.equals("user_account")) {
> > xtUser.setUserAccount(value);
> > }
> > map.put(xtUser.getUserAccount(), xtUser);
> > }
> > }
> > } catch (Exception e) {
> > throw new AccumuloException("获取用户消息异常", e);
> > }
> > return map.get(userAccount);
> > }
> >
> > Method 1, method 2 will modify the user's permission, if concurrent case,
> > is
> > also the method 1, method 2 and is called, should be how to deal with?
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-accumulo.1065345.n5.nabble.com/accumulo-query-delete-tp12965p12969.html
> > Sent from the Developers mailing list archive at Nabble.com.
> >
>
>
>
>
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-accumulo.1065345.n5.nabble.com/accumulo-query-delete-tp12965p12976.html
> To unsubscribe from accumulo query delete, click here.
> NAML
>
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/accumulo-query-delete-tp12965p12993.html
> Sent from the Developers mailing list archive at Nabble.com.
>
Re: Re: accumulo query delete
Posted by "panqingcui@163.com" <pa...@163.com>.
HI ,thank you for this eamil, My question is if multiple threads modifying user permissions at the same time, will lead to a problem? For example, a users will need to admin, B users will need to root, assuming the two at the same time, will lead to a problem.
panqingcui@163.com
From: Eric Newton [via Apache Accumulo]
Date: 2015-01-22 00:32
To: panqingcui@163.com
Subject: Re: accumulo query delete
Ah. OK.
All users' permissions and Authorizations are stored in zookeeper and
update servers asynchronously. The assumption is that these operations are
not needed to be fast and atomic: they are performed infrequently. That is,
a few seconds update delay is acceptable, caching the data is good.
If you need row-level atomic updates, please look into ConditionalWriter.
In some cases, an IsolatedScanner may work better than the
WholeRowIterator, though for small records, the WholeRowIterator works fine.
You probably want the priority of the WholeRowIterator to be higher than
the VersioningIterator, so that you do not get multiple versions for a
particular cell of the table.
-Eric
On Wed, Jan 21, 2015 at 3:12 AM, [hidden email] <[hidden email]>
wrote:
> Thank your reply this question.
> 1、 public void deleteUser(String licenseKey, String userId) throws
> AccumuloException {
> Connector conn = AccumuloClientSingleton.INSTANCE.getConnector();
> BatchDeleter bd = null;
> try {
>
>
> conn.securityOperations().changeUserAuthorizations(AccumuloConstants.USER_NAME,
> new Authorizations(licenseKey));
> bd = conn.createBatchDeleter(AccumuloConstants.XT_USER, new
> Authorizations(licenseKey), 10,
> new BatchWriterConfig());
> Set<Range> ranges = new HashSet<Range>();
> ranges.add(new Range(userId));
> bd.setRanges(ranges);
> bd.delete();
> } catch (Exception e) {
> throw new AccumuloException("删除用户信息时异常", e);
> } finally {
> if (bd != null)
> bd.close();
> }
> }
> 2、 public XtUser getUser(String licenseKey, String userAccount) throws
> AccumuloException {
> Connector conn = AccumuloClientSingleton.INSTANCE.getConnector();
> Map<String, XtUser> map = new HashMap<String, XtUser>();
> try {
>
>
> conn.securityOperations().changeUserAuthorizations(AccumuloConstants.USER_NAME,
> new Authorizations(licenseKey));
> Scanner scanner =
> conn.createScanner(AccumuloConstants.USER_NAME, new
> Authorizations(licenseKey));
> // 行迭代器
> IteratorSetting it = new IteratorSetting(1, "WholeRowIterator",
> WholeRowIterator.class);
> scanner.addScanIterator(it);
> for (Entry<Key, Value> entry : scanner) {
> XtUser xtUser = new XtUser();
> for (Entry<Key, Value> actualEntry :
> WholeRowIterator.decodeRow(entry.getKey(), entry.getValue())
> .entrySet()) {
> String qualifier =
> actualEntry.getKey().getColumnFamily().toString();
> String value = actualEntry.getValue().toString();
> if (qualifier.equals("role_id")) {
> xtUser.setRoleId(value);
> } else if (qualifier.equals("role_name")) {
> xtUser.setRoleName("role_name");
> } else if (qualifier.equals("user_account")) {
> xtUser.setUserAccount(value);
> }
> map.put(xtUser.getUserAccount(), xtUser);
> }
> }
> } catch (Exception e) {
> throw new AccumuloException("获取用户消息异常", e);
> }
> return map.get(userAccount);
> }
>
> Method 1, method 2 will modify the user's permission, if concurrent case,
> is
> also the method 1, method 2 and is called, should be how to deal with?
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/accumulo-query-delete-tp12965p12969.html
> Sent from the Developers mailing list archive at Nabble.com.
>
If you reply to this email, your message will be added to the discussion below:
http://apache-accumulo.1065345.n5.nabble.com/accumulo-query-delete-tp12965p12976.html
To unsubscribe from accumulo query delete, click here.
NAML
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/accumulo-query-delete-tp12965p12993.html
Sent from the Developers mailing list archive at Nabble.com.
Re: accumulo query delete
Posted by Eric Newton <er...@gmail.com>.
Ah. OK.
All users' permissions and Authorizations are stored in zookeeper and
update servers asynchronously. The assumption is that these operations are
not needed to be fast and atomic: they are performed infrequently. That is,
a few seconds update delay is acceptable, caching the data is good.
If you need row-level atomic updates, please look into ConditionalWriter.
In some cases, an IsolatedScanner may work better than the
WholeRowIterator, though for small records, the WholeRowIterator works fine.
You probably want the priority of the WholeRowIterator to be higher than
the VersioningIterator, so that you do not get multiple versions for a
particular cell of the table.
-Eric
On Wed, Jan 21, 2015 at 3:12 AM, panqingcui@163.com <pa...@163.com>
wrote:
> Thank your reply this question.
> 1、 public void deleteUser(String licenseKey, String userId) throws
> AccumuloException {
> Connector conn = AccumuloClientSingleton.INSTANCE.getConnector();
> BatchDeleter bd = null;
> try {
>
>
> conn.securityOperations().changeUserAuthorizations(AccumuloConstants.USER_NAME,
> new Authorizations(licenseKey));
> bd = conn.createBatchDeleter(AccumuloConstants.XT_USER, new
> Authorizations(licenseKey), 10,
> new BatchWriterConfig());
> Set<Range> ranges = new HashSet<Range>();
> ranges.add(new Range(userId));
> bd.setRanges(ranges);
> bd.delete();
> } catch (Exception e) {
> throw new AccumuloException("删除用户信息时异常", e);
> } finally {
> if (bd != null)
> bd.close();
> }
> }
> 2、 public XtUser getUser(String licenseKey, String userAccount) throws
> AccumuloException {
> Connector conn = AccumuloClientSingleton.INSTANCE.getConnector();
> Map<String, XtUser> map = new HashMap<String, XtUser>();
> try {
>
>
> conn.securityOperations().changeUserAuthorizations(AccumuloConstants.USER_NAME,
> new Authorizations(licenseKey));
> Scanner scanner =
> conn.createScanner(AccumuloConstants.USER_NAME, new
> Authorizations(licenseKey));
> // 行迭代器
> IteratorSetting it = new IteratorSetting(1, "WholeRowIterator",
> WholeRowIterator.class);
> scanner.addScanIterator(it);
> for (Entry<Key, Value> entry : scanner) {
> XtUser xtUser = new XtUser();
> for (Entry<Key, Value> actualEntry :
> WholeRowIterator.decodeRow(entry.getKey(), entry.getValue())
> .entrySet()) {
> String qualifier =
> actualEntry.getKey().getColumnFamily().toString();
> String value = actualEntry.getValue().toString();
> if (qualifier.equals("role_id")) {
> xtUser.setRoleId(value);
> } else if (qualifier.equals("role_name")) {
> xtUser.setRoleName("role_name");
> } else if (qualifier.equals("user_account")) {
> xtUser.setUserAccount(value);
> }
> map.put(xtUser.getUserAccount(), xtUser);
> }
> }
> } catch (Exception e) {
> throw new AccumuloException("获取用户消息异常", e);
> }
> return map.get(userAccount);
> }
>
> Method 1, method 2 will modify the user's permission, if concurrent case,
> is
> also the method 1, method 2 and is called, should be how to deal with?
>
>
>
> --
> View this message in context:
> http://apache-accumulo.1065345.n5.nabble.com/accumulo-query-delete-tp12965p12969.html
> Sent from the Developers mailing list archive at Nabble.com.
>
Re: accumulo query delete
Posted by "panqingcui@163.com" <pa...@163.com>.
Thank your reply this question.
1、 public void deleteUser(String licenseKey, String userId) throws
AccumuloException {
Connector conn = AccumuloClientSingleton.INSTANCE.getConnector();
BatchDeleter bd = null;
try {
conn.securityOperations().changeUserAuthorizations(AccumuloConstants.USER_NAME,
new Authorizations(licenseKey));
bd = conn.createBatchDeleter(AccumuloConstants.XT_USER, new
Authorizations(licenseKey), 10,
new BatchWriterConfig());
Set<Range> ranges = new HashSet<Range>();
ranges.add(new Range(userId));
bd.setRanges(ranges);
bd.delete();
} catch (Exception e) {
throw new AccumuloException("删除用户信息时异常", e);
} finally {
if (bd != null)
bd.close();
}
}
2、 public XtUser getUser(String licenseKey, String userAccount) throws
AccumuloException {
Connector conn = AccumuloClientSingleton.INSTANCE.getConnector();
Map<String, XtUser> map = new HashMap<String, XtUser>();
try {
conn.securityOperations().changeUserAuthorizations(AccumuloConstants.USER_NAME,
new Authorizations(licenseKey));
Scanner scanner =
conn.createScanner(AccumuloConstants.USER_NAME, new
Authorizations(licenseKey));
// 行迭代器
IteratorSetting it = new IteratorSetting(1, "WholeRowIterator",
WholeRowIterator.class);
scanner.addScanIterator(it);
for (Entry<Key, Value> entry : scanner) {
XtUser xtUser = new XtUser();
for (Entry<Key, Value> actualEntry :
WholeRowIterator.decodeRow(entry.getKey(), entry.getValue())
.entrySet()) {
String qualifier =
actualEntry.getKey().getColumnFamily().toString();
String value = actualEntry.getValue().toString();
if (qualifier.equals("role_id")) {
xtUser.setRoleId(value);
} else if (qualifier.equals("role_name")) {
xtUser.setRoleName("role_name");
} else if (qualifier.equals("user_account")) {
xtUser.setUserAccount(value);
}
map.put(xtUser.getUserAccount(), xtUser);
}
}
} catch (Exception e) {
throw new AccumuloException("获取用户消息异常", e);
}
return map.get(userAccount);
}
Method 1, method 2 will modify the user's permission, if concurrent case, is
also the method 1, method 2 and is called, should be how to deal with?
--
View this message in context: http://apache-accumulo.1065345.n5.nabble.com/accumulo-query-delete-tp12965p12969.html
Sent from the Developers mailing list archive at Nabble.com.