You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@usergrid.apache.org by shawnfeldman <gi...@git.apache.org> on 2014/12/08 16:44:55 UTC
[GitHub] incubator-usergrid pull request: fix deletes so we remove from all
GitHub user shawnfeldman opened a pull request:
https://github.com/apache/incubator-usergrid/pull/117
fix deletes so we remove from all
add index seek from aliases so we can find the indexes for an alias then delete from all
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-usergrid delete_from_all_indexes
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-usergrid/pull/117.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #117
----
commit 4b90e97e16253559d51479dd57a59aacf3f61466
Author: Shawn Feldman <sf...@apache.org>
Date: 2014-12-08T15:40:54Z
fix deletes so we remove from all
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-usergrid pull request: fix deletes so we remove from all
Posted by shawnfeldman <gi...@git.apache.org>.
Github user shawnfeldman commented on a diff in the pull request:
https://github.com/apache/incubator-usergrid/pull/117#discussion_r21489794
--- Diff: stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/impl/EsEntityIndexImpl.java ---
@@ -424,7 +440,7 @@ public void refresh() {
@Override
public boolean doOp() {
try {
- esProvider.getClient().admin().indices().prepareRefresh( alias.getWriteAlias() ).execute().actionGet();
+ esProvider.getClient().admin().indices().prepareRefresh( alias.getReadAlias() ).execute().actionGet();
--- End diff --
refresh all indexes
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-usergrid pull request: fix deletes so we remove from all
Posted by tnine <gi...@git.apache.org>.
Github user tnine commented on a diff in the pull request:
https://github.com/apache/incubator-usergrid/pull/117#discussion_r21563902
--- Diff: stack/core/src/main/java/org/apache/usergrid/corepersistence/CpEntityManagerFactory.java ---
@@ -702,7 +703,10 @@ public void rebuildCollectionIndex(UUID appId, String collection, ProgressObserv
@Override
public void addIndex(final UUID applicationId,final String indexSuffix,final int shards,final int replicas){
- getManagerCache().getEntityIndex(CpNamingUtils.getApplicationScope( applicationId )).addIndex(indexSuffix,shards,replicas);
+ EntityIndex entityIndex = getManagerCache().getEntityIndex(CpNamingUtils.getApplicationScope(applicationId));
+ if(entityIndex instanceof AliasedEntityIndex) {
+ ((AliasedEntityIndex)entityIndex).addIndex(indexSuffix, shards, replicas);
+ }
--- End diff --
What if it's not the right instance? Shouldn't we throw an UnsupportedOperationException? Or, is there a way to just create it from the factory, so we know it's always the right instance type? Since this happens very rarely, I don't think we'll need to worry about getting it from the cache.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-usergrid pull request: fix deletes so we remove from all
Posted by tnine <gi...@git.apache.org>.
Github user tnine commented on a diff in the pull request:
https://github.com/apache/incubator-usergrid/pull/117#discussion_r21574468
--- Diff: stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/impl/EsEntityIndexImpl.java ---
@@ -440,8 +443,15 @@ public void refresh() {
@Override
public boolean doOp() {
try {
- esProvider.getClient().admin().indices().prepareRefresh( alias.getWriteAlias() )
- .execute().actionGet();
+ String[] indexes = getIndexes(AliasType.Read);
+ Observable.from(indexes).subscribeOn(Schedulers.io()).map(new Func1<String, String>() {
--- End diff --
IF we're not doing this in parallel, we won't want to subscribe on anything. Just let it run in the caller's thread.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-usergrid pull request: fix deletes so we remove from all
Posted by shawnfeldman <gi...@git.apache.org>.
Github user shawnfeldman commented on a diff in the pull request:
https://github.com/apache/incubator-usergrid/pull/117#discussion_r21489837
--- Diff: stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/EntityIndex.java ---
@@ -47,6 +47,13 @@
public void addIndex(final String indexSuffix, final int shards, final int replicas);
/**
+ * Get the indexes for an alias
+ * @param aliasName name of alias
+ * @return list of index names
+ */
+ public String[] getIndexes(final AliasType aliasType);
--- End diff --
move to new interface
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-usergrid pull request: fix deletes so we remove from all
Posted by tnine <gi...@git.apache.org>.
Github user tnine commented on a diff in the pull request:
https://github.com/apache/incubator-usergrid/pull/117#discussion_r21489983
--- Diff: stack/corepersistence/queryindex/src/main/java/org/apache/usergrid/persistence/index/impl/EsEntityIndexImpl.java ---
@@ -198,6 +197,23 @@ public void addAlias(final String indexSuffix) {
}
}
+ @Override
+ public String[] getIndexes(final AliasType aliasType) {
+ final String aliasName = aliasType == AliasType.Read ? alias.getReadAlias() : alias.getWriteAlias();
+ return getIndexes(aliasName);
+ }
+
+ /**
+ * get indexes for alias
+ * @param aliasName
+ * @return
+ */
+ private String[] getIndexes(final String aliasName){
+ final AdminClient adminClient = esProvider.getClient().admin();
+ //remove write alias, can only have one
+ ImmutableOpenMap<String,List<AliasMetaData>> aliasMap = adminClient.indices().getAliases(new GetAliasesRequest(aliasName)).actionGet().getAliases();
+ return aliasMap.keys().toArray(String.class);
--- End diff --
Can we verify which node this hits? If only the master responds to this operation, can we hide it behind a cache with an internal refresh worker pool, and a timeout that's configurable? I'm thinking 5 min or so should be sufficient if that's the case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-usergrid pull request: fix deletes so we remove from all
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-usergrid/pull/117
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---