You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucene.apache.org by er...@apache.org on 2017/12/29 04:41:56 UTC
lucene-solr:master: SOLR-7733: remove optimize from the UI.
Repository: lucene-solr
Updated Branches:
refs/heads/master fbea59b08 -> 8e439a0a5
SOLR-7733: remove optimize from the UI.
Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/8e439a0a
Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/8e439a0a
Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/8e439a0a
Branch: refs/heads/master
Commit: 8e439a0a5c37a95ae632719b8901b225462b80bf
Parents: fbea59b
Author: Erick Erickson <er...@apache.org>
Authored: Thu Dec 28 20:41:48 2017 -0800
Committer: Erick Erickson <er...@apache.org>
Committed: Thu Dec 28 20:41:48 2017 -0800
----------------------------------------------------------------------
solr/CHANGES.txt | 2 ++
solr/solr-ref-guide/src/documents-screen.adoc | 2 +-
solr/solr-ref-guide/src/index-replication.adoc | 6 +++++-
solr/solr-ref-guide/src/indexupgrader-tool.adoc | 2 +-
.../src/uploading-data-with-index-handlers.adoc | 6 +++++-
.../webapp/web/js/angular/controllers/collections.js | 1 -
.../web/js/angular/controllers/core-overview.js | 12 ------------
solr/webapp/web/js/angular/controllers/cores.js | 15 ---------------
solr/webapp/web/js/angular/controllers/dataimport.js | 1 -
solr/webapp/web/js/angular/services.js | 7 ++-----
solr/webapp/web/partials/core_overview.html | 4 ----
solr/webapp/web/partials/cores.html | 5 -----
12 files changed, 16 insertions(+), 47 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/CHANGES.txt
----------------------------------------------------------------------
diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt
index e14d771..c8e36f2 100644
--- a/solr/CHANGES.txt
+++ b/solr/CHANGES.txt
@@ -93,6 +93,8 @@ Other Changes
* SOLR-11793: Reduce code duplication w.r.t. RestTestHarness(es). (Christine Poerschke)
+* SOLR-7733: remove "optimize" from the UI. (Erick Erickson)
+
================== 7.2.0 ==================
Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/solr-ref-guide/src/documents-screen.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/documents-screen.adoc b/solr/solr-ref-guide/src/documents-screen.adoc
index abeeb9e..83da713 100644
--- a/solr/solr-ref-guide/src/documents-screen.adoc
+++ b/solr/solr-ref-guide/src/documents-screen.adoc
@@ -68,7 +68,7 @@ Then you can choose when documents should be added to the index (Commit Within),
== Solr Command
-The Solr Command option allows you use XML or JSON to perform specific actions on documents, such as defining documents to be added or deleted, updating only certain fields of documents, or commit and optimize commands on the index.
+The Solr Command option allows you use XML or JSON to perform specific actions on documents, such as defining documents to be added or deleted, updating only certain fields of documents, or commit commands on the index.
The documents should be structured as they would be if using `/update` on the command line.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/solr-ref-guide/src/index-replication.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/index-replication.adoc b/solr/solr-ref-guide/src/index-replication.adoc
index de1e8d6..7e07e49 100644
--- a/solr/solr-ref-guide/src/index-replication.adoc
+++ b/solr/solr-ref-guide/src/index-replication.adoc
@@ -54,7 +54,7 @@ Distribution::
The copying of an index from the master server to all slaves. The distribution process takes advantage of Lucene's index file structure.
Inserts and Deletes::
-As inserts and deletes occur in the index, the directory remains unchanged. Documents are always inserted into newly created files. Documents that are deleted are not removed from the files. They are flagged in the file, deletable, and are not removed from the files until the index is optimized.
+As inserts and deletes occur in the index, the directory remains unchanged. Documents are always inserted into newly created segment files. Documents that are deleted are not removed from the segment files. They are flagged in the file, deletable, and are not removed from the segments until the segment is merged as part of normal index updates.
Master and Slave::
A Solr replication master is a single node which receives all updates initially and keeps everything organized. Solr replication slave nodes receive no updates directly, instead all changes (such as inserts, updates, deletes, etc.) are made against the single master node. Changes made on the master are distributed to all the slave nodes which service all query requests from the clients.
@@ -65,6 +65,8 @@ An update is a single change request against a single Solr instance. It may be a
Optimization::
A process that compacts the index and merges segments in order to improve query performance. Optimization should only be run on the master nodes. An optimized index may give query performance gains compared to an index that has become fragmented over a period of time with many updates. Distributing an optimized index requires a much longer time than the distribution of new segments to an un-optimized index.
+WARNING: optimizing is not recommended unless it can be performed regularly as it may lead to a significantly larger portion of the index consisting of deleted documents than would normally be the case.
+
Segments::
A self contained subset of an index consisting of some documents and data structures related to the inverted index of terms in those documents.
@@ -356,3 +358,5 @@ Consider this example: on a three-slave one-master configuration, distributing a
Optimizing on the master allows for a straight-forward optimization operation. No query slaves need to be taken out of service. The optimized index can be distributed in the background as queries are being normally serviced. The optimization can occur at any time convenient to the application providing index updates.
While optimizing may have some benefits in some situations, a rapidly changing index will not retain those benefits for long, and since optimization is an intensive process, it may be better to consider other options, such as lowering the merge factor (discussed in the section on <<indexconfig-in-solrconfig.adoc#merge-factors,Index Configuration>>).
+
+TIP: Do not elect to optimize your index unless you have tangible evidence that it will significantly improve your search performance. Recent changes in Solr/Lucene have dramatically lessened the need to optimize as discussed at the above link.
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/solr-ref-guide/src/indexupgrader-tool.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/indexupgrader-tool.adoc b/solr/solr-ref-guide/src/indexupgrader-tool.adoc
index ff60c42..bb01eea 100644
--- a/solr/solr-ref-guide/src/indexupgrader-tool.adoc
+++ b/solr/solr-ref-guide/src/indexupgrader-tool.adoc
@@ -42,5 +42,5 @@ Upgrading large indexes may take a long time. As a rule of thumb, the upgrade pr
[WARNING]
====
-This tool may reorder documents if the index was partially upgraded before execution (e.g., documents were added). If your application relies on monotonicity of document IDs (i.e., the order in which the documents were added to the index is preserved), do a full forceMerge instead.
+This tool may reorder documents if the index was partially upgraded before execution (e.g., documents were added). If your application relies on monotonicity of document IDs (i.e., the order in which the documents were added to the index is preserved), do a full optimize instead.
====
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/solr-ref-guide/src/uploading-data-with-index-handlers.adoc
----------------------------------------------------------------------
diff --git a/solr/solr-ref-guide/src/uploading-data-with-index-handlers.adoc b/solr/solr-ref-guide/src/uploading-data-with-index-handlers.adoc
index c872485..a240dcf 100644
--- a/solr/solr-ref-guide/src/uploading-data-with-index-handlers.adoc
+++ b/solr/solr-ref-guide/src/uploading-data-with-index-handlers.adoc
@@ -87,7 +87,9 @@ The `<commit>` operation writes all documents loaded since the last commit to on
Commits may be issued explicitly with a `<commit/>` message, and can also be triggered from `<autocommit>` parameters in `solrconfig.xml`.
-The `<optimize>` operation requests Solr to merge internal data structures in order to improve search performance. For a large index, optimization will take some time to complete, but by merging many small segment files into a larger one, search performance will improve. If you are using Solr's replication mechanism to distribute searches across many systems, be aware that after an optimize, a complete index will need to be transferred. In contrast, post-commit transfers are usually much smaller.
+The `<optimize>` operation requests Solr to merge internal data structures. For a large index, optimization will take some time to complete, but by merging many small segment files into a larger one, search performance may improve. If you are using Solr's replication mechanism to distribute searches across many systems, be aware that after an optimize, a complete index will need to be transferred.
+
+WARNING: You should only consider using optimize on static indexes, i.e. indexes that can be optimized as part of the regular update process (say once-a-day updates). Applications requiring NRT functionalty are discouraged from using optimize.
The `<commit>` and `<optimize>` elements accept these optional attributes:
@@ -96,6 +98,8 @@ Default is `true`. Blocks until a new searcher is opened and registered as the m
`expungeDeletes`:: (commit only) Default is `false`. Merges segments that have more than 10% deleted docs, expunging them in the process.
+WARNING: expungeDeletes is "less expensive" than optimize, but the same warnings apply.
+
`maxSegments`:: (optimize only) Default is `1`. Merges the segments down to no more than this number of segments.
Here are examples of <commit> and <optimize> using optional attributes:
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/webapp/web/js/angular/controllers/collections.js
----------------------------------------------------------------------
diff --git a/solr/webapp/web/js/angular/controllers/collections.js b/solr/webapp/web/js/angular/controllers/collections.js
index 22c5e7f..251298a 100644
--- a/solr/webapp/web/js/angular/controllers/collections.js
+++ b/solr/webapp/web/js/angular/controllers/collections.js
@@ -15,7 +15,6 @@
limitations under the License.
*/
-// @todo test optimize (delete stuff, watch button appear, test button/form)
solrAdminApp.controller('CollectionsController',
function($scope, $routeParams, $location, $timeout, Collections, Zookeeper, Constants){
$scope.resetMenu("collections", Constants.IS_ROOT_PAGE);
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/webapp/web/js/angular/controllers/core-overview.js
----------------------------------------------------------------------
diff --git a/solr/webapp/web/js/angular/controllers/core-overview.js b/solr/webapp/web/js/angular/controllers/core-overview.js
index 7aa9685..b11329c 100644
--- a/solr/webapp/web/js/angular/controllers/core-overview.js
+++ b/solr/webapp/web/js/angular/controllers/core-overview.js
@@ -30,18 +30,6 @@ function($scope, $rootScope, $routeParams, Luke, CoreSystem, Update, Replication
);
};
- $scope.optimizeIndex = function(core) {
- Update.optimize({core: $routeParams.core},
- function(response) {
- $scope.refresh();
- delete $scope.indexMessage;
- },
- function(error) {
- $scope.statisticsDisabled = true;
- $scope.indexMessage = "Optimize broken.";
- });
- };
-
$scope.refreshReplication = function() {
Replication.details({core: $routeParams.core},
function(data) {
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/webapp/web/js/angular/controllers/cores.js
----------------------------------------------------------------------
diff --git a/solr/webapp/web/js/angular/controllers/cores.js b/solr/webapp/web/js/angular/controllers/cores.js
index 0188617..202b622 100644
--- a/solr/webapp/web/js/angular/controllers/cores.js
+++ b/solr/webapp/web/js/angular/controllers/cores.js
@@ -15,7 +15,6 @@
limitations under the License.
*/
-// @todo test optimize (delete stuff, watch button appear, test button/form)
solrAdminApp.controller('CoreAdminController',
function($scope, $routeParams, $location, $timeout, $route, Cores, Update, Constants){
$scope.resetMenu("cores", Constants.IS_ROOT_PAGE);
@@ -176,20 +175,6 @@ solrAdminApp.controller('CoreAdminController',
$scope.showSwap = false;
};
- $scope.optimizeCore = function() {
- Update.optimize({core: $scope.selectedCore},
- function(successData) {
- $scope.optimizeSuccess = true;
- $timeout(function() {$scope.optimizeSuccess=false}, 1000);
- $scope.refresh();
- },
- function(failureData) {
- $scope.optimizeFailure = true;
- $timeout(function () {$scope.optimizeFailure=false}, 1000);
- $scope.refresh();
- });
- };
-
$scope.refresh();
}
);
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/webapp/web/js/angular/controllers/dataimport.js
----------------------------------------------------------------------
diff --git a/solr/webapp/web/js/angular/controllers/dataimport.js b/solr/webapp/web/js/angular/controllers/dataimport.js
index 3a26324..cebf27a 100644
--- a/solr/webapp/web/js/angular/controllers/dataimport.js
+++ b/solr/webapp/web/js/angular/controllers/dataimport.js
@@ -98,7 +98,6 @@ solrAdminApp.controller('DataImportController',
verbose: false,
clean: true,
commit: true,
- optimize: false,
showDebug: false,
custom: "",
core: $routeParams.core
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/webapp/web/js/angular/services.js
----------------------------------------------------------------------
diff --git a/solr/webapp/web/js/angular/services.js b/solr/webapp/web/js/angular/services.js
index a1df37e..4d89347 100644
--- a/solr/webapp/web/js/angular/services.js
+++ b/solr/webapp/web/js/angular/services.js
@@ -35,8 +35,7 @@ solrAdminServices.factory('System',
"deleteReplica": {params:{action: "DELETEREPLICA"}},
"addReplica": {params:{action: "ADDREPLICA"}},
"deleteShard": {params:{action: "DELETESHARD"}},
- "reload": {method: "GET", params:{action:"RELOAD", core: "@core"}},
- "optimize": {params:{}}
+ "reload": {method: "GET", params:{action:"RELOAD", core: "@core"}}
});
}])
.factory('Cores',
@@ -49,8 +48,7 @@ solrAdminServices.factory('System',
"unload": {params:{action: "UNLOAD", core: "@core"}},
"rename": {params:{action: "RENAME"}},
"swap": {params:{action: "SWAP"}},
- "reload": {method: "GET", params:{action:"RELOAD", core: "@core"}, headers:{doNotIntercept: "true"}},
- "optimize": {params:{}}
+ "reload": {method: "GET", params:{action:"RELOAD", core: "@core"}, headers:{doNotIntercept: "true"}}
});
}])
.factory('Logging',
@@ -106,7 +104,6 @@ solrAdminServices.factory('System',
.factory('Update',
['$resource', function($resource) {
return $resource(':core/:handler', {core: '@core', wt:'json', _:Date.now(), handler:'update'}, {
- "optimize": {params: { optimize: "true"}},
"commit": {params: {commit: "true"}},
"post": {headers: {'Content-type': 'application/json'}, method: "POST", params: {handler: '@handler'}},
"postJson": {headers: {'Content-type': 'application/json'}, method: "POST", params: {handler: '@handler'}},
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/webapp/web/partials/core_overview.html
----------------------------------------------------------------------
diff --git a/solr/webapp/web/partials/core_overview.html b/solr/webapp/web/partials/core_overview.html
index 3b92fc3..f1826f6 100644
--- a/solr/webapp/web/partials/core_overview.html
+++ b/solr/webapp/web/partials/core_overview.html
@@ -51,10 +51,6 @@ limitations under the License.
<dt class="index_segmentCount">Segment Count:</dt>
<dd class="index_segmentCount value">{{index.segmentCount}}</dd>
- <dt class="index_optimized">Optimized:</dt>
- <dd class="index_optimized ico value" ng-class="index.hasDeletions ? 'ico-0' : 'ico-1'"><span></span>
- <a ng-click="optimizeIndex()" ng-show="index.hasDeletions">optimize now</a></dd>
-
<dt class="index_current">Current:</dt>
<dd class="ico" ng-class="index.current ?'ico-1' : 'ico-0'"><span></span></dd>
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/8e439a0a/solr/webapp/web/partials/cores.html
----------------------------------------------------------------------
diff --git a/solr/webapp/web/partials/cores.html b/solr/webapp/web/partials/cores.html
index 7a989ba..e426e4c 100644
--- a/solr/webapp/web/partials/cores.html
+++ b/solr/webapp/web/partials/cores.html
@@ -198,11 +198,6 @@ limitations under the License.
<dd>{{core.index.deletedDocs}}</dd>
</dl></li>
- <li class="optimized"><dl class="clearfix">
- <dt><span>optimized:</span></dt>
- <dd class="ico" ng-class="core.index.hasDeletions ? 'ico-0' : 'ico-1'"><span></span></dd>
- </dl></li>
-
<li class="current"><dl class="clearfix">
<dt><span>current:</span></dt>
<dd class="ico" ng-class="core.index.current ?'ico-1' : 'ico-0'"><span></span></dd>