You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@geode.apache.org by kh...@apache.org on 2017/03/17 20:10:36 UTC

[01/49] geode git commit: GEODE-2614: fix javadoc warnings - spotless

Repository: geode
Updated Branches:
  refs/heads/feature/GEODE-2420 abc0dd2fe -> 2bcba1daa


GEODE-2614: fix javadoc warnings - spotless


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/608ce117
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/608ce117
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/608ce117

Branch: refs/heads/feature/GEODE-2420
Commit: 608ce117bbb35f46346127f76493d1ee95ced155
Parents: abc0dd2
Author: Jinmei Liao <ji...@pivotal.io>
Authored: Wed Mar 8 10:28:39 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:01:51 2017 -0700

----------------------------------------------------------------------
 .../apache/geode/management/internal/cli/CliAroundInterceptor.java  | 1 +
 1 file changed, 1 insertion(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/608ce117/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
index f552460..7a95306 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
@@ -38,6 +38,7 @@ public interface CliAroundInterceptor {
 
   /**
    * called by the OperationInvoker after the command is executed
+   * 
    * @param tempFile if the command's isFileDownloadOverHttp is true, the is the File downloaded
    *        after the http response is processed.
    */


[07/49] geode git commit: GEODE-1195 Improve "Synchronizing Your Offline Disk Store with Your Cache" docs This closes #423

Posted by kh...@apache.org.
GEODE-1195 Improve "Synchronizing Your Offline Disk Store with Your Cache" docs
This closes #423


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/257f0eaa
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/257f0eaa
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/257f0eaa

Branch: refs/heads/feature/GEODE-2420
Commit: 257f0eaab2a84e7c8494e7b3251193b0ca0d5a2c
Parents: 019ee40
Author: Dave Barnes <db...@pivotal.io>
Authored: Thu Mar 9 14:28:15 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 ...eping_offline_disk_store_in_sync.html.md.erb | 23 +++++++++++++++++---
 1 file changed, 20 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/257f0eaa/geode-docs/managing/disk_storage/keeping_offline_disk_store_in_sync.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/managing/disk_storage/keeping_offline_disk_store_in_sync.html.md.erb b/geode-docs/managing/disk_storage/keeping_offline_disk_store_in_sync.html.md.erb
index 0da24a8..1d32d8c 100644
--- a/geode-docs/managing/disk_storage/keeping_offline_disk_store_in_sync.html.md.erb
+++ b/geode-docs/managing/disk_storage/keeping_offline_disk_store_in_sync.html.md.erb
@@ -20,13 +20,30 @@ limitations under the License.
 -->
 
 <a id="syncing_offline_disk_store__section_7D01550D750E48289EFBA9BBDB5A334E"></a>
-You can take several actions to optimize disk store use and data loading at startup.
+Recovering data from an offline disk store proceeds most quickly when the configuration of the offline data matches that of the online data.
+
+Whenever you change or remove persistent regions (by modifying your cache.xml or the code that configures
+the regions), then you should alter the corresponding offline disk-store to match. If you don't, then the next time
+this disk-store is recovered it will recover all of that region's data into a temporary region using the
+old configuration. The old configuration will still consume the old configured resources (heap
+memory, off-heap memory). If those resources are no longer available (for example the old
+configuration of the region was off-heap but you decide to no longer configure off-heap memory on
+the JVM), the disk-store recovery will fail.
+
+It is common practice to have more than one off-line disk store, because each member of the cluster usually has its own copy.
+Be sure to apply the same `alter disk-store` command to each offline copy of the disk store.
 
 ## <a id="syncing_offline_disk_store__section_7B95B20F07BD40699CDB7F3D6A93B905" class="no-quick-link"></a>Change Region Configuration
 
-When your disk store is offline, you can keep the configuration for its regions up-to-date with your `cache.xml` and API settings. The disk store retains region capacity and load settings, including entry map settings (initial capacity, concurrency level, load factor), LRU eviction settings, and the statistics enabled boolean. If the configurations do not match at startup, the `cache.xml` and API override any disk store settings and the disk store is automatically updated to match. So you do not need to modify your disk store to keep your cache configuration and disk store synchronized, but you will save startup time and memory if you do.
+When your disk store is offline, you can keep the configuration for its regions up-to-date with your
+`cache.xml` and API settings. The disk store retains a subset of the region configuration
+attributes.  (For a list of the retained attributes, see [alter
+disk-store](../../tools_modules/gfsh/command-pages/alter.html#topic_99BCAD98BDB5470189662D2F308B68EB)).  If the configurations do not
+match at startup, the `cache.xml` and API override any disk store settings and the disk store is
+automatically updated to match. So you do not need to modify your disk store to keep your cache
+configuration and disk store synchronized, but you will save startup time and memory if you do.
 
-For example, to change the initial capacity of the disk store:
+For example, to change the initial capacity of the region named "partitioned_region" in the disk store:
 
 ``` pre
 gfsh>alter disk-store --name=myDiskStoreName --region=partitioned_region 


[23/49] geode git commit: GEODE-2592 Document Lucene-related gfsh commands

Posted by kh...@apache.org.
GEODE-2592 Document Lucene-related gfsh commands


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/f4f9b6b3
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/f4f9b6b3
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/f4f9b6b3

Branch: refs/heads/feature/GEODE-2420
Commit: f4f9b6b39a647397e588f87042a9509d8eb69107
Parents: 4f307c7
Author: Dave Barnes <db...@pivotal.io>
Authored: Wed Mar 8 16:00:58 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../source/subnavs/geode-subnav.erb             |  18 +++
 .../gfsh/command-pages/create.html.md.erb       |  99 ++++++++++------
 .../gfsh/command-pages/describe.html.md.erb     |  81 +++++++++----
 .../gfsh/command-pages/destroy.html.md.erb      |  65 ++++++++---
 .../gfsh/command-pages/list.html.md.erb         |  70 +++++++++---
 .../gfsh/command-pages/search.html.md.erb       | 114 +++++++++++++++++++
 .../gfsh/gfsh_quick_reference.html.md.erb       |  32 +++---
 .../gfsh/quick_ref_commands_by_area.html.md.erb |  20 +++-
 .../lucene_integration.html.md.erb              |  24 ++--
 9 files changed, 405 insertions(+), 118 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/f4f9b6b3/geode-book/master_middleman/source/subnavs/geode-subnav.erb
----------------------------------------------------------------------
diff --git a/geode-book/master_middleman/source/subnavs/geode-subnav.erb b/geode-book/master_middleman/source/subnavs/geode-subnav.erb
index d3472de..eb9afae 100644
--- a/geode-book/master_middleman/source/subnavs/geode-subnav.erb
+++ b/geode-book/master_middleman/source/subnavs/geode-subnav.erb
@@ -1766,6 +1766,9 @@ gfsh</a>
                                         <a href="/docs/guide/11/tools_modules/gfsh/quick_ref_commands_by_area.html#topic_1C82E6F1B2AF4A65A8DA6B3C846BAC13">Locator Commands</a>
                                     </li>
                                     <li>
+                                        <a href="/docs/guide/11/tools_modules/gfsh/quick_ref_commands_by_area.html#topic_lucene_commands">Lucene Commands</a>
+                                    </li>
+                                    <li>
                                         <a href="/docs/guide/11/tools_modules/gfsh/quick_ref_commands_by_area.html#topic_cvg_bls_5q">PDX Commands</a>
                                     </li>
                                     <li>
@@ -1852,6 +1855,9 @@ gfsh</a>
                                                 <a href="/docs/guide/11/tools_modules/gfsh/command-pages/create.html#topic_960A5B6FD3D84E1881EE118E299DD12D">create index</a>
                                             </li>
                                             <li>
+                                                <a href="/docs/guide/11/tools_modules/gfsh/command-pages/create.html#create_lucene_index">create lucene index</a>
+                                            </li>
+                                            <li>
                                                 <a href="/docs/guide/11/tools_modules/gfsh/command-pages/create.html#topic_54B0985FEC5241CA9D26B0CE0A5EA863">create region</a>
                                             </li>
                                         </ul>
@@ -1882,6 +1888,9 @@ gfsh</a>
                                                 <a href="/docs/guide/11/tools_modules/gfsh/command-pages/describe.html#topic_C635B500BE6A4F1D9572D0BC98A224F2">describe disk-store</a>
                                             </li>
                                             <li>
+                                                <a href="/docs/guide/11/tools_modules/gfsh/command-pages/describe.html#describe_lucene_index">describe lucene index</a>
+                                            </li>
+                                            <li>
                                                 <a href="/docs/guide/11/tools_modules/gfsh/command-pages/describe.html#topic_D62F3D42B1D84CF68F03D54D5122806A">describe member</a>
                                             </li>
                                             <li>
@@ -1906,6 +1915,9 @@ gfsh</a>
                                                 <a href="/docs/guide/11/tools_modules/gfsh/command-pages/destroy.html#topic_D00219CCD6F64C1582A0802AC5CDF3F3">destroy index</a>
                                             </li>
                                             <li>
+                                                <a href="/docs/guide/11/tools_modules/gfsh/command-pages/destroy.html#destroy_lucene_index">destroy lucene index</a>
+                                            </li>
+                                            <li>
                                                 <a href="/docs/guide/11/tools_modules/gfsh/command-pages/destroy.html#topic_BEDACECF4599407794ACBC0E56B30F65">destroy region</a>
                                             </li>
                                         </ul>
@@ -2004,6 +2016,9 @@ gfsh</a>
                                                 <a href="/docs/guide/11/tools_modules/gfsh/command-pages/list.html#topic_B3B51B6DEA484EE086C4F657EC9831F2">list indexes</a>
                                             </li>
                                             <li>
+                                                <a href="/docs/guide/11/tools_modules/gfsh/command-pages/list.html#list_lucene_indexes">list lucene indexes</a>
+                                            </li>
+                                            <li>
                                                 <a href="/docs/guide/11/tools_modules/gfsh/command-pages/list.html#topic_5B5BFB2E5F314210858641BE3A689637">list members</a>
                                             </li>
                                             <li>
@@ -2048,6 +2063,9 @@ gfsh</a>
                                         <a href="/docs/guide/11/tools_modules/gfsh/command-pages/run.html">run</a>
                                     </li>
                                     <li>
+                                        <a href="/docs/guide/11/tools_modules/gfsh/command-pages/search.html">search lucene</a>
+                                    </li>
+                                    <li>
                                         <a href="/docs/guide/11/tools_modules/gfsh/command-pages/set.html">set variable</a>
                                     </li>
                                     <li>

http://git-wip-us.apache.org/repos/asf/geode/blob/f4f9b6b3/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb b/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb
index b901bda..4fb4c8c 100644
--- a/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb
+++ b/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb
@@ -18,36 +18,38 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 -->
-<a id="topic_9663C9448D2745B387FC08F84818DDEC"></a>
-
 
 Create async-event-queues, disk-stores, gateway receivers, gateway senders, indexes, and regions.
 
--   **[create async-event-queue](../../../tools_modules/gfsh/command-pages/create.html#topic_ryz_pb1_dk)**
+-   **[create async-event-queue](#topic_ryz_pb1_dk)**
 
     Creates an asynchronous event queue for batching events before they are delivered by a gateway sender.
 
--   **[create defined indexes](../../../tools_modules/gfsh/command-pages/create.html#topic_w2t_l3m_qq)**
+-   **[create defined indexes](#topic_w2t_l3m_qq)**
 
     Creates all the defined indexes.
 
--   **[create disk-store](../../../tools_modules/gfsh/command-pages/create.html#topic_bkn_zty_ck)**
+-   **[create disk-store](#topic_bkn_zty_ck)**
 
     Defines a pool of one or more disk stores, which can be used by regions and client subscription queues, and gateway sender queues for WAN distribution.
 
--   **[create gateway-receiver](../../../tools_modules/gfsh/command-pages/create.html#topic_a4x_pb1_dk)**
+-   **[create gateway-receiver](#topic_a4x_pb1_dk)**
 
     Creates a gateway receiver. You can only have one gateway receiver on each member, and unlike a gateway sender, you do not need to specify an identifier for the gateway receiver .
 
--   **[create gateway-sender](../../../tools_modules/gfsh/command-pages/create.html#topic_hg2_bjz_ck)**
+-   **[create gateway-sender](#topic_hg2_bjz_ck)**
 
     Creates a gateway sender on one or more members of a distributed system.
 
--   **[create index](../../../tools_modules/gfsh/command-pages/create.html#topic_960A5B6FD3D84E1881EE118E299DD12D)**
+-   **[create index](#topic_960A5B6FD3D84E1881EE118E299DD12D)**
 
     Create an index that can be used when executing queries.
 
--   **[create region](../../../tools_modules/gfsh/command-pages/create.html#topic_54B0985FEC5241CA9D26B0CE0A5EA863)**
+-   **[create lucene index](#create_lucene_index)**
+
+    Create a region with given path and configuration.
+
+-   **[create region](#topic_54B0985FEC5241CA9D26B0CE0A5EA863)**
 
     Create a region with given path and configuration.
 
@@ -72,10 +74,8 @@ create async-event-queue --id=value --listener=value [--group=value(nullvalue)*]
     [--listener-param=value(,value)*]
 ```
 
-<a id="topic_ryz_pb1_dk__table_jzz_pb1_dk"></a>
-
+**Parameters, create async-event-queue:**
 <table>
-<caption><span class="tablecap">Table 1. Create Async-Event-Queue Parameters</span></caption>
 <colgroup>
 <col width="33%" />
 <col width="34%" />
@@ -179,8 +179,6 @@ create async-event-queue --id=value --listener=value [--group=value(nullvalue)*]
 </tbody>
 </table>
 
-<span class="tablecap">Table 1. Create Async-Event-Queue Parameters</span>
-
 **Example Commands:**
 
 ``` pre
@@ -201,14 +199,13 @@ See also [define index](define.html) and [clear defined indexes](clear.html).
 create defined indexes [--member=value] [--group=value]
 ```
 
-<a id="topic_w2t_l3m_qq__table_cgy_3jm_qq"></a>
+**Parameters, create defined indexes:**
 
 | Name                                           | Description                                                        | Default |
 |------------------------------------------------|--------------------------------------------------------------------|---------|
 | <span class="keyword parmname">\\-\\-member</span> | Name/Id of the member on which index will be created.              | �       |
 | <span class="keyword parmname">\\-\\-group</span>  | The index will be created on all the members in this member group. | �       |
 
-<span class="tablecap">Table 2. Create Defined Indexes Parameters</span>
 
 **Example Commands:**
 
@@ -250,10 +247,10 @@ create disk-store --name=value --dir=value(,value)* [--allow-force-compaction(=v
 [--disk-usage-warning-percentage=value] [--disk-usage-critical-percentage=value]
 ```
 
-<a id="topic_bkn_zty_ck__table_lkn_zty_ck"></a>
+
+**Parameters, create disk-store:**
 
 <table>
-<caption><span class="tablecap">Table 3. Create Disk-Store Parameters</span></caption>
 <colgroup>
 <col width="25%" />
 <col width="50%" />
@@ -335,8 +332,6 @@ If the specified directory does not exist, the command will create the directory
 </tbody>
 </table>
 
-<span class="tablecap">Table 3. Create Disk-Store Parameters</span>
-
 **Example Commands:**
 
 ``` pre
@@ -369,10 +364,9 @@ create gateway-receiver [--group=value(,value)*] [--member=value(,value)*]
   [--gateway-transport-filter=value(,value)*]
 ```
 
-<a id="topic_a4x_pb1_dk__table_v4x_pb1_dk"></a>
+**Parameters, create gateway-receiver:**
 
 <table>
-<caption><span class="tablecap">Table 4. Create Gateway-Receiver Parameters</span></caption>
 <colgroup>
 <col width="33%" />
 <col width="33%" />
@@ -436,12 +430,10 @@ create gateway-receiver [--group=value(,value)*] [--member=value(,value)*]
 </tbody>
 </table>
 
-<span class="tablecap">Table 4. Create Gateway-Receiver Parameters</span>
-
 **Example Commands:**
 
 ``` pre
-gfsh>create gatweway-receiver --member=server1
+gfsh>create gateway-receiver --member=server1
 ```
 
 **Sample Output:**
@@ -477,10 +469,9 @@ create gateway-sender --id=value --remote-distributed-system-id=value
    [--gateway-transport-filter=value(,value)*]
 ```
 
-<a id="topic_hg2_bjz_ck__table_ah2_bjz_ck"></a>
+**Parameters, create gateway-sender:**
 
 <table>
-<caption><span class="tablecap">Table 5. Create Gateway-Sender Parameters</span></caption>
 <colgroup>
 <col width="30%" />
 <col width="50%" />
@@ -606,8 +597,6 @@ create gateway-sender --id=value --remote-distributed-system-id=value
 </tbody>
 </table>
 
-<span class="tablecap">Table 5. Create Gateway-Sender Parameters</span>
-
 **Example Commands:**
 
 ``` pre
@@ -638,18 +627,17 @@ create index --name=value --expression=value --region=value
 [--member=value] [--type=value] [--group=value]
 ```
 
-<a id="topic_960A5B6FD3D84E1881EE118E299DD12D__table_9E1C010276AC45A59446A767A517D39F"></a>
+**Parameters: create index:**
 
 | Name                                               | Description                                                                            | Default |
 |----------------------------------------------------|----------------------------------------------------------------------------------------|---------|
 | <span class="keyword parmname">\\-\\-name</span>       | *Required.* Name of the index to create.                                               | �       |
-| <span class="keyword parmname">\\-\\-expression</span> | *Required.* Field of the region values that are referenced by the index.               | �       |
+| <span class="keyword parmname">&#8209;&#8209;expression</span> | *Required.* Field of the region values that are referenced by the index.               | �       |
 | <span class="keyword parmname">\\-\\-region</span>     | *Required.* Name/Path of the region which corresponds to the "from" clause in a query. | �       |
 | <span class="keyword parmname">\\-\\-member</span>     | Name/Id of the member on which index will be created.                                  | �       |
 | <span class="keyword parmname">\\-\\-type</span>       | Type of the index. Valid values are: `range`, `key` and `hash`.                        | `range` |
 | <span class="keyword parmname">\\-\\-group</span>      | The index will be created on all the members in this member group.                     | �       |
 
-<span class="tablecap">Table 6. Create Index Parameters</span>
 
 **Example Commands:**
 
@@ -675,6 +663,48 @@ Occurred on following members
 1. ubuntu(server1:17682)<v1>:27574
 ```
 
+## <a id="create_lucene_index" class="no-quick-link"></a>create lucene index
+
+Create a Lucene index.
+
+See also [describe lucene index](describe.html#describe_lucene_index), [destroy lucene index](destroy.html#destroy_lucene_index), [list lucene indexes](list.html#list_lucene_indexes) and [search lucene](search.html#search_lucene).
+
+**Availability:** Online. You must be connected in <span class="keyword parmname">gfsh</span> to a JMX Manager member to use this command.
+
+**Syntax:**
+
+``` pre
+create lucene index --name=value --region=value --field=value(,value)* [--analyzer=value(,value)*] [--group=value(,value)*]
+```
+
+**Parameters, create lucene index:**
+
+| Name                                               | Description                                                                            | Default |
+|----------------------------------------------------|----------------------------------------------------------------------------------------|---------|
+| <span class="keyword parmname">\\-\\-name</span>       | *Required.* Name of the index to create.                                               | �       |
+| <span class="keyword parmname">\\-\\-region</span>     | *Required.* Name/Path of the region which corresponds to the "from" clause in a query. | �       |
+| <span class="keyword parmname">\\-\\-field</span>      | *Required.* Field of the region values that are referenced by the index.               | �       |
+| <span class="keyword parmname">&#8209;&#8209;analyzer</span>   | Analyzer to extract terms from text                                  | �       |
+| <span class="keyword parmname">\\-\\-group</span>      | The index will be created on all the members in the specified member groups.                     | �       |
+
+
+**Example Commands:**
+
+``` pre
+create region --name=Person --type=PARTITION_REDUNDANT_PERSISTENT
+create lucene index --name=customerIndex --region=/Customer --field=symbol,revenue,SSN,name,email,address,__REGION_VALUE_FIELD
+create lucene index --name=analyzerIndex --region=/Person --field=name,email,address,revenue --analyzer=null,org.apache.lucene.analysis.core.KeywordAnalyzer,examples.MyCharacterAnalyzer,null
+```
+
+**Sample Output:**
+
+``` pre
+gfsh>create lucene index --name=testIndex --region=testRegion --field=__REGION_VALUE_FIELD
+                 Member                  | Status
+---------------------------------------- | ---------------------------------
+192.168.1.23(server50505:17200)<v1>:1025 | Successfully created lucene index
+```
+
 ## <a id="topic_54B0985FEC5241CA9D26B0CE0A5EA863" class="no-quick-link"></a>create region
 
 Create a region with given path and configuration.
@@ -711,10 +741,9 @@ See [Region Data Storage and Distribution](../../../developing/region_options/ch
     [--total-num-buckets=value] [--compressor=value] [--off-heap(=value)]
 ```
 
-<a id="topic_54B0985FEC5241CA9D26B0CE0A5EA863__table_4CDCDC115DD043D5B8ACAA8D4CCFB145"></a>
+**Parameters, create region:**
 
 <table>
-<caption><span class="tablecap">Table 7. Create Region Parameters</span></caption>
 <colgroup>
 <col width="20%" />
 <col width="50%" />
@@ -917,8 +946,6 @@ See [Region Data Storage and Distribution](../../../developing/region_options/ch
 </tbody>
 </table>
 
-<span class="tablecap">Table 7. Create Region Parameters</span>
-
 **Example Commands:**
 
 ``` pre

http://git-wip-us.apache.org/repos/asf/geode/blob/f4f9b6b3/geode-docs/tools_modules/gfsh/command-pages/describe.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/tools_modules/gfsh/command-pages/describe.html.md.erb b/geode-docs/tools_modules/gfsh/command-pages/describe.html.md.erb
index 1c7e7c4..149d0ac 100644
--- a/geode-docs/tools_modules/gfsh/command-pages/describe.html.md.erb
+++ b/geode-docs/tools_modules/gfsh/command-pages/describe.html.md.erb
@@ -18,36 +18,38 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 -->
-<a id="concept_B8A3E21674294C65AE9F9A60F37A284E"></a>
-
 
 Display details of a member's configuration, shell connection, disk-stores, members, or regions.
 
--   **[describe client](../../../tools_modules/gfsh/command-pages/describe.html#topic_gyr_jgz_ck)**
+-   **[describe client](#topic_gyr_jgz_ck)**
 
     Displays details about a specified client.
 
--   **[describe config](../../../tools_modules/gfsh/command-pages/describe.html#topic_3C2C817D999C4E40AF788808B7B6AF99)**
+-   **[describe config](#topic_3C2C817D999C4E40AF788808B7B6AF99)**
 
     Display the configuration of a member.
 
--   **[describe connection](../../../tools_modules/gfsh/command-pages/describe.html#topic_591DC6B781B641168E6173E69AC6D201)**
+-   **[describe connection](#topic_591DC6B781B641168E6173E69AC6D201)**
 
     Display connection information details.
 
--   **[describe disk-store](../../../tools_modules/gfsh/command-pages/describe.html#topic_C635B500BE6A4F1D9572D0BC98A224F2)**
+-   **[describe disk-store](#topic_C635B500BE6A4F1D9572D0BC98A224F2)**
 
     Display information about a member's disk store.
 
--   **[describe member](../../../tools_modules/gfsh/command-pages/describe.html#topic_D62F3D42B1D84CF68F03D54D5122806A)**
+-   **[describe lucene index](#describe_lucene_index)**
+
+    Display information about a Lucene index.
+
+-   **[describe member](#topic_D62F3D42B1D84CF68F03D54D5122806A)**
 
     Display details of a member with given name/id.
 
--   **[describe offline-disk-store](../../../tools_modules/gfsh/command-pages/describe.html#topic_kys_yvk_2l)**
+-   **[describe offline-disk-store](#topic_kys_yvk_2l)**
 
     Display information about an offline member's disk store.
 
--   **[describe region](../../../tools_modules/gfsh/command-pages/describe.html#topic_DECF7D3D33F54071B6B8AD4EA7E3F90B)**
+-   **[describe region](#topic_DECF7D3D33F54071B6B8AD4EA7E3F90B)**
 
     Display the attributes and key information of a region.
 
@@ -63,13 +65,12 @@ Displays details about a specified client.
 describe client --clientID=value
 ```
 
-<a id="topic_gyr_jgz_ck__table_pzd_qdh_2w"></a>
+**Parameters, describe client:**
 
 | Name                                             | Description                                                                                                                                     |
 |--------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------|
 | <span class="keyword parmname">&#8209;&#8209;clientID</span> | *Required.* ID of the client. To find a client ID, you can use the `list clients` command to display a list of connected clients and their IDs. |
 
-<span class="tablecap">Table 1. Describe Client Parameters</span>
 
 **Example Commands:**
 
@@ -108,14 +109,13 @@ Display the configuration of a member.
 describe config --member=value [--hide-defaults(=value)?]
 ```
 
-<a id="topic_3C2C817D999C4E40AF788808B7B6AF99__table_ohx_tdh_2w"></a>
+**Parameters, describe config:**
 
 | Name                                                  | Description                                                                      | Default Value |
 |-------------------------------------------------------|----------------------------------------------------------------------------------|---------------|
 | <span class="keyword parmname">\\-\\-member </span>       | Name or ID of a member whose configuration is to be shown.                       | �             |
 | <span class="keyword parmname">\\-\\-hide-defaults</span> | Whether to hide configuration information for properties with the default value. | true          |
 
-<span class="tablecap">Table 2. Describe Config Parameters</span>
 
 **Example Commands:**
 
@@ -192,14 +192,13 @@ Display information about a member's disk store.
 describe disk-store --member=value --name=value
 ```
 
-<a id="topic_C635B500BE6A4F1D9572D0BC98A224F2__table_xs1_12h_2w"></a>
+**Parameters, describe disk-store:**
 
 | Name                                           | Description                                                            |
 |------------------------------------------------|------------------------------------------------------------------------|
 | <span class="keyword parmname">\\-\\-member</span> | *Required.* Name/ID of the member with the disk store to be described. |
 | <span class="keyword parmname">\\-\\-name </span>  | *Required*. Name of the disk store to be described.                    |
 
-<span class="tablecap">Table 3. Describe Disk-Store Parameters</span>
 
 **Example Commands:**
 
@@ -233,6 +232,49 @@ PDX Serialization Meta-Data Stored : No
 
 ```
 
+## <a id="describe_lucene_index" class="no-quick-link"></a>describe lucene index
+
+Describe a Lucene index.
+
+See also [create lucene index](create.html#create_lucene_index), [destroy lucene index](destroy.html#destroy_lucene_index), [list lucene indexes](list.html#list_lucene_indexes) and [search lucene](search.html#search_lucene).
+
+**Availability:** Online. You must be connected in `gfsh` to a JMX Manager member to use this command.
+
+**Syntax:**
+
+``` pre
+describe lucene index --name=value --region=value
+```
+
+**Parameters, describe lucene index:**
+
+| Name                                               | Description                                                                  |
+|----------------------------------------------------|------------------------------------------------------------------------------|
+| <span class="keyword parmname">\\-\\-name</span>       | *Required.* Name of the Lucene index to describe                          |
+| <span class="keyword parmname">\\-\\-region</span>     | *Required.* Name and path of the region in which the Lucene index exists               |
+
+
+
+**Example Commands:**
+
+``` pre
+gfsh>describe lucene index --name=personIndex --region=/Person
+```
+
+**Sample Output:**
+
+``` pre
+gfsh>describe lucene index --name=personIndex --region=/Person
+Index Name  | Region Path |                 Indexed Fields                 | Field Analyzer |   Status    | Query Executions | Updates | Commits | Documents
+----------- | ----------- | ---------------------------------------------- | -------------- | ----------- | ---------------- | ------- | ------- | ---------
+personIndex | /Person     | [name, email, address, streetAddress, revenue] | {}             | Initialized | 339              | 1008    | 962     | 1004
+
+gfsh>describe lucene index --name=analyzerIndex --region=/Person
+ Index Name   | Region Path |     Indexed Fields     |            Field Analyzer             |   Status    | Query Executions | Updates | Commits | Documents
+------------- | ----------- | ---------------------- | ------------------------------------- | ----------- | ---------------- | ------- | ------- | ---------
+analyzerIndex | /Person     | [address, name, email] | {address=MyCharacterAnalyzer, email.. | Initialized | 1695             | 1008    | 962     | 1004
+```
+
 ## <a id="topic_D62F3D42B1D84CF68F03D54D5122806A" class="no-quick-link"></a>describe member
 
 Display details of a member with given name/id.
@@ -245,13 +287,12 @@ Display details of a member with given name/id.
 describe member --name=value
 ```
 
-<a id="topic_D62F3D42B1D84CF68F03D54D5122806A__table_psn_c2h_2w"></a>
+**Parameters, describe member:**
 
 | Name                                          | Description                                                                               |
 |-----------------------------------------------|-------------------------------------------------------------------------------------------|
 | <span class="keyword parmname">&#8209;&#8209;name </span> | *Required.* Display information about a member, including name, ID, groups, regions, etc. |
 
-<span class="tablecap">Table 4. Describe Member Parameters</span>
 
 **Example Commands:**
 
@@ -298,7 +339,7 @@ Display information about an offline member's disk store.
 describe offline-disk-store --name=value --disk-dirs=value(,value)* [--pdx=value] [--region=value]
 ```
 
-<a id="topic_kys_yvk_2l__table_rlg_f2h_2w"></a>
+**Parameters, describe offline-disk-store:**
 
 | Name                                               | Description                                                                  |
 |----------------------------------------------------|------------------------------------------------------------------------------|
@@ -307,7 +348,6 @@ describe offline-disk-store --name=value --disk-dirs=value(,value)* [--pdx=value
 | <span class="keyword parmname">\\-\\-pdx</span>        | If set (or set to true), display all the pdx types stored in the disk store. |
 | <span class="keyword parmname">\\-\\-region</span>     | Name and path of the region in the disk store to be described.               |
 
-<span class="tablecap">Table 5. Describe Offline-Disk-Store Parameters</span>
 
 **Example Commands:**
 
@@ -375,13 +415,12 @@ Display the attributes and key information of a region.
 describe region --name=value
 ```
 
-<a id="topic_DECF7D3D33F54071B6B8AD4EA7E3F90B__table_wvt_32h_2w"></a>
+**Parameters, describe region:**
 
 | Name                                          | Description                                          |
 |-----------------------------------------------|------------------------------------------------------|
 | <span class="keyword parmname">\\-\\-name </span> | *Required.* Name/Path of the region to be described. |
 
-<span class="tablecap">Table 6. Describe Region Parameters</span>
 
 **Example Commands:**
 

http://git-wip-us.apache.org/repos/asf/geode/blob/f4f9b6b3/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb b/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb
index 5338abc..e6de426 100644
--- a/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb
+++ b/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb
@@ -18,24 +18,27 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 See the License for the specific language governing permissions and
 limitations under the License.
 -->
-<a id="concept_B8A3E21674294C65AE9F9A60F37A284E"></a>
 
 
 Delete or unregister functions, remove indexes, disk stores and regions.
 
--   **[destroy disk-store](../../../tools_modules/gfsh/command-pages/destroy.html#topic_yfr_l2z_ck)**
+-   **[destroy disk-store](#topic_yfr_l2z_ck)**
 
     Deletes a disk store and all files on disk used by the disk store. Data for closed regions that previously used this disk store is lost.
 
--   **[destroy function](../../../tools_modules/gfsh/command-pages/destroy.html#topic_E48C2DF809054C12A162026D8A2139BB)**
+-   **[destroy function](#topic_E48C2DF809054C12A162026D8A2139BB)**
 
     Destroy or unregister a function.
 
--   **[destroy index](../../../tools_modules/gfsh/command-pages/destroy.html#topic_D00219CCD6F64C1582A0802AC5CDF3F3)**
+-   **[destroy index](#topic_D00219CCD6F64C1582A0802AC5CDF3F3)**
 
     Destroy or remove the specified index.
 
--   **[destroy region](../../../tools_modules/gfsh/command-pages/destroy.html#topic_BEDACECF4599407794ACBC0E56B30F65)**
+-   **[destroy lucene index](#destroy_lucene_index)**
+
+    Destroy or remove the specified Lucene index.
+
+-   **[destroy region](#topic_BEDACECF4599407794ACBC0E56B30F65)**
 
     Destroy or remove a region.
 
@@ -51,14 +54,13 @@ Deletes a disk store and all files on disk used by the disk store. Data for clos
 destroy disk-store --name=value [--group=value(,value)*]
 ```
 
-<a id="topic_yfr_l2z_ck__table_ypc_1dh_2w"></a>
+**Parameters, destroy disk-store:**
 
 | Name                                          | Description                                                                                                                          |
 |-----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------|
 | <span class="keyword parmname">\\-\\-name</span>  | *Required.* Name of the disk store to be deleted.                                                                                    |
 | <span class="keyword parmname">&#8209;&#8209;group</span> | Group(s) of members on which the disk store will be destroyed. If no group is specified, the disk store is destroyed on all members. |
 
-<span class="tablecap">Table 1. Destroy Disk-Store Parameters</span>
 
 **Example Commands:**
 
@@ -89,7 +91,7 @@ The default is for the function to be unregistered from all members.
 destroy function --id=value [--group=value] [--member=value]
 ```
 
-<a id="topic_E48C2DF809054C12A162026D8A2139BB__table_rq1_ddh_2w"></a>
+**Parameters, destroy function:**
 
 | Name                                           | Description                                                                                |
 |------------------------------------------------|--------------------------------------------------------------------------------------------|
@@ -97,7 +99,6 @@ destroy function --id=value [--group=value] [--member=value]
 | <span class="keyword parmname">\\-\\-groups</span> | One or more groups of members from which this function will be unregistered.               |
 | <span class="keyword parmname">&#8209;&#8209;member</span> | Name or ID of the member from which this function will be unregistered.                    |
 
-<span class="tablecap">Table 2. Destroy Function Parameters</span>
 
 **Example Commands:**
 
@@ -123,16 +124,15 @@ destroy index [--name=value] [--region=value] [--member=value]
 **Note:**
 You must specify at least one of the parameter options. If you enter `destroy index` without any parameters, the command will ask you to specify at least one option.
 
-<a id="topic_D00219CCD6F64C1582A0802AC5CDF3F3__table_n24_fdh_2w"></a>
+**Parameters, destroy index:**
 
 | Name                                           | Description                                                                  |
 |------------------------------------------------|------------------------------------------------------------------------------|
-| <span class="keyword parmname">\\-\\-name</span>   | Name for the index to be removed.                                            |
-| <span class="keyword parmname">\\-\\-member</span> | Id of the member on which index is to be created.                            |
-| <span class="keyword parmname">\\-\\-region</span> | Name of the region , from where an index or all indexes are to be destroyed. |
-| <span class="keyword parmname">\\-\\-group</span>  | The index will be created on all the members in this member group.           |
+| <span class="keyword parmname">\\-\\-name</span>   | Name of the index to be removed.                                            |
+| <span class="keyword parmname">\\-\\-member</span> | Id of the member on which index is to be removed.                            |
+| <span class="keyword parmname">\\-\\-region</span> | Name of the region from which an index or all indexes are to be destroyed. |
+| <span class="keyword parmname">\\-\\-group</span>  | The index will be removed on all the members in this member group.           |
 
-<span class="tablecap">Table 3. Destroy Index Parameters</span>
 
 **Example Commands:**
 
@@ -141,6 +141,38 @@ destroy index --member=server2
 destroy index --name=MyKeyIndex
 ```
 
+## <a id="destroy_lucene_index" class="no-quick-link"></a>destroy lucene index
+
+Destroy or remove the specified Lucene index.
+
+See also [create lucene index](create.html#create_lucene_index), [describe lucene index](describe.html#describe_lucene_index), [list lucene indexes](list.html#list_lucene_indexes) and [search lucene](search.html#search_lucene).
+
+**Availability:** Online. You must be connected in `gfsh` to a JMX Manager member to use this command.
+
+**Syntax:**
+
+``` pre
+destroy lucene index [--name=value] [--region=value]
+```
+
+**Note:**
+You must specify at least one of the parameter options. If you enter `destroy lucene index` without any parameters, the command will ask you to specify at least one option.
+
+**Parameters, destroy lucene index:**
+
+| Name                                           | Description                                                                  |
+|------------------------------------------------|------------------------------------------------------------------------------|
+| <span class="keyword parmname">\\-\\-name</span>   | Name of the index to be removed.                                            |
+| <span class="keyword parmname">\\-\\-region</span> | Name of the region from which an index or all indexes are to be removed. |
+
+
+**Example Commands:**
+
+``` pre
+destroy lucene index --member=server2
+destroy lucene index --name=MyKeyIndex
+```
+
 ## <a id="topic_BEDACECF4599407794ACBC0E56B30F65" class="no-quick-link"></a>destroy region
 
 Destroy or remove a region.
@@ -153,13 +185,12 @@ Destroy or remove a region.
 destroy region --name=value
 ```
 
-<a id="topic_BEDACECF4599407794ACBC0E56B30F65__table_yb5_3dh_2w"></a>
+**Parameters, destroy region:**
 
 | Name                                         | Description                                            |
 |----------------------------------------------|--------------------------------------------------------|
 | <span class="keyword parmname">\\-\\-name</span> | *Required.* Name and path of the region to be removed. |
 
-<span class="tablecap">Table 4. Destroy Region Parameters</span>
 
 **Example Commands:**
 

http://git-wip-us.apache.org/repos/asf/geode/blob/f4f9b6b3/geode-docs/tools_modules/gfsh/command-pages/list.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/tools_modules/gfsh/command-pages/list.html.md.erb b/geode-docs/tools_modules/gfsh/command-pages/list.html.md.erb
index e6238da..6851b0a 100644
--- a/geode-docs/tools_modules/gfsh/command-pages/list.html.md.erb
+++ b/geode-docs/tools_modules/gfsh/command-pages/list.html.md.erb
@@ -19,7 +19,6 @@ See the License for the specific language governing permissions and
 limitations under the License.
 -->
 
-<a id="topic_AE5B406FAE7F4152B2919EDF45F4EE80"></a>
 
 
 List existing Geode resources such as deployed applications, disk-stores, functions, members, servers, and regions.
@@ -56,6 +55,10 @@ List existing Geode resources such as deployed applications, disk-stores, functi
 
     Display the list of indexes created for all members.
 
+-   **[list lucene indexes](#list_lucene_indexes)**
+
+    List Lucene indexes created for all members.
+
 -   **[list members](#topic_5B5BFB2E5F314210858641BE3A689637)**
 
     Display all or a subset of members.
@@ -127,13 +130,12 @@ Display a list of JARs that were deployed to members using the deploy command.
 list deployed [--group=value(,value)*]
 ```
 
-<a id="topic_59DF60DE71AD4097B281749425254BFF__table_ch5_b2g_2w"></a>
+**Parameters, list deployed:**
 
 | Name                                          | Description                                                                                                          |
 |-----------------------------------------------|----------------------------------------------------------------------------------------------------------------------|
 | <span class="keyword parmname">&#8209;&#8209;group</span> | Group(s) of members for which deployed JARs will be displayed. If not specified, JARs for all members are displayed. |
 
-<span class="tablecap">Table 1. List Deployed Parameters</span>
 
 **Example Commands:**
 
@@ -211,7 +213,7 @@ List durable client CQs associated with the specified durable client id.
 list durable-cqs --durable-client-id=value [--member=value] [--group=value]
 ```
 
-<a id="topic_66016A698C334F4EBA19B99F51B0204B__table_esj_32g_2w"></a>
+**Parameters, list durable-cqs:**
 
 | Name                                                       | Description                                                                                            |
 |------------------------------------------------------------|--------------------------------------------------------------------------------------------------------|
@@ -219,7 +221,6 @@ list durable-cqs --durable-client-id=value [--member=value] [--group=value]
 | <span class="keyword parmname">\\-\\-member</span>             | Name or Id of the member for which the durable client is registered and durable CQs will be displayed. |
 | <span class="keyword parmname">\\-\\-group</span>              | Group of members for which the durable client is registered and durable CQs will be displayed.         |
 
-<span class="tablecap">Table 2. List Durable-CQs Parameters</span>
 
 **Example Commands**:
 
@@ -267,7 +268,7 @@ list functions [--matches=value] [--group=value(,value)*]
 [--member=value(,value)*]
 ```
 
-<a id="topic_DCC7CCBBEF5942B783A8F2A4A5B2FABF__table_d3y_l2g_2w"></a>
+**Parameters, list functions:**
 
 | Name                                             | Description                                                                                                                                                                                     |
 |--------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
@@ -275,7 +276,6 @@ list functions [--matches=value] [--group=value(,value)*]
 | <span class="keyword parmname">\\-\\-group</span>    | Group(s) of members for which functions will be displayed. Use a comma separated list for multiple groups.                                                                                      |
 | <span class="keyword parmname">&#8209;&#8209;member </span>  | Name or ID of the member(s) for which functions will be displayed. Use a comma separated list for multiple members.                                                                             |
 
-<span class="tablecap">Table 3. List Functions Parameters</span>
 
 **Example Commands:**
 
@@ -325,14 +325,13 @@ Displays the gateway senders and receivers for a member or members.
 list gateways [--group=value(,value)*]
 ```
 
-<a id="topic_B1D89671C7B74074899C7D52F15849ED__table_n3w_q2g_2w"></a>
+**Parameters, list gateways:**
 
 | Name                                           | Description                                                                                                                    |
 |------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------|
 | <span class="keyword parmname">&#8209;&#8209;member</span> | Member(s) whose gateways senders and receiver display.                                                                         |
 | <span class="keyword parmname">\\-\\-group</span>  | Group(s) of members for which Gateway Senders and Receivers will be displayed. Use a comma separated list for multiple groups. |
 
-<span class="tablecap">Table 4. List Gateways Parameters</span>
 
 **Example Commands:**
 
@@ -352,13 +351,12 @@ Display the list of indexes created for all members.
 list indexes [--with-stats(=value)?]
 ```
 
-<a id="topic_B3B51B6DEA484EE086C4F657EC9831F2__table_hth_t2g_2w"></a>
+**Parameters, list indexes:**
 
 | Name                                               | Description                                            | Default Value |
 |----------------------------------------------------|--------------------------------------------------------|---------------|
 | <span class="keyword parmname">\\-\\-with-stats</span> | Specifies whether statistics should also be displayed. | false         |
 
-<span class="tablecap">Table 5. List Indexes Parameters </span>
 
 **Example Commands:**
 
@@ -398,6 +396,50 @@ gfsh> list indexes
 No Indexes Found
 ```
 
+## <a id="list_lucene_indexes" class="no-quick-link"></a>list lucene indexes
+
+Display the list of Lucene indexes created for all members. The optional `--with-stats` qualifier shows activity on the indexes.
+
+See also [create lucene index](create.html#create_lucene_index), [describe lucene index](describe.html#describe_lucene_index), [destroy lucene index](destroy.html#destroy_lucene_index) and [search lucene](search.html#search_lucene).
+
+**Availability:** Online. You must be connected in `gfsh` to a JMX Manager member to use this command.
+
+**Syntax:**
+
+``` pre
+list lucene indexes [--with-stats(=value)]
+```
+
+**Parameters, list lucene indexes:**
+
+| Name                                               | Description                                            | Default Value |
+|----------------------------------------------------|--------------------------------------------------------|---------------|
+| <span class="keyword parmname">\\-\\-with-stats</span> | Specifies whether statistics should also be displayed. | false if not specified, true if specified         |
+
+
+**Example Commands:**
+
+``` pre
+list lucene indexes
+```
+
+**Sample Output:**
+
+``` pre
+gfsh>list lucene indexes --with-stats
+Index Name | Region Path |     Indexed Fields     | Field Analy.. | Status  | Query Executions | Updates | Commits | Documents
+---------- | ----------- | ---------------------- | ------------- | ------- | ---------------- | ------- | ------- | ---------
+testIndex  | /testRegion | [__REGION_VALUE_FIELD] | {__REGION_V.. | Defined | NA               | NA      | NA      | NA
+
+gfsh>list lucene indexes
+ Index Name   | Region Path |                           Indexed Fields                           | Field Analy.. | Status
+------------- | ----------- | ------------------------------------------------------------------ | ------------- | -----------
+analyzerIndex | /Person     | [revenue, address, name, email]                                    | {revenue=St.. | Initialized
+customerIndex | /Customer   | [symbol, revenue, SSN, name, email, address, __REGION_VALUE_FIELD] | {}            | Initialized
+pageIndex     | /Page       | [id, title, content]                                               | {}            | Initialized
+personIndex   | /Person     | [name, email, address, revenue]                                    | {}            | Initialized
+```
+
 ## <a id="topic_5B5BFB2E5F314210858641BE3A689637" class="no-quick-link"></a>list members
 
 Display all or a subset of members.
@@ -410,13 +452,12 @@ Display all or a subset of members.
 list members [--group=value]
 ```
 
-<a id="topic_5B5BFB2E5F314210858641BE3A689637__table_r2h_x2g_2w"></a>
+**Parameters, list members:**
 
 | Name                                          | Description                                     |
 |-----------------------------------------------|-------------------------------------------------|
 | <span class="keyword parmname">\\-\\-group</span> | Group name for which members will be displayed. |
 
-<span class="tablecap">Table 6. List Members Parameters</span>
 
 **Example Commands:**
 
@@ -446,14 +487,13 @@ Display regions of a member or members. If no parameter is specified, all region
 list regions [--group=value] [--member=value]
 ```
 
-<a id="topic_F0ECEFF26086474498598035DD83C588__table_ckh_1fg_2w"></a>
+**Parameters, list regions:**
 
 | Name                                            | Description                                                                 |
 |-------------------------------------------------|-----------------------------------------------------------------------------|
 | <span class="keyword parmname">\\-\\-group</span>   | Group of members for which regions will be displayed.                       |
 | <span class="keyword parmname">\\-\\-member </span> | Name or ID of the member of the member for which regions will be displayed. |
 
-<span class="tablecap">Table 7. List Regions Parameters</span>
 
 **Example Commands:**
 

http://git-wip-us.apache.org/repos/asf/geode/blob/f4f9b6b3/geode-docs/tools_modules/gfsh/command-pages/search.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/tools_modules/gfsh/command-pages/search.html.md.erb b/geode-docs/tools_modules/gfsh/command-pages/search.html.md.erb
new file mode 100644
index 0000000..7f239e9
--- /dev/null
+++ b/geode-docs/tools_modules/gfsh/command-pages/search.html.md.erb
@@ -0,0 +1,114 @@
+---
+title:  search lucene
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements.  See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License.  You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+## <a id="search_lucene" class="no-quick-link"></a>search lucene
+
+Search a Lucene index
+
+See also [create lucene index](create.html#create_lucene_index), [describe lucene index](describe.html#describe_lucene_index), [destroy lucene index](destroy.html#destroy_lucene_index) and [list lucene indexes](list.html#list_lucene_indexes).
+
+**Availability:** Online.
+
+**Syntax:**
+
+``` pre
+search lucene --name=value --region=value --queryStrings=value --defaultField=value
+    [--limit=value] [--pageSize=value] [--keys-only=value]
+```
+
+**Parameters, search lucene:**
+
+<table>
+<colgroup>
+<col width="33%" />
+<col width="34%" />
+<col width="33%" />
+</colgroup>
+<thead>
+<tr class="header">
+<th>Name</th>
+<th>Description</th>
+<th>Default Value</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td><span class="keyword parmname">\-\-name</span></td>
+<td><em>Required</em>. Name of the lucene index to search.</td>
+<td> </td>
+</tr>
+<tr>
+<td><span class="keyword parmname">\-\-region</span></td>
+<td><em>Required</em>. Name/Path of the region where the lucene index exists.</td>
+<td> </td>
+</tr>
+<tr>
+<td><span class="keyword parmname" style="whitespace:nowrap;">&#8209;&#8208;queryStrings</span></td>
+<td><em>Required</em>. Query string to search the lucene index.</td>
+<td> </td>
+</tr>
+<tr>
+<td><span class="keyword parmname" style="whitespace:nowrap;">&#8209;&#8208;defaultField</span></td>
+<td><em>Required</em>. Default field to search in.</td>
+<td> </td>
+</tr>
+<tr>
+<td><span class="keyword parmname">\-\-limit</span></td>
+<td>Number of search results needed.</td>
+<td>If the parameter is not specified: -1</td>
+</tr>
+<tr>
+<td><span class="keyword parmname">\-\-pageSize</span></td>
+<td>Number of results to be returned in a page.</td>
+<td>If the parameter is not specified: -1</td>
+</tr>
+<td><span class="keyword parmname">\-\-keys-only</span></td>
+<td>Return only keys of search results.</td>
+<td>If the parameter is not specified: false</td>
+</tr>
+</table>
+
+**Example Commands:**
+
+``` pre
+gfsh> search lucene --name=testIndex --region=/testRegion --queryStrings=value1 --defaultField=__REGION_VALUE_FIELD
+ 
+```
+
+**Sample Output:**
+
+``` pre
+gfsh>search lucene --name=testIndex --region=/testRegion --queryStrings=value* --defaultField=__REGION_VALUE_FIELD
+key | value  | score
+--- | ------ | -----
+3   | value3 | 1
+2   | value2 | 1
+1   | value1 | 1
+
+gfsh>search lucene --region=/Person --name=analyzerIndex --defaultField=address --queryStrings="97763"
+ key   |                                                   value                                                   | score
+------ | --------------------------------------------------------------------------------------------------------- | ---------
+key763 | Person{name='Fred Freeloader', email='ffl@example.com', address='763 Miles Dv, Portland_OR_97763', re.. | 1.6694657
+
+
+```
+
+

http://git-wip-us.apache.org/repos/asf/geode/blob/f4f9b6b3/geode-docs/tools_modules/gfsh/gfsh_quick_reference.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/tools_modules/gfsh/gfsh_quick_reference.html.md.erb b/geode-docs/tools_modules/gfsh/gfsh_quick_reference.html.md.erb
index 7dbc5bb..4c78194 100644
--- a/geode-docs/tools_modules/gfsh/gfsh_quick_reference.html.md.erb
+++ b/geode-docs/tools_modules/gfsh/gfsh_quick_reference.html.md.erb
@@ -25,34 +25,36 @@ This quick reference sorts all commands into functional areas.
 
 Click a command to see additional information, including syntax, a list of options, and examples.
 
--   **[Basic Geode gfsh Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_77DA6E3929404EB4AC24230CC7C21493)**
+-   **[Basic Geode gfsh Commands](quick_ref_commands_by_area.html#topic_77DA6E3929404EB4AC24230CC7C21493)**
 
--   **[Configuration Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_EB854534301A477BB01058B3B142AE1D)**
+-   **[Configuration Commands](quick_ref_commands_by_area.html#topic_EB854534301A477BB01058B3B142AE1D)**
 
--   **[Data Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_C7DB8A800D6244AE8FF3ADDCF139DCE4)**
+-   **[Data Commands](quick_ref_commands_by_area.html#topic_C7DB8A800D6244AE8FF3ADDCF139DCE4)**
 
--   **[Deployment Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_1B47A0E110124EB6BF08A467EB510412)**
+-   **[Deployment Commands](quick_ref_commands_by_area.html#topic_1B47A0E110124EB6BF08A467EB510412)**
 
--   **[Disk Store Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_1ACC91B493EE446E89EC7DBFBBAE00EA)**
+-   **[Disk Store Commands](quick_ref_commands_by_area.html#topic_1ACC91B493EE446E89EC7DBFBBAE00EA)**
 
--   **[Durable CQ and Client Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_10613D4850F04A3EB507F6B441AD3413)**
+-   **[Durable CQ and Client Commands](quick_ref_commands_by_area.html#topic_10613D4850F04A3EB507F6B441AD3413)**
 
--   **[Function Execution Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_8BB061D1A7A9488C819FE2B7881A1278)**
+-   **[Function Execution Commands](quick_ref_commands_by_area.html#topic_8BB061D1A7A9488C819FE2B7881A1278)**
 
--   **[Gateway (WAN) Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_F0AE5CE40D6D49BF92247F5EF4F871D2)**
+-   **[Gateway (WAN) Commands](quick_ref_commands_by_area.html#topic_F0AE5CE40D6D49BF92247F5EF4F871D2)**
 
--   **[Geode Monitoring Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_B742E9E862BA457082E2346581C97D03)**
+-   **[Geode Monitoring Commands](quick_ref_commands_by_area.html#topic_B742E9E862BA457082E2346581C97D03)**
 
--   **[Index Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_688C66526B4649AFA51C0F72F34FA45E)**
+-   **[Index Commands](quick_ref_commands_by_area.html#topic_688C66526B4649AFA51C0F72F34FA45E)**
 
--   **[JMX Connection Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_2A6DA4078E4E496A9F725A29BC4CFD0D)**
+-   **[JMX Connection Commands](quick_ref_commands_by_area.html#topic_2A6DA4078E4E496A9F725A29BC4CFD0D)**
 
--   **[Locator Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_1C82E6F1B2AF4A65A8DA6B3C846BAC13)**
+-   **[Locator Commands](quick_ref_commands_by_area.html#topic_1C82E6F1B2AF4A65A8DA6B3C846BAC13)**
 
--   **[PDX Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_cvg_bls_5q)**
+-   **[Lucene Commands](quick_ref_commands_by_area.html#topic_lucene_commands)**
 
--   **[Region Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_EF03119A40EE492984F3B6248596E1DD)**
+-   **[PDX Commands](quick_ref_commands_by_area.html#topic_cvg_bls_5q)**
 
--   **[Server Commands](../../tools_modules/gfsh/quick_ref_commands_by_area.html#topic_8A341FF86958466E9E64CF06CD21FED9)**
+-   **[Region Commands](quick_ref_commands_by_area.html#topic_EF03119A40EE492984F3B6248596E1DD)**
+
+-   **[Server Commands](quick_ref_commands_by_area.html#topic_8A341FF86958466E9E64CF06CD21FED9)**
 
 

http://git-wip-us.apache.org/repos/asf/geode/blob/f4f9b6b3/geode-docs/tools_modules/gfsh/quick_ref_commands_by_area.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/tools_modules/gfsh/quick_ref_commands_by_area.html.md.erb b/geode-docs/tools_modules/gfsh/quick_ref_commands_by_area.html.md.erb
index c93f488..f707a64 100644
--- a/geode-docs/tools_modules/gfsh/quick_ref_commands_by_area.html.md.erb
+++ b/geode-docs/tools_modules/gfsh/quick_ref_commands_by_area.html.md.erb
@@ -277,6 +277,20 @@ limitations under the License.
 
 <span class="tablecap">Table 13. Locator Commands</span>
 
+## <a id="topic_lucene_commands" class="no-quick-link"></a>Lucene Commands
+
+<a id="topic_1C82E6F1B2AF4A65A8DA6B3C846BAC13__table_hy4_4z1_3l"></a>
+
+| Command                                                                                                                                                          | Description                                                                                                                                                                                                                                                                     | Availability    |
+|------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------|
+| [create lucene index](command-pages/create.html#create_lucene_index)                                | Create a Lucene index. | online |
+| [describe lucene index](command-pages/describe.html#describe_lucene_index) | Describe a Lucene index. | online |
+| [destroy lucene index](command-pages/destroy.html#destroy_lucene_index) | Destroy a Lucene index. | online |
+| [list lucene indexes](command-pages/list.html#list_lucene_indexes) | List Lucene indexes created for all members. The optional `--with-stats` qualifier shows activity on the indexes. | online |
+| [search lucene](command-pages/search.html#search_lucene) | Search a Lucene index. | online |
+
+<span class="tablecap">Table 14. Lucene Commands</span>
+
 ## <a id="topic_cvg_bls_5q" class="no-quick-link"></a>PDX Commands
 
 <a id="topic_cvg_bls_5q__table_tdf_cls_5q"></a>
@@ -286,7 +300,7 @@ limitations under the License.
 | [configure pdx](command-pages/configure.html#topic_jdkdiqbgphqh)                         | Configure Portable Data eXchange for all the cache(s) in the cluster. | online, offline |
 | [pdx rename](command-pages/pdx.html) | Renames PDX types in an offline disk store.                           | online, offline |
 
-<span class="tablecap">Table 14. PDX Commands</span>
+<span class="tablecap">Table 15. PDX Commands</span>
 
 ## <a id="topic_EF03119A40EE492984F3B6248596E1DD" class="no-quick-link"></a>Region Commands
 
@@ -301,7 +315,7 @@ limitations under the License.
 | [list regions](command-pages/list.html#topic_F0ECEFF26086474498598035DD83C588) | Display regions of a member or members. If no parameter is specified, all regions in the Geode distributed system are listed. | online       |
 | [rebalance](command-pages/rebalance.html)                                                                                                                                     | Rebalance partitioned regions.                                                                                                                             | online       |
 
-<span class="tablecap">Table 15. Region Commands</span>
+<span class="tablecap">Table 16. Region Commands</span>
 
 ## <a id="topic_8A341FF86958466E9E64CF06CD21FED9" class="no-quick-link"></a>Server Commands
 
@@ -313,6 +327,6 @@ limitations under the License.
 | [status server](command-pages/status.html#topic_E5DB49044978404D9D6B1971BF5D400D) | Display the status of the specified Geode cache server. | online, offline |
 | [stop server](command-pages/stop.html#topic_723EE395A63A40D6819618AFC2902115)                                  | Stop a Geode cache server.                              | online, offline |
 
-<span class="tablecap">Table 16. Server Commands</span>
+<span class="tablecap">Table 17. Server Commands</span>
 
 

http://git-wip-us.apache.org/repos/asf/geode/blob/f4f9b6b3/geode-docs/tools_modules/lucene_integration.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/tools_modules/lucene_integration.html.md.erb b/geode-docs/tools_modules/lucene_integration.html.md.erb
index aa19e43..4ccf806 100644
--- a/geode-docs/tools_modules/lucene_integration.html.md.erb
+++ b/geode-docs/tools_modules/lucene_integration.html.md.erb
@@ -82,22 +82,24 @@ Collection<Person> results = query.findValues();
 
 ## <a id="gfsh-api" class="no-quick-link"></a>Gfsh API
 
-The gfsh command-line utility supports four Apache Lucene actions:
-
-<dt><b>create lucene index</b></dt>
-    <dd>Create a lucene index that can be used to execute queries.</dd>
-<dt><b>describe lucene index</b></dt>
-    <dd>Display the describe of lucene indexes created for all members.</dd>
-<dt><b>list lucene indexes [with-stats]</b></dt>
-    <dd>Display the list of lucene indexes created for all members. The optional `with-stats` qualifier shows activity on the indexes.</dd>
-<dt><b>search lucene</b></dt>
-    <dd>Search lucene index</dd>
+The gfsh command-line utility supports five Apache Lucene actions:
+
+<dt><a href="gfsh/command-pages/create.html#create_lucene_index"><b>create lucene index</b></a></dt>
+    <dd>Create a Lucene index that can be used to execute queries.</dd>
+<dt><a href="gfsh/command-pages/describe.html#describe_lucene_index"><b>describe lucene index</b></a></dt>
+    <dd>Describe a Lucene index.</dd>
+<dt><a href="gfsh/command-pages/destroy.html#destroy_lucene_index"><b>destroy lucene index</b></a></dt>
+    <dd>Destroy a Lucene index.</dd>
+<dt><a href="gfsh/command-pages/list.html#list_lucene_indexes"><b>list lucene indexes</b></a></dt>
+    <dd>List Lucene indexes created for all members.</dd>
+<dt><a href="gfsh/command-pages/search.html#search_lucene"><b>search lucene</b></a></dt>
+    <dd>Search a Lucene index</dd>
 
 **Gfsh command-line examples:**
 
 ``` pre
 // List Index
-gfsh> list lucene indexes [with-stats]
+gfsh> list lucene indexes --with-stats
 
 // Create Index
 gfsh>create lucene index --name=indexName --region=/orders --field=customer,tags


[10/49] geode git commit: GEODE-2614: fix javadoc warnings - spotless

Posted by kh...@apache.org.
GEODE-2614: fix javadoc warnings - spotless


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/c6c6459a
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/c6c6459a
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/c6c6459a

Branch: refs/heads/feature/GEODE-2420
Commit: c6c6459a1a2fc6e4cf272655fd29b633d3108834
Parents: 04f9350
Author: Jinmei Liao <ji...@pivotal.io>
Authored: Wed Mar 8 10:28:39 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../apache/geode/management/internal/cli/CliAroundInterceptor.java  | 1 +
 1 file changed, 1 insertion(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/c6c6459a/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
index f552460..7a95306 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
@@ -38,6 +38,7 @@ public interface CliAroundInterceptor {
 
   /**
    * called by the OperationInvoker after the command is executed
+   * 
    * @param tempFile if the command's isFileDownloadOverHttp is true, the is the File downloaded
    *        after the http response is processed.
    */


[19/49] geode git commit: GEODE-2595: Change LuceneService.createIndex to use a factory

Posted by kh...@apache.org.
GEODE-2595: Change LuceneService.createIndex to use a factory

Changing LuceneService.createIndex to createIndexFactory and
using a factory pattern to create the index.

This allows us to introduce new options to the index create without
breaking backwards compatibility in the future.


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/4f307c71
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/4f307c71
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/4f307c71

Branch: refs/heads/feature/GEODE-2420
Commit: 4f307c7175cbd7c41e5a696768aa638fd9048bdf
Parents: 72e152f
Author: Dan Smith <up...@apache.org>
Authored: Wed Mar 8 17:15:44 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../geode/cache/lucene/LuceneIndexFactory.java  | 86 ++++++++++++++++++++
 .../geode/cache/lucene/LuceneService.java       | 57 +++++--------
 .../lucene/internal/LuceneIndexFactory.java     | 25 ------
 .../lucene/internal/LuceneIndexFactoryImpl.java | 66 +++++++++++++++
 .../lucene/internal/LuceneIndexImplFactory.java | 25 ++++++
 .../lucene/internal/LuceneRawIndexFactory.java  |  2 +-
 .../lucene/internal/LuceneServiceImpl.java      | 18 ++--
 .../functions/LuceneCreateIndexFunction.java    | 16 ++--
 .../lucene/LuceneIndexCreationDUnitTest.java    | 32 ++++----
 .../LuceneIndexCreationIntegrationTest.java     | 19 ++---
 ...IndexCreationPersistenceIntegrationTest.java |  9 +-
 .../lucene/LuceneIndexDestroyDUnitTest.java     |  6 +-
 .../LuceneIndexMaintenanceIntegrationTest.java  | 21 +++--
 .../cache/lucene/LuceneQueriesDUnitTest.java    |  8 +-
 .../lucene/LuceneQueriesIntegrationTest.java    | 25 +++---
 ...LuceneQueriesPersistenceIntegrationTest.java |  2 +-
 .../geode/cache/lucene/PaginationDUnitTest.java |  6 +-
 .../geode/cache/lucene/RebalanceDUnitTest.java  | 21 +----
 .../RebalanceWithRedundancyDUnitTest.java       |  9 +-
 .../lucene/internal/LuceneIndexFactorySpy.java  |  4 +-
 .../LuceneIndexRecoveryHAIntegrationTest.java   |  3 +-
 .../internal/LuceneServiceImplJUnitTest.java    |  2 +-
 .../cli/LuceneIndexCommandsDUnitTest.java       | 15 +---
 .../LuceneCreateIndexFunctionJUnitTest.java     | 23 +++---
 .../DumpDirectoryFilesIntegrationTest.java      |  3 +-
 .../management/LuceneManagementDUnitTest.java   |  3 +-
 .../IndexRepositoryImplPerformanceTest.java     |  2 +-
 ...neIndexXmlGeneratorIntegrationJUnitTest.java |  2 +-
 .../cache/lucene/test/LuceneTestUtilities.java  | 10 +--
 29 files changed, 313 insertions(+), 207 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneIndexFactory.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneIndexFactory.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneIndexFactory.java
new file mode 100644
index 0000000..2472ec0
--- /dev/null
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneIndexFactory.java
@@ -0,0 +1,86 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.lucene;
+
+import org.apache.geode.annotations.Experimental;
+import org.apache.lucene.analysis.Analyzer;
+
+import java.util.Map;
+
+/**
+ * A factory for creating a lucene index on the current member. Obtain a factory from
+ * {@link LuceneService#createIndexFactory()}.
+ *
+ * Configure the index using the add methods, and then call {@link #create(String, String)} to
+ * create the index.
+ */
+@Experimental
+public interface LuceneIndexFactory {
+
+  /**
+   * Add a field to be indexed
+   * 
+   * @param name A field of the object to index. Only fields listed here will be stored in the
+   *        index. Fields should map to PDX fieldNames if the object is serialized with PDX, or to
+   *        java fields on the object otherwise. The special field name
+   *        {@link LuceneService#REGION_VALUE_FIELD} indicates that the entire value should be
+   *        stored as a single field in the index.
+   */
+  LuceneIndexFactory addField(String name);
+
+  /**
+   * Set the list of fields to be indexed.
+   * 
+   * @param fields Fields of the object to index. Only fields listed here will be stored in the
+   *        index. Fields should map to PDX fieldNames if the object is serialized with PDX, or to
+   *        java fields on the object otherwise. The special field name
+   *        {@link LuceneService#REGION_VALUE_FIELD} indicates that the entire value should be
+   *        stored as a single field in the index.
+   */
+  LuceneIndexFactory setFields(String... fields);
+
+  /**
+   * Add a field to be indexed, using the specified analyzer.
+   *
+   * @param name A field of the object to index. Only fields listed here will be stored in the
+   *        index. Fields should map to PDX fieldNames if the object is serialized with PDX, or to
+   *        java fields on the object otherwise. The special field name
+   *        {@link LuceneService#REGION_VALUE_FIELD} indicates that the entire value should be
+   *        stored as a single field in the index.
+   * @param analyzer The analyzer to use for this this field. Analyzers are used by Lucene to
+   *        tokenize your field into individual words.
+   */
+  LuceneIndexFactory addField(String name, Analyzer analyzer);
+
+  /**
+   * Set the list of fields to be indexed.
+   * 
+   * @param fieldMap Fields of the object to index, with the analyzer to be used for each field.
+   *        Only fields listed here will be stored in the index. Fields should map to PDX fieldNames
+   *        if the object is serialized with PDX, or to java fields on the object otherwise. The
+   *        special field name {@link LuceneService#REGION_VALUE_FIELD} indicates that the entire
+   *        value should be stored as a single field in the index.
+   */
+  LuceneIndexFactory setFields(Map<String, Analyzer> fieldMap);
+
+  /**
+   * Create the index on this member.
+   * 
+   * @param indexName name of the index.
+   * @param regionPath The region to index. The entries added to this region will be indexes.
+   */
+  void create(String indexName, String regionPath);
+
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java
index f035442..24ca902 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java
@@ -14,17 +14,17 @@
  */
 package org.apache.geode.cache.lucene;
 
-import java.util.Collection;
-import java.util.Map;
-import java.util.concurrent.TimeUnit;
-
+import org.apache.geode.annotations.Experimental;
 import org.apache.geode.cache.Cache;
 import org.apache.geode.cache.DataPolicy;
+import org.apache.geode.cache.GemFireCache;
+import org.apache.geode.cache.lucene.LuceneIndexFactory;
 import org.apache.lucene.analysis.Analyzer;
+import org.apache.lucene.analysis.standard.StandardAnalyzer;
 
-import org.apache.geode.annotations.Experimental;
-import org.apache.geode.cache.GemFireCache;
-import org.apache.geode.cache.lucene.internal.LuceneIndexCreationProfile;
+import java.util.Collection;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
 
 /**
  *
@@ -37,27 +37,26 @@ import org.apache.geode.cache.lucene.internal.LuceneIndexCreationProfile;
  * </p>
  * <p>
  * Lucene indexes can be created using gfsh, xml, or the java API. Below is an example of creating a
- * Lucene index with the java API. The Lucene index created on each member that will host data for
- * the region.
+ * Lucene index with the java API. The Lucene index should be created on each member that has the
+ * region that is being indexed.
  * </p>
  * 
  * <pre>
  * {
  *   &#64;code
- *   LuceneIndex index =
- *       luceneService.createIndex(indexName, regionName, "field1", "field2", "field3");
+*       luceneService.createCreateIndexFactory()
+ *        .addField("name")
+ *        .addField("zipcode")
+ *        .addField("email", new KeywordAnalyzer())
+ *        .create(indexName, regionName);
  * }
  * </pre>
  * <p>
- * You can also specify what {@link Analyzer} to use for each field.
+ * You can also specify what {@link Analyzer} to use for each field. In the example above, email is
+ * being tokenized with the KeywordAnalyzer so it is treated as a single word. The default analyzer
+ * if non is specified is the {@link StandardAnalyzer}
  * </p>
  * 
- * <pre>
- * {
- *   &#64;code
- *   LuceneIndex index = luceneService.createIndex(indexName, regionName, analyzerPerField);
- * }
- * </pre>
  *
  * Indexes should be created on all peers that host the region being indexed. Clients do not need to
  * define the index, they can directly execute queries using this service.
@@ -107,27 +106,9 @@ public interface LuceneService {
   public String REGION_VALUE_FIELD = "__REGION_VALUE_FIELD";
 
   /**
-   * Create a Lucene index using default analyzer.
-   * 
-   * @param fields The fields of the object to index. Only fields listed here will be stored in the
-   *        index. Fields should map to PDX fieldNames if the object is serialized with PDX, or to
-   *        java fields on the object otherwise. The special field name {@link #REGION_VALUE_FIELD}
-   *        indicates that the entire value should be stored as a single field in the index.
-   */
-  public void createIndex(String indexName, String regionPath, String... fields);
-
-  /**
-   * Create a Lucene index using specified {@link Analyzer} per field. Analyzers are used by Lucene
-   * to tokenize your field into individual words.
-   * 
-   * @param indexName index name
-   * @param regionPath region name
-   * @param analyzerPerField A map of fields to analyzers. See
-   *        {@link #createIndex(String, String, String...)} for details on valid values for fields.
-   *        Each field will be tokenized using the provided Analyzer.
+   * Get a factory for creating a lucene index on this member.
    */
-  public void createIndex(String indexName, String regionPath,
-      Map<String, Analyzer> analyzerPerField);
+  public LuceneIndexFactory createIndexFactory();
 
   /**
    * Destroy the Lucene index

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactory.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactory.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactory.java
deleted file mode 100755
index fe1fc52..0000000
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactory.java
+++ /dev/null
@@ -1,25 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- * 
- * http://www.apache.org/licenses/LICENSE-2.0
- * 
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-package org.apache.geode.cache.lucene.internal;
-
-import org.apache.geode.internal.cache.GemFireCacheImpl;
-
-public class LuceneIndexFactory {
-  public LuceneIndexFactory() {}
-
-  public LuceneIndexImpl create(String indexName, String regionPath, GemFireCacheImpl cache) {
-    return new LuceneIndexForPartitionedRegion(indexName, regionPath, cache);
-  }
-}

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactoryImpl.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactoryImpl.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactoryImpl.java
new file mode 100644
index 0000000..23f2206
--- /dev/null
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactoryImpl.java
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.lucene.internal;
+
+import org.apache.geode.cache.lucene.LuceneIndexFactory;
+import org.apache.lucene.analysis.Analyzer;
+import org.apache.lucene.analysis.standard.StandardAnalyzer;
+
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.Map;
+
+public class LuceneIndexFactoryImpl implements org.apache.geode.cache.lucene.LuceneIndexFactory {
+  private final LuceneServiceImpl service;
+  private final Map<String, Analyzer> fields = new LinkedHashMap<String, Analyzer>();
+
+
+  public LuceneIndexFactoryImpl(final LuceneServiceImpl luceneService) {
+    this.service = luceneService;
+  }
+
+  @Override
+  public LuceneIndexFactory addField(final String name) {
+    return addField(name, new StandardAnalyzer());
+  }
+
+  @Override
+  public LuceneIndexFactory setFields(final String... fields) {
+    this.fields.clear();
+    for (String field : fields) {
+      addField(field);
+    }
+    return this;
+  }
+
+  @Override
+  public LuceneIndexFactory addField(final String name, final Analyzer analyzer) {
+    fields.put(name, analyzer);
+    return this;
+  }
+
+  @Override
+  public LuceneIndexFactory setFields(final Map<String, Analyzer> fieldMap) {
+    this.fields.clear();
+    this.fields.putAll(fieldMap);
+    return this;
+  }
+
+  @Override
+  public void create(final String indexName, final String regionPath) {
+    service.createIndex(indexName, regionPath, fields);
+
+  }
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImplFactory.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImplFactory.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImplFactory.java
new file mode 100755
index 0000000..5da2732
--- /dev/null
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImplFactory.java
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ * 
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * 
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.lucene.internal;
+
+import org.apache.geode.internal.cache.GemFireCacheImpl;
+
+public class LuceneIndexImplFactory {
+  public LuceneIndexImplFactory() {}
+
+  public LuceneIndexImpl create(String indexName, String regionPath, GemFireCacheImpl cache) {
+    return new LuceneIndexForPartitionedRegion(indexName, regionPath, cache);
+  }
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneRawIndexFactory.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneRawIndexFactory.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneRawIndexFactory.java
index 825786c..cf445c0 100755
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneRawIndexFactory.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneRawIndexFactory.java
@@ -16,7 +16,7 @@ package org.apache.geode.cache.lucene.internal;
 
 import org.apache.geode.internal.cache.GemFireCacheImpl;
 
-public class LuceneRawIndexFactory extends LuceneIndexFactory {
+public class LuceneRawIndexFactory extends LuceneIndexImplFactory {
   public LuceneIndexImpl create(String indexName, String regionPath, GemFireCacheImpl cache) {
     return new LuceneRawIndex(indexName, regionPath, cache);
   }

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java
index a212094..41d2991 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java
@@ -69,7 +69,7 @@ import org.apache.geode.internal.logging.LogService;
  * @since GemFire 8.5
  */
 public class LuceneServiceImpl implements InternalLuceneService {
-  public static LuceneIndexFactory luceneIndexFactory = new LuceneIndexFactory();
+  public static LuceneIndexImplFactory luceneIndexFactory = new LuceneIndexImplFactory();
   private static final Logger logger = LogService.getLogger();
 
   private GemFireCacheImpl cache;
@@ -82,6 +82,11 @@ public class LuceneServiceImpl implements InternalLuceneService {
   }
 
   @Override
+  public org.apache.geode.cache.lucene.LuceneIndexFactory createIndexFactory() {
+    return new LuceneIndexFactoryImpl(this);
+  }
+
+  @Override
   public Cache getCache() {
     return this.cache;
   }
@@ -127,17 +132,6 @@ public class LuceneServiceImpl implements InternalLuceneService {
     return getUniqueIndexName(indexName, regionPath) + regionSuffix;
   }
 
-  @Override
-  public void createIndex(String indexName, String regionPath, String... fields) {
-    if (fields == null || fields.length == 0) {
-      throw new IllegalArgumentException("At least one field must be indexed");
-    }
-    StandardAnalyzer analyzer = new StandardAnalyzer();
-
-    createIndex(indexName, regionPath, analyzer, null, fields);
-  }
-
-  @Override
   public void createIndex(String indexName, String regionPath,
       Map<String, Analyzer> fieldAnalyzers) {
     if (fieldAnalyzers == null || fieldAnalyzers.isEmpty()) {

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunction.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunction.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunction.java
index 075bbd7..f1b9c82 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunction.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunction.java
@@ -15,15 +15,11 @@
 
 package org.apache.geode.cache.lucene.internal.cli.functions;
 
-import java.util.HashMap;
-import java.util.HashSet;
-import java.util.Map;
-import java.util.Set;
-
 import org.apache.geode.cache.Cache;
 import org.apache.geode.cache.CacheFactory;
 import org.apache.geode.cache.execute.FunctionAdapter;
 import org.apache.geode.cache.execute.FunctionContext;
+import org.apache.geode.cache.lucene.LuceneIndexFactory;
 import org.apache.geode.cache.lucene.LuceneService;
 import org.apache.geode.cache.lucene.LuceneServiceProvider;
 import org.apache.geode.cache.lucene.internal.cli.LuceneCliStrings;
@@ -73,18 +69,20 @@ public class LuceneCreateIndexFunction extends FunctionAdapter implements Intern
       String[] fields = indexInfo.getSearchableFieldNames();
       String[] analyzerName = indexInfo.getFieldAnalyzers();
 
+      final LuceneIndexFactory indexFactory = service.createIndexFactory();
       if (analyzerName == null || analyzerName.length == 0) {
-        service.createIndex(indexInfo.getIndexName(), indexInfo.getRegionPath(), fields);
+        for (String field : fields) {
+          indexFactory.addField(field);
+        }
       } else {
         if (analyzerName.length != fields.length)
           throw new Exception("Mismatch in lengths of fields and analyzers");
-        Map<String, Analyzer> fieldAnalyzer = new HashMap<>();
         for (int i = 0; i < fields.length; i++) {
           Analyzer analyzer = toAnalyzer(analyzerName[i]);
-          fieldAnalyzer.put(fields[i], analyzer);
+          indexFactory.addField(fields[i], analyzer);
         }
-        service.createIndex(indexInfo.getIndexName(), indexInfo.getRegionPath(), fieldAnalyzer);
       }
+      indexFactory.create(indexInfo.getIndexName(), indexInfo.getRegionPath());
 
       // TODO - update cluster configuration by returning a valid XmlEntity
       XmlEntity xmlEntity = null;

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationDUnitTest.java
index 60f8b65..a16646e 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationDUnitTest.java
@@ -186,13 +186,13 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
   public void verifyDifferentIndexNamesFails(RegionTestableType regionType) {
     SerializableRunnableIF createIndex1 = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME + "1", REGION_NAME, "field1");
+      luceneService.createIndexFactory().addField("field1").create(INDEX_NAME + "1", REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex1, regionType));
 
     SerializableRunnableIF createIndex2 = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME + "2", REGION_NAME, "field1");
+      luceneService.createIndexFactory().addField("field1").create(INDEX_NAME + "2", REGION_NAME);
     };
     dataStore2.invoke(
         () -> initDataStore(createIndex2, regionType, CANNOT_CREATE_LUCENE_INDEX_DIFFERENT_NAMES));
@@ -219,8 +219,8 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
 
     SerializableRunnableIF createIndex2 = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "field1");
-      luceneService.createIndex(INDEX_NAME + "2", REGION_NAME, "field2");
+      luceneService.createIndexFactory().addField("field1").create(INDEX_NAME, REGION_NAME);
+      luceneService.createIndexFactory().addField("field2").create(INDEX_NAME + "2", REGION_NAME);
     };
     dataStore2.invoke(() -> initDataStore(createIndex2, regionType,
         CANNOT_CREATE_LUCENE_INDEX_DIFFERENT_INDEXES_2));
@@ -362,22 +362,23 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
   protected SerializableRunnableIF getFieldsIndexWithOneField() {
     return () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "field1");
+      luceneService.createIndexFactory().addField("field1").create(INDEX_NAME, REGION_NAME);
     };
   }
 
   protected SerializableRunnableIF getFieldsIndexWithTwoFields() {
     return () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "field1", "field2");
+      luceneService.createIndexFactory().addField("field1").addField("field2").create(INDEX_NAME,
+          REGION_NAME);
     };
   }
 
   protected SerializableRunnableIF get2FieldsIndexes() {
     return () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME + "_1", REGION_NAME, "field1");
-      luceneService.createIndex(INDEX_NAME + "_2", REGION_NAME, "field2");
+      luceneService.createIndexFactory().addField("field1").create(INDEX_NAME + "_1", REGION_NAME);
+      luceneService.createIndexFactory().addField("field2").create(INDEX_NAME + "_2", REGION_NAME);
     };
   }
 
@@ -385,7 +386,8 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
     return () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
       for (int count = 1; count <= numberOfIndexes; count++) {
-        luceneService.createIndex(INDEX_NAME + "_" + count, REGION_NAME, "field" + count);
+        luceneService.createIndexFactory().addField("field" + count)
+            .create(INDEX_NAME + "_" + count, REGION_NAME);
       }
     };
   }
@@ -410,7 +412,7 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
       Map<String, Analyzer> analyzers = new HashMap<>();
       analyzers.put("field1", null);
       analyzers.put("field2", new KeywordAnalyzer());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, analyzers);
+      luceneService.createIndexFactory().setFields(analyzers).create(INDEX_NAME, REGION_NAME);
     };
   }
 
@@ -420,7 +422,7 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
       Map<String, Analyzer> analyzers = new HashMap<>();
       analyzers.put("field1", new KeywordAnalyzer());
       analyzers.put("field2", null);
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, analyzers);
+      luceneService.createIndexFactory().setFields(analyzers).create(INDEX_NAME, REGION_NAME);
     };
   }
 
@@ -430,7 +432,7 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
       Map<String, Analyzer> analyzers = new HashMap<>();
       analyzers.put("field1", analyzerClass.newInstance());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, analyzers);
+      luceneService.createIndexFactory().setFields(analyzers).create(INDEX_NAME, REGION_NAME);
     };
   }
 
@@ -439,7 +441,7 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
       Map<String, Analyzer> analyzers = new HashMap<>();
       analyzers.put("field1", new KeywordAnalyzer());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, analyzers);
+      luceneService.createIndexFactory().setFields(analyzers).create(INDEX_NAME, REGION_NAME);
     };
   }
 
@@ -449,7 +451,7 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
       Map<String, Analyzer> analyzers = new HashMap<>();
       analyzers.put("field1", new KeywordAnalyzer());
       analyzers.put("field2", new KeywordAnalyzer());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, analyzers);
+      luceneService.createIndexFactory().setFields(analyzers).create(INDEX_NAME, REGION_NAME);
     };
   }
 
@@ -459,7 +461,7 @@ public class LuceneIndexCreationDUnitTest extends LuceneDUnitTest {
       Map<String, Analyzer> analyzers = new HashMap<>();
       analyzers.put("field1", new StandardAnalyzer());
       analyzers.put("field2", new KeywordAnalyzer());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, analyzers);
+      luceneService.createIndexFactory().setFields(analyzers).create(INDEX_NAME, REGION_NAME);
     };
   }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java
index b696676..80e77e9 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java
@@ -15,9 +15,7 @@
 
 package org.apache.geode.cache.lucene;
 
-import static org.apache.geode.cache.RegionShortcut.*;
 import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.*;
-import static junitparams.JUnitParamsRunner.*;
 import static org.junit.Assert.*;
 
 import java.io.IOException;
@@ -28,9 +26,7 @@ import java.util.Collections;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
-import java.util.concurrent.TimeUnit;
 import java.util.function.Consumer;
-import java.util.stream.Collector;
 import java.util.stream.Collectors;
 
 import org.apache.geode.cache.EvictionAction;
@@ -42,7 +38,7 @@ import org.apache.geode.cache.Region;
 import org.apache.geode.cache.RegionFactory;
 import org.apache.geode.cache.RegionShortcut;
 import org.apache.geode.cache.lucene.internal.LuceneIndexCreationProfile;
-import org.apache.geode.cache.lucene.internal.LuceneIndexFactory;
+import org.apache.geode.cache.lucene.internal.LuceneIndexImplFactory;
 import org.apache.geode.cache.lucene.internal.LuceneRawIndex;
 import org.apache.geode.cache.lucene.internal.LuceneRawIndexFactory;
 import org.apache.geode.cache.lucene.internal.LuceneServiceImpl;
@@ -52,7 +48,6 @@ import org.apache.geode.internal.cache.BucketNotFoundException;
 import org.apache.geode.internal.cache.LocalRegion;
 import org.apache.geode.internal.cache.PartitionedRegion;
 import org.apache.geode.test.junit.categories.IntegrationTest;
-import org.awaitility.Awaitility;
 
 import org.apache.lucene.analysis.Analyzer;
 import org.apache.lucene.analysis.core.KeywordTokenizer;
@@ -90,7 +85,7 @@ public class LuceneIndexCreationIntegrationTest extends LuceneIntegrationTest {
     final RecordingAnalyzer field2Analyzer = new RecordingAnalyzer();
     analyzers.put("field1", field1Analyzer);
     analyzers.put("field2", field2Analyzer);
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, analyzers);
+    luceneService.createIndexFactory().setFields(analyzers).create(INDEX_NAME, REGION_NAME);
     Region region = createRegion();
     final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
     region.put("key1", new TestObject());
@@ -166,7 +161,7 @@ public class LuceneIndexCreationIntegrationTest extends LuceneIntegrationTest {
     analyzers.put("field2", field2Analyzer);
     LuceneServiceImpl.luceneIndexFactory = new LuceneRawIndexFactory();
     try {
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, analyzers);
+      luceneService.createIndexFactory().setFields(analyzers).create(INDEX_NAME, REGION_NAME);
       Region region = createRegion();
       final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
       assertTrue(index instanceof LuceneRawIndex);
@@ -176,7 +171,7 @@ public class LuceneIndexCreationIntegrationTest extends LuceneIntegrationTest {
       assertEquals(Arrays.asList("field1"), field1Analyzer.analyzedfields);
       assertEquals(Arrays.asList("field2"), field2Analyzer.analyzedfields);
     } finally {
-      LuceneServiceImpl.luceneIndexFactory = new LuceneIndexFactory();
+      LuceneServiceImpl.luceneIndexFactory = new LuceneIndexImplFactory();
     }
   }
 
@@ -225,8 +220,10 @@ public class LuceneIndexCreationIntegrationTest extends LuceneIntegrationTest {
   @Test
   public void shouldReturnAllDefinedIndexes() {
     LuceneServiceImpl luceneServiceImpl = (LuceneServiceImpl) luceneService;
-    luceneServiceImpl.createIndex(INDEX_NAME, REGION_NAME, "field1", "field2", "field3");
-    luceneServiceImpl.createIndex("index2", "region2", "field4", "field5", "field6");
+    luceneServiceImpl.createIndexFactory().setFields("field1", "field2", "field3")
+        .create(INDEX_NAME, REGION_NAME);
+    luceneServiceImpl.createIndexFactory().setFields("field4", "field5", "field6").create("index2",
+        "region2");
     final Collection<LuceneIndexCreationProfile> indexList =
         luceneServiceImpl.getAllDefinedIndexes();
 

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationPersistenceIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationPersistenceIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationPersistenceIntegrationTest.java
index 4b27ee6..8f65574 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationPersistenceIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationPersistenceIntegrationTest.java
@@ -164,8 +164,10 @@ public class LuceneIndexCreationPersistenceIntegrationTest extends LuceneIntegra
   @Test
   @Parameters(method = "getRegionShortcuts")
   public void shouldHandleMultipleIndexes(RegionShortcut shortcut) throws Exception {
-    LuceneServiceProvider.get(this.cache).createIndex(INDEX_NAME + "_1", REGION_NAME, "field1");
-    LuceneServiceProvider.get(this.cache).createIndex(INDEX_NAME + "_2", REGION_NAME, "field2");
+    LuceneServiceProvider.get(this.cache).createIndexFactory().setFields("field1")
+        .create(INDEX_NAME + "_1", REGION_NAME);
+    LuceneServiceProvider.get(this.cache).createIndexFactory().setFields("field2")
+        .create(INDEX_NAME + "_2", REGION_NAME);
     Region region = cache.createRegionFactory(shortcut).create(REGION_NAME);
     region.put("key1", new TestObject());
     verifyQueryResultSize(INDEX_NAME + "_1", REGION_NAME, "field1:world", DEFAULT_FIELD, 1);
@@ -175,7 +177,8 @@ public class LuceneIndexCreationPersistenceIntegrationTest extends LuceneIntegra
   @Test
   @Parameters(method = "getRegionShortcuts")
   public void shouldCreateInternalRegionsForIndex(RegionShortcut shortcut) {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "field1", "field2");
+    luceneService.createIndexFactory().setFields("field1", "field2").create(INDEX_NAME,
+        REGION_NAME);
 
     // Create partitioned region
     createRegion(REGION_NAME, shortcut);

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
index 496cdeb..cf79248 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
@@ -331,15 +331,15 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
   private SerializableRunnableIF createIndex(String indexName, String field) {
     return () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(indexName, REGION_NAME, field);
+      luceneService.createIndexFactory().setFields(field).create(INDEX_NAME, REGION_NAME);
     };
   }
 
   private SerializableRunnableIF createIndexes() {
     return () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME + "0", REGION_NAME, "text");
-      luceneService.createIndex(INDEX_NAME + "1", REGION_NAME, "text");
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME + "0", REGION_NAME);
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME + "1", REGION_NAME);
     };
   }
 

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java
index 5ac01b8..00abb99 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java
@@ -43,7 +43,8 @@ public class LuceneIndexMaintenanceIntegrationTest extends LuceneIntegrationTest
 
   @Test
   public void indexIsNotUpdatedIfTransactionHasNotCommittedYet() throws Exception {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "title", "description");
+    luceneService.createIndexFactory().setFields("title", "description").create(INDEX_NAME,
+        REGION_NAME);
 
     Region region = createRegion(REGION_NAME, RegionShortcut.PARTITION);
     region.put("object-1", new TestObject("title 1", "hello world"));
@@ -69,7 +70,8 @@ public class LuceneIndexMaintenanceIntegrationTest extends LuceneIntegrationTest
 
   @Test
   public void indexIsUpdatedAfterTransactionHasCommitted() throws Exception {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "title", "description");
+    luceneService.createIndexFactory().setFields("title", "description").create(INDEX_NAME,
+        REGION_NAME);
 
     Region region = createRegion(REGION_NAME, RegionShortcut.PARTITION);
     region.put("object-1", new TestObject("title 1", "hello world"));
@@ -96,7 +98,8 @@ public class LuceneIndexMaintenanceIntegrationTest extends LuceneIntegrationTest
 
   @Test
   public void indexIsNotUpdatedAfterTransactionRollback() throws Exception {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "title", "description");
+    luceneService.createIndexFactory().setFields("title", "description").create(INDEX_NAME,
+        REGION_NAME);
 
     Region region = createRegion(REGION_NAME, RegionShortcut.PARTITION);
     region.put("object-1", new TestObject("title 1", "hello world"));
@@ -123,7 +126,8 @@ public class LuceneIndexMaintenanceIntegrationTest extends LuceneIntegrationTest
 
   @Test
   public void statsAreUpdatedAfterACommit() throws Exception {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "title", "description");
+    luceneService.createIndexFactory().setFields("title", "description").create(INDEX_NAME,
+        REGION_NAME);
 
     Region region = createRegion(REGION_NAME, RegionShortcut.PARTITION);
     region.put("object-1", new TestObject("title 1", "hello world"));
@@ -146,7 +150,8 @@ public class LuceneIndexMaintenanceIntegrationTest extends LuceneIntegrationTest
 
   @Test
   public void indexShouldBeUpdatedWithRegionExpirationDestroyOperation() throws Exception {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "title", "description");
+    luceneService.createIndexFactory().setFields("title", "description").create(INDEX_NAME,
+        REGION_NAME);
 
     // Configure PR with expiration operation set to destroy
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION)
@@ -173,7 +178,8 @@ public class LuceneIndexMaintenanceIntegrationTest extends LuceneIntegrationTest
 
   @Test
   public void nullValuesShouldNotCauseAnException() throws Exception {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "title", "description");
+    luceneService.createIndexFactory().setFields("title", "description").create(INDEX_NAME,
+        REGION_NAME);
 
     // Configure PR with expiration operation set to destroy
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
@@ -192,7 +198,8 @@ public class LuceneIndexMaintenanceIntegrationTest extends LuceneIntegrationTest
 
   @Test
   public void entriesFlushedToIndexAfterWaitForFlushCalled() throws InterruptedException {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "title", "description");
+    luceneService.createIndexFactory().setFields("title", "description").create(INDEX_NAME,
+        REGION_NAME);
 
     Region region = createRegion(REGION_NAME, RegionShortcut.PARTITION);
     LuceneTestUtilities.pauseSender(cache);

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesDUnitTest.java
index 5db2ed4..d259205 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesDUnitTest.java
@@ -50,7 +50,7 @@ public class LuceneQueriesDUnitTest extends LuceneQueriesAccessorBase {
       RegionTestableType regionTestType) {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().addField("text").create(INDEX_NAME, REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
     dataStore2.invoke(() -> initDataStore(createIndex, regionTestType));
@@ -68,7 +68,7 @@ public class LuceneQueriesDUnitTest extends LuceneQueriesAccessorBase {
       RegionTestableType regionTestType) {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().addField("text").create(INDEX_NAME, REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
     dataStore2.invoke(() -> initDataStore(createIndex, regionTestType));
@@ -85,7 +85,7 @@ public class LuceneQueriesDUnitTest extends LuceneQueriesAccessorBase {
   public void canQueryWithCustomLuceneQueryObject(RegionTestableType regionTestType) {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().addField("text").create(INDEX_NAME, REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
     dataStore2.invoke(() -> initDataStore(createIndex, regionTestType));
@@ -113,7 +113,7 @@ public class LuceneQueriesDUnitTest extends LuceneQueriesAccessorBase {
       throws InterruptedException {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().addField("text").create(INDEX_NAME, REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
     dataStore2.invoke(() -> initDataStore(createIndex, regionTestType));

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java
index 6420307..5b85c19 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java
@@ -44,7 +44,6 @@ import org.apache.lucene.search.Query;
 import org.junit.Ignore;
 import org.junit.Rule;
 import org.junit.Test;
-import org.junit.contrib.java.lang.system.SystemOutRule;
 import org.junit.experimental.categories.Category;
 import org.junit.rules.ExpectedException;
 
@@ -54,10 +53,8 @@ import org.apache.geode.cache.RegionShortcut;
 import org.apache.geode.cache.lucene.test.TestObject;
 import org.apache.geode.internal.DataSerializableFixedID;
 import org.apache.geode.internal.Version;
-import org.apache.geode.internal.cache.PartitionAttributesImpl;
 import org.apache.geode.pdx.JSONFormatter;
 import org.apache.geode.pdx.PdxInstance;
-import org.apache.geode.pdx.internal.AutoSerializableManager.ObjectArrayField;
 import org.apache.geode.test.junit.categories.IntegrationTest;
 
 /**
@@ -76,7 +73,7 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
     Map<String, Analyzer> fields = new HashMap<String, Analyzer>();
     fields.put("field1", new StandardAnalyzer());
     fields.put("field2", new KeywordAnalyzer());
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, fields);
+    luceneService.createIndexFactory().setFields(fields).create(INDEX_NAME, REGION_NAME);
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
     final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
 
@@ -134,7 +131,8 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
   public void shouldQueryUsingIntRangeQueryProvider() throws Exception {
     // Note: range query on numeric field has some limitations. But IntRangeQueryProvider
     // provided basic functionality
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, LuceneService.REGION_VALUE_FIELD);
+    luceneService.createIndexFactory().setFields(LuceneService.REGION_VALUE_FIELD)
+        .create(INDEX_NAME, REGION_NAME);
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
     final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
 
@@ -153,7 +151,8 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
   public void queryParserCannotQueryByRange() throws Exception {
     // Note: range query on numeric field has some limitations. But IntRangeQueryProvider
     // provided basic functionality
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, LuceneService.REGION_VALUE_FIELD);
+    luceneService.createIndexFactory().setFields(LuceneService.REGION_VALUE_FIELD)
+        .create(INDEX_NAME, REGION_NAME);
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
     final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
 
@@ -208,7 +207,8 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
 
   private LuceneQuery<Object, Object> addValuesAndCreateQuery(int pagesize)
       throws InterruptedException {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "field1", "field2");
+    luceneService.createIndexFactory().setFields("field1", "field2").create(INDEX_NAME,
+        REGION_NAME);
     region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
     final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
 
@@ -236,7 +236,7 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
     // Note: fields has to contain "field1", otherwise, field1 will not be tokenized
     fields.put("field1", null);
     fields.put("field2", new MyCharacterAnalyzer());
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, fields);
+    luceneService.createIndexFactory().setFields(fields).create(INDEX_NAME, REGION_NAME);
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
     final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
 
@@ -261,7 +261,7 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
     Map<String, Analyzer> fields = new HashMap<String, Analyzer>();
     fields.put("field1", null);
     fields.put("field2", null);
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, fields);
+    luceneService.createIndexFactory().setFields(fields).create(INDEX_NAME, REGION_NAME);
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
     final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
 
@@ -279,7 +279,7 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
     fields.put("name", null);
     fields.put("lastName", null);
     fields.put("address", null);
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, fields);
+    luceneService.createIndexFactory().setFields(fields).create(INDEX_NAME, REGION_NAME);
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
     final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
 
@@ -297,7 +297,8 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
 
   @Test()
   public void shouldAllowQueryOnRegionWithStringValue() throws Exception {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, LuceneService.REGION_VALUE_FIELD);
+    luceneService.createIndexFactory().setFields(LuceneService.REGION_VALUE_FIELD)
+        .create(INDEX_NAME, REGION_NAME);
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
     final LuceneIndex index = luceneService.getIndex(INDEX_NAME, REGION_NAME);
 
@@ -310,7 +311,7 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
   @Test()
   public void throwFunctionExceptionWhenGivenBadQuery() throws Exception {
     LuceneService luceneService = LuceneServiceProvider.get(cache);
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+    luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
     Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(REGION_NAME);
 
     // Create a query that throws an exception

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java
index 5fe2df5..f7bb5ab 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java
@@ -63,7 +63,7 @@ public class LuceneQueriesPersistenceIntegrationTest extends LuceneIntegrationTe
     String aeqId = LuceneServiceImpl.getUniqueIndexName(INDEX_NAME, REGION_NAME);
 
     LuceneService service = LuceneServiceProvider.get(cache);
-    service.createIndex(INDEX_NAME, REGION_NAME, Type1.fields);
+    service.createIndexFactory().setFields(Type1.fields).create(INDEX_NAME, REGION_NAME);
 
     RegionFactory<String, Type1> regionFactory =
         cache.createRegionFactory(RegionShortcut.PARTITION);

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java
index 3ff86f7..ad6727e 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java
@@ -64,7 +64,7 @@ public class PaginationDUnitTest extends LuceneQueriesAccessorBase {
       RegionTestableType regionTestType) {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
     };
 
 
@@ -104,7 +104,7 @@ public class PaginationDUnitTest extends LuceneQueriesAccessorBase {
       RegionTestableType regionTestType) {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
     };
 
 
@@ -145,7 +145,7 @@ public class PaginationDUnitTest extends LuceneQueriesAccessorBase {
       RegionTestableType regionTestType) {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
     };
 
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceDUnitTest.java
index 0142b37..ad5595b 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceDUnitTest.java
@@ -15,34 +15,21 @@
 
 package org.apache.geode.cache.lucene;
 
-import static org.apache.geode.cache.lucene.test.IndexRepositorySpy.doOnce;
 import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.*;
 import static org.junit.Assert.*;
 import static org.mockito.Matchers.any;
 
 import java.util.stream.IntStream;
 
-import org.apache.geode.cache.lucene.internal.LuceneIndexFactorySpy;
-import org.apache.geode.distributed.internal.membership.InternalDistributedMember;
-import org.apache.geode.internal.cache.InitialImageOperation;
-import org.apache.geode.internal.cache.PartitionedRegion;
-import org.apache.geode.internal.cache.partitioned.BecomePrimaryBucketMessage;
-import org.apache.geode.internal.cache.partitioned.BecomePrimaryBucketMessage.BecomePrimaryBucketResponse;
-import org.apache.geode.test.dunit.Host;
 import org.apache.geode.test.junit.categories.DistributedTest;
 import org.junit.After;
 import org.junit.Test;
 
 import org.apache.geode.cache.Cache;
 import org.apache.geode.cache.Region;
-import org.apache.geode.cache.control.RebalanceOperation;
-import org.apache.geode.cache.control.RebalanceResults;
-import org.apache.geode.cache.lucene.test.IndexRepositorySpy;
 import org.apache.geode.cache.lucene.test.LuceneTestUtilities;
-import org.apache.geode.cache.partition.PartitionRegionHelper;
 import org.apache.geode.distributed.DistributedMember;
 import org.apache.geode.test.dunit.SerializableRunnableIF;
-import org.apache.geode.test.dunit.VM;
 import org.junit.experimental.categories.Category;
 import org.junit.runner.RunWith;
 
@@ -125,7 +112,7 @@ public class RebalanceDUnitTest extends LuceneQueriesAccessorBase {
       RegionTestableType regionTestType) throws InterruptedException {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
     accessor.invoke(() -> initAccessor(createIndex, regionTestType));
@@ -147,7 +134,7 @@ public class RebalanceDUnitTest extends LuceneQueriesAccessorBase {
       RegionTestableType regionTestType) throws InterruptedException {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
     accessor.invoke(() -> initAccessor(createIndex, regionTestType));
@@ -168,7 +155,7 @@ public class RebalanceDUnitTest extends LuceneQueriesAccessorBase {
   protected void putEntriesAndValidateQueryResults(RegionTestableType regionTestType) {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
     accessor.invoke(() -> initAccessor(createIndex, regionTestType));
@@ -181,7 +168,7 @@ public class RebalanceDUnitTest extends LuceneQueriesAccessorBase {
 
     assertTrue(waitForFlushBeforeExecuteTextSearch(dataStore1, 60000));
 
-    // dataStore3.invoke(() -> initDataStore(createIndex, regionType));
+    // dataStore3.invoke(() -> initDataStore(create, regionType));
     executeTextSearch(accessor, "world", "text", NUM_BUCKETS);
   }
 

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java
index 8ee14cb..7406162 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java
@@ -28,15 +28,10 @@ import org.apache.geode.cache.lucene.test.IndexRegionSpy;
 import org.apache.geode.cache.lucene.test.IndexRepositorySpy;
 import org.apache.geode.cache.lucene.test.LuceneTestUtilities;
 import org.apache.geode.distributed.DistributedMember;
-import org.apache.geode.distributed.internal.membership.InternalDistributedMember;
 import org.apache.geode.internal.cache.InitialImageOperation;
 import org.apache.geode.internal.cache.InitialImageOperation.GIITestHook;
 import org.apache.geode.internal.cache.InitialImageOperation.GIITestHookType;
-import org.apache.geode.internal.cache.PartitionedRegion;
-import org.apache.geode.internal.cache.partitioned.BecomePrimaryBucketMessage;
-import org.apache.geode.internal.cache.partitioned.BecomePrimaryBucketMessage.BecomePrimaryBucketResponse;
 import org.apache.geode.test.dunit.SerializableRunnableIF;
-import org.apache.geode.test.dunit.VM;
 import org.apache.geode.test.junit.categories.DistributedTest;
 
 import org.awaitility.Awaitility;
@@ -120,7 +115,7 @@ public class RebalanceWithRedundancyDUnitTest extends LuceneQueriesAccessorBase
       RegionTestableType regionTestType) throws InterruptedException {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
     accessor.invoke(() -> initAccessor(createIndex, regionTestType));
@@ -157,7 +152,7 @@ public class RebalanceWithRedundancyDUnitTest extends LuceneQueriesAccessorBase
   private void putEntriesAndValidateResultsWithRedundancy(RegionTestableType regionTestType) {
     SerializableRunnableIF createIndex = () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
     };
     dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
     dataStore2.invoke(() -> initDataStore(createIndex, regionTestType));

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactorySpy.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactorySpy.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactorySpy.java
index 5c6d256..b3db8ec 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactorySpy.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexFactorySpy.java
@@ -26,7 +26,7 @@ import org.mockito.stubbing.Answer;
 
 import java.util.function.Consumer;
 
-public class LuceneIndexFactorySpy extends LuceneIndexFactory {
+public class LuceneIndexFactorySpy extends LuceneIndexImplFactory {
 
   public static LuceneIndexFactorySpy injectSpy() {
     LuceneIndexFactorySpy factory = new LuceneIndexFactorySpy();
@@ -35,7 +35,7 @@ public class LuceneIndexFactorySpy extends LuceneIndexFactory {
   }
 
   public static void remove() {
-    LuceneServiceImpl.luceneIndexFactory = new LuceneIndexFactory();
+    LuceneServiceImpl.luceneIndexFactory = new LuceneIndexImplFactory();
   }
 
 

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java
index b50db98..f853baf 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java
@@ -31,7 +31,6 @@ import org.apache.lucene.analysis.standard.StandardAnalyzer;
 import org.junit.After;
 import org.junit.Assert;
 import org.junit.Before;
-import org.junit.Test;
 import org.junit.experimental.categories.Category;
 
 import java.io.IOException;
@@ -77,7 +76,7 @@ public class LuceneIndexRecoveryHAIntegrationTest {
   public void recoverRepoInANewNode()
       throws BucketNotFoundException, IOException, InterruptedException {
     LuceneServiceImpl service = (LuceneServiceImpl) LuceneServiceProvider.get(cache);
-    service.createIndex("index1", "/userRegion", indexedFields);
+    service.createIndexFactory().setFields(indexedFields).create("index1", "/userRegion");
     PartitionAttributes<String, String> attrs =
         new PartitionAttributesFactory().setTotalNumBuckets(1).create();
     RegionFactory<String, String> regionfactory =

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneServiceImplJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneServiceImplJUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneServiceImplJUnitTest.java
index d6ec545..35923e7 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneServiceImplJUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneServiceImplJUnitTest.java
@@ -52,7 +52,7 @@ public class LuceneServiceImplJUnitTest {
   @Test
   public void shouldThrowIllegalArgumentExceptionIfFieldsAreMissing() {
     thrown.expect(IllegalArgumentException.class);
-    service.createIndex("index", "region");
+    service.createIndexFactory().create("index", "region");
   }
 
   @Test

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/LuceneIndexCommandsDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/LuceneIndexCommandsDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/LuceneIndexCommandsDUnitTest.java
index efc11ab..a3e2a90 100755
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/LuceneIndexCommandsDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/LuceneIndexCommandsDUnitTest.java
@@ -15,7 +15,6 @@
 package org.apache.geode.cache.lucene.internal.cli;
 
 import org.apache.geode.cache.*;
-import org.apache.geode.cache.Region.Entry;
 import org.apache.geode.cache.lucene.LuceneIndex;
 import org.apache.geode.cache.lucene.LuceneQuery;
 import org.apache.geode.cache.lucene.LuceneService;
@@ -24,7 +23,6 @@ import org.apache.geode.cache.lucene.internal.LuceneIndexCreationProfile;
 import org.apache.geode.cache.lucene.internal.LuceneIndexImpl;
 import org.apache.geode.cache.lucene.internal.LuceneServiceImpl;
 import org.apache.geode.distributed.ConfigurationProperties;
-import org.apache.geode.internal.lang.StringUtils;
 import org.apache.geode.management.cli.Result.Status;
 import org.apache.geode.management.internal.cli.CommandManager;
 import org.apache.geode.management.internal.cli.commands.CliCommandTestBase;
@@ -39,33 +37,24 @@ import org.awaitility.Awaitility;
 import org.apache.lucene.analysis.Analyzer;
 import org.apache.lucene.analysis.core.KeywordAnalyzer;
 import org.apache.lucene.analysis.standard.StandardAnalyzer;
-import org.apache.lucene.search.Collector;
 import org.junit.Before;
-import org.junit.Rule;
 import org.junit.Test;
 import org.junit.experimental.categories.Category;
-import org.junit.rules.ExpectedException;
 import org.junit.runner.RunWith;
 
 import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.*;
 import static org.apache.geode.test.dunit.Assert.*;
-import static junitparams.JUnitParamsRunner.$;
 
 import java.io.Serializable;
 import java.util.ArrayList;
-import java.util.Arrays;
-import java.util.Collection;
 import java.util.Collections;
-import java.util.Comparator;
 import java.util.HashMap;
 import java.util.List;
 import java.util.Map;
 import java.util.Properties;
 import java.util.concurrent.TimeUnit;
-import java.util.stream.Collectors;
 
 import junitparams.JUnitParamsRunner;
-import junitparams.Parameters;
 
 @Category(DistributedTest.class)
 @RunWith(JUnitParamsRunner.class)
@@ -623,7 +612,7 @@ public class LuceneIndexCommandsDUnitTest extends CliCommandTestBase {
       fieldAnalyzers.put("field1", new StandardAnalyzer());
       fieldAnalyzers.put("field2", new KeywordAnalyzer());
       fieldAnalyzers.put("field3", null);
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, fieldAnalyzers);
+      luceneService.createIndexFactory().setFields(fieldAnalyzers).create(INDEX_NAME, REGION_NAME);
       createRegion();
     });
   }
@@ -635,7 +624,7 @@ public class LuceneIndexCommandsDUnitTest extends CliCommandTestBase {
       fieldAnalyzers.put("field1", new StandardAnalyzer());
       fieldAnalyzers.put("field2", new KeywordAnalyzer());
       fieldAnalyzers.put("field3", null);
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, fieldAnalyzers);
+      luceneService.createIndexFactory().setFields(fieldAnalyzers).create(INDEX_NAME, REGION_NAME);
     });
   }
 

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunctionJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunctionJUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunctionJUnitTest.java
index d974730..5f062d9 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunctionJUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/cli/functions/LuceneCreateIndexFunctionJUnitTest.java
@@ -22,7 +22,7 @@ import java.util.List;
 import java.util.Map;
 import java.util.Set;
 
-import org.apache.lucene.analysis.Analyzer;
+import org.apache.geode.cache.lucene.LuceneIndexFactory;
 import org.apache.lucene.analysis.core.KeywordAnalyzer;
 import org.apache.lucene.analysis.standard.StandardAnalyzer;
 import org.junit.Before;
@@ -50,6 +50,7 @@ public class LuceneCreateIndexFunctionJUnitTest {
   FunctionContext context;
   ResultSender resultSender;
   CliFunctionResult expectedResult;
+  private LuceneIndexFactory factory;
 
   @Before
   public void prepare() {
@@ -58,7 +59,8 @@ public class LuceneCreateIndexFunctionJUnitTest {
     member = ds.getDistributedMember().getId();
     service = mock(InternalLuceneService.class);
     when(cache.getService(InternalLuceneService.class)).thenReturn(service);
-    doNothing().when(service).createIndex(anyString(), anyString(), anyMap());
+    factory = mock(LuceneIndexFactory.class);
+    when(service.createIndexFactory()).thenReturn(factory);
 
     context = mock(FunctionContext.class);
     resultSender = mock(ResultSender.class);
@@ -87,12 +89,11 @@ public class LuceneCreateIndexFunctionJUnitTest {
     function.execute(context);
 
     ArgumentCaptor<Map> analyzersCaptor = ArgumentCaptor.forClass(Map.class);
-    verify(service).createIndex(eq("index1"), eq("/region1"), analyzersCaptor.capture());
-    Map<String, Analyzer> analyzerPerField = analyzersCaptor.getValue();
-    assertEquals(3, analyzerPerField.size());
-    assertTrue(analyzerPerField.get("field1") instanceof StandardAnalyzer);
-    assertTrue(analyzerPerField.get("field2") instanceof KeywordAnalyzer);
-    assertTrue(analyzerPerField.get("field3") instanceof StandardAnalyzer);
+    verify(service).createIndexFactory();
+    verify(factory).addField(eq("field1"), isA(StandardAnalyzer.class));
+    verify(factory).addField(eq("field2"), isA(KeywordAnalyzer.class));
+    verify(factory).addField(eq("field3"), isA(StandardAnalyzer.class));
+    verify(factory).create(eq("index1"), eq("/region1"));
 
     ArgumentCaptor<Set> resultCaptor = ArgumentCaptor.forClass(Set.class);
     verify(resultSender).lastResult(resultCaptor.capture());
@@ -113,8 +114,10 @@ public class LuceneCreateIndexFunctionJUnitTest {
     doReturn(cache).when(function).getCache();
     function.execute(context);
 
-    verify(service).createIndex(eq("index1"), eq("/region1"), eq("field1"), eq("field2"),
-        eq("field3"));
+    verify(factory).addField(eq("field1"));
+    verify(factory).addField(eq("field2"));
+    verify(factory).addField(eq("field3"));
+    verify(factory).create(eq("index1"), eq("/region1"));
 
     ArgumentCaptor<Set> resultCaptor = ArgumentCaptor.forClass(Set.class);
     verify(resultSender).lastResult(resultCaptor.capture());

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/DumpDirectoryFilesIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/DumpDirectoryFilesIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/DumpDirectoryFilesIntegrationTest.java
index eb6c882..93f1aff 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/DumpDirectoryFilesIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/DumpDirectoryFilesIntegrationTest.java
@@ -45,7 +45,8 @@ public class DumpDirectoryFilesIntegrationTest extends LuceneIntegrationTest {
 
   @Test
   public void shouldDumpReadableLuceneIndexFile() throws Exception {
-    luceneService.createIndex(INDEX_NAME, REGION_NAME, "title", "description");
+    luceneService.createIndexFactory().setFields("title", "description").create(INDEX_NAME,
+        REGION_NAME);
 
     Region region = createRegion(REGION_NAME, RegionShortcut.PARTITION);
     region.put(0, new TestObject("title 1", "hello world"));

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/management/LuceneManagementDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/management/LuceneManagementDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/management/LuceneManagementDUnitTest.java
index 36a8e55..79c011d 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/management/LuceneManagementDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/management/LuceneManagementDUnitTest.java
@@ -184,7 +184,8 @@ public class LuceneManagementDUnitTest extends ManagementTestBase {
   private void createIndexes(String regionName, int numIndexes) {
     LuceneService luceneService = LuceneServiceProvider.get(getCache());
     for (int i = 0; i < numIndexes; i++) {
-      luceneService.createIndex(INDEX_NAME + "_" + i, regionName, "field" + i);
+      luceneService.createIndexFactory().setFields("field" + i).create(INDEX_NAME + "_" + i,
+          regionName);
     }
   }
 

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java
index e3e2787..9067376 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java
@@ -160,7 +160,7 @@ public class IndexRepositoryImplPerformanceTest {
       public void init() throws Exception {
         cache = new CacheFactory().set(MCAST_PORT, "0").set(LOG_LEVEL, "warning").create();
         service = LuceneServiceProvider.get(cache);
-        service.createIndex("index", "/region", "text");
+        service.createIndexFactory().addField("test").create("index", "/region");
         region =
             cache.<String, TestObject>createRegionFactory(RegionShortcut.PARTITION)
                 .setPartitionAttributes(

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/xml/LuceneIndexXmlGeneratorIntegrationJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/xml/LuceneIndexXmlGeneratorIntegrationJUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/xml/LuceneIndexXmlGeneratorIntegrationJUnitTest.java
index 8df05b6..361b7a9 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/xml/LuceneIndexXmlGeneratorIntegrationJUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/xml/LuceneIndexXmlGeneratorIntegrationJUnitTest.java
@@ -52,7 +52,7 @@ public class LuceneIndexXmlGeneratorIntegrationJUnitTest {
   public void generateWithFields() {
     cache = new CacheFactory().set(MCAST_PORT, "0").create();
     LuceneService service = LuceneServiceProvider.get(cache);
-    service.createIndex("index", "region", "a", "b", "c");
+    service.createIndexFactory().setFields("a", "b", "c").create("index", "region");
     cache.createRegionFactory(RegionShortcut.PARTITION).create("region");
 
 

http://git-wip-us.apache.org/repos/asf/geode/blob/4f307c71/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
index 329dee9..e2d3abc 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
@@ -14,7 +14,6 @@
  */
 package org.apache.geode.cache.lucene.test;
 
-import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.REGION_NAME;
 import static org.junit.Assert.*;
 
 import java.util.ArrayList;
@@ -36,6 +35,7 @@ import org.apache.geode.cache.Region;
 import org.apache.geode.cache.asyncqueue.AsyncEventQueue;
 import org.apache.geode.cache.asyncqueue.internal.AsyncEventQueueImpl;
 import org.apache.geode.cache.lucene.LuceneIndex;
+import org.apache.geode.cache.lucene.LuceneIndexFactory;
 import org.apache.geode.cache.lucene.LuceneQuery;
 import org.apache.geode.cache.lucene.LuceneQueryException;
 import org.apache.geode.cache.lucene.PageableLuceneQueryResults;
@@ -43,13 +43,8 @@ import org.apache.geode.cache.lucene.LuceneService;
 import org.apache.geode.cache.lucene.LuceneServiceProvider;
 import org.apache.geode.cache.lucene.internal.LuceneIndexForPartitionedRegion;
 import org.apache.geode.cache.lucene.internal.LuceneServiceImpl;
-import org.apache.geode.cache.persistence.PartitionOfflineException;
-import org.apache.geode.internal.cache.ForceReattemptException;
 import org.apache.geode.internal.cache.LocalRegion;
 import org.apache.geode.internal.cache.wan.AbstractGatewaySender;
-import org.apache.geode.pdx.JSONFormatter;
-import org.apache.geode.pdx.PdxInstance;
-import org.apache.geode.test.dunit.IgnoredException;
 import org.apache.geode.test.dunit.VM;
 
 public class LuceneTestUtilities {
@@ -186,7 +181,8 @@ public class LuceneTestUtilities {
   }
 
   public static void createIndex(Cache cache, String... fieldNames) {
-    LuceneServiceProvider.get(cache).createIndex(INDEX_NAME, REGION_NAME, fieldNames);
+    final LuceneIndexFactory indexFactory = LuceneServiceProvider.get(cache).createIndexFactory();
+    indexFactory.setFields(fieldNames).create(INDEX_NAME, REGION_NAME);
   }
 
   public static void verifyIndexFinishFlushing(Cache cache, String indexName, String regionName)


[30/49] geode git commit: GEODE-2379 Document new behavior of export logs This closes #425

Posted by kh...@apache.org.
GEODE-2379 Document new behavior of export logs
This closes #425


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/e39aff22
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/e39aff22
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/e39aff22

Branch: refs/heads/feature/GEODE-2420
Commit: e39aff223d662c018464a40a2deb357e00e16e9f
Parents: 2e2992a
Author: Dave Barnes <db...@pivotal.io>
Authored: Tue Mar 14 12:48:52 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../gfsh/command-pages/export.html.md.erb       | 46 ++++++++------------
 1 file changed, 19 insertions(+), 27 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/e39aff22/geode-docs/tools_modules/gfsh/command-pages/export.html.md.erb
----------------------------------------------------------------------
diff --git a/geode-docs/tools_modules/gfsh/command-pages/export.html.md.erb b/geode-docs/tools_modules/gfsh/command-pages/export.html.md.erb
index 690bf07..d8dbb1b 100644
--- a/geode-docs/tools_modules/gfsh/command-pages/export.html.md.erb
+++ b/geode-docs/tools_modules/gfsh/command-pages/export.html.md.erb
@@ -23,27 +23,27 @@ limitations under the License.
 
 Export configurations, data, logs and stack-traces.
 
--   **[export cluster-configuration](../../../tools_modules/gfsh/command-pages/export.html#topic_mdv_jgz_ck)**
+-   **[export cluster-configuration](#topic_mdv_jgz_ck)**
 
     Exports a cluster configuration zip file that contains the `cache.xml` files, `gemfire.properties` files, and application jar files needed to configure and operate a Geode distributed system.
 
--   **[export config](../../../tools_modules/gfsh/command-pages/export.html#topic_C7C69306F93743459E65D46537F4A1EE)**
+-   **[export config](#topic_C7C69306F93743459E65D46537F4A1EE)**
 
     Export configuration properties for a member or members.
 
--   **[export data](../../../tools_modules/gfsh/command-pages/export.html#topic_263B70069BFC4A7185F86B3272011734)**
+-   **[export data](#topic_263B70069BFC4A7185F86B3272011734)**
 
     Export user data from a region to a file.
 
--   **[export logs](../../../tools_modules/gfsh/command-pages/export.html#topic_B80978CC659244AE91E2B8CE56EBDFE3)**
+-   **[export logs](#topic_B80978CC659244AE91E2B8CE56EBDFE3)**
 
     Export logs to a given directory.
 
--   **[export offline-disk-store](../../../tools_modules/gfsh/command-pages/export.html#topic_sjg_bvt_gq)**
+-   **[export offline-disk-store](#topic_sjg_bvt_gq)**
 
     Export region data from an offline disk store into gemfire snapshot files.
 
--   **[export stack-traces](../../../tools_modules/gfsh/command-pages/export.html#topic_195D27B8B2B64A4E84CF2256636D54BD)**
+-   **[export stack-traces](#topic_195D27B8B2B64A4E84CF2256636D54BD)**
 
     Export the stack trace for a member or members.
 
@@ -61,8 +61,6 @@ See [Overview of the Cluster Configuration Service](../../../configuring/cluster
 export cluster-configuration --zip-file-name=value [--dir=value]
 ```
 
-<a id="topic_mdv_jgz_ck__table_qyw_hbh_2w"></a>
-
 | Name                                                  | Description                                                                          | Default Value     |
 |-------------------------------------------------------|--------------------------------------------------------------------------------------|-------------------|
 | <span class="keyword parmname">\\-\\-zip-file-name</span> | *Required.* File name of the zip file to contain the exported cluster configuration. | �                 |
@@ -98,8 +96,6 @@ export config [--member=value(,value)*] [--group=value(,value)*]
 [--dir=value]
 ```
 
-<a id="topic_C7C69306F93743459E65D46537F4A1EE__table_u5w_2bh_2w"></a>
-
 | Name                                           | Description                                                          |
 |------------------------------------------------|----------------------------------------------------------------------|
 | <span class="keyword parmname">\\-\\-member</span> | Name/Id of the member(s) whose configuration will be exported.       |
@@ -135,8 +131,6 @@ Export user data from a region to a file.
 export data --region=value --file=value --member=value
 ```
 
-<a id="topic_263B70069BFC4A7185F86B3272011734__table_lds_lbh_2w"></a>
-
 | Name                                           | Description                                                                                                                                     |
 |------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------|
 | <span class="keyword parmname">\\-\\-region</span> | *Required.* Region from which data is to be exported.                                                                                           |
@@ -167,28 +161,30 @@ Export logs to a given directory.
 
 All files that have logs in the specified time range will be exported. If no time range is specified, all logs will be exported.
 
+The `--dir` parameter specifies a local directory to which log files will be written. This is used only when you are exporting logs using an http connection.  If executed over http, the zip archive will be saved in the specified directory on the user's client machine. If not specified, logs are written to the location specified by the `user.dir` system property. When the command is executed over JMX, logs will be saved as `exportedlogs_xxx.zip` in the connected locator's working directory.
+
 **Availability:** Online. You must be connected in `gfsh` to a JMX Manager member to use this command.
 
 **Syntax:**
 
 ``` pre
-export logs --dir=value [--group=value(,value)*] [--member=value] [--log
--level=value] [--only-log-level=value] [--merge-log=value] [--start-time=value]
-[--end-time=value]
+export logs [--dir=value] [--group=value(,value)*] [--member=value(,value)*] 
+[--log-level=value] [--only-log-level=value] [--merge-log=value] 
+[--start-time=value] [--end-time=value] [logs-only(=value)?] [--stats-only(=value)?]
 ```
 
-<a id="topic_B80978CC659244AE91E2B8CE56EBDFE3__table_xw5_4bh_2w"></a>
-
 | Name                                                   | Description                                                                                                                | Default Value |
 |--------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------|---------------|
-| <span class="keyword parmname">\\-\\-dir</span>            | *Required.* Directory to which log files will be written.                                                                  | �             |
+| <span class="keyword parmname">\\-\\-dir</span>            | Local directory to which log files will be written when logs are exported using an http connection. Ignored when the command is executed over JMX.     | �             | 
 | <span class="keyword parmname">\\-\\-group</span>          | Group of members whose log files will be exported.                                                                         | �             |
 | <span class="keyword parmname">&#8209;&#8209;member</span>         | Name/ID of the member whose log files will be exported.                                                                    | �             |
-| <span class="keyword parmname">\\-\\-log-level</span>      | Minimum level of log entries to export. Valid values are: `none`, `error`, `info`, `config`, `fine`, `finer` and `finest`. | `info`        |
-| <span class="keyword parmname">\\-\\-only-log-level</span> | Whether to only include those entries that exactly match the <span class="keyword parmname">\\-\\-log-level</span> specified.  | false         |
-| <span class="keyword parmname">&#8209;&#8209;merge&#8209;log</span>      | Whether to merge logs after exporting to the target directory.                                                             | false         |
+| <span class="keyword parmname">\\-\\-log-level</span>      | Minimum level of log entries to export. Valid values are: `fatal`, `error`, `warn`, `info`, `debug`, `trace`, and `all`. | `info`        |
+| <span class="keyword parmname">\\-\\-only-log-level</span> | Whether to only include only entries that exactly match the <span class="keyword parmname">\\-\\-log-level</span> specified.  | false         |
+| <span class="keyword parmname">&#8209;&#8209;merge&#8209;log</span>      | Whether to merge logs after exporting to the target directory (deprecated).                                                             | false         |
 | <span class="keyword parmname">\\-\\-start-time</span>     | Log entries that occurred after this time will be exported. Format: yyyy/MM/dd/HH/mm/ss/SSS/z OR yyyy/MM/dd                | no limit      |
 | <span class="keyword parmname">\\-\\-end-time</span>       | Log entries that occurred before this time will be exported. Format: yyyy/MM/dd/HH/mm/ss/SSS/z OR yyyy/MM/dd               | no limit      |
+| <span class="keyword parmname">\\-\\-logs-only</span>       | Whether to export only logs (not statistics)               | If parameter not specified: false. If parameter specified without a value: true      |
+| <span class="keyword parmname">\\-\\-stats-only</span>       | Whether to export only statistics (not logs)               | If parameter not specified: false. If parameter specified without a value: true      |
 
 <span class="tablecap">Table 4. Export Logs Parameters</span>
 
@@ -201,8 +197,8 @@ export logs --dir=data/logs
 **Sample Output:**
 
 ``` pre
-gfsh>export logs --dir=data/logs
-Successfully exported to directory data/logs
+gfsh>export logs
+Logs exported to the connected member's file system: /data/my-locator/exportedLogs_1489513007261.zip
 ```
 
 ## <a id="topic_sjg_bvt_gq" class="no-quick-link"></a>export offline-disk-store
@@ -217,8 +213,6 @@ Export region data from an offline disk store into gemfire snapshot files.
 export offline-disk-store --name=value --disk-dirs=value(,value)* --dir=value
 ```
 
-<a id="topic_sjg_bvt_gq__table_gzr_rbh_2w"></a>
-
 | Name                                              | Description                                        |
 |---------------------------------------------------|----------------------------------------------------|
 | <span class="keyword parmname">\\-\\-name</span>      | *Required.* Name of the disk store to be exported. |
@@ -246,8 +240,6 @@ Export the stack trace for a member or members.
 export stack-traces --file=value [--member=value] [--group=value]
 ```
 
-<a id="topic_195D27B8B2B64A4E84CF2256636D54BD__table_cjr_wbh_2w"></a>
-
 | Name                                           | Description                                                     |
 |------------------------------------------------|-----------------------------------------------------------------|
 | <span class="keyword parmname">\\-\\-file</span>   | *Required.* Filename to which the stack-traces will be written. |


[04/49] geode git commit: GEODE-2539: Upgrading Jetty causes RestSecurityItegrationTest to fail

Posted by kh...@apache.org.
GEODE-2539: Upgrading Jetty causes RestSecurityItegrationTest to fail


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/caf0767e
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/caf0767e
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/caf0767e

Branch: refs/heads/feature/GEODE-2420
Commit: caf0767e349870aceb1bc5e22d0ffb9903ad4aab
Parents: 54eab9a
Author: Kevin J. Duling <kd...@pivotal.io>
Authored: Mon Mar 6 12:08:28 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:08:56 2017 -0700

----------------------------------------------------------------------
 .../web/RestSecurityIntegrationTest.java        | 144 ++++++++++---------
 .../web/RestSecurityPostProcessorTest.java      |  20 ++-
 gradle/dependency-versions.properties           |   2 +-
 3 files changed, 90 insertions(+), 76 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/caf0767e/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityIntegrationTest.java b/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityIntegrationTest.java
index 85fc5be..dee004f 100644
--- a/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityIntegrationTest.java
+++ b/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityIntegrationTest.java
@@ -18,6 +18,7 @@ import static org.apache.geode.distributed.ConfigurationProperties.HTTP_SERVICE_
 import static org.apache.geode.distributed.ConfigurationProperties.HTTP_SERVICE_PORT;
 import static org.apache.geode.distributed.ConfigurationProperties.SECURITY_MANAGER;
 import static org.apache.geode.distributed.ConfigurationProperties.START_DEV_REST_API;
+import static org.assertj.core.api.Assertions.assertThat;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertNotNull;
 import static org.junit.Assert.assertTrue;
@@ -71,104 +72,107 @@ public class RestSecurityIntegrationTest {
     String json = "{\"@type\":\"double\",\"@value\":210}";
 
     HttpResponse response = restClient.doGet("/functions", "unknown-user", "1234567");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doGet("/functions", "stranger", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
     response = restClient.doGet("/functions", "dataReader", "1234567");
-    assertEquals(200, restClient.getCode(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
     response.getEntity();
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, restClient.getContentType(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
 
     response = restClient.doPost("/functions/AddFreeItemsToOrder", "unknown-user", "1234567", json);
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doPost("/functions/AddFreeItemsToOrder", "dataReader", "1234567", json);
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
     response = restClient.doPost("/functions/AddFreeItemsToOrder?onRegion=" + REGION_NAME,
         "dataWriter", "1234567", json);
     // because we're only testing the security of the endpoint, not the endpoint functionality, a
     // 500 is acceptable
-    assertEquals(500, restClient.getCode(response));
+    assertEquals(500, GeodeRestClient.getCode(response));
   }
 
   @Test
   public void testQueries() throws Exception {
     HttpResponse response = restClient.doGet("/queries", "unknown-user", "1234567");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doGet("/queries", "stranger", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
     response = restClient.doGet("/queries", "dataReader", "1234567");
-    assertEquals(200, restClient.getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, restClient.getContentType(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
   }
 
   @Test
   public void testAdhocQuery() throws Exception {
     HttpResponse response = restClient.doGet("/queries/adhoc?q=", "unknown-user", "1234567");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doGet("/queries/adhoc?q=", "stranger", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
     response = restClient.doGet("/queries/adhoc?q=", "dataReader", "1234567");
     // because we're only testing the security of the endpoint, not the endpoint functionality, a
     // 500 is acceptable
-    assertEquals(500, restClient.getCode(response));
+    assertEquals(500, GeodeRestClient.getCode(response));
   }
 
   @Test
   public void testPostQuery() throws Exception {
     HttpResponse response = restClient.doPost("/queries?id=0&q=", "unknown-user", "1234567", "");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doPost("/queries?id=0&q=", "stranger", "1234567", "");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
     response = restClient.doPost("/queries?id=0&q=", "dataReader", "1234567", "");
     // because we're only testing the security of the endpoint, not the endpoint functionality, a
     // 500 is acceptable
-    assertEquals(500, restClient.getCode(response));
+    assertEquals(500, GeodeRestClient.getCode(response));
   }
 
   @Test
   public void testPostQuery2() throws Exception {
     HttpResponse response =
         restClient.doPost("/queries/id", "unknown-user", "1234567", "{\"id\" : \"foo\"}");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doPost("/queries/id", "stranger", "1234567", "{\"id\" : \"foo\"}");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
     response = restClient.doPost("/queries/id", "dataReader", "1234567", "{\"id\" : \"foo\"}");
     // We should get a 404 because we're trying to update a query that doesn't exist
-    assertEquals(404, restClient.getCode(response));
+    assertEquals(404, GeodeRestClient.getCode(response));
   }
 
   @Test
   public void testPutQuery() throws Exception {
     HttpResponse response =
         restClient.doPut("/queries/id", "unknown-user", "1234567", "{\"id\" : \"foo\"}");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doPut("/queries/id", "stranger", "1234567", "{\"id\" : \"foo\"}");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
     response = restClient.doPut("/queries/id", "dataReader", "1234567", "{\"id\" : \"foo\"}");
     // We should get a 404 because we're trying to update a query that doesn't exist
-    assertEquals(404, restClient.getCode(response));
+    assertEquals(404, GeodeRestClient.getCode(response));
   }
 
   @Test
   public void testDeleteQuery() throws Exception {
     HttpResponse response = restClient.doDelete("/queries/id", "unknown-user", "1234567");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doDelete("/queries/id", "stranger", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
     response = restClient.doDelete("/queries/id", "dataWriter", "1234567");
     // We should get a 404 because we're trying to delete a query that doesn't exist
-    assertEquals(404, restClient.getCode(response));
+    assertEquals(404, GeodeRestClient.getCode(response));
   }
 
   @Test
   public void testServers() throws Exception {
     HttpResponse response = restClient.doGet("/servers", "unknown-user", "1234567");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doGet("/servers", "stranger", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
     response = restClient.doGet("/servers", "super-user", "1234567");
-    assertEquals(200, restClient.getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, restClient.getContentType(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
   }
 
   /**
@@ -178,14 +182,14 @@ public class RestSecurityIntegrationTest {
   @Test
   public void testPing() throws Exception {
     HttpResponse response = restClient.doHEAD("/ping", "stranger", "1234567");
-    assertEquals(200, restClient.getCode(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
     response = restClient.doGet("/ping", "stranger", "1234567");
-    assertEquals(200, restClient.getCode(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
 
     response = restClient.doHEAD("/ping", "super-user", "1234567");
-    assertEquals(200, restClient.getCode(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
     response = restClient.doGet("/ping", "super-user", "1234567");
-    assertEquals(200, restClient.getCode(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
   }
 
   /**
@@ -194,10 +198,11 @@ public class RestSecurityIntegrationTest {
   @Test
   public void getRegions() throws Exception {
     HttpResponse response = restClient.doGet("", "dataReader", "1234567");
-    assertEquals("A '200 - OK' was expected", 200, restClient.getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, restClient.getContentType(response));
+    assertEquals("A '200 - OK' was expected", 200, GeodeRestClient.getCode(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
 
-    JSONObject jsonObject = restClient.getJsonObject(response);
+    JSONObject jsonObject = GeodeRestClient.getJsonObject(response);
     JSONArray regions = jsonObject.getJSONArray("regions");
     assertNotNull(regions);
     assertTrue(regions.length() > 0);
@@ -207,11 +212,11 @@ public class RestSecurityIntegrationTest {
 
     // List regions with an unknown user - 401
     response = restClient.doGet("", "unknown-user", "badpassword");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
 
     // list regions with insufficent rights - 403
     response = restClient.doGet("", "authRegionReader", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
   }
 
   /**
@@ -221,16 +226,17 @@ public class RestSecurityIntegrationTest {
   public void getRegion() throws Exception {
     // Test an unknown user - 401 error
     HttpResponse response = restClient.doGet("/" + REGION_NAME, "unknown-user", "1234567");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
 
     // Test a user with insufficient rights - 403
     response = restClient.doGet("/" + REGION_NAME, "stranger", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
 
     // Test an authorized user - 200
     response = restClient.doGet("/" + REGION_NAME, "super-user", "1234567");
-    assertEquals(200, restClient.getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, restClient.getContentType(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
   }
 
   /**
@@ -240,15 +246,15 @@ public class RestSecurityIntegrationTest {
   public void headRegion() throws Exception {
     // Test an unknown user - 401 error
     HttpResponse response = restClient.doHEAD("/" + REGION_NAME, "unknown-user", "1234567");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
 
     // Test a user with insufficient rights - 403
     response = restClient.doHEAD("/" + REGION_NAME, "stranger", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
 
     // Test an authorized user - 200
     response = restClient.doHEAD("/" + REGION_NAME, "super-user", "1234567");
-    assertEquals(200, restClient.getCode(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
   }
 
   /**
@@ -258,11 +264,11 @@ public class RestSecurityIntegrationTest {
   public void deleteRegion() throws Exception {
     // Test an unknown user - 401 error
     HttpResponse response = restClient.doDelete("/" + REGION_NAME, "unknown-user", "1234567");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
 
     // Test a user with insufficient rights - 403
     response = restClient.doDelete("/" + REGION_NAME, "dataReader", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
   }
 
   /**
@@ -272,11 +278,12 @@ public class RestSecurityIntegrationTest {
   public void getRegionKeys() throws Exception {
     // Test an authorized user
     HttpResponse response = restClient.doGet("/" + REGION_NAME + "/keys", "super-user", "1234567");
-    assertEquals(200, restClient.getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, restClient.getContentType(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
     // Test an unauthorized user
     response = restClient.doGet("/" + REGION_NAME + "/keys", "dataWriter", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
   }
 
   /**
@@ -286,12 +293,13 @@ public class RestSecurityIntegrationTest {
   public void getRegionKey() throws Exception {
     // Test an authorized user
     HttpResponse response = restClient.doGet("/" + REGION_NAME + "/key1", "key1User", "1234567");
-    assertEquals(200, restClient.getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, restClient.getContentType(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
 
     // Test an unauthorized user
     response = restClient.doGet("/" + REGION_NAME + "/key1", "dataWriter", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
   }
 
   /**
@@ -302,15 +310,15 @@ public class RestSecurityIntegrationTest {
     // Test an unknown user - 401 error
     HttpResponse response =
         restClient.doDelete("/" + REGION_NAME + "/key1", "unknown-user", "1234567");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
 
     // Test a user with insufficient rights - 403
     response = restClient.doDelete("/" + REGION_NAME + "/key1", "dataReader", "1234567");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
 
     // Test an authorized user - 200
     response = restClient.doDelete("/" + REGION_NAME + "/key1", "key1User", "1234567");
-    assertEquals(200, restClient.getCode(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
   }
 
   /**
@@ -321,17 +329,17 @@ public class RestSecurityIntegrationTest {
     // Test an unknown user - 401 error
     HttpResponse response = restClient.doPost("/" + REGION_NAME + "?key9", "unknown", "1234567",
         "{ \"key9\" : \"foo\" }");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
 
     // Test a user with insufficient rights - 403
     response = restClient.doPost("/" + REGION_NAME + "?key9", "dataReader", "1234567",
         "{ \"key9\" : \"foo\" }");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
 
     // Test an authorized user - 200
     response = restClient.doPost("/" + REGION_NAME + "?key9", "dataWriter", "1234567",
         "{ \"key9\" : \"foo\" }");
-    assertEquals(201, restClient.getCode(response));
+    assertEquals(201, GeodeRestClient.getCode(response));
   }
 
   /**
@@ -347,33 +355,33 @@ public class RestSecurityIntegrationTest {
     // Test an unknown user - 401 error
     HttpResponse response = restClient.doPut("/" + REGION_NAME + "/key1?op=PUT", "unknown-user",
         "1234567", "{ \"key9\" : \"foo\" }");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
 
     response = restClient.doPut("/" + REGION_NAME + "/key1?op=CAS", "unknown-user", "1234567",
         "{ \"key9\" : \"foo\" }");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
     response = restClient.doPut("/" + REGION_NAME + "/key1?op=REPLACE", "unknown-user", "1234567",
         "{ \"@old\" : \"value1\", \"@new\" : \"CASvalue\" }");
-    assertEquals(401, restClient.getCode(response));
+    assertEquals(401, GeodeRestClient.getCode(response));
 
     response = restClient.doPut("/" + REGION_NAME + "/key1?op=PUT", "dataReader", "1234567",
         "{ \"key1\" : \"foo\" }");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
 
     response = restClient.doPut("/" + REGION_NAME + "/key1?op=REPLACE", "dataReader", "1234567",
         "{ \"key1\" : \"foo\" }");
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
 
     response =
         restClient.doPut("/" + REGION_NAME + "/key1?op=CAS", "dataReader", "1234567", casJSON);
-    assertEquals(403, restClient.getCode(response));
+    assertEquals(403, GeodeRestClient.getCode(response));
 
     response = restClient.doPut("/" + REGION_NAME + "/key1?op=PUT", "key1User", "1234567",
         "{ \"key1\" : \"foo\" }");
-    assertEquals(200, restClient.getCode(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
 
     response =
         restClient.doPut("/" + REGION_NAME + "/key1?op=REPLACE", "key1User", "1234567", json);
-    assertEquals(200, restClient.getCode(response));
+    assertEquals(200, GeodeRestClient.getCode(response));
   }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/caf0767e/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityPostProcessorTest.java
----------------------------------------------------------------------
diff --git a/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityPostProcessorTest.java b/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityPostProcessorTest.java
index 82e9b7f..ab21094 100644
--- a/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityPostProcessorTest.java
+++ b/geode-assembly/src/test/java/org/apache/geode/rest/internal/web/RestSecurityPostProcessorTest.java
@@ -20,9 +20,9 @@ import static org.apache.geode.distributed.ConfigurationProperties.SECURITY_MANA
 import static org.apache.geode.distributed.ConfigurationProperties.SECURITY_POST_PROCESSOR;
 import static org.apache.geode.distributed.ConfigurationProperties.START_DEV_REST_API;
 import static org.apache.geode.rest.internal.web.GeodeRestClient.getCode;
-import static org.apache.geode.rest.internal.web.GeodeRestClient.getContentType;
 import static org.apache.geode.rest.internal.web.GeodeRestClient.getJsonArray;
 import static org.apache.geode.rest.internal.web.GeodeRestClient.getJsonObject;
+import static org.assertj.core.api.Assertions.assertThat;
 import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertTrue;
 
@@ -86,7 +86,8 @@ public class RestSecurityPostProcessorTest {
     // Test a single key
     HttpResponse response = restClient.doGet("/customers/1", "dataReader", "1234567");
     assertEquals(200, getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, getContentType(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
 
     // Ensure SSN is hidden
     JSONObject jsonObject = getJsonObject(response);
@@ -96,7 +97,8 @@ public class RestSecurityPostProcessorTest {
     // Try with super-user
     response = restClient.doGet("/customers/1", "super-user", "1234567");
     assertEquals(200, getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, getContentType(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
 
     // ensure SSN is readable
     jsonObject = getJsonObject(response);
@@ -109,7 +111,8 @@ public class RestSecurityPostProcessorTest {
   public void getMultipleRegionKeys() throws Exception {
     HttpResponse response = restClient.doGet("/customers/1,3", "dataReader", "1234567");
     assertEquals(200, getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, getContentType(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
 
     JSONObject jsonObject = getJsonObject(response);
     JSONArray jsonArray = jsonObject.getJSONArray("customers");
@@ -127,7 +130,8 @@ public class RestSecurityPostProcessorTest {
   public void getRegion() throws Exception {
     HttpResponse response = restClient.doGet("/customers", "dataReader", "1234567");
     assertEquals(200, getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, getContentType(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
 
     JSONObject jsonObject = getJsonObject(response);
     JSONArray jsonArray = jsonObject.getJSONArray("customers");
@@ -145,7 +149,8 @@ public class RestSecurityPostProcessorTest {
         + URLEncoder.encode("SELECT * FROM /customers order by customerId", "UTF-8");
     HttpResponse response = restClient.doGet(query, "dataReader", "1234567");
     assertEquals(200, getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, getContentType(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
 
     JSONArray jsonArray = getJsonArray(response);
     final int length = jsonArray.length();
@@ -171,7 +176,8 @@ public class RestSecurityPostProcessorTest {
     String query = "/queries";
     response = restClient.doGet(query, "dataReader", "1234567");
     assertEquals(200, getCode(response));
-    assertEquals(MediaType.APPLICATION_JSON_UTF8_VALUE, getContentType(response));
+    assertThat(GeodeRestClient.getContentType(response))
+        .containsIgnoringCase(MediaType.APPLICATION_JSON_UTF8_VALUE);
 
     // Execute the query
     response = restClient.doPost("/queries/selectCustomer", "dataReader", "1234567",

http://git-wip-us.apache.org/repos/asf/geode/blob/caf0767e/gradle/dependency-versions.properties
----------------------------------------------------------------------
diff --git a/gradle/dependency-versions.properties b/gradle/dependency-versions.properties
index 34e5d67..90601ac 100644
--- a/gradle/dependency-versions.properties
+++ b/gradle/dependency-versions.properties
@@ -52,7 +52,7 @@ javax.resource-api.version = 1.7
 javax.servlet-api.version = 3.1.0
 javax.transaction-api.version = 1.2
 jedis.version = 2.9.0
-jetty.version = 9.3.6.v20151106
+jetty.version = 9.4.2.v20170220
 jgroups.version = 3.6.10.Final
 jmock.version = 2.8.2
 jna.version = 4.0.0


[11/49] geode git commit: GEODE-2618: Fixing spotlessApply errors

Posted by kh...@apache.org.
GEODE-2618: Fixing spotlessApply errors


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/016f07a9
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/016f07a9
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/016f07a9

Branch: refs/heads/feature/GEODE-2420
Commit: 016f07a9a26672061e299042172f5a08f2aff633
Parents: 88fa1af
Author: nabarun <nn...@pivotal.io>
Authored: Fri Mar 10 13:55:19 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../geode/cache/client/internal/ConnectionPoolImplJUnitTest.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/016f07a9/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java b/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java
index 45a4494..d99b31a 100644
--- a/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java
@@ -150,7 +150,7 @@ public class ConnectionPoolImplJUnitTest {
     }
     LiveServerPinger lsp = new LiveServerPinger(pool);
     long NANOS_PER_MS = 1000000L;
-    assertEquals(((pool.getPingInterval() + 1)/ 2) * NANOS_PER_MS, lsp.pingIntervalNanos  );
+    assertEquals(((pool.getPingInterval() + 1) / 2) * NANOS_PER_MS, lsp.pingIntervalNanos);
   }
 
   @Test


[06/49] geode git commit: GEODE-2642 Fixed ping thread issue on client.

Posted by kh...@apache.org.
GEODE-2642 Fixed ping thread issue on client.

Client ping thread wakes up every "ping-interval" and then it checks
whether someone already did some operation on that server or not, in the
last ping-interval. If there is some operation, then it doesn't ping
server and wakes up again after ping-interval.Otherwise ot pings the
server. But if operation happens just start of the previous ping-interval
then client thread will ping server after more than ping-interval.
This can cause an issue. Thus to avoid this race condition, now ping
thread will wake up in ping-interval/2 and will make sure it pings server.


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/de724d34
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/de724d34
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/de724d34

Branch: refs/heads/feature/GEODE-2420
Commit: de724d34a485d1a1fb47109c8b8d6645fc46adc2
Parents: 257f0ea
Author: Hitesh Khamesra <hk...@pivotal.io>
Authored: Fri Mar 10 13:31:25 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../org/apache/geode/cache/client/internal/LiveServerPinger.java  | 2 +-
 .../geode/cache/client/internal/ConnectionPoolImplJUnitTest.java  | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/de724d34/geode-core/src/main/java/org/apache/geode/cache/client/internal/LiveServerPinger.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/cache/client/internal/LiveServerPinger.java b/geode-core/src/main/java/org/apache/geode/cache/client/internal/LiveServerPinger.java
index bcb8eec..32d0f8e 100644
--- a/geode-core/src/main/java/org/apache/geode/cache/client/internal/LiveServerPinger.java
+++ b/geode-core/src/main/java/org/apache/geode/cache/client/internal/LiveServerPinger.java
@@ -42,7 +42,7 @@ public class LiveServerPinger extends EndpointListenerAdapter {
 
   public LiveServerPinger(InternalPool pool) {
     this.pool = pool;
-    this.pingIntervalNanos = pool.getPingInterval() * NANOS_PER_MS;
+    this.pingIntervalNanos = ((pool.getPingInterval() + 1) / 2) * NANOS_PER_MS;
   }
 
   @Override

http://git-wip-us.apache.org/repos/asf/geode/blob/de724d34/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java b/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java
index 272580d..45a4494 100644
--- a/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/cache/client/internal/ConnectionPoolImplJUnitTest.java
@@ -148,6 +148,9 @@ public class ConnectionPoolImplJUnitTest {
       assertEquals(port, addr.getPort());
       assertEquals("localhost", addr.getHostName());
     }
+    LiveServerPinger lsp = new LiveServerPinger(pool);
+    long NANOS_PER_MS = 1000000L;
+    assertEquals(((pool.getPingInterval() + 1)/ 2) * NANOS_PER_MS, lsp.pingIntervalNanos  );
   }
 
   @Test


[48/49] geode git commit: refactor for testability

Posted by kh...@apache.org.
refactor for testability

Extract inner Interceptor class from ExportLogsCommand


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/2bcba1da
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/2bcba1da
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/2bcba1da

Branch: refs/heads/feature/GEODE-2420
Commit: 2bcba1daae43247ee6dde673c36adba4f7142932
Parents: 26700b5
Author: Ken Howe <kh...@pivotal.io>
Authored: Fri Mar 17 12:59:59 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:46 2017 -0700

----------------------------------------------------------------------
 .../internal/cli/commands/ExportLogCommand.java | 251 -------------------
 .../cli/commands/ExportLogsCommand.java         | 192 ++++++++++++++
 .../cli/commands/ExportLogsInterceptor.java     | 108 ++++++++
 .../cli/functions/ExportLogsFunction.java       |   6 +-
 .../cli/commands/ExportLogsCommandTest.java     |  47 ++++
 .../cli/commands/ExportLogsDUnitTest.java       |   4 +-
 .../ExportLogsInterceptorJUnitTest.java         |   4 +-
 .../cli/commands/ExportLogsStatsDUnitTest.java  |   2 +-
 8 files changed, 355 insertions(+), 259 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/2bcba1da/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogCommand.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogCommand.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogCommand.java
deleted file mode 100644
index 7dfc63d..0000000
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogCommand.java
+++ /dev/null
@@ -1,251 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-
-package org.apache.geode.management.internal.cli.commands;
-
-import org.apache.commons.io.FileUtils;
-import org.apache.commons.lang.StringUtils;
-import org.apache.geode.cache.Region;
-import org.apache.geode.distributed.DistributedMember;
-import org.apache.geode.internal.cache.GemFireCacheImpl;
-import org.apache.geode.internal.logging.LogService;
-import org.apache.geode.management.cli.CliMetaData;
-import org.apache.geode.management.cli.ConverterHint;
-import org.apache.geode.management.cli.Result;
-import org.apache.geode.management.internal.cli.AbstractCliAroundInterceptor;
-import org.apache.geode.management.internal.cli.CliUtil;
-import org.apache.geode.management.internal.cli.GfshParseResult;
-import org.apache.geode.management.internal.cli.functions.ExportLogsFunction;
-import org.apache.geode.management.internal.cli.i18n.CliStrings;
-import org.apache.geode.management.internal.cli.result.ResultBuilder;
-import org.apache.geode.management.internal.cli.shell.Gfsh;
-import org.apache.geode.management.internal.cli.util.ExportLogsCacheWriter;
-import org.apache.geode.management.internal.configuration.utils.ZipUtils;
-import org.apache.geode.management.internal.security.ResourceOperation;
-import org.apache.geode.security.ResourcePermission;
-import org.apache.logging.log4j.Level;
-import org.apache.logging.log4j.Logger;
-import org.springframework.shell.core.CommandMarker;
-import org.springframework.shell.core.annotation.CliCommand;
-import org.springframework.shell.core.annotation.CliOption;
-
-import java.io.File;
-import java.io.IOException;
-import java.nio.file.Files;
-import java.nio.file.Path;
-import java.nio.file.Paths;
-import java.time.LocalDateTime;
-import java.util.HashMap;
-import java.util.Map;
-import java.util.Set;
-
-public class ExportLogCommand implements CommandMarker {
-  private static final Logger logger = LogService.getLogger();
-  public static final String FORMAT = "yyyy/MM/dd/HH/mm/ss/SSS/z";
-  public static final String ONLY_DATE_FORMAT = "yyyy/MM/dd";
-
-  @CliCommand(value = CliStrings.EXPORT_LOGS, help = CliStrings.EXPORT_LOGS__HELP)
-  @CliMetaData(shellOnly = false, isFileDownloadOverHttp = true,
-      interceptor = "org.apache.geode.management.internal.cli.commands.ExportLogCommand$ExportLogsInterceptor",
-      relatedTopic = {CliStrings.TOPIC_GEODE_SERVER, CliStrings.TOPIC_GEODE_DEBUG_UTIL})
-  @ResourceOperation(resource = ResourcePermission.Resource.CLUSTER,
-      operation = ResourcePermission.Operation.READ)
-  public Result exportLogs(
-      @CliOption(key = CliStrings.EXPORT_LOGS__DIR, help = CliStrings.EXPORT_LOGS__DIR__HELP,
-          mandatory = false) String dirName,
-      @CliOption(key = CliStrings.EXPORT_LOGS__GROUP,
-          unspecifiedDefaultValue = CliMetaData.ANNOTATION_NULL_VALUE,
-          optionContext = ConverterHint.MEMBERGROUP,
-          help = CliStrings.EXPORT_LOGS__GROUP__HELP) String[] groups,
-      @CliOption(key = CliStrings.EXPORT_LOGS__MEMBER,
-          unspecifiedDefaultValue = CliMetaData.ANNOTATION_NULL_VALUE,
-          optionContext = ConverterHint.ALL_MEMBER_IDNAME,
-          help = CliStrings.EXPORT_LOGS__MEMBER__HELP) String[] memberIds,
-      @CliOption(key = CliStrings.EXPORT_LOGS__LOGLEVEL,
-          unspecifiedDefaultValue = LogService.DEFAULT_LOG_LEVEL,
-          optionContext = ConverterHint.LOG_LEVEL,
-          help = CliStrings.EXPORT_LOGS__LOGLEVEL__HELP) String logLevel,
-      @CliOption(key = CliStrings.EXPORT_LOGS__UPTO_LOGLEVEL, unspecifiedDefaultValue = "false",
-          help = CliStrings.EXPORT_LOGS__UPTO_LOGLEVEL__HELP) boolean onlyLogLevel,
-      @CliOption(key = CliStrings.EXPORT_LOGS__MERGELOG, unspecifiedDefaultValue = "false",
-          help = CliStrings.EXPORT_LOGS__MERGELOG__HELP) boolean mergeLog,
-      @CliOption(key = CliStrings.EXPORT_LOGS__STARTTIME,
-          unspecifiedDefaultValue = CliMetaData.ANNOTATION_NULL_VALUE,
-          help = CliStrings.EXPORT_LOGS__STARTTIME__HELP) String start,
-      @CliOption(key = CliStrings.EXPORT_LOGS__ENDTIME,
-          unspecifiedDefaultValue = CliMetaData.ANNOTATION_NULL_VALUE,
-          help = CliStrings.EXPORT_LOGS__ENDTIME__HELP) String end,
-      @CliOption(key = CliStrings.EXPORT_LOGS__LOGSONLY, unspecifiedDefaultValue = "false",
-          specifiedDefaultValue = "true",
-          help = CliStrings.EXPORT_LOGS__LOGSONLY__HELP) boolean logsOnly,
-      @CliOption(key = CliStrings.EXPORT_LOGS__STATSONLY, unspecifiedDefaultValue = "false",
-          specifiedDefaultValue = "true",
-          help = CliStrings.EXPORT_LOGS__STATSONLY__HELP) boolean statsOnly) {
-    Result result = null;
-    GemFireCacheImpl cache = GemFireCacheImpl.getInstance();
-    try {
-      Set<DistributedMember> targetMembers =
-          CliUtil.findMembersIncludingLocators(groups, memberIds);
-
-      if (targetMembers.isEmpty()) {
-        return ResultBuilder.createUserErrorResult(CliStrings.NO_MEMBERS_FOUND_MESSAGE);
-      }
-
-      Map<String, Path> zipFilesFromMembers = new HashMap<>();
-      for (DistributedMember server : targetMembers) {
-        Region region = ExportLogsFunction.createOrGetExistingExportLogsRegion(true, cache);
-
-        ExportLogsCacheWriter cacheWriter =
-            (ExportLogsCacheWriter) region.getAttributes().getCacheWriter();
-
-        cacheWriter.startFile(server.getName());
-
-        CliUtil.executeFunction(new ExportLogsFunction(),
-            new ExportLogsFunction.Args(start, end, logLevel, onlyLogLevel, logsOnly, statsOnly),
-            server).getResult();
-        Path zipFile = cacheWriter.endFile();
-        ExportLogsFunction.destroyExportLogsRegion(cache);
-
-        // only put the zipfile in the map if it is not null
-        if (zipFile != null) {
-          logger.info("Received zip file from member {}: {}", server.getId(), zipFile);
-          zipFilesFromMembers.put(server.getId(), zipFile);
-        }
-      }
-
-      if (zipFilesFromMembers.isEmpty()) {
-        return ResultBuilder.createUserErrorResult("No files to be exported.");
-      }
-
-      Path tempDir = Files.createTempDirectory("exportedLogs");
-      // make sure the directory is created, so that even if there is no files unzipped to this dir,
-      // we can
-      // still zip it and send an empty zip file back to the client
-      Path exportedLogsDir = tempDir.resolve("exportedLogs");
-      FileUtils.forceMkdir(exportedLogsDir.toFile());
-
-      for (Path zipFile : zipFilesFromMembers.values()) {
-        Path unzippedMemberDir =
-            exportedLogsDir.resolve(zipFile.getFileName().toString().replace(".zip", ""));
-        ZipUtils.unzip(zipFile.toAbsolutePath().toString(), unzippedMemberDir.toString());
-        FileUtils.deleteQuietly(zipFile.toFile());
-      }
-
-      Path workingDir = Paths.get(System.getProperty("user.dir"));
-      Path exportedLogsZipFile = workingDir
-          .resolve("exportedLogs_" + System.currentTimeMillis() + ".zip").toAbsolutePath();
-
-      logger.info("Zipping into: " + exportedLogsZipFile.toString());
-      ZipUtils.zipDirectory(exportedLogsDir, exportedLogsZipFile);
-      FileUtils.deleteDirectory(tempDir.toFile());
-
-      // TODO:GEODE-2420: warn user if exportedLogsZipFile size > threshold
-      if (isOverDiskSpaceThreshold()) {
-        // append warning to exportedLogsZipFile.toString()??
-      }
-
-      result = ResultBuilder.createInfoResult(exportedLogsZipFile.toString());
-    } catch (Exception ex) {
-      logger.error(ex, ex);
-      result = ResultBuilder.createGemFireErrorResult(ex.getMessage());
-    } finally {
-      ExportLogsFunction.destroyExportLogsRegion(cache);
-    }
-    logger.debug("Exporting logs returning = {}", result);
-    return result;
-  }
-
-  protected static Gfsh getGfsh() {
-    return Gfsh.getCurrentInstance();
-  }
-
-  boolean isOverDiskSpaceThreshold() {
-    return false;
-  }
-
-  /**
-   * after the export logs, will need to copy the tempFile to the desired location and delete the
-   * temp file.
-   */
-  public static class ExportLogsInterceptor extends AbstractCliAroundInterceptor {
-    @Override
-    public Result preExecution(GfshParseResult parseResult) {
-      // the arguments are in the order of it's being declared
-      Map<String, String> arguments = parseResult.getParamValueStrings();
-
-      // validates groupId and memberIds not both set
-      if (arguments.get("group") != null && arguments.get("member") != null) {
-        return ResultBuilder.createUserErrorResult("Can't specify both group and member.");
-      }
-
-      // validate log level
-      String logLevel = arguments.get("log-level");
-      if (StringUtils.isBlank(logLevel) || Level.getLevel(logLevel.toUpperCase()) == null) {
-        return ResultBuilder.createUserErrorResult("Invalid log level: " + logLevel);
-      }
-
-      // validate start date and end date
-      String start = arguments.get("start-time");
-      String end = arguments.get("end-time");
-      if (start != null && end != null) {
-        // need to make sure end is later than start
-        LocalDateTime startTime = ExportLogsFunction.parseTime(start);
-        LocalDateTime endTime = ExportLogsFunction.parseTime(end);
-        if (startTime.isAfter(endTime)) {
-          return ResultBuilder.createUserErrorResult("start-time has to be earlier than end-time.");
-        }
-      }
-
-      // validate onlyLogs and onlyStats
-      boolean onlyLogs = Boolean.parseBoolean(arguments.get("logs-only"));
-      boolean onlyStats = Boolean.parseBoolean(arguments.get("stats-only"));
-      if (onlyLogs && onlyStats) {
-        return ResultBuilder.createUserErrorResult("logs-only and stats-only can't both be true");
-      }
-
-      return ResultBuilder.createInfoResult("");
-    }
-
-    @Override
-    public Result postExecution(GfshParseResult parseResult, Result commandResult, Path tempFile) {
-      // in the command over http case, the command result is in the downloaded temp file
-      if (tempFile != null) {
-        Path dirPath;
-        String dirName = parseResult.getParamValueStrings().get("dir");
-        if (StringUtils.isBlank(dirName)) {
-          dirPath = Paths.get(System.getProperty("user.dir"));
-        } else {
-          dirPath = Paths.get(dirName);
-        }
-        String fileName = "exportedLogs_" + System.currentTimeMillis() + ".zip";
-        File exportedLogFile = dirPath.resolve(fileName).toFile();
-        try {
-          FileUtils.copyFile(tempFile.toFile(), exportedLogFile);
-          FileUtils.deleteQuietly(tempFile.toFile());
-          commandResult = ResultBuilder
-              .createInfoResult("Logs exported to: " + exportedLogFile.getAbsolutePath());
-        } catch (IOException e) {
-          logger.error(e.getMessage(), e);
-          commandResult = ResultBuilder.createGemFireErrorResult(e.getMessage());
-        }
-      } else if (commandResult.getStatus() == Result.Status.OK) {
-        commandResult = ResultBuilder.createInfoResult(
-            "Logs exported to the connected member's file system: " + commandResult.nextLine());
-      }
-      return commandResult;
-    }
-  }
-
-}

http://git-wip-us.apache.org/repos/asf/geode/blob/2bcba1da/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogsCommand.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogsCommand.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogsCommand.java
new file mode 100644
index 0000000..9dd64d7
--- /dev/null
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogsCommand.java
@@ -0,0 +1,192 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.geode.management.internal.cli.commands;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.geode.cache.Region;
+import org.apache.geode.distributed.DistributedMember;
+import org.apache.geode.internal.cache.GemFireCacheImpl;
+import org.apache.geode.internal.logging.LogService;
+import org.apache.geode.management.cli.CliMetaData;
+import org.apache.geode.management.cli.ConverterHint;
+import org.apache.geode.management.cli.Result;
+import org.apache.geode.management.internal.cli.AbstractCliAroundInterceptor;
+import org.apache.geode.management.internal.cli.CliUtil;
+import org.apache.geode.management.internal.cli.GfshParseResult;
+import org.apache.geode.management.internal.cli.functions.ExportLogsFunction;
+import org.apache.geode.management.internal.cli.i18n.CliStrings;
+import org.apache.geode.management.internal.cli.result.ResultBuilder;
+import org.apache.geode.management.internal.cli.shell.Gfsh;
+import org.apache.geode.management.internal.cli.util.ExportLogsCacheWriter;
+import org.apache.geode.management.internal.configuration.utils.ZipUtils;
+import org.apache.geode.management.internal.security.ResourceOperation;
+import org.apache.geode.security.ResourcePermission;
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.Logger;
+import org.springframework.shell.core.CommandMarker;
+import org.springframework.shell.core.annotation.CliCommand;
+import org.springframework.shell.core.annotation.CliOption;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.time.LocalDateTime;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+public class ExportLogsCommand implements CommandMarker {
+
+  private static final Logger logger = LogService.getLogger();
+
+  public static final String FORMAT = "yyyy/MM/dd/HH/mm/ss/SSS/z";
+  public static final String ONLY_DATE_FORMAT = "yyyy/MM/dd";
+
+  private GemFireCacheImpl cache;
+
+  public ExportLogsCommand() {
+    cache = GemFireCacheImpl.getInstance();
+  }
+
+  @CliCommand(value = CliStrings.EXPORT_LOGS, help = CliStrings.EXPORT_LOGS__HELP)
+  @CliMetaData(shellOnly = false, isFileDownloadOverHttp = true,
+      interceptor = "org.apache.geode.management.internal.cli.commands.ExportLogsInterceptor",
+      relatedTopic = {CliStrings.TOPIC_GEODE_SERVER, CliStrings.TOPIC_GEODE_DEBUG_UTIL})
+  @ResourceOperation(resource = ResourcePermission.Resource.CLUSTER,
+      operation = ResourcePermission.Operation.READ)
+  public Result exportLogs(
+      @CliOption(key = CliStrings.EXPORT_LOGS__DIR, help = CliStrings.EXPORT_LOGS__DIR__HELP,
+          mandatory = false) String dirName,
+      @CliOption(key = CliStrings.EXPORT_LOGS__GROUP,
+          unspecifiedDefaultValue = CliMetaData.ANNOTATION_NULL_VALUE,
+          optionContext = ConverterHint.MEMBERGROUP,
+          help = CliStrings.EXPORT_LOGS__GROUP__HELP) String[] groups,
+      @CliOption(key = CliStrings.EXPORT_LOGS__MEMBER,
+          unspecifiedDefaultValue = CliMetaData.ANNOTATION_NULL_VALUE,
+          optionContext = ConverterHint.ALL_MEMBER_IDNAME,
+          help = CliStrings.EXPORT_LOGS__MEMBER__HELP) String[] memberIds,
+      @CliOption(key = CliStrings.EXPORT_LOGS__LOGLEVEL,
+          unspecifiedDefaultValue = LogService.DEFAULT_LOG_LEVEL,
+          optionContext = ConverterHint.LOG_LEVEL,
+          help = CliStrings.EXPORT_LOGS__LOGLEVEL__HELP) String logLevel,
+      @CliOption(key = CliStrings.EXPORT_LOGS__UPTO_LOGLEVEL, unspecifiedDefaultValue = "false",
+          help = CliStrings.EXPORT_LOGS__UPTO_LOGLEVEL__HELP) boolean onlyLogLevel,
+      @CliOption(key = CliStrings.EXPORT_LOGS__MERGELOG, unspecifiedDefaultValue = "false",
+          help = CliStrings.EXPORT_LOGS__MERGELOG__HELP) boolean mergeLog,
+      @CliOption(key = CliStrings.EXPORT_LOGS__STARTTIME,
+          unspecifiedDefaultValue = CliMetaData.ANNOTATION_NULL_VALUE,
+          help = CliStrings.EXPORT_LOGS__STARTTIME__HELP) String start,
+      @CliOption(key = CliStrings.EXPORT_LOGS__ENDTIME,
+          unspecifiedDefaultValue = CliMetaData.ANNOTATION_NULL_VALUE,
+          help = CliStrings.EXPORT_LOGS__ENDTIME__HELP) String end,
+      @CliOption(key = CliStrings.EXPORT_LOGS__LOGSONLY, unspecifiedDefaultValue = "false",
+          specifiedDefaultValue = "true",
+          help = CliStrings.EXPORT_LOGS__LOGSONLY__HELP) boolean logsOnly,
+      @CliOption(key = CliStrings.EXPORT_LOGS__STATSONLY, unspecifiedDefaultValue = "false",
+          specifiedDefaultValue = "true",
+          help = CliStrings.EXPORT_LOGS__STATSONLY__HELP) boolean statsOnly) {
+    Result result = null;
+    GemFireCacheImpl cache = this.cache;
+    if (cache == null || cache.isClosed()) {
+      new Exception("KEN: cache is null").printStackTrace();
+      cache = GemFireCacheImpl.getInstance();
+    } else {
+      new Exception("KEN: cache has been created").printStackTrace();
+    }
+    try {
+      Set<DistributedMember> targetMembers =
+          CliUtil.findMembersIncludingLocators(groups, memberIds);
+
+      if (targetMembers.isEmpty()) {
+        return ResultBuilder.createUserErrorResult(CliStrings.NO_MEMBERS_FOUND_MESSAGE);
+      }
+
+      Map<String, Path> zipFilesFromMembers = new HashMap<>();
+      for (DistributedMember server : targetMembers) {
+        Region region = ExportLogsFunction.createOrGetExistingExportLogsRegion(true, cache);
+
+        ExportLogsCacheWriter cacheWriter =
+            (ExportLogsCacheWriter) region.getAttributes().getCacheWriter();
+
+        cacheWriter.startFile(server.getName());
+
+        CliUtil.executeFunction(new ExportLogsFunction(),
+            new ExportLogsFunction.Args(start, end, logLevel, onlyLogLevel, logsOnly, statsOnly),
+            server).getResult();
+        Path zipFile = cacheWriter.endFile();
+        ExportLogsFunction.destroyExportLogsRegion(cache);
+
+        // only put the zipfile in the map if it is not null
+        if (zipFile != null) {
+          logger.info("Received zip file from member {}: {}", server.getId(), zipFile);
+          zipFilesFromMembers.put(server.getId(), zipFile);
+        }
+      }
+
+      if (zipFilesFromMembers.isEmpty()) {
+        return ResultBuilder.createUserErrorResult("No files to be exported.");
+      }
+
+      Path tempDir = Files.createTempDirectory("exportedLogs");
+      // make sure the directory is created, so that even if there is no files unzipped to this dir,
+      // we can
+      // still zip it and send an empty zip file back to the client
+      Path exportedLogsDir = tempDir.resolve("exportedLogs");
+      FileUtils.forceMkdir(exportedLogsDir.toFile());
+
+      for (Path zipFile : zipFilesFromMembers.values()) {
+        Path unzippedMemberDir =
+            exportedLogsDir.resolve(zipFile.getFileName().toString().replace(".zip", ""));
+        ZipUtils.unzip(zipFile.toAbsolutePath().toString(), unzippedMemberDir.toString());
+        FileUtils.deleteQuietly(zipFile.toFile());
+      }
+
+      Path workingDir = Paths.get(System.getProperty("user.dir"));
+      Path exportedLogsZipFile = workingDir
+          .resolve("exportedLogs_" + System.currentTimeMillis() + ".zip").toAbsolutePath();
+
+      logger.info("Zipping into: " + exportedLogsZipFile.toString());
+      ZipUtils.zipDirectory(exportedLogsDir, exportedLogsZipFile);
+      FileUtils.deleteDirectory(tempDir.toFile());
+
+      // TODO:GEODE-2420: warn user if exportedLogsZipFile size > threshold
+      if (isOverDiskSpaceThreshold()) {
+        // append warning to exportedLogsZipFile.toString()??
+      }
+
+      result = ResultBuilder.createInfoResult(exportedLogsZipFile.toString());
+    } catch (Exception ex) {
+      logger.error(ex, ex);
+      result = ResultBuilder.createGemFireErrorResult(ex.getMessage());
+    } finally {
+      ExportLogsFunction.destroyExportLogsRegion(cache);
+    }
+    logger.debug("Exporting logs returning = {}", result);
+    return result;
+  }
+
+  protected static Gfsh getGfsh() {
+    return Gfsh.getCurrentInstance();
+  }
+
+  boolean isOverDiskSpaceThreshold() {
+    return false;
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/2bcba1da/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogsInterceptor.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogsInterceptor.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogsInterceptor.java
new file mode 100644
index 0000000..8427c83
--- /dev/null
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogsInterceptor.java
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.management.internal.cli.commands;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.geode.internal.logging.LogService;
+import org.apache.geode.management.cli.Result;
+import org.apache.geode.management.internal.cli.AbstractCliAroundInterceptor;
+import org.apache.geode.management.internal.cli.GfshParseResult;
+import org.apache.geode.management.internal.cli.functions.ExportLogsFunction;
+import org.apache.geode.management.internal.cli.result.ResultBuilder;
+import org.apache.logging.log4j.Level;
+import org.apache.logging.log4j.Logger;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.time.LocalDateTime;
+import java.util.Map;
+
+/**
+ * after the export logs, will need to copy the tempFile to the desired location and delete the
+ * temp file.
+ */
+public class ExportLogsInterceptor extends AbstractCliAroundInterceptor {
+  private static final Logger logger = LogService.getLogger();
+
+  @Override
+  public Result preExecution(GfshParseResult parseResult) {
+    // the arguments are in the order of it's being declared
+    Map<String, String> arguments = parseResult.getParamValueStrings();
+
+    // validates groupId and memberIds not both set
+    if (arguments.get("group") != null && arguments.get("member") != null) {
+      return ResultBuilder.createUserErrorResult("Can't specify both group and member.");
+    }
+
+    // validate log level
+    String logLevel = arguments.get("log-level");
+    if (StringUtils.isBlank(logLevel) || Level.getLevel(logLevel.toUpperCase()) == null) {
+      return ResultBuilder.createUserErrorResult("Invalid log level: " + logLevel);
+    }
+
+    // validate start date and end date
+    String start = arguments.get("start-time");
+    String end = arguments.get("end-time");
+    if (start != null && end != null) {
+      // need to make sure end is later than start
+      LocalDateTime startTime = ExportLogsFunction.parseTime(start);
+      LocalDateTime endTime = ExportLogsFunction.parseTime(end);
+      if (startTime.isAfter(endTime)) {
+        return ResultBuilder.createUserErrorResult("start-time has to be earlier than end-time.");
+      }
+    }
+
+    // validate onlyLogs and onlyStats
+    boolean onlyLogs = Boolean.parseBoolean(arguments.get("logs-only"));
+    boolean onlyStats = Boolean.parseBoolean(arguments.get("stats-only"));
+    if (onlyLogs && onlyStats) {
+      return ResultBuilder.createUserErrorResult("logs-only and stats-only can't both be true");
+    }
+
+    return ResultBuilder.createInfoResult("");
+  }
+
+  @Override
+  public Result postExecution(GfshParseResult parseResult, Result commandResult, Path tempFile) {
+    // in the command over http case, the command result is in the downloaded temp file
+    if (tempFile != null) {
+      Path dirPath;
+      String dirName = parseResult.getParamValueStrings().get("dir");
+      if (StringUtils.isBlank(dirName)) {
+        dirPath = Paths.get(System.getProperty("user.dir"));
+      } else {
+        dirPath = Paths.get(dirName);
+      }
+      String fileName = "exportedLogs_" + System.currentTimeMillis() + ".zip";
+      File exportedLogFile = dirPath.resolve(fileName).toFile();
+      try {
+        FileUtils.copyFile(tempFile.toFile(), exportedLogFile);
+        FileUtils.deleteQuietly(tempFile.toFile());
+        commandResult = ResultBuilder
+            .createInfoResult("Logs exported to: " + exportedLogFile.getAbsolutePath());
+      } catch (IOException e) {
+        logger.error(e.getMessage(), e);
+        commandResult = ResultBuilder.createGemFireErrorResult(e.getMessage());
+      }
+    } else if (commandResult.getStatus() == Result.Status.OK) {
+      commandResult = ResultBuilder.createInfoResult(
+          "Logs exported to the connected member's file system: " + commandResult.nextLine());
+    }
+    return commandResult;
+  }
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/2bcba1da/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunction.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunction.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunction.java
index 62026fb..516a9a7 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunction.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunction.java
@@ -28,7 +28,7 @@ import org.apache.geode.internal.InternalEntity;
 import org.apache.geode.internal.cache.GemFireCacheImpl;
 import org.apache.geode.internal.cache.InternalRegionArguments;
 import org.apache.geode.internal.logging.LogService;
-import org.apache.geode.management.internal.cli.commands.ExportLogCommand;
+import org.apache.geode.management.internal.cli.commands.ExportLogsCommand;
 import org.apache.geode.management.internal.cli.util.ExportLogsCacheWriter;
 import org.apache.geode.management.internal.cli.util.LogExporter;
 import org.apache.geode.management.internal.cli.util.LogFilter;
@@ -207,11 +207,11 @@ public class ExportLogsFunction implements Function, InternalEntity {
     }
 
     try {
-      SimpleDateFormat df = new SimpleDateFormat(ExportLogCommand.FORMAT);
+      SimpleDateFormat df = new SimpleDateFormat(ExportLogsCommand.FORMAT);
       return df.parse(dateString).toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();
     } catch (ParseException e) {
       try {
-        SimpleDateFormat df = new SimpleDateFormat(ExportLogCommand.ONLY_DATE_FORMAT);
+        SimpleDateFormat df = new SimpleDateFormat(ExportLogsCommand.ONLY_DATE_FORMAT);
         return df.parse(dateString).toInstant().atZone(ZoneId.systemDefault()).toLocalDateTime();
       } catch (ParseException e1) {
         return null;

http://git-wip-us.apache.org/repos/asf/geode/blob/2bcba1da/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsCommandTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsCommandTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsCommandTest.java
new file mode 100644
index 0000000..21843f6
--- /dev/null
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsCommandTest.java
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.management.internal.cli.commands;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.Mockito.*;
+
+import org.apache.geode.cache.Cache;
+import org.apache.geode.test.junit.categories.UnitTest;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.mockito.InjectMocks;
+import org.mockito.Mock;
+
+@Category(UnitTest.class)
+public class ExportLogsCommandTest {
+
+  @Mock
+  private Cache cache;
+
+  @Before
+  public void before() throws Exception {
+    // customize behavior
+    this.cache = mock(Cache.class);
+  }
+
+  @Test
+  public void noLogsIsNotOverThreshold() throws Exception {
+//    ExportLogsCommand command = new ExportLogsCommand(this.cache);
+//
+//    assertThat(command.isOverDiskSpaceThreshold()).isFalse();
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/2bcba1da/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsDUnitTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsDUnitTest.java
index 95edd42..d0180d0 100644
--- a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsDUnitTest.java
@@ -19,8 +19,8 @@ package org.apache.geode.management.internal.cli.commands;
 import static java.util.stream.Collectors.joining;
 import static java.util.stream.Collectors.toList;
 import static java.util.stream.Collectors.toSet;
-import static org.apache.geode.management.internal.cli.commands.ExportLogCommand.FORMAT;
-import static org.apache.geode.management.internal.cli.commands.ExportLogCommand.ONLY_DATE_FORMAT;
+import static org.apache.geode.management.internal.cli.commands.ExportLogsCommand.FORMAT;
+import static org.apache.geode.management.internal.cli.commands.ExportLogsCommand.ONLY_DATE_FORMAT;
 import static org.assertj.core.api.Assertions.assertThat;
 
 import org.apache.commons.io.FileUtils;

http://git-wip-us.apache.org/repos/asf/geode/blob/2bcba1da/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsInterceptorJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsInterceptorJUnitTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsInterceptorJUnitTest.java
index 97ed686..9709aa6 100644
--- a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsInterceptorJUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsInterceptorJUnitTest.java
@@ -31,14 +31,14 @@ import java.util.Map;
 
 @Category(UnitTest.class)
 public class ExportLogsInterceptorJUnitTest {
-  private ExportLogCommand.ExportLogsInterceptor interceptor;
+  private ExportLogsInterceptor interceptor;
   private GfshParseResult parseResult;
   private Map<String, String> arguments;
   private Result result;
 
   @Before
   public void before() {
-    interceptor = new ExportLogCommand.ExportLogsInterceptor();
+    interceptor = new ExportLogsInterceptor();
     parseResult = Mockito.mock(GfshParseResult.class);
     arguments = new HashMap<>();
     arguments.put("log-level", "info");

http://git-wip-us.apache.org/repos/asf/geode/blob/2bcba1da/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsDUnitTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsDUnitTest.java
index de2ea64..92d0268 100644
--- a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsDUnitTest.java
@@ -19,7 +19,7 @@ import static org.apache.geode.distributed.ConfigurationProperties.HTTP_SERVICE_
 import static org.apache.geode.distributed.ConfigurationProperties.HTTP_SERVICE_PORT;
 import static org.apache.geode.distributed.ConfigurationProperties.JMX_MANAGER_PORT;
 import static org.apache.geode.distributed.ConfigurationProperties.STATISTIC_ARCHIVE_FILE;
-import static org.apache.geode.management.internal.cli.commands.ExportLogCommand.ONLY_DATE_FORMAT;
+import static org.apache.geode.management.internal.cli.commands.ExportLogsCommand.ONLY_DATE_FORMAT;
 import static org.assertj.core.api.Assertions.assertThat;
 
 import com.google.common.collect.Sets;


[12/49] geode git commit: GEODE-2595: Fixing failure from LuceneIndexDestroyDUnitTest

Posted by kh...@apache.org.
GEODE-2595: Fixing failure from LuceneIndexDestroyDUnitTest


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/3f2de087
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/3f2de087
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/3f2de087

Branch: refs/heads/feature/GEODE-2420
Commit: 3f2de0877e6f29478bc27e91ec47edf2ccab2f13
Parents: 25fc337
Author: Dan Smith <up...@apache.org>
Authored: Fri Mar 10 11:15:14 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/3f2de087/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
index cf79248..037c99f 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
@@ -331,7 +331,7 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
   private SerializableRunnableIF createIndex(String indexName, String field) {
     return () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndexFactory().setFields(field).create(INDEX_NAME, REGION_NAME);
+      luceneService.createIndexFactory().setFields(field).create(indexName, REGION_NAME);
     };
   }
 


[42/49] geode git commit: GEODE-2522: Added missing teardown method in test

Posted by kh...@apache.org.
GEODE-2522: Added missing teardown method in test


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/1e806c69
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/1e806c69
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/1e806c69

Branch: refs/heads/feature/GEODE-2420
Commit: 1e806c69c0060faa5bcbbb53511d12774421d71b
Parents: eeb01bb
Author: Jason Huynh <hu...@gmail.com>
Authored: Tue Mar 14 11:23:44 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java  | 7 +++++++
 1 file changed, 7 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/1e806c69/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java
index 7406162..f866e51 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/RebalanceWithRedundancyDUnitTest.java
@@ -36,6 +36,7 @@ import org.apache.geode.test.junit.categories.DistributedTest;
 
 import org.awaitility.Awaitility;
 
+import org.junit.After;
 import org.junit.Test;
 import org.junit.experimental.categories.Category;
 import org.junit.runner.RunWith;
@@ -47,6 +48,12 @@ import junitparams.Parameters;
 @RunWith(JUnitParamsRunner.class)
 public class RebalanceWithRedundancyDUnitTest extends LuceneQueriesAccessorBase {
 
+  @After
+  public void cleanupRebalanceCallback() {
+    removeCallback(dataStore1);
+    removeCallback(dataStore2);
+  }
+
   @Override
   protected RegionTestableType[] getListOfRegionTestTypes() {
     return new RegionTestableType[] {RegionTestableType.PARTITION_REDUNDANT,


[02/49] geode git commit: GEODE-2593: add port range to AvailablePortHelper to fix testUDPPortRange

Posted by kh...@apache.org.
GEODE-2593: add port range to AvailablePortHelper to fix testUDPPortRange


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/749a92f2
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/749a92f2
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/749a92f2

Branch: refs/heads/feature/GEODE-2420
Commit: 749a92f26c9a39e7f9823222fe0baef64b5120f2
Parents: 608ce11
Author: Kirk Lund <kl...@apache.org>
Authored: Fri Mar 3 17:36:07 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:06:58 2017 -0700

----------------------------------------------------------------------
 .../distributed/DistributedSystemDUnitTest.java | 79 +++++++++-----------
 1 file changed, 37 insertions(+), 42 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/749a92f2/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java b/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
index 9a64f53..8b669f3 100644
--- a/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
@@ -14,32 +14,9 @@
  */
 package org.apache.geode.distributed;
 
-import static java.lang.Integer.parseInt;
-import static java.net.NetworkInterface.getNetworkInterfaces;
 import static org.apache.geode.distributed.ConfigurationProperties.*;
-import static org.apache.geode.distributed.internal.DistributionConfig.*;
-import static org.apache.geode.distributed.internal.DistributionManager.*;
 import static org.apache.geode.internal.AvailablePort.*;
-import static org.apache.geode.internal.AvailablePortHelper.*;
-import static org.apache.geode.internal.net.SocketCreator.getLocalHost;
-import static org.apache.geode.test.dunit.DistributedTestUtils.getDUnitLocatorPort;
-import static org.apache.geode.test.dunit.Host.getHost;
-import static org.apache.geode.test.dunit.LogWriterUtils.getLogWriter;
-import static org.assertj.core.api.Assertions.*;
-
-import org.junit.After;
-import org.junit.Before;
-import org.junit.Test;
-import org.junit.experimental.categories.Category;
-
-import java.net.Inet4Address;
-import java.net.Inet6Address;
-import java.net.InetAddress;
-import java.net.NetworkInterface;
-import java.net.SocketException;
-import java.net.UnknownHostException;
-import java.util.Enumeration;
-import java.util.Properties;
+import static org.junit.Assert.*;
 
 import org.apache.geode.CancelException;
 import org.apache.geode.GemFireConfigException;
@@ -47,6 +24,9 @@ import org.apache.geode.cache.AttributesFactory;
 import org.apache.geode.cache.Cache;
 import org.apache.geode.cache.CacheFactory;
 import org.apache.geode.cache.Region;
+import org.apache.geode.cache30.CacheSerializableRunnable;
+import org.apache.geode.distributed.internal.DistributionConfig;
+import org.apache.geode.distributed.internal.DistributionException;
 import org.apache.geode.distributed.internal.DistributionManager;
 import org.apache.geode.distributed.internal.InternalDistributedSystem;
 import org.apache.geode.distributed.internal.SerialDistributionMessage;
@@ -55,33 +35,40 @@ import org.apache.geode.distributed.internal.membership.InternalDistributedMembe
 import org.apache.geode.distributed.internal.membership.gms.MembershipManagerHelper;
 import org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger;
 import org.apache.geode.distributed.internal.membership.gms.mgr.GMSMembershipManager;
+import org.apache.geode.internal.AvailablePort;
+import org.apache.geode.internal.AvailablePortHelper;
+import org.apache.geode.internal.net.SocketCreator;
+import org.apache.geode.test.dunit.DistributedTestUtils;
+import org.apache.geode.test.dunit.Host;
+import org.apache.geode.test.dunit.LogWriterUtils;
+import org.apache.geode.test.dunit.RMIException;
+import org.apache.geode.test.dunit.VM;
 import org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase;
 import org.apache.geode.test.junit.categories.DistributedTest;
 import org.apache.geode.test.junit.categories.MembershipTest;
 
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import java.net.Inet4Address;
+import java.net.Inet6Address;
+import java.net.InetAddress;
+import java.net.NetworkInterface;
+import java.util.Enumeration;
+import java.util.Properties;
+import java.util.concurrent.TimeoutException;
+
 /**
  * Tests the functionality of the {@link DistributedSystem} class.
  */
 @Category({DistributedTest.class, MembershipTest.class})
 public class DistributedSystemDUnitTest extends JUnit4DistributedTestCase {
 
-  private int mcastPort;
-  private int locatorPort;
-  private int tcpPort;
-  private int lowerBoundOfPortRange;
-  private int upperBoundOfPortRange;
-
   @Before
   public void before() throws Exception {
     disconnectAllFromDS();
-
-    this.mcastPort = getRandomAvailablePort(MULTICAST);
-    this.locatorPort = getRandomAvailablePort(SOCKET);
-    this.tcpPort = getRandomAvailablePort(SOCKET);
-
-    int[] portRange = getRandomAvailableTCPPortRange(3, true);
-    this.lowerBoundOfPortRange = portRange[0];
-    this.upperBoundOfPortRange = portRange[portRange.length - 1];
   }
 
   @After
@@ -248,14 +235,22 @@ public class DistributedSystemDUnitTest extends JUnit4DistributedTestCase {
 
   @Test
   public void testUDPPortRange() throws Exception {
+    int[] unicastPort = AvailablePortHelper.getRandomAvailableTCPPortRange(3, true);
+
     Properties config = new Properties();
-    config.put(LOCATORS, "localhost[" + getDUnitLocatorPort() + "]");
-    config.setProperty(MEMBERSHIP_PORT_RANGE,
-        this.lowerBoundOfPortRange + "-" + this.upperBoundOfPortRange);
+    config.put(LOCATORS, "localhost[" + DistributedTestUtils.getDUnitLocatorPort() + "]");
+    // Minimum 3 ports required in range for UDP, FD_SOCK and TcpConduit.
+    config.setProperty(MEMBERSHIP_PORT_RANGE, unicastPort[0] + "-" + unicastPort[2]);
 
     InternalDistributedSystem system = getSystem(config);
     DistributionManager dm = (DistributionManager) system.getDistributionManager();
-    InternalDistributedMember member = dm.getDistributionManagerId();
+    InternalDistributedMember idm = dm.getDistributionManagerId();
+
+    assertTrue(unicastPort[0] <= idm.getPort() && idm.getPort() <= unicastPort[2]);
+    assertTrue(unicastPort[0] <= idm.getPort() && idm.getDirectChannelPort() <= unicastPort[2]);
+
+    system.disconnect();
+  }
 
     verifyMembershipPortsInRange(member, this.lowerBoundOfPortRange, this.upperBoundOfPortRange);
   }


[47/49] geode git commit: GEODE-2633: When turning on fine logging, GEODE logs the keystore password in clear text

Posted by kh...@apache.org.
GEODE-2633: When turning on fine logging, GEODE logs the keystore password in clear text


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/c02970b4
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/c02970b4
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/c02970b4

Branch: refs/heads/feature/GEODE-2420
Commit: c02970b4fdb8f3238e9bec10f9b5692eed6006df
Parents: 22750cf
Author: Kevin J. Duling <kd...@pivotal.io>
Authored: Thu Mar 9 15:21:23 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../geode/internal/net/SocketCreator.java       | 59 ++++++++++----------
 .../geode/internal/util/ArgumentRedactor.java   |  9 +--
 2 files changed, 32 insertions(+), 36 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/c02970b4/geode-core/src/main/java/org/apache/geode/internal/net/SocketCreator.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/net/SocketCreator.java b/geode-core/src/main/java/org/apache/geode/internal/net/SocketCreator.java
index 742e7f3..7a8f3ad 100755
--- a/geode-core/src/main/java/org/apache/geode/internal/net/SocketCreator.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/net/SocketCreator.java
@@ -14,6 +14,32 @@
  */
 package org.apache.geode.internal.net;
 
+import org.apache.commons.lang.StringUtils;
+import org.apache.geode.GemFireConfigException;
+import org.apache.geode.SystemConnectException;
+import org.apache.geode.SystemFailure;
+import org.apache.geode.admin.internal.InetAddressUtil;
+import org.apache.geode.cache.wan.GatewaySender;
+import org.apache.geode.cache.wan.GatewayTransportFilter;
+import org.apache.geode.distributed.ClientSocketFactory;
+import org.apache.geode.distributed.internal.DistributionConfig;
+import org.apache.geode.distributed.internal.DistributionConfigImpl;
+import org.apache.geode.distributed.internal.InternalDistributedSystem;
+import org.apache.geode.internal.ClassPathLoader;
+import org.apache.geode.internal.ConnectionWatcher;
+import org.apache.geode.internal.GfeConsoleReaderFactory;
+import org.apache.geode.internal.GfeConsoleReaderFactory.GfeConsoleReader;
+import org.apache.geode.internal.admin.SSLConfig;
+import org.apache.geode.internal.cache.wan.TransportFilterServerSocket;
+import org.apache.geode.internal.cache.wan.TransportFilterSocketFactory;
+import org.apache.geode.internal.i18n.LocalizedStrings;
+import org.apache.geode.internal.logging.LogService;
+import org.apache.geode.internal.logging.log4j.LocalizedMessage;
+import org.apache.geode.internal.security.SecurableCommunicationChannel;
+import org.apache.geode.internal.util.ArgumentRedactor;
+import org.apache.geode.internal.util.PasswordUtil;
+import org.apache.logging.log4j.Logger;
+
 import java.io.FileInputStream;
 import java.io.IOException;
 import java.net.BindException;
@@ -71,32 +97,6 @@ import javax.net.ssl.TrustManager;
 import javax.net.ssl.TrustManagerFactory;
 import javax.net.ssl.X509ExtendedKeyManager;
 
-import org.apache.commons.lang.StringUtils;
-import org.apache.logging.log4j.Logger;
-
-import org.apache.geode.GemFireConfigException;
-import org.apache.geode.SystemConnectException;
-import org.apache.geode.SystemFailure;
-import org.apache.geode.admin.internal.InetAddressUtil;
-import org.apache.geode.cache.wan.GatewaySender;
-import org.apache.geode.cache.wan.GatewayTransportFilter;
-import org.apache.geode.distributed.ClientSocketFactory;
-import org.apache.geode.distributed.internal.DistributionConfig;
-import org.apache.geode.distributed.internal.DistributionConfigImpl;
-import org.apache.geode.distributed.internal.InternalDistributedSystem;
-import org.apache.geode.internal.ClassPathLoader;
-import org.apache.geode.internal.ConnectionWatcher;
-import org.apache.geode.internal.GfeConsoleReaderFactory;
-import org.apache.geode.internal.GfeConsoleReaderFactory.GfeConsoleReader;
-import org.apache.geode.internal.admin.SSLConfig;
-import org.apache.geode.internal.cache.wan.TransportFilterServerSocket;
-import org.apache.geode.internal.cache.wan.TransportFilterSocketFactory;
-import org.apache.geode.internal.i18n.LocalizedStrings;
-import org.apache.geode.internal.logging.LogService;
-import org.apache.geode.internal.logging.log4j.LocalizedMessage;
-import org.apache.geode.internal.security.SecurableCommunicationChannel;
-import org.apache.geode.internal.util.PasswordUtil;
-
 /**
  * Analyze configuration data (gemfire.properties) and configure sockets accordingly for SSL.
  * <p>
@@ -1126,13 +1126,14 @@ public class SocketCreator {
   private void printConfig() {
     if (!configShown && logger.isDebugEnabled()) {
       configShown = true;
-      StringBuffer sb = new StringBuffer();
+      StringBuilder sb = new StringBuilder();
       sb.append("SSL Configuration: \n");
-      sb.append("  ssl-enabled = " + this.sslConfig.isEnabled()).append("\n");
+      sb.append("  ssl-enabled = ").append(this.sslConfig.isEnabled()).append("\n");
       // add other options here....
       for (String key : System.getProperties().stringPropertyNames()) { // fix for 46822
         if (key.startsWith("javax.net.ssl")) {
-          sb.append("  ").append(key).append(" = ").append(System.getProperty(key)).append("\n");
+          String redactedString = ArgumentRedactor.redact(key, System.getProperty(key));
+          sb.append("  ").append(key).append(" = ").append(redactedString).append("\n");
         }
       }
       logger.debug(sb.toString());

http://git-wip-us.apache.org/repos/asf/geode/blob/c02970b4/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java b/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java
index 419f3f9..8873a52 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java
@@ -63,10 +63,10 @@ public class ArgumentRedactor {
 
   /**
    * Parse a string to find key=value pairs and redact the values if necessary. If more than one
-   * key=value pair exists in the input, each pair must be preceeded by a hyphen '-' to delineate
+   * key=value pair exists in the input, each pair must be preceded by a hyphen '-' to delineate
    * the pairs. <br>
    * Example:<br>
-   * Single value: "password=secret" or "--password=secret" Mulitple values: "-Dflag -Dkey=value
+   * Single value: "password=secret" or "--password=secret" Multiple values: "-Dflag -Dkey=value
    * --classpath=."
    * 
    * @param line The input to be parsed
@@ -145,10 +145,5 @@ public class ArgumentRedactor {
       compareKey = compareKey.substring(2);
     }
     return compareKey.toLowerCase().contains("password");
-    // return compareKey
-    // .startsWith(DistributionConfig.GEMFIRE_PREFIX + DistributionConfig.SECURITY_PREFIX_NAME)
-    // || compareKey.startsWith(
-    // DistributionConfigImpl.SECURITY_SYSTEM_PREFIX + DistributionConfig.SECURITY_PREFIX_NAME)
-    // || compareKey.toLowerCase().contains("password");
   }
 }


[03/49] geode git commit: GEODE-1198: refactor test with before/after and cleanup use of ports

Posted by kh...@apache.org.
GEODE-1198: refactor test with before/after and cleanup use of ports


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/8afd0d6e
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/8afd0d6e
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/8afd0d6e

Branch: refs/heads/feature/GEODE-2420
Commit: 8afd0d6ebe1455d930c87d830bda6f1c466404be
Parents: 749a92f
Author: Kirk Lund <kl...@apache.org>
Authored: Tue Mar 7 12:32:36 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:08:55 2017 -0700

----------------------------------------------------------------------
 .../distributed/DistributedSystemDUnitTest.java | 81 ++++++++++----------
 1 file changed, 42 insertions(+), 39 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/8afd0d6e/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java b/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
index 8b669f3..dfe4588 100644
--- a/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
@@ -14,9 +14,32 @@
  */
 package org.apache.geode.distributed;
 
+import static java.lang.Integer.parseInt;
+import static java.net.NetworkInterface.getNetworkInterfaces;
 import static org.apache.geode.distributed.ConfigurationProperties.*;
+import static org.apache.geode.distributed.internal.DistributionConfig.*;
+import static org.apache.geode.distributed.internal.DistributionManager.*;
 import static org.apache.geode.internal.AvailablePort.*;
-import static org.junit.Assert.*;
+import static org.apache.geode.internal.AvailablePortHelper.*;
+import static org.apache.geode.internal.net.SocketCreator.getLocalHost;
+import static org.apache.geode.test.dunit.DistributedTestUtils.getDUnitLocatorPort;
+import static org.apache.geode.test.dunit.Host.getHost;
+import static org.apache.geode.test.dunit.LogWriterUtils.getLogWriter;
+import static org.assertj.core.api.Assertions.*;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import java.net.Inet4Address;
+import java.net.Inet6Address;
+import java.net.InetAddress;
+import java.net.NetworkInterface;
+import java.net.SocketException;
+import java.net.UnknownHostException;
+import java.util.Enumeration;
+import java.util.Properties;
 
 import org.apache.geode.CancelException;
 import org.apache.geode.GemFireConfigException;
@@ -24,9 +47,6 @@ import org.apache.geode.cache.AttributesFactory;
 import org.apache.geode.cache.Cache;
 import org.apache.geode.cache.CacheFactory;
 import org.apache.geode.cache.Region;
-import org.apache.geode.cache30.CacheSerializableRunnable;
-import org.apache.geode.distributed.internal.DistributionConfig;
-import org.apache.geode.distributed.internal.DistributionException;
 import org.apache.geode.distributed.internal.DistributionManager;
 import org.apache.geode.distributed.internal.InternalDistributedSystem;
 import org.apache.geode.distributed.internal.SerialDistributionMessage;
@@ -35,40 +55,33 @@ import org.apache.geode.distributed.internal.membership.InternalDistributedMembe
 import org.apache.geode.distributed.internal.membership.gms.MembershipManagerHelper;
 import org.apache.geode.distributed.internal.membership.gms.messenger.JGroupsMessenger;
 import org.apache.geode.distributed.internal.membership.gms.mgr.GMSMembershipManager;
-import org.apache.geode.internal.AvailablePort;
-import org.apache.geode.internal.AvailablePortHelper;
-import org.apache.geode.internal.net.SocketCreator;
-import org.apache.geode.test.dunit.DistributedTestUtils;
-import org.apache.geode.test.dunit.Host;
-import org.apache.geode.test.dunit.LogWriterUtils;
-import org.apache.geode.test.dunit.RMIException;
-import org.apache.geode.test.dunit.VM;
 import org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase;
 import org.apache.geode.test.junit.categories.DistributedTest;
 import org.apache.geode.test.junit.categories.MembershipTest;
 
-import org.junit.After;
-import org.junit.Before;
-import org.junit.Test;
-import org.junit.experimental.categories.Category;
-
-import java.net.Inet4Address;
-import java.net.Inet6Address;
-import java.net.InetAddress;
-import java.net.NetworkInterface;
-import java.util.Enumeration;
-import java.util.Properties;
-import java.util.concurrent.TimeoutException;
-
 /**
  * Tests the functionality of the {@link DistributedSystem} class.
  */
 @Category({DistributedTest.class, MembershipTest.class})
 public class DistributedSystemDUnitTest extends JUnit4DistributedTestCase {
 
+  private int mcastPort;
+  private int locatorPort;
+  private int tcpPort;
+  private int lowerBoundOfPortRange;
+  private int upperBoundOfPortRange;
+
   @Before
   public void before() throws Exception {
     disconnectAllFromDS();
+
+    this.mcastPort = getRandomAvailablePort(MULTICAST);
+    this.locatorPort = getRandomAvailablePort(SOCKET);
+    this.tcpPort = getRandomAvailablePort(SOCKET);
+
+    int[] portRange = getRandomAvailableTCPPortRange(3, true);
+    this.lowerBoundOfPortRange = portRange[0];
+    this.upperBoundOfPortRange = portRange[portRange.length - 1];
   }
 
   @After
@@ -111,8 +124,6 @@ public class DistributedSystemDUnitTest extends JUnit4DistributedTestCase {
       }
     });
 
-    // TODO: assert that queue was created or this test is just broken
-
     assertThat(distributionManager.getMembershipManager().waitForDeparture(member))
         .as("expected the serial queue to be flushed").isTrue();
   }
@@ -235,22 +246,14 @@ public class DistributedSystemDUnitTest extends JUnit4DistributedTestCase {
 
   @Test
   public void testUDPPortRange() throws Exception {
-    int[] unicastPort = AvailablePortHelper.getRandomAvailableTCPPortRange(3, true);
-
     Properties config = new Properties();
-    config.put(LOCATORS, "localhost[" + DistributedTestUtils.getDUnitLocatorPort() + "]");
-    // Minimum 3 ports required in range for UDP, FD_SOCK and TcpConduit.
-    config.setProperty(MEMBERSHIP_PORT_RANGE, unicastPort[0] + "-" + unicastPort[2]);
+    config.put(LOCATORS, "localhost[" + getDUnitLocatorPort() + "]");
+    config.setProperty(MEMBERSHIP_PORT_RANGE,
+        this.lowerBoundOfPortRange + "-" + this.upperBoundOfPortRange);
 
     InternalDistributedSystem system = getSystem(config);
     DistributionManager dm = (DistributionManager) system.getDistributionManager();
-    InternalDistributedMember idm = dm.getDistributionManagerId();
-
-    assertTrue(unicastPort[0] <= idm.getPort() && idm.getPort() <= unicastPort[2]);
-    assertTrue(unicastPort[0] <= idm.getPort() && idm.getDirectChannelPort() <= unicastPort[2]);
-
-    system.disconnect();
-  }
+    InternalDistributedMember member = dm.getDistributionManagerId();
 
     verifyMembershipPortsInRange(member, this.lowerBoundOfPortRange, this.upperBoundOfPortRange);
   }


[34/49] geode git commit: GEODE-2674: Changing the lucene listener to fetch the value from the region

Posted by kh...@apache.org.
GEODE-2674: Changing the lucene listener to fetch the value from the region

Rather than use the value that is in the queue, use the latest value
from the region to update the lucene index.

This ensures that even if the queue contains spurious events due to
retries or other issues, we put the correct value in the index.

This also potentially saves memory and disk space for the queue, because
the queue does not need to hold the value for the entry.


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/94eaca7c
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/94eaca7c
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/94eaca7c

Branch: refs/heads/feature/GEODE-2420
Commit: 94eaca7c62d970f085e8457e39b7912885637c7d
Parents: e39aff2
Author: Dan Smith <up...@apache.org>
Authored: Wed Mar 15 13:23:20 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../lucene/internal/LuceneEventListener.java    | 23 ++++++------
 .../internal/LuceneEventSubstitutionFilter.java | 37 ++++++++++++++++++++
 .../cache/lucene/internal/LuceneIndexImpl.java  |  1 +
 .../internal/LuceneEventListenerJUnitTest.java  | 24 ++++++-------
 4 files changed, 63 insertions(+), 22 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/94eaca7c/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventListener.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventListener.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventListener.java
index 2943ce8..0f55533 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventListener.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventListener.java
@@ -20,6 +20,8 @@ import java.util.HashSet;
 import java.util.List;
 import java.util.Set;
 
+import org.apache.geode.cache.EntryDestroyedException;
+import org.apache.geode.cache.Region.Entry;
 import org.apache.geode.internal.cache.wan.parallel.ParallelGatewaySenderQueue;
 import org.apache.logging.log4j.Logger;
 import org.apache.geode.cache.CacheClosedException;
@@ -87,19 +89,20 @@ public class LuceneEventListener implements AsyncEventListener {
 
         IndexRepository repository = repositoryManager.getRepository(region, key, callbackArgument);
 
-        Operation op = event.getOperation();
+        final Entry entry = region.getEntry(key);
+        Object value;
+        try {
+          value = entry == null ? null : entry.getValue();
+        } catch (EntryDestroyedException e) {
+          value = null;
+        }
 
-        if (op.isCreate()) {
-          repository.update(key, event.getDeserializedValue());
-        } else if (op.isUpdate()) {
-          repository.update(key, event.getDeserializedValue());
-        } else if (op.isDestroy()) {
-          repository.delete(key);
-        } else if (op.isInvalidate()) {
-          repository.delete(key);
+        if (value != null) {
+          repository.update(key, value);
         } else {
-          throw new InternalGemFireError("Unhandled operation " + op + " on " + event.getRegion());
+          repository.delete(key);
         }
+
         affectedRepos.add(repository);
       }
 

http://git-wip-us.apache.org/repos/asf/geode/blob/94eaca7c/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventSubstitutionFilter.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventSubstitutionFilter.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventSubstitutionFilter.java
new file mode 100644
index 0000000..04a465a
--- /dev/null
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneEventSubstitutionFilter.java
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.geode.cache.lucene.internal;
+
+import org.apache.geode.cache.EntryEvent;
+import org.apache.geode.cache.wan.GatewayEventSubstitutionFilter;
+import org.apache.geode.internal.cache.Token;
+
+/**
+ * A substitution filter which throws away the value of the entry and replaces it with an empty
+ * string. For the lucene index, we will just fetch the new value from the region, so we don't need
+ * the value.
+ */
+public class LuceneEventSubstitutionFilter implements GatewayEventSubstitutionFilter {
+  @Override
+  public Object getSubstituteValue(final EntryEvent event) {
+    return "";
+  }
+
+  @Override
+  public void close() {
+
+  }
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/94eaca7c/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
index 0c9d220..66dc1f9 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
@@ -180,6 +180,7 @@ public abstract class LuceneIndexImpl implements InternalLuceneIndex {
       return null;
     }
     LuceneEventListener listener = new LuceneEventListener(repositoryManager);
+    factory.setGatewayEventSubstitutionListener(new LuceneEventSubstitutionFilter());
     String aeqId = LuceneServiceImpl.getUniqueIndexName(getName(), regionPath);
     AsyncEventQueue indexQueue = factory.create(aeqId, listener);
     return indexQueue;

http://git-wip-us.apache.org/repos/asf/geode/blob/94eaca7c/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneEventListenerJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneEventListenerJUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneEventListenerJUnitTest.java
index 0320068..79de29a 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneEventListenerJUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneEventListenerJUnitTest.java
@@ -27,6 +27,8 @@ import java.util.List;
 import java.util.concurrent.atomic.AtomicReference;
 
 import org.apache.geode.InternalGemFireError;
+import org.apache.geode.cache.Region.Entry;
+import org.apache.geode.internal.cache.RegionEntry;
 import org.junit.After;
 import org.junit.Before;
 import org.junit.Test;
@@ -80,18 +82,16 @@ public class LuceneEventListenerJUnitTest {
       Mockito.when(event.getKey()).thenReturn(i);
       Mockito.when(event.getCallbackArgument()).thenReturn(callback);
 
-      switch (i % 3) {
+      switch (i % 4) {
         case 0:
-          Mockito.when(event.getOperation()).thenReturn(Operation.CREATE);
-          Mockito.when(event.getDeserializedValue()).thenReturn(i);
-          break;
         case 1:
-          Mockito.when(event.getOperation()).thenReturn(Operation.UPDATE);
-          Mockito.when(event.getDeserializedValue()).thenReturn(i);
+          final Entry entry = mock(Entry.class);
+          when(entry.getValue()).thenReturn(i);
+          when(region.getEntry(eq(i))).thenReturn(entry);
           break;
         case 2:
-          Mockito.when(event.getOperation()).thenReturn(Operation.DESTROY);
-          Mockito.when(event.getDeserializedValue()).thenThrow(new AssertionError());
+        case 3:
+          // Do nothing, get value will return a destroy
           break;
       }
 
@@ -100,10 +100,10 @@ public class LuceneEventListenerJUnitTest {
 
     listener.processEvents(events);
 
-    verify(repo1, atLeast(numEntries / 6)).delete(any());
-    verify(repo1, atLeast(numEntries / 3)).update(any(), any());
-    verify(repo2, atLeast(numEntries / 6)).delete(any());
-    verify(repo2, atLeast(numEntries / 3)).update(any(), any());
+    verify(repo1, atLeast(numEntries / 4)).delete(any());
+    verify(repo1, atLeast(numEntries / 4)).update(any(), any());
+    verify(repo2, atLeast(numEntries / 4)).delete(any());
+    verify(repo2, atLeast(numEntries / 4)).update(any(), any());
     verify(repo1, times(1)).commit();
     verify(repo2, times(1)).commit();
   }


[33/49] geode git commit: GEODE-2548: Removing the test alternativelyCloseDataStoresAfterGettingAPageAndThenValidateTheContentsOfTheResults

Posted by kh...@apache.org.
GEODE-2548: Removing the test alternativelyCloseDataStoresAfterGettingAPageAndThenValidateTheContentsOfTheResults

	* Removed the test as it was shutting down one datastore before the other datastore could recover from it.
	* The second datastore just keeps waiting (hang) to recover but the other datastore was shut down


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/22750cf7
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/22750cf7
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/22750cf7

Branch: refs/heads/feature/GEODE-2420
Commit: 22750cf78ba6e4fc96d0d5fdd76cc6f1876dc64f
Parents: 755661a
Author: nabarun <nn...@pivotal.io>
Authored: Mon Mar 13 12:03:07 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../geode/cache/lucene/PaginationDUnitTest.java | 54 --------------------
 1 file changed, 54 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/22750cf7/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java
index ad6727e..11d5cd0 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/PaginationDUnitTest.java
@@ -138,60 +138,6 @@ public class PaginationDUnitTest extends LuceneQueriesAccessorBase {
     });
   }
 
-  @Test
-  @Ignore
-  @Parameters(method = "getListOfRegionTestTypes")
-  public void alternativelyCloseDataStoresAfterGettingAPageAndThenValidateTheContentsOfTheResults(
-      RegionTestableType regionTestType) {
-    SerializableRunnableIF createIndex = () -> {
-      LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
-    };
-
-    dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
-    dataStore2.invoke(() -> initDataStore(createIndex, regionTestType));
-    accessor.invoke(() -> initAccessor(createIndex, regionTestType));
-
-    putEntryInEachBucket();
-
-    assertTrue(waitForFlushBeforeExecuteTextSearch(dataStore1, FLUSH_WAIT_TIME_MS));
-
-    accessor.invoke(() -> {
-      List<LuceneResultStruct<Integer, TestObject>> combinedResult =
-          new ArrayList<LuceneResultStruct<Integer, TestObject>>();
-      Cache cache = getCache();
-      LuceneService service = LuceneServiceProvider.get(cache);
-      LuceneQuery<Integer, TestObject> query;
-      query = service.createLuceneQueryFactory().setResultLimit(1000).setPageSize(PAGE_SIZE)
-          .create(INDEX_NAME, REGION_NAME, "world", "text");
-      PageableLuceneQueryResults<Integer, TestObject> pages = query.findPages();
-      assertTrue(pages.hasNext());
-
-      dataStore1.invoke(() -> closeCache());
-      dataStore2.invoke(() -> closeCache());
-
-      for (int i = 0; i < (NUM_BUCKETS / PAGE_SIZE); i++) {
-        List<LuceneResultStruct<Integer, TestObject>> page;
-        if (i % 2 == 0) {
-          // Bring up dataStore2 and shutdown dataStore1
-          dataStore2.invoke(() -> initDataStore(createIndex, regionTestType));
-          assertTrue(waitForFlushBeforeExecuteTextSearch(dataStore2, FLUSH_WAIT_TIME_MS));
-          dataStore1.invoke(() -> closeCache());
-          page = pages.next();
-        } else {
-          // Bring up dataStore1 and shutdown dataStore2
-          dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
-          assertTrue(waitForFlushBeforeExecuteTextSearch(dataStore1, FLUSH_WAIT_TIME_MS));
-          dataStore2.invoke(() -> closeCache());
-          page = pages.next();
-        }
-        assertEquals(PAGE_SIZE, page.size());
-        combinedResult.addAll(page);
-      }
-      validateTheCombinedResult(combinedResult);
-    });
-  }
-
   private void validateTheCombinedResult(
       final List<LuceneResultStruct<Integer, TestObject>> combinedResult) {
     Map<Integer, TestObject> resultMap = combinedResult.stream()


[22/49] geode git commit: GEODE-2576: delete the zip file on locator after it's being streamed to client.

Posted by kh...@apache.org.
GEODE-2576: delete the zip file on locator after it's being streamed to client.


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/a4b32303
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/a4b32303
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/a4b32303

Branch: refs/heads/feature/GEODE-2420
Commit: a4b3230325cddf8ea431f5d2dbf249e475c0ddae
Parents: f4f9b6b
Author: Jinmei Liao <ji...@pivotal.io>
Authored: Wed Mar 8 16:41:34 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../web/controllers/ExportLogController.java    |  6 ++++-
 .../ExportLogsFunctionIntegrationTest.java      |  8 ++-----
 .../commands/ExportLogsOverHttpDUnitTest.java   |  5 ++++
 .../commands/QueryNamesOverHttpDUnitTest.java   | 24 ++++++++------------
 4 files changed, 21 insertions(+), 22 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/a4b32303/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java b/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java
index f3a5934..0351573 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java
@@ -15,6 +15,7 @@
 
 package org.apache.geode.management.internal.web.controllers;
 
+import org.apache.commons.io.FileUtils;
 import org.apache.geode.internal.lang.StringUtils;
 import org.apache.geode.management.internal.cli.i18n.CliStrings;
 import org.apache.geode.management.internal.cli.result.ResultBuilder;
@@ -91,11 +92,14 @@ public class ExportLogController extends AbstractCommandsController {
     String filePath = ResultBuilder.fromJson(result).nextLine().trim();
 
     HttpHeaders respHeaders = new HttpHeaders();
+    File zipFile = new File(filePath);
     try {
-      InputStreamResource isr = new InputStreamResource(new FileInputStream(new File(filePath)));
+      InputStreamResource isr = new InputStreamResource(new FileInputStream(zipFile));
       return new ResponseEntity<InputStreamResource>(isr, respHeaders, HttpStatus.OK);
     } catch (Exception ex) {
       throw new RuntimeException("IOError writing file to output stream", ex);
+    } finally {
+      FileUtils.deleteQuietly(zipFile);
     }
   }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/a4b32303/geode-core/src/test/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunctionIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunctionIntegrationTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunctionIntegrationTest.java
index 72c39dc..cdc5766 100644
--- a/geode-core/src/test/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunctionIntegrationTest.java
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/cli/functions/ExportLogsFunctionIntegrationTest.java
@@ -25,7 +25,6 @@ import org.apache.geode.cache.execute.FunctionContext;
 import org.apache.geode.cache.execute.ResultSender;
 import org.apache.geode.internal.cache.GemFireCacheImpl;
 import org.apache.geode.internal.cache.execute.FunctionContextImpl;
-import org.apache.geode.test.dunit.rules.Server;
 import org.apache.geode.test.dunit.rules.ServerStarterRule;
 import org.apache.geode.test.junit.categories.IntegrationTest;
 import org.apache.logging.log4j.Level;
@@ -41,15 +40,12 @@ import java.io.IOException;
 public class ExportLogsFunctionIntegrationTest {
 
   @Rule
-  public ServerStarterRule serverStarterRule = new ServerStarterRule();
-
-  private Server server;
+  public ServerStarterRule serverStarterRule = new ServerStarterRule().startServer();
   private File serverWorkingDir;
 
   @Before
   public void setup() throws Exception {
-    server = serverStarterRule.startServer();
-    serverWorkingDir = server.getWorkingDir();
+    serverWorkingDir = serverStarterRule.getWorkingDir();
   }
 
   @Test

http://git-wip-us.apache.org/repos/asf/geode/blob/a4b32303/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpDUnitTest.java b/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpDUnitTest.java
index 8c9442a..cc4ae28 100644
--- a/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpDUnitTest.java
+++ b/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpDUnitTest.java
@@ -25,6 +25,7 @@ import org.junit.experimental.categories.Category;
 import org.junit.rules.TemporaryFolder;
 
 import java.io.File;
+import java.util.Arrays;
 import java.util.Set;
 import java.util.stream.Collectors;
 import java.util.zip.ZipEntry;
@@ -58,6 +59,10 @@ public class ExportLogsOverHttpDUnitTest extends ExportStatsDUnitTest {
         new ZipFile(zipPath).stream().map(ZipEntry::getName).collect(Collectors.toSet());
 
     assertThat(actualZipEntries).isEqualTo(expectedZipEntries);
+
+    // also verify that the zip file on locator is deleted
+    assertThat(Arrays.stream(locator.getWorkingDir().listFiles())
+        .filter(file -> file.getName().endsWith(".zip")).collect(Collectors.toSet())).isEmpty();
   }
 
   protected String getZipPathFromCommandResult(String message) {

http://git-wip-us.apache.org/repos/asf/geode/blob/a4b32303/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/QueryNamesOverHttpDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/QueryNamesOverHttpDUnitTest.java b/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/QueryNamesOverHttpDUnitTest.java
index 4ec9c0d..22ca6e3 100644
--- a/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/QueryNamesOverHttpDUnitTest.java
+++ b/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/QueryNamesOverHttpDUnitTest.java
@@ -29,7 +29,6 @@ import org.apache.geode.management.internal.web.http.HttpMethod;
 import org.apache.geode.management.internal.web.shell.RestHttpOperationInvoker;
 import org.apache.geode.test.dunit.rules.LocatorStarterRule;
 import org.apache.geode.test.junit.categories.IntegrationTest;
-import org.junit.Before;
 import org.junit.Rule;
 import org.junit.Test;
 import org.junit.experimental.categories.Category;
@@ -48,21 +47,16 @@ public class QueryNamesOverHttpDUnitTest {
   protected static int jmxPort = ports[0];
   protected static int httpPort = ports[1];
 
-  @Rule
-  public LocatorStarterRule locatorRule = new LocatorStarterRule();
-
-  @Before
-  public void before() throws Exception {
-    Properties locatorProps = new Properties() {
-      {
-        setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
-        setProperty(HTTP_SERVICE_PORT, httpPort + "");
-        setProperty(JMX_MANAGER_PORT, jmxPort + "");
-      }
-    };
-    locatorRule.startLocator(locatorProps);
-  }
+  protected static Properties locatorProps = new Properties() {
+    {
+      setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
+      setProperty(HTTP_SERVICE_PORT, httpPort + "");
+      setProperty(JMX_MANAGER_PORT, jmxPort + "");
+    }
+  };
 
+  @Rule
+  public LocatorStarterRule locatorRule = new LocatorStarterRule().startLocator(locatorProps);
 
   @Test
   public void testQueryNameOverHttp() throws Exception {


[09/49] geode git commit: GEODE-2618: PrimaryBucketExceptions handled in LuceneQueryFunction.execute

Posted by kh...@apache.org.
GEODE-2618: PrimaryBucketExceptions handled in LuceneQueryFunction.execute

	* PrimaryBucketException while executing a LuceneQuery will now be caught and wrapped as a InternalFunctionInvocationTargetException
	* This will trigger a re-execution of the LuceneQuery on the member which will have the primary bucket.

	This closes #416


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/88fa1aff
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/88fa1aff
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/88fa1aff

Branch: refs/heads/feature/GEODE-2420
Commit: 88fa1aff274aad73b505093a6d172439424c818b
Parents: de724d3
Author: nabarun <nn...@pivotal.io>
Authored: Tue Mar 7 14:55:54 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../cache/lucene/internal/distributed/LuceneQueryFunction.java   | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/88fa1aff/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/distributed/LuceneQueryFunction.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/distributed/LuceneQueryFunction.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/distributed/LuceneQueryFunction.java
index e3b5297..428301f 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/distributed/LuceneQueryFunction.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/distributed/LuceneQueryFunction.java
@@ -23,6 +23,7 @@ import org.apache.geode.cache.CacheClosedException;
 import org.apache.geode.cache.execute.Function;
 import org.apache.geode.cache.lucene.internal.LuceneIndexImpl;
 import org.apache.geode.cache.lucene.internal.LuceneIndexStats;
+import org.apache.geode.internal.cache.PrimaryBucketException;
 import org.apache.geode.internal.cache.execute.InternalFunctionInvocationTargetException;
 import org.apache.logging.log4j.Logger;
 import org.apache.lucene.search.Query;
@@ -112,7 +113,8 @@ public class LuceneQueryFunction implements Function, InternalEntity {
       }
       stats.incNumberOfQueryExecuted();
       resultSender.lastResult(mergedResult);
-    } catch (IOException | BucketNotFoundException | CacheClosedException e) {
+    } catch (IOException | BucketNotFoundException | CacheClosedException
+        | PrimaryBucketException e) {
       logger.debug("Exception during lucene query function", e);
       throw new InternalFunctionInvocationTargetException(e);
     }


[20/49] geode git commit: GEODE-2612: Added option to invoke callbacks during snapshot

Posted by kh...@apache.org.
GEODE-2612: Added option to invoke callbacks during snapshot


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/8c371be8
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/8c371be8
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/8c371be8

Branch: refs/heads/feature/GEODE-2420
Commit: 8c371be82ba28074c06e8df9268c207017600199
Parents: aa9ced5
Author: Barry Oglesby <bo...@pivotal.io>
Authored: Wed Mar 8 14:35:23 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../geode/cache/snapshot/SnapshotOptions.java   |  16 ++
 .../snapshot/RegionSnapshotServiceImpl.java     |   4 +-
 .../cache/snapshot/SnapshotOptionsImpl.java     |  15 ++
 .../geode/cache/snapshot/SnapshotDUnitTest.java | 208 +++++++++++++++++--
 .../codeAnalysis/sanctionedSerializables.txt    |   2 +-
 .../LuceneIndexForPartitionedRegion.java        |   2 +-
 .../cache/lucene/internal/LuceneIndexImpl.java  |  10 +-
 .../lucene/LuceneIndexDestroyDUnitTest.java     | 201 ++++++++++++++++--
 .../geode/cache/lucene/test/TestObject.java     |   4 +
 9 files changed, 424 insertions(+), 38 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/8c371be8/geode-core/src/main/java/org/apache/geode/cache/snapshot/SnapshotOptions.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/cache/snapshot/SnapshotOptions.java b/geode-core/src/main/java/org/apache/geode/cache/snapshot/SnapshotOptions.java
index 1562b3a..4c819b8 100644
--- a/geode-core/src/main/java/org/apache/geode/cache/snapshot/SnapshotOptions.java
+++ b/geode-core/src/main/java/org/apache/geode/cache/snapshot/SnapshotOptions.java
@@ -55,4 +55,20 @@ public interface SnapshotOptions<K, V> extends Serializable {
    * @return the filter, or null if the filter is not set
    */
   SnapshotFilter<K, V> getFilter();
+
+  /**
+   * Sets whether to invoke callbacks when loading a snapshot. The default is false.
+   *
+   * @param invokeCallbacks
+   *
+   * @return the snapshot options
+   */
+  SnapshotOptions<K, V> invokeCallbacks(boolean invokeCallbacks);
+
+  /**
+   * Returns whether loading a snapshot causes callbacks to be invoked
+   *
+   * @return whether loading a snapshot causes callbacks to be invoked
+   */
+  boolean shouldInvokeCallbacks();
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/8c371be8/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/RegionSnapshotServiceImpl.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/RegionSnapshotServiceImpl.java b/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/RegionSnapshotServiceImpl.java
index e8b28c4..bfdef05 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/RegionSnapshotServiceImpl.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/RegionSnapshotServiceImpl.java
@@ -257,7 +257,7 @@ public class RegionSnapshotServiceImpl<K, V> implements RegionSnapshotService<K,
                 .getDistributionManager().getWaitingThreadPool().submit(new Runnable() {
                   @Override
                   public void run() {
-                    local.basicImportPutAll(copy, true);
+                    local.basicImportPutAll(copy, !options.shouldInvokeCallbacks());
                   }
                 });
 
@@ -270,7 +270,7 @@ public class RegionSnapshotServiceImpl<K, V> implements RegionSnapshotService<K,
 
       // send off any remaining entries
       if (!buffer.isEmpty()) {
-        local.basicImportPutAll(buffer, true);
+        local.basicImportPutAll(buffer, !options.shouldInvokeCallbacks());
       }
 
       // wait for completion and check for errors

http://git-wip-us.apache.org/repos/asf/geode/blob/8c371be8/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/SnapshotOptionsImpl.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/SnapshotOptionsImpl.java b/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/SnapshotOptionsImpl.java
index d56535f..9f4fcd8 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/SnapshotOptionsImpl.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/snapshot/SnapshotOptionsImpl.java
@@ -30,6 +30,9 @@ public class SnapshotOptionsImpl<K, V> implements SnapshotOptions<K, V> {
   /** the entry filter */
   private volatile SnapshotFilter<K, V> filter;
 
+  /** true if callbacks should be invoked on load */
+  private volatile boolean invokeCallbacks;
+
   /** true if parallel mode is enabled */
   private volatile boolean parallel;
 
@@ -38,6 +41,7 @@ public class SnapshotOptionsImpl<K, V> implements SnapshotOptions<K, V> {
 
   public SnapshotOptionsImpl() {
     filter = null;
+    invokeCallbacks = false;
   }
 
   @Override
@@ -51,6 +55,17 @@ public class SnapshotOptionsImpl<K, V> implements SnapshotOptions<K, V> {
     return filter;
   }
 
+  @Override
+  public SnapshotOptions<K, V> invokeCallbacks(boolean invokeCallbacks) {
+    this.invokeCallbacks = invokeCallbacks;
+    return this;
+  }
+
+  @Override
+  public boolean shouldInvokeCallbacks() {
+    return this.invokeCallbacks;
+  }
+
   /**
    * Enables parallel mode for snapshot operations. This will cause each member of a partitioned
    * region to save its local data set (ignoring redundant copies) to a separate snapshot file.

http://git-wip-us.apache.org/repos/asf/geode/blob/8c371be8/geode-core/src/test/java/org/apache/geode/cache/snapshot/SnapshotDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/cache/snapshot/SnapshotDUnitTest.java b/geode-core/src/test/java/org/apache/geode/cache/snapshot/SnapshotDUnitTest.java
index 1b6d426..196a88b 100644
--- a/geode-core/src/test/java/org/apache/geode/cache/snapshot/SnapshotDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/cache/snapshot/SnapshotDUnitTest.java
@@ -14,21 +14,29 @@
  */
 package org.apache.geode.cache.snapshot;
 
+import org.apache.geode.cache.AttributesMutator;
+import org.apache.geode.cache.DiskStoreFactory;
+import org.apache.geode.cache.asyncqueue.AsyncEvent;
+import org.apache.geode.cache.asyncqueue.AsyncEventListener;
+import org.apache.geode.cache.asyncqueue.AsyncEventQueue;
+import org.apache.geode.cache.asyncqueue.AsyncEventQueueFactory;
+import org.awaitility.Awaitility;
 import org.junit.experimental.categories.Category;
 import org.junit.Test;
 
 import static org.junit.Assert.*;
 
 import org.apache.geode.test.dunit.cache.internal.JUnit4CacheTestCase;
-import org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase;
 import org.apache.geode.test.junit.categories.DistributedTest;
 
 import java.io.File;
-import java.io.FilenameFilter;
+import java.util.ArrayList;
 import java.util.HashMap;
+import java.util.List;
 import java.util.Map;
 import java.util.Map.Entry;
-import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
 
 import com.examples.snapshot.MyObject;
 import com.examples.snapshot.MyPdxSerializer;
@@ -42,12 +50,14 @@ import org.apache.geode.cache.snapshot.RegionGenerator.SerializationType;
 import org.apache.geode.cache.snapshot.SnapshotOptions.SnapshotFormat;
 import org.apache.geode.cache.util.CacheListenerAdapter;
 import org.apache.geode.cache.util.CacheWriterAdapter;
-import org.apache.geode.cache30.CacheTestCase;
 import org.apache.geode.test.dunit.Host;
 import org.apache.geode.test.dunit.SerializableCallable;
 
 @Category(DistributedTest.class)
 public class SnapshotDUnitTest extends JUnit4CacheTestCase {
+
+  private static final int NUM_ENTRIES = 1000;
+
   public SnapshotDUnitTest() {
     super();
   }
@@ -60,17 +70,8 @@ public class SnapshotDUnitTest extends JUnit4CacheTestCase {
     // save all regions
     getCache().getSnapshotService().save(dir, SnapshotFormat.GEMFIRE);
 
-    for (final RegionType rt : RegionType.values()) {
-      for (final SerializationType st : SerializationType.values()) {
-        String name = "test-" + rt.name() + "-" + st.name();
-
-        // overwrite region with bad data
-        Region<Integer, MyObject> region = getCache().getRegion(name);
-        for (Entry<Integer, MyObject> entry : region.entrySet()) {
-          region.put(entry.getKey(), new MyObject(Integer.MAX_VALUE, "bad!!"));
-        }
-      }
-    }
+    // update regions with data to be overwritten by import
+    updateRegions();
 
     SerializableCallable callbacks = new SerializableCallable() {
       @Override
@@ -106,8 +107,83 @@ public class SnapshotDUnitTest extends JUnit4CacheTestCase {
     forEachVm(callbacks, true);
 
     // load all regions
+    loadRegions(dir, null);
+  }
+
+  @Test
+  public void testExportAndImportWithInvokeCallbacksEnabled() throws Exception {
+    File dir = new File(getDiskDirs()[0], "callbacks");
+    dir.mkdir();
+
+    // save all regions
+    CacheSnapshotService service = getCache().getSnapshotService();
+    service.save(dir, SnapshotFormat.GEMFIRE);
+
+    // update regions with data to be overwritten by import
+    updateRegions();
+
+    SerializableCallable callbacks = new SerializableCallable() {
+      @Override
+      public Object call() throws Exception {
+        for (final RegionType rt : RegionType.values()) {
+          for (final SerializationType st : SerializationType.values()) {
+            String name = "test-" + rt.name() + "-" + st.name();
+            Cache cache = getCache();
+            Region<Integer, MyObject> region = cache.getRegion(name);
+            // add CacheWriter and CacheListener
+            AttributesMutator mutator = region.getAttributesMutator();
+            mutator.setCacheWriter(new CountingCacheWriter());
+            mutator.addCacheListener(new CountingCacheListener());
+            // add AsyncEventQueue
+            addAsyncEventQueue(region, name);
+          }
+        }
+        return null;
+      }
+    };
+
+    // add callbacks
+    forEachVm(callbacks, true);
+
+    // load all regions with invoke callbacks enabled
+    SnapshotOptions options = service.createOptions();
+    options.invokeCallbacks(true);
+    loadRegions(dir, options);
+
+    // verify callbacks were invoked
+    verifyCallbacksInvoked();
+  }
+
+  private void addAsyncEventQueue(Region region, String name) {
+    DiskStoreFactory dsFactory = getCache().createDiskStoreFactory();
+    dsFactory.create(name);
+    AsyncEventQueueFactory aeqFactory = getCache().createAsyncEventQueueFactory();
+    aeqFactory.setDiskStoreName(name);
+    aeqFactory.create(name, new CountingAsyncEventListener());
+    region.getAttributesMutator().addAsyncEventQueueId(name);
+  }
+
+  private void updateRegions() {
+    for (final RegionType rt : RegionType.values()) {
+      for (final SerializationType st : SerializationType.values()) {
+        String name = "test-" + rt.name() + "-" + st.name();
+
+        // overwrite region with bad data
+        Region<Integer, MyObject> region = getCache().getRegion(name);
+        for (Entry<Integer, MyObject> entry : region.entrySet()) {
+          region.put(entry.getKey(), new MyObject(Integer.MAX_VALUE, "bad!!"));
+        }
+      }
+    }
+  }
+
+  private void loadRegions(File dir, SnapshotOptions options) throws Exception {
     RegionGenerator rgen = new RegionGenerator();
-    getCache().getSnapshotService().load(dir, SnapshotFormat.GEMFIRE);
+    if (options != null) {
+      getCache().getSnapshotService().load(dir.listFiles(), SnapshotFormat.GEMFIRE, options);
+    } else {
+      getCache().getSnapshotService().load(dir, SnapshotFormat.GEMFIRE);
+    }
     for (final RegionType rt : RegionType.values()) {
       for (final SerializationType st : SerializationType.values()) {
         Region<Integer, MyObject> region =
@@ -120,6 +196,51 @@ public class SnapshotDUnitTest extends JUnit4CacheTestCase {
     }
   }
 
+  private void verifyCallbacksInvoked() throws Exception {
+    for (final RegionType rt : RegionType.values()) {
+      for (final SerializationType st : SerializationType.values()) {
+        SerializableCallable counts = new SerializableCallable() {
+          @Override
+          public Object call() throws Exception {
+            String name = "test-" + rt.name() + "-" + st.name();
+            Region<Integer, MyObject> region = getCache().getRegion(name);
+            // get CacheWriter and CacheListener events
+            CountingCacheWriter writer =
+                (CountingCacheWriter) region.getAttributes().getCacheWriter();
+            CountingCacheListener listener =
+                (CountingCacheListener) region.getAttributes().getCacheListener();
+            // get AsyncEventListener events
+            int numAeqEvents = 0;
+            AsyncEventQueue aeq = getCache().getAsyncEventQueue(name);
+            CountingAsyncEventListener aeqListener =
+                (CountingAsyncEventListener) aeq.getAsyncEventListener();
+            if (aeq.isPrimary()) {
+              Awaitility.waitAtMost(60, TimeUnit.SECONDS)
+                  .until(() -> aeqListener.getEvents() == NUM_ENTRIES);
+              numAeqEvents = aeqListener.getEvents();
+            }
+            return new int[] {writer.getEvents(), listener.getEvents(), numAeqEvents};
+          }
+        };
+        Object result = forEachVm(counts, true);
+        int totalWriterUpdates = 0, totalListenerUpdates = 0, totalAeqEvents = 0;
+        List<int[]> list = (List) result;
+        for (int[] vmResult : list) {
+          totalWriterUpdates += vmResult[0];
+          totalListenerUpdates += vmResult[1];
+          totalAeqEvents += vmResult[2];
+        }
+        if (rt.name().contains("PARTITION")) {
+          assertEquals(NUM_ENTRIES, totalListenerUpdates);
+        } else {
+          assertEquals(NUM_ENTRIES * (Host.getHost(0).getVMCount() + 1), totalListenerUpdates);
+        }
+        assertEquals(NUM_ENTRIES, totalWriterUpdates);
+        assertEquals(NUM_ENTRIES, totalAeqEvents);
+      }
+    }
+  }
+
   @Test
   public void testCacheExportFilterException() throws Exception {
     SnapshotFilter<Object, Object> oops = new SnapshotFilter<Object, Object>() {
@@ -190,7 +311,7 @@ public class SnapshotDUnitTest extends JUnit4CacheTestCase {
   public static Map<Integer, MyObject> createExpected(SerializationType type,
       RegionGenerator rgen) {
     Map<Integer, MyObject> expected = new HashMap<Integer, MyObject>();
-    for (int i = 0; i < 1000; i++) {
+    for (int i = 0; i < NUM_ENTRIES; i++) {
       expected.put(i, rgen.createData(type, i, "The number is " + i));
     }
     return expected;
@@ -222,16 +343,63 @@ public class SnapshotDUnitTest extends JUnit4CacheTestCase {
   }
 
   public static Object forEachVm(SerializableCallable call, boolean local) throws Exception {
+    List result = new ArrayList();
     Host host = Host.getHost(0);
     int vms = host.getVMCount();
 
     for (int i = 0; i < vms; ++i) {
-      host.getVM(i).invoke(call);
+      result.add(host.getVM(i).invoke(call));
     }
 
     if (local) {
-      return call.call();
+      result.add(call.call());
     }
-    return null;
+    return result;
+  }
+
+  private static class CountingCacheListener extends CacheListenerAdapter<Integer, MyObject> {
+
+    private final AtomicInteger events = new AtomicInteger();
+
+    @Override
+    public void afterUpdate(EntryEvent<Integer, MyObject> event) {
+      events.incrementAndGet();
+    }
+
+    private int getEvents() {
+      return events.get();
+    }
+  }
+
+  private static class CountingCacheWriter extends CacheWriterAdapter<Integer, MyObject> {
+
+    private final AtomicInteger events = new AtomicInteger();
+
+    @Override
+    public void beforeUpdate(EntryEvent<Integer, MyObject> event) {
+      events.incrementAndGet();
+    }
+
+    private int getEvents() {
+      return events.get();
+    }
+  }
+
+  private static final class CountingAsyncEventListener implements AsyncEventListener {
+
+    private final AtomicInteger events = new AtomicInteger();
+
+    @Override
+    public boolean processEvents(final List<AsyncEvent> list) {
+      events.addAndGet(list.size());
+      return true;
+    }
+
+    private int getEvents() {
+      return events.get();
+    }
+
+    @Override
+    public void close() {}
   }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/8c371be8/geode-core/src/test/resources/org/apache/geode/codeAnalysis/sanctionedSerializables.txt
----------------------------------------------------------------------
diff --git a/geode-core/src/test/resources/org/apache/geode/codeAnalysis/sanctionedSerializables.txt b/geode-core/src/test/resources/org/apache/geode/codeAnalysis/sanctionedSerializables.txt
index a11c7ad..397ec14 100755
--- a/geode-core/src/test/resources/org/apache/geode/codeAnalysis/sanctionedSerializables.txt
+++ b/geode-core/src/test/resources/org/apache/geode/codeAnalysis/sanctionedSerializables.txt
@@ -335,7 +335,7 @@ org/apache/geode/internal/cache/snapshot/RegionSnapshotServiceImpl$1,true,1
 org/apache/geode/internal/cache/snapshot/RegionSnapshotServiceImpl$ParallelArgs,true,1,file:java/io/File,format:org/apache/geode/cache/snapshot/SnapshotOptions$SnapshotFormat,options:org/apache/geode/internal/cache/snapshot/SnapshotOptionsImpl
 org/apache/geode/internal/cache/snapshot/RegionSnapshotServiceImpl$ParallelExportFunction,false
 org/apache/geode/internal/cache/snapshot/RegionSnapshotServiceImpl$ParallelImportFunction,false
-org/apache/geode/internal/cache/snapshot/SnapshotOptionsImpl,true,1,filter:org/apache/geode/cache/snapshot/SnapshotFilter,mapper:org/apache/geode/internal/cache/snapshot/SnapshotFileMapper,parallel:boolean
+org/apache/geode/internal/cache/snapshot/SnapshotOptionsImpl,true,1,filter:org/apache/geode/cache/snapshot/SnapshotFilter,invokeCallbacks:boolean,mapper:org/apache/geode/internal/cache/snapshot/SnapshotFileMapper,parallel:boolean
 org/apache/geode/internal/cache/snapshot/WindowedExporter$WindowedArgs,true,1,exporter:org/apache/geode/distributed/DistributedMember,options:org/apache/geode/cache/snapshot/SnapshotOptions
 org/apache/geode/internal/cache/snapshot/WindowedExporter$WindowedExportFunction,true,1
 org/apache/geode/internal/cache/tier/BatchException,true,-6707074107791305564,_index:int

http://git-wip-us.apache.org/repos/asf/geode/blob/8c371be8/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
index f24c6d6..7274d6a 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
@@ -251,7 +251,7 @@ public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
   private void destroyOnRemoteMembers() {
     PartitionedRegion pr = (PartitionedRegion) getDataRegion();
     DM dm = pr.getDistributionManager();
-    Set<InternalDistributedMember> recipients = pr.getRegionAdvisor().adviseDataStore();
+    Set<InternalDistributedMember> recipients = pr.getRegionAdvisor().adviseAllPRNodes();
     if (!recipients.isEmpty()) {
       if (logger.isDebugEnabled()) {
         logger.debug("LuceneIndexForPartitionedRegion: About to send destroy message recipients="

http://git-wip-us.apache.org/repos/asf/geode/blob/8c371be8/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
index b5b13c1..0c9d220 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
@@ -244,7 +244,10 @@ public abstract class LuceneIndexImpl implements InternalLuceneIndex {
     AsyncEventQueueImpl aeq = (AsyncEventQueueImpl) cache.getAsyncEventQueue(aeqId);
 
     // Stop the AsyncEventQueue (this stops the AsyncEventQueue's underlying GatewaySender)
-    aeq.stop();
+    // The AsyncEventQueue can be null in an accessor member
+    if (aeq != null) {
+      aeq.stop();
+    }
 
     // Remove the id from the dataRegion's AsyncEventQueue ids
     // Note: The region may already have been destroyed by a remote member
@@ -254,7 +257,10 @@ public abstract class LuceneIndexImpl implements InternalLuceneIndex {
     }
 
     // Destroy the aeq (this also removes it from the GemFireCacheImpl)
-    aeq.destroy();
+    // The AsyncEventQueue can be null in an accessor member
+    if (aeq != null) {
+      aeq.destroy();
+    }
     if (logger.isDebugEnabled()) {
       logger.debug("Destroyed aeqId=" + aeqId);
     }

http://git-wip-us.apache.org/repos/asf/geode/blob/8c371be8/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
index 1afde6a..496cdeb 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
@@ -20,10 +20,13 @@ import org.apache.geode.cache.Region;
 import org.apache.geode.cache.lucene.internal.LuceneIndexForPartitionedRegion;
 import org.apache.geode.cache.lucene.internal.LuceneServiceImpl;
 import org.apache.geode.cache.lucene.test.TestObject;
+import org.apache.geode.cache.snapshot.RegionSnapshotService;
+import org.apache.geode.cache.snapshot.SnapshotOptions;
 import org.apache.geode.internal.cache.LocalRegion;
 import org.apache.geode.test.dunit.AsyncInvocation;
+import org.apache.geode.test.dunit.Host;
 import org.apache.geode.test.dunit.SerializableRunnableIF;
-import org.apache.geode.test.dunit.ThreadUtils;
+import org.apache.geode.test.dunit.VM;
 import org.apache.geode.test.junit.categories.DistributedTest;
 import org.awaitility.Awaitility;
 import org.junit.Ignore;
@@ -31,12 +34,14 @@ import org.junit.Test;
 import org.junit.experimental.categories.Category;
 import org.junit.runner.RunWith;
 
-import java.io.InterruptedIOException;
+import java.io.File;
+import java.util.Collection;
 import java.util.concurrent.TimeUnit;
 
 import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.INDEX_NAME;
 import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.REGION_NAME;
 import static org.apache.geode.internal.Assert.fail;
+import static org.junit.Assert.assertEquals;
 import static org.junit.Assert.assertFalse;
 import static org.junit.Assert.assertNotNull;
 import static org.junit.Assert.assertNull;
@@ -48,6 +53,14 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
 
   private volatile boolean STOP_PUTS = false;
 
+  protected VM accessor;
+
+  @Override
+  public void postSetUp() throws Exception {
+    super.postSetUp();
+    accessor = Host.getHost(0).getVM(3);
+  }
+
   private final Object[] parametersForIndexDestroys() {
     String[] destroyDataRegionParameters = {"true", "false"};
     RegionTestableType[] regionTestTypes = getListOfRegionTestTypes();
@@ -146,17 +159,31 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
 
   @Test
   @Parameters(method = "getListOfRegionTestTypes")
-  public void verifyDestroyRecreateSingleIndex(RegionTestableType regionType) {
+  public void verifyDestroyRecreateIndexSameName(RegionTestableType regionType) {
     // Create index and region
-    dataStore1.invoke(() -> initDataStore(createIndex(), regionType));
-    dataStore2.invoke(() -> initDataStore(createIndex(), regionType));
+    SerializableRunnableIF createIndex = createIndex();
+    dataStore1.invoke(() -> initDataStore(createIndex, regionType));
+    dataStore2.invoke(() -> initDataStore(createIndex, regionType));
+    accessor.invoke(() -> initAccessor(createIndex, regionType));
 
     // Verify index created
     dataStore1.invoke(() -> verifyIndexCreated());
     dataStore2.invoke(() -> verifyIndexCreated());
+    accessor.invoke(() -> verifyIndexCreated());
 
     // Do puts to cause IndexRepositories to be created
-    dataStore1.invoke(() -> doPuts(10));
+    int numPuts = 10;
+    accessor.invoke(() -> doPuts(numPuts));
+
+    // Wait until queue is flushed
+    dataStore1.invoke(() -> waitUntilFlushed(INDEX_NAME));
+    dataStore2.invoke(() -> waitUntilFlushed(INDEX_NAME));
+
+    // Execute query and verify results
+    accessor.invoke(() -> executeQuery(INDEX_NAME, "field1Value", "field1", numPuts));
+
+    // Export entries from region
+    accessor.invoke(() -> exportData(regionType));
 
     // Destroy indexes (only needs to be done on one member)
     dataStore1.invoke(() -> destroyIndexes());
@@ -169,22 +196,142 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
     dataStore1.invoke(() -> destroyDataRegion(true));
 
     // Recreate index and region
-    dataStore1.invoke(() -> initDataStore(createIndex(), regionType));
-    dataStore2.invoke(() -> initDataStore(createIndex(), regionType));
+    dataStore1.invoke(() -> initDataStore(createIndex, regionType));
+    dataStore2.invoke(() -> initDataStore(createIndex, regionType));
+    accessor.invoke(() -> initAccessor(createIndex, regionType));
 
-    // Do puts to cause IndexRepositories to be recreated
-    dataStore1.invoke(() -> doPuts(10));
+    // Import entries into region
+    accessor.invoke(() -> importData(regionType, numPuts));
 
     // Wait until queue is flushed
     // This verifies there are no deadlocks
     dataStore1.invoke(() -> waitUntilFlushed(INDEX_NAME));
     dataStore2.invoke(() -> waitUntilFlushed(INDEX_NAME));
+
+    // re-execute query and verify results
+    accessor.invoke(() -> executeQuery(INDEX_NAME, "field1Value", "field1", numPuts));
+  }
+
+  @Test
+  @Parameters(method = "getListOfRegionTestTypes")
+  public void verifyDestroyRecreateIndexDifferentName(RegionTestableType regionType) {
+    // Create index and region
+    SerializableRunnableIF createIndex = createIndex();
+    dataStore1.invoke(() -> initDataStore(createIndex, regionType));
+    dataStore2.invoke(() -> initDataStore(createIndex, regionType));
+    accessor.invoke(() -> initAccessor(createIndex, regionType));
+
+    // Verify index created
+    dataStore1.invoke(() -> verifyIndexCreated());
+    dataStore2.invoke(() -> verifyIndexCreated());
+    accessor.invoke(() -> verifyIndexCreated());
+
+    // Do puts to cause IndexRepositories to be created
+    int numPuts = 10;
+    accessor.invoke(() -> doPuts(numPuts));
+
+    // Wait until queue is flushed
+    dataStore1.invoke(() -> waitUntilFlushed(INDEX_NAME));
+    dataStore2.invoke(() -> waitUntilFlushed(INDEX_NAME));
+
+    // Execute query and verify results
+    accessor.invoke(() -> executeQuery(INDEX_NAME, "field1Value", "field1", numPuts));
+
+    // Export entries from region
+    accessor.invoke(() -> exportData(regionType));
+
+    // Destroy indexes (only needs to be done on one member)
+    dataStore1.invoke(() -> destroyIndexes());
+
+    // Verify indexes destroyed
+    dataStore1.invoke(() -> verifyIndexesDestroyed());
+    dataStore2.invoke(() -> verifyIndexesDestroyed());
+
+    // Destroy data region
+    dataStore1.invoke(() -> destroyDataRegion(true));
+
+    // Recreate index and region
+    String newIndexName = INDEX_NAME + "+_1";
+    SerializableRunnableIF createIndexNewName = createIndex(newIndexName, "field1");
+    dataStore1.invoke(() -> initDataStore(createIndexNewName, regionType));
+    dataStore2.invoke(() -> initDataStore(createIndexNewName, regionType));
+    accessor.invoke(() -> initAccessor(createIndexNewName, regionType));
+
+    // Import entries into region
+    accessor.invoke(() -> importData(regionType, numPuts));
+
+    // Wait until queue is flushed
+    // This verifies there are no deadlocks
+    dataStore1.invoke(() -> waitUntilFlushed(newIndexName));
+    dataStore2.invoke(() -> waitUntilFlushed(newIndexName));
+
+    // re-execute query and verify results
+    accessor.invoke(() -> executeQuery(newIndexName, "field1Value", "field1", numPuts));
+  }
+
+  @Test
+  @Parameters(method = "getListOfRegionTestTypes")
+  public void verifyDestroyRecreateDifferentIndex(RegionTestableType regionType) {
+    SerializableRunnableIF createIndex = createIndex();
+    dataStore1.invoke(() -> initDataStore(createIndex, regionType));
+    dataStore2.invoke(() -> initDataStore(createIndex, regionType));
+    accessor.invoke(() -> initAccessor(createIndex, regionType));
+
+    // Verify index created
+    dataStore1.invoke(() -> verifyIndexCreated());
+    dataStore2.invoke(() -> verifyIndexCreated());
+    accessor.invoke(() -> verifyIndexCreated());
+
+    // Do puts to cause IndexRepositories to be created
+    int numPuts = 10;
+    accessor.invoke(() -> doPuts(numPuts));
+
+    // Wait until queue is flushed
+    dataStore1.invoke(() -> waitUntilFlushed(INDEX_NAME));
+    dataStore2.invoke(() -> waitUntilFlushed(INDEX_NAME));
+
+    // Execute query and verify results
+    accessor.invoke(() -> executeQuery(INDEX_NAME, "field1Value", "field1", numPuts));
+
+    // Export entries from region
+    accessor.invoke(() -> exportData(regionType));
+
+    // Destroy indexes (only needs to be done on one member)
+    dataStore1.invoke(() -> destroyIndexes());
+
+    // Verify indexes destroyed
+    dataStore1.invoke(() -> verifyIndexesDestroyed());
+    dataStore2.invoke(() -> verifyIndexesDestroyed());
+
+    // Destroy data region
+    dataStore1.invoke(() -> destroyDataRegion(true));
+
+    // Create new index and region
+    SerializableRunnableIF createNewIndex = createIndex(INDEX_NAME, "field2");
+    dataStore1.invoke(() -> initDataStore(createNewIndex, regionType));
+    dataStore2.invoke(() -> initDataStore(createNewIndex, regionType));
+    accessor.invoke(() -> initAccessor(createNewIndex, regionType));
+
+    // Import entries into region
+    accessor.invoke(() -> importData(regionType, numPuts));
+
+    // Wait until queue is flushed
+    // This verifies there are no deadlocks
+    dataStore1.invoke(() -> waitUntilFlushed(INDEX_NAME));
+    dataStore2.invoke(() -> waitUntilFlushed(INDEX_NAME));
+
+    // re-execute query and verify results
+    accessor.invoke(() -> executeQuery(INDEX_NAME, "field2Value", "field2", numPuts));
   }
 
   private SerializableRunnableIF createIndex() {
+    return createIndex(INDEX_NAME, "field1");
+  }
+
+  private SerializableRunnableIF createIndex(String indexName, String field) {
     return () -> {
       LuceneService luceneService = LuceneServiceProvider.get(getCache());
-      luceneService.createIndex(INDEX_NAME, REGION_NAME, "text");
+      luceneService.createIndex(indexName, REGION_NAME, field);
     };
   }
 
@@ -216,7 +363,7 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
   private void doPuts(int numPuts) throws Exception {
     Region region = getCache().getRegion(REGION_NAME);
     for (int i = 0; i < numPuts; i++) {
-      region.put(i, new TestObject());
+      region.put(i, new TestObject("field1Value", "field2Value"));
     }
   }
 
@@ -238,6 +385,15 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
         .until(() -> getCache().getRegion(REGION_NAME).size() > 0);
   }
 
+  private void executeQuery(String indexName, String queryString, String field,
+      int expectedResultsSize) throws LuceneQueryException {
+    LuceneService luceneService = LuceneServiceProvider.get(getCache());
+    LuceneQuery query =
+        luceneService.createLuceneQueryFactory().create(indexName, REGION_NAME, queryString, field);
+    Collection results = query.findValues();
+    assertEquals(expectedResultsSize, results.size());
+  }
+
   private void destroyDataRegion(boolean shouldSucceed) {
     Region region = getCache().getRegion(REGION_NAME);
     assertNotNull(region);
@@ -296,4 +452,25 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
     LocalRegion region = (LocalRegion) getCache().getRegion(REGION_NAME);
     assertFalse(region.getExtensionPoint().getExtensions().iterator().hasNext());
   }
+
+  private void exportData(RegionTestableType regionType) throws Exception {
+    Region region = getCache().getRegion(REGION_NAME);
+    RegionSnapshotService service = region.getSnapshotService();
+    service.save(getSnapshotFile(getDiskDirs()[0], regionType),
+        SnapshotOptions.SnapshotFormat.GEMFIRE);
+  }
+
+  private void importData(RegionTestableType regionType, int expectedRegionSize) throws Exception {
+    Region region = getCache().getRegion(REGION_NAME);
+    RegionSnapshotService service = region.getSnapshotService();
+    SnapshotOptions options = service.createOptions();
+    options.invokeCallbacks(true);
+    service.load(getSnapshotFile(getDiskDirs()[0], regionType),
+        SnapshotOptions.SnapshotFormat.GEMFIRE, options);
+    assertEquals(expectedRegionSize, region.size());
+  }
+
+  private File getSnapshotFile(File baseDirectory, RegionTestableType regionType) {
+    return new File(baseDirectory, REGION_NAME + "_" + regionType.name() + "_snapshot.gfd");
+  }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/8c371be8/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/TestObject.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/TestObject.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/TestObject.java
index be91b6d..24e330b 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/TestObject.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/TestObject.java
@@ -52,4 +52,8 @@ public class TestObject implements Serializable {
     return (testObject.field1.equals(field1) && testObject.field2.equals(field2));
   }
 
+  public String toString() {
+    return new StringBuilder().append(getClass().getSimpleName()).append("[").append("field1=")
+        .append(field1).append("; field2=").append(field2).append("]").toString();
+  }
 }


[32/49] geode git commit: GEODE-2612: Added gfsh support for invoking callbacks during snapshot load

Posted by kh...@apache.org.
GEODE-2612: Added gfsh support for invoking callbacks during snapshot load


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/6cc8a113
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/6cc8a113
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/6cc8a113

Branch: refs/heads/feature/GEODE-2420
Commit: 6cc8a1139768eaae41cddf8ec0e392583c26ad92
Parents: f7aad2e
Author: Barry Oglesby <bo...@pivotal.io>
Authored: Thu Mar 9 15:49:14 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../internal/cli/commands/DataCommands.java     |   7 +-
 .../cli/functions/ImportDataFunction.java       |  15 ++-
 .../internal/cli/i18n/CliStrings.java           |   3 +
 .../commands/GemfireDataCommandsDUnitTest.java  | 117 +++++++++++++++++--
 .../cli/commands/golden-help-offline.properties |   6 +-
 5 files changed, 129 insertions(+), 19 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/6cc8a113/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/DataCommands.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/DataCommands.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/DataCommands.java
index a938987..484bbf2 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/DataCommands.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/DataCommands.java
@@ -841,7 +841,10 @@ public class DataCommands implements CommandMarker {
       @CliOption(key = CliStrings.IMPORT_DATA__MEMBER, mandatory = true,
           unspecifiedDefaultValue = CliMetaData.ANNOTATION_NULL_VALUE,
           optionContext = ConverterHint.MEMBERIDNAME,
-          help = CliStrings.IMPORT_DATA__MEMBER__HELP) String memberNameOrId) {
+          help = CliStrings.IMPORT_DATA__MEMBER__HELP) String memberNameOrId,
+      @CliOption(key = CliStrings.IMPORT_DATA__INVOKE_CALLBACKS, mandatory = false,
+          unspecifiedDefaultValue = "false",
+          help = CliStrings.IMPORT_DATA__INVOKE_CALLBACKS__HELP) boolean invokeCallbacks) {
 
     this.securityService.authorizeRegionWrite(regionName);
 
@@ -856,7 +859,7 @@ public class DataCommands implements CommandMarker {
             .format(CliStrings.INVALID_FILE_EXTENSION, CliStrings.GEODE_DATA_FILE_EXTENSION));
       }
       if (targetMember != null) {
-        final String args[] = {regionName, filePath};
+        final Object args[] = {regionName, filePath, invokeCallbacks};
         ResultCollector<?, ?> rc = CliUtil.executeFunction(importDataFunction, args, targetMember);
         List<Object> results = (List<Object>) rc.getResult();
 

http://git-wip-us.apache.org/repos/asf/geode/blob/6cc8a113/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ImportDataFunction.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ImportDataFunction.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ImportDataFunction.java
index 6444662..13949c8 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ImportDataFunction.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/functions/ImportDataFunction.java
@@ -22,6 +22,7 @@ import org.apache.geode.cache.Region;
 import org.apache.geode.cache.execute.FunctionAdapter;
 import org.apache.geode.cache.execute.FunctionContext;
 import org.apache.geode.cache.snapshot.RegionSnapshotService;
+import org.apache.geode.cache.snapshot.SnapshotOptions;
 import org.apache.geode.cache.snapshot.SnapshotOptions.SnapshotFormat;
 import org.apache.geode.internal.InternalEntity;
 import org.apache.geode.management.internal.cli.i18n.CliStrings;
@@ -36,9 +37,13 @@ public class ImportDataFunction extends FunctionAdapter implements InternalEntit
   private static final long serialVersionUID = 1L;
 
   public void execute(FunctionContext context) {
-    final String[] args = (String[]) context.getArguments();
-    final String regionName = args[0];
-    final String importFileName = args[1];
+    final Object[] args = (Object[]) context.getArguments();
+    final String regionName = (String) args[0];
+    final String importFileName = (String) args[1];
+    boolean invokeCallbacks = false;
+    if (args.length > 2) {
+      invokeCallbacks = (boolean) args[2];
+    }
 
     try {
       final Cache cache = CacheFactory.getAnyInstance();
@@ -46,8 +51,10 @@ public class ImportDataFunction extends FunctionAdapter implements InternalEntit
       final String hostName = cache.getDistributedSystem().getDistributedMember().getHost();
       if (region != null) {
         RegionSnapshotService<?, ?> snapshotService = region.getSnapshotService();
+        SnapshotOptions options = snapshotService.createOptions();
+        options.invokeCallbacks(invokeCallbacks);
         File importFile = new File(importFileName);
-        snapshotService.load(new File(importFileName), SnapshotFormat.GEMFIRE);
+        snapshotService.load(new File(importFileName), SnapshotFormat.GEMFIRE, options);
         String successMessage = CliStrings.format(CliStrings.IMPORT_DATA__SUCCESS__MESSAGE,
             importFile.getCanonicalPath(), hostName, regionName);
         context.getResultSender().lastResult(successMessage);

http://git-wip-us.apache.org/repos/asf/geode/blob/6cc8a113/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
index 51d39d6..b835e8b 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
@@ -1566,6 +1566,9 @@ public class CliStrings {
   public static final String IMPORT_DATA__REGION__NOT__FOUND = "Region {0} not found.";
   public static final String IMPORT_DATA__SUCCESS__MESSAGE =
       "Data imported from file : {0} on host : {1} to region : {2}";
+  public static final String IMPORT_DATA__INVOKE_CALLBACKS = "invoke-callbacks";
+  public static final String IMPORT_DATA__INVOKE_CALLBACKS__HELP =
+      "Whether callbacks should be invoked";
 
   /* 'list async-event-queues' command */
   public static final String LIST_ASYNC_EVENT_QUEUES = "list async-event-queues";

http://git-wip-us.apache.org/repos/asf/geode/blob/6cc8a113/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GemfireDataCommandsDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GemfireDataCommandsDUnitTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GemfireDataCommandsDUnitTest.java
index 8a20631..e99a7fb 100644
--- a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GemfireDataCommandsDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/GemfireDataCommandsDUnitTest.java
@@ -31,8 +31,11 @@ import java.util.Map;
 import java.util.Properties;
 import java.util.Random;
 import java.util.Set;
+import java.util.concurrent.atomic.AtomicInteger;
 
-import org.junit.Ignore;
+import org.apache.geode.cache.CacheListener;
+import org.apache.geode.cache.EntryEvent;
+import org.apache.geode.cache.util.CacheListenerAdapter;
 import org.junit.Test;
 import org.junit.experimental.categories.Category;
 
@@ -1730,18 +1733,19 @@ public class GemfireDataCommandsDUnitTest extends CliCommandTestBase {
 
       setUpJmxManagerOnVm0ThenConnect(null);
 
-      manager.invoke(new SerializableCallable() {
-        public Object call() {
-          Region region = createParReg(regionName, getCache());
-          return region.put("Manager", "ASD");
+      manager.invoke(new SerializableRunnable() {
+        public void run() {
+          createParReg(regionName, getCache());
         }
       });
 
-      vm1.invoke(new SerializableCallable() {
+      vm1.invoke(new SerializableRunnable() {
         @Override
-        public Object call() throws Exception {
+        public void run() throws Exception {
           Region region = createParReg(regionName, getCache());
-          return region.put("VM1", "QWE");
+          for (int i = 0; i < 100; i++) {
+            region.put(i, i);
+          }
         }
       });
 
@@ -1760,12 +1764,20 @@ public class GemfireDataCommandsDUnitTest extends CliCommandTestBase {
       vm1.invoke(new SerializableRunnable() {
         public void run() {
           Region region = getCache().getRegion(regionName);
-          region.destroy("Manager");
-          region.destroy("VM1");
+          for (int i = 0; i < 100; i++) {
+            region.destroy(i);
+          }
         }
       });
 
       /**
+       * Add CacheListener
+       */
+
+      manager.invoke(addCacheListenerInvocations(regionName));
+      vm1.invoke(addCacheListenerInvocations(regionName));
+
+      /**
        * Import the data
        */
 
@@ -1789,11 +1801,49 @@ public class GemfireDataCommandsDUnitTest extends CliCommandTestBase {
       manager.invoke(new SerializableRunnable() {
         public void run() {
           Region region = getCache().getRegion(regionName);
-          assertEquals(region.get("Manager"), "ASD");
-          assertEquals(region.get("VM1"), "QWE");
+          for (int i = 0; i < 100; i++) {
+            assertEquals(i, region.get(i));
+          }
+        }
+      });
+
+      /**
+       * Verify callbacks were not invoked
+       */
+
+      manager.invoke(verifyCacheListenerInvocations(regionName, false));
+      vm1.invoke(verifyCacheListenerInvocations(regionName, false));
+
+      /**
+       * Import the data with invokeCallbacks=true
+       */
+
+      vm1.invoke(new SerializableRunnable() {
+        public void run() {
+          Region region = getCache().getRegion(regionName);
+          for (int i = 0; i < 100; i++) {
+            region.destroy(i);
+          }
         }
       });
 
+      csb = new CommandStringBuilder(CliStrings.IMPORT_DATA);
+      csb.addOption(CliStrings.IMPORT_DATA__REGION, regionName);
+      csb.addOption(CliStrings.IMPORT_DATA__FILE, filePath);
+      csb.addOption(CliStrings.IMPORT_DATA__MEMBER, "Manager");
+      csb.addOption(CliStrings.IMPORT_DATA__INVOKE_CALLBACKS, "true");
+      commandString = csb.toString();
+      cmdResult = executeCommand(commandString);
+      commandResultToString(cmdResult);
+      assertEquals(Result.Status.OK, cmdResult.getStatus());
+
+      /**
+       * Verify callbacks were invoked
+       */
+
+      manager.invoke(verifyCacheListenerInvocations(regionName, true));
+      vm1.invoke(verifyCacheListenerInvocations(regionName, true));
+
       // Test for bad input
       csb = new CommandStringBuilder(CliStrings.EXPORT_DATA);
       csb.addOption(CliStrings.EXPORT_DATA__REGION, "FDSERW");
@@ -1824,6 +1874,35 @@ public class GemfireDataCommandsDUnitTest extends CliCommandTestBase {
     }
   }
 
+  private SerializableRunnable addCacheListenerInvocations(final String regionName) {
+    return new SerializableRunnable() {
+      public void run() {
+        Region region = getCache().getRegion(regionName);
+        region.getAttributesMutator().addCacheListener(new CountingCacheListener());
+      }
+    };
+  }
+
+  private SerializableRunnable verifyCacheListenerInvocations(final String regionName,
+      boolean callbacksShouldHaveBeenInvoked) {
+    return new SerializableRunnable() {
+      public void run() {
+        Region region = getCache().getRegion(regionName);
+        CacheListener<?, ?>[] listeners = region.getAttributes().getCacheListeners();
+        for (CacheListener<?, ?> listener : listeners) {
+          if (listener instanceof CountingCacheListener) {
+            CountingCacheListener ccl = (CountingCacheListener) listener;
+            if (callbacksShouldHaveBeenInvoked) {
+              assertNotEquals(0, ccl.getEvents());
+            } else {
+              assertEquals(0, ccl.getEvents());
+            }
+          }
+        }
+      }
+    };
+  }
+
   void setupWith2Regions() {
     final VM vm1 = Host.getHost(0).getVM(1);
     final VM vm2 = Host.getHost(0).getVM(2);
@@ -2329,4 +2408,18 @@ public class GemfireDataCommandsDUnitTest extends CliCommandTestBase {
         + memSizeFromFunctionCall);
     assertTrue(memSizeFromFunctionCall.equals(memSizeFromMbean));
   }
+
+  private static class CountingCacheListener extends CacheListenerAdapter {
+
+    private final AtomicInteger events = new AtomicInteger();
+
+    @Override
+    public void afterCreate(EntryEvent event) {
+      events.incrementAndGet();
+    }
+
+    private int getEvents() {
+      return events.get();
+    }
+  }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/6cc8a113/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
----------------------------------------------------------------------
diff --git a/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties b/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
index 17b7515..8cc8aba 100644
--- a/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
+++ b/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
@@ -1682,7 +1682,7 @@ IS AVAILABLE\n\
 SYNOPSIS\n\
 \ \ \ \ Import user data from a file to a region.\n\
 SYNTAX\n\
-\ \ \ \ import data --region=value --file=value --member=value\n\
+\ \ \ \ import data --region=value --file=value --member=value [--invoke-callbacks=value]\n\
 PARAMETERS\n\
 \ \ \ \ region\n\
 \ \ \ \ \ \ \ \ Region into which data will be imported.\n\
@@ -1694,6 +1694,10 @@ PARAMETERS\n\
 \ \ \ \ \ \ \ \ Name/Id of a member which hosts the region. The data will be imported from the specified\n\
 \ \ \ \ \ \ \ \ file on the host where the member is running.\n\
 \ \ \ \ \ \ \ \ Required: true\n\
+\ \ \ \ invoke-callbacks\n\
+\ \ \ \ \ \ \ \ Whether callbacks should be invoked\n\
+\ \ \ \ \ \ \ \ Required: false\n\
+\ \ \ \ \ \ \ \ Default (if the parameter is not specified): false\n\
 
 list-async-event-queues.help=\
 NAME\n\


[18/49] geode git commit: GEODE-2638: GatewaySenders now start even if configured with locators on unknown hosts

Posted by kh...@apache.org.
GEODE-2638: GatewaySenders now start even if configured with locators on unknown hosts


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/25fc337e
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/25fc337e
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/25fc337e

Branch: refs/heads/feature/GEODE-2420
Commit: 25fc337ecc21f054491a452e707c2b106f345591
Parents: a4b3230
Author: Barry Oglesby <bo...@pivotal.io>
Authored: Thu Mar 9 09:28:39 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../geode/internal/i18n/LocalizedStrings.java   |  5 ++
 .../cache/wan/AbstractRemoteGatewaySender.java  | 15 ++++-
 .../geode/internal/cache/wan/WANTestBase.java   | 23 +++++++
 .../wan/misc/WanAutoDiscoveryDUnitTest.java     | 63 ++++++++++++++++++++
 4 files changed, 103 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/25fc337e/geode-core/src/main/java/org/apache/geode/internal/i18n/LocalizedStrings.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/i18n/LocalizedStrings.java b/geode-core/src/main/java/org/apache/geode/internal/i18n/LocalizedStrings.java
index 47ae0c5..1d288ec 100755
--- a/geode-core/src/main/java/org/apache/geode/internal/i18n/LocalizedStrings.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/i18n/LocalizedStrings.java
@@ -7672,6 +7672,11 @@ public class LocalizedStrings {
       new StringId(6651, "Lucene index {0} was not found in region {1}.");
   public static final StringId LuceneService_DESTROYED_INDEX_0_FROM_REGION_1 =
       new StringId(6652, "Destroyed Lucene index {0} from region {1}.");
+
+  public static final StringId PoolFactoryImpl_CAUGHT_EXCEPTION_ATTEMPTING_TO_ADD_REMOTE_LOCATOR_0 =
+      new StringId(6653,
+          "Caught the following exception attempting to add remote locator {0}. The locator will be ignored.");
+
   /** Testing strings, messageId 90000-99999 **/
 
   /**

http://git-wip-us.apache.org/repos/asf/geode/blob/25fc337e/geode-wan/src/main/java/org/apache/geode/internal/cache/wan/AbstractRemoteGatewaySender.java
----------------------------------------------------------------------
diff --git a/geode-wan/src/main/java/org/apache/geode/internal/cache/wan/AbstractRemoteGatewaySender.java b/geode-wan/src/main/java/org/apache/geode/internal/cache/wan/AbstractRemoteGatewaySender.java
index 918c723..68eb56c 100644
--- a/geode-wan/src/main/java/org/apache/geode/internal/cache/wan/AbstractRemoteGatewaySender.java
+++ b/geode-wan/src/main/java/org/apache/geode/internal/cache/wan/AbstractRemoteGatewaySender.java
@@ -99,9 +99,18 @@ public abstract class AbstractRemoteGatewaySender extends AbstractGatewaySender
           StringBuffer strBuffer = new StringBuffer();
           Iterator<String> itr = response.getLocators().iterator();
           while (itr.hasNext()) {
-            DistributionLocatorId locatorId = new DistributionLocatorId(itr.next());
-            pf.addLocator(locatorId.getHost().getHostName(), locatorId.getPort());
-            locatorCount++;
+            String remoteLocator = itr.next();
+            try {
+              DistributionLocatorId locatorId = new DistributionLocatorId(remoteLocator);
+              pf.addLocator(locatorId.getHost().getHostName(), locatorId.getPort());
+              locatorCount++;
+            } catch (Exception e) {
+              if (logProxyFailure()) {
+                logger.warn(LocalizedMessage.create(
+                    LocalizedStrings.PoolFactoryImpl_CAUGHT_EXCEPTION_ATTEMPTING_TO_ADD_REMOTE_LOCATOR_0,
+                    new Object[] {remoteLocator}), e);
+              }
+            }
           }
           break;
         }

http://git-wip-us.apache.org/repos/asf/geode/blob/25fc337e/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/WANTestBase.java
----------------------------------------------------------------------
diff --git a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/WANTestBase.java b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/WANTestBase.java
index c8de7dc..0656dd8 100644
--- a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/WANTestBase.java
+++ b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/WANTestBase.java
@@ -67,6 +67,7 @@ import org.apache.geode.cache.asyncqueue.internal.AsyncEventQueueImpl;
 import org.apache.geode.cache.client.Pool;
 import org.apache.geode.cache.client.PoolManager;
 import org.apache.geode.cache.client.internal.LocatorDiscoveryCallbackAdapter;
+import org.apache.geode.cache.client.internal.PoolImpl;
 import org.apache.geode.cache.client.internal.locator.wan.LocatorMembershipListener;
 import org.apache.geode.cache.persistence.PartitionOfflineException;
 import org.apache.geode.cache.server.CacheServer;
@@ -3032,6 +3033,16 @@ public class WANTestBase extends JUnit4DistributedTestCase {
     return System.currentTimeMillis();
   }
 
+  public static void putRemoteSiteLocators(int remoteDsId, Set<String> remoteLocators) {
+    List<Locator> locatorsConfigured = Locator.getLocators();
+    Locator locator = locatorsConfigured.get(0);
+    if (remoteLocators != null) {
+      // Add fake remote locators to the locators map
+      ((InternalLocator) locator).getlocatorMembershipListener().getAllServerLocatorsInfo()
+          .put(remoteDsId, remoteLocators);
+    }
+  }
+
   public static void checkLocatorsinSender(String senderId, InetSocketAddress locatorToWaitFor)
       throws InterruptedException {
 
@@ -3426,6 +3437,18 @@ public class WANTestBase extends JUnit4DistributedTestCase {
     assertTrue(sender.isRunning());
   }
 
+  public static void verifyPool(String senderId, boolean poolShouldExist,
+      int expectedPoolLocatorsSize) {
+    AbstractGatewaySender sender = (AbstractGatewaySender) cache.getGatewaySender(senderId);
+    PoolImpl pool = sender.getProxy();
+    if (poolShouldExist) {
+      assertNotNull(pool);
+      assertEquals(expectedPoolLocatorsSize, pool.getLocators().size());
+    } else {
+      assertNull(pool);
+    }
+  }
+
   public static void removeSenderFromTheRegion(String senderId, String regionName) {
     Region region = cache.getRegion(regionName);
     assertNotNull(region);

http://git-wip-us.apache.org/repos/asf/geode/blob/25fc337e/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/misc/WanAutoDiscoveryDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/misc/WanAutoDiscoveryDUnitTest.java b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/misc/WanAutoDiscoveryDUnitTest.java
index 388760e..3e94697 100644
--- a/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/misc/WanAutoDiscoveryDUnitTest.java
+++ b/geode-wan/src/test/java/org/apache/geode/internal/cache/wan/misc/WanAutoDiscoveryDUnitTest.java
@@ -14,6 +14,7 @@
  */
 package org.apache.geode.internal.cache.wan.misc;
 
+import org.apache.geode.test.dunit.IgnoredException;
 import org.junit.Ignore;
 import org.junit.experimental.categories.Category;
 import org.junit.Test;
@@ -581,4 +582,66 @@ public class WanAutoDiscoveryDUnitTest extends WANTestBase {
       fail(builder.toString());
     }
   }
+
+  @Test
+  public void testNoRemoteLocators() {
+    IgnoredException ie = IgnoredException
+        .addIgnoredException("could not get remote locator information for remote site");
+    try {
+      testRemoteLocators(null, false, 0);
+    } finally {
+      ie.remove();
+    }
+  }
+
+  @Test
+  public void testValidHostRemoteLocators() {
+    Set<String> remoteLocators = new HashSet();
+    remoteLocators.add("localhost[12345]");
+    testRemoteLocators(remoteLocators, true, 1);
+  }
+
+  @Test
+  public void testInvalidHostRemoteLocators() {
+    IgnoredException ie = IgnoredException
+        .addIgnoredException("could not get remote locator information for remote site");
+    try {
+      Set<String> remoteLocators = new HashSet();
+      remoteLocators.add("unknown[12345]");
+      testRemoteLocators(remoteLocators, false, 0);
+    } finally {
+      ie.remove();
+    }
+  }
+
+  @Test
+  public void testValidAndInvalidHostRemoteLocators() {
+    Set<String> remoteLocators = new HashSet();
+    remoteLocators.add("localhost[12345]");
+    remoteLocators.add("unknown[12345]");
+    testRemoteLocators(remoteLocators, true, 1);
+  }
+
+  private void testRemoteLocators(Set<String> remoteLocators, boolean poolShouldExist,
+      int expectedPoolLocatorsSize) {
+    // Start locator
+    Integer lnLocPort = (Integer) vm0.invoke(() -> WANTestBase.createFirstLocatorWithDSId(1));
+
+    // Add remote locators
+    int remoteDsId = 2;
+    vm0.invoke(() -> WANTestBase.putRemoteSiteLocators(remoteDsId, remoteLocators));
+
+    // Create cache
+    vm2.invoke(() -> WANTestBase.createCache(lnLocPort));
+
+    // Create sender
+    vm2.invoke(() -> WANTestBase.createSender("ln", remoteDsId, false, 100, 10, false, false, null,
+        false));
+
+    // Verify sender is running
+    vm2.invoke(() -> WANTestBase.verifySenderRunningState("ln"));
+
+    // Verify pool exists or not
+    vm2.invoke(() -> WANTestBase.verifyPool("ln", poolShouldExist, expectedPoolLocatorsSize));
+  }
 }


[45/49] geode git commit: GEODE-2639: Added DUnit tests for expiration on Lucene

Posted by kh...@apache.org.
GEODE-2639: Added DUnit tests for expiration on Lucene

	* Tests added to validate the effect of the expiration of Lucene Index entries.
	* Once the entries expire, the lucene index entries must be updated.

	This closes #422


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/5be70038
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/5be70038
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/5be70038

Branch: refs/heads/feature/GEODE-2420
Commit: 5be700386bd5f1483868f0a821c58d9642743699
Parents: 16bf532
Author: nabarun <nn...@pivotal.io>
Authored: Fri Mar 10 11:00:09 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../geode/cache/lucene/ExpirationDUnitTest.java | 89 ++++++++++++++++++++
 .../geode/cache/lucene/LuceneDUnitTest.java     | 34 ++++++--
 2 files changed, 116 insertions(+), 7 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/5be70038/geode-lucene/src/test/java/org/apache/geode/cache/lucene/ExpirationDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/ExpirationDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/ExpirationDUnitTest.java
new file mode 100644
index 0000000..92c7370
--- /dev/null
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/ExpirationDUnitTest.java
@@ -0,0 +1,89 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.lucene;
+
+import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.INDEX_NAME;
+import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.REGION_NAME;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+import org.apache.geode.cache.Cache;
+import org.apache.geode.cache.Region;
+import org.apache.geode.test.dunit.SerializableRunnableIF;
+import org.apache.geode.test.junit.categories.DistributedTest;
+import org.awaitility.Awaitility;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+
+import java.util.Collection;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.IntStream;
+import junitparams.JUnitParamsRunner;
+import junitparams.Parameters;
+
+@Category(DistributedTest.class)
+@RunWith(JUnitParamsRunner.class)
+public class ExpirationDUnitTest extends LuceneQueriesAccessorBase {
+
+  protected final static int EXTRA_WAIT_TIME_SEC = 15;
+
+  protected RegionTestableType[] getPartitionRegionsWithExpirationSet() {
+    return new RegionTestableType[] {RegionTestableType.PARTITION_WITH_EXPIRATION_DESTROY,
+        RegionTestableType.PARTITION_REDUNDANT_WITH_EXPIRATION_DESTROY,
+        RegionTestableType.PARTITION_REDUNDANT_PERSISTENT_WITH_EXPIRATION_DESTROY};
+  }
+
+  @Test
+  @Parameters(method = "getPartitionRegionsWithExpirationSet")
+  public void regionWithExpirationSetMustAlsoRemoveLuceneIndexEntries(
+      RegionTestableType regionTestType) {
+    SerializableRunnableIF createIndex = () -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
+    };
+
+    dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
+    dataStore2.invoke(() -> initDataStore(createIndex, regionTestType));
+    accessor.invoke(() -> initDataStore(createIndex, regionTestType));
+
+    accessor.invoke(() -> {
+      Cache cache = getCache();
+      Region region = cache.getRegion(REGION_NAME);
+      IntStream.range(0, NUM_BUCKETS).forEach(i -> region.put(i, new TestObject("hello world")));
+    });
+
+    assertTrue(waitForFlushBeforeExecuteTextSearch(accessor, 60000));
+
+    accessor.invoke(() -> Awaitility.await()
+        .atMost(EXPIRATION_TIMEOUT_SEC + EXTRA_WAIT_TIME_SEC, TimeUnit.SECONDS).until(() -> {
+          LuceneService luceneService = LuceneServiceProvider.get(getCache());
+          LuceneQuery<Integer, TestObject> luceneQuery = luceneService.createLuceneQueryFactory()
+              .setResultLimit(100).create(INDEX_NAME, REGION_NAME, "world", "text");
+
+          Collection luceneResultList = null;
+          try {
+            luceneResultList = luceneQuery.findKeys();
+          } catch (LuceneQueryException e) {
+            e.printStackTrace();
+            fail();
+          }
+          assertEquals(0, luceneResultList.size());
+        }));
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/5be70038/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java
index fa3c470..b80f5de 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java
@@ -17,6 +17,8 @@ package org.apache.geode.cache.lucene;
 import org.apache.geode.cache.Cache;
 import org.apache.geode.cache.EvictionAction;
 import org.apache.geode.cache.EvictionAttributes;
+import org.apache.geode.cache.ExpirationAction;
+import org.apache.geode.cache.ExpirationAttributes;
 import org.apache.geode.cache.PartitionAttributes;
 import org.apache.geode.cache.PartitionAttributesFactory;
 import org.apache.geode.cache.Region;
@@ -37,6 +39,7 @@ public abstract class LuceneDUnitTest extends JUnit4CacheTestCase {
 
   protected static int NUM_BUCKETS = 10;
   protected static int MAX_ENTRIES_FOR_EVICTION = 20;
+  protected static int EXPIRATION_TIMEOUT_SEC = 5;
 
   @Override
   public void postSetUp() throws Exception {
@@ -98,12 +101,16 @@ public abstract class LuceneDUnitTest extends JUnit4CacheTestCase {
             EvictionAction.LOCAL_DESTROY)),
     PARTITION_PERSISTENT_EVICTION_LOCAL_DESTROY(RegionShortcut.PARTITION_PROXY,
         RegionShortcut.PARTITION_PERSISTENT, EvictionAttributes
-            .createLRUEntryAttributes(MAX_ENTRIES_FOR_EVICTION, EvictionAction.LOCAL_DESTROY))
-
-
-    ;
-
-
+            .createLRUEntryAttributes(MAX_ENTRIES_FOR_EVICTION, EvictionAction.LOCAL_DESTROY)),
+    PARTITION_REDUNDANT_WITH_EXPIRATION_DESTROY(RegionShortcut.PARTITION_PROXY_REDUNDANT,
+        RegionShortcut.PARTITION_REDUNDANT, EXPIRATION_TIMEOUT_SEC, ExpirationAction.DESTROY),
+    PARTITION_WITH_EXPIRATION_DESTROY(RegionShortcut.PARTITION_PROXY, RegionShortcut.PARTITION,
+        EXPIRATION_TIMEOUT_SEC, ExpirationAction.DESTROY),
+    PARTITION_REDUNDANT_PERSISTENT_WITH_EXPIRATION_DESTROY(RegionShortcut.PARTITION_PROXY_REDUNDANT,
+        RegionShortcut.PARTITION_REDUNDANT_PERSISTENT, EXPIRATION_TIMEOUT_SEC,
+        ExpirationAction.DESTROY);
+
+    ExpirationAttributes expirationAttributes = null;
     EvictionAttributes evictionAttributes = null;
     private RegionShortcut serverRegionShortcut;
     private RegionShortcut clientRegionShortcut;
@@ -117,6 +124,15 @@ public abstract class LuceneDUnitTest extends JUnit4CacheTestCase {
       this.clientRegionShortcut = clientRegionShortcut;
       this.serverRegionShortcut = serverRegionShortcut;
       this.evictionAttributes = evictionAttributes;
+      this.expirationAttributes = null;
+    }
+
+    RegionTestableType(RegionShortcut clientRegionShortcut, RegionShortcut serverRegionShortcut,
+        int timeout, ExpirationAction expirationAction) {
+      this.clientRegionShortcut = clientRegionShortcut;
+      this.serverRegionShortcut = serverRegionShortcut;
+      this.evictionAttributes = null;
+      this.expirationAttributes = new ExpirationAttributes(timeout, expirationAction);
     }
 
     public Region createDataStore(Cache cache, String regionName) {
@@ -128,7 +144,11 @@ public abstract class LuceneDUnitTest extends JUnit4CacheTestCase {
           return null;
         }
       }
-      if (evictionAttributes == null) {
+      if (expirationAttributes != null) {
+        return cache.createRegionFactory(serverRegionShortcut)
+            .setEntryTimeToLive(expirationAttributes)
+            .setPartitionAttributes(getPartitionAttributes(false)).create(regionName);
+      } else if (evictionAttributes == null) {
         return cache.createRegionFactory(serverRegionShortcut)
             .setPartitionAttributes(getPartitionAttributes(false)).create(regionName);
       } else {


[49/49] geode git commit: GEODE-2536: Remove the inappropriate implementation of methods in DiskId for persistent regions.

Posted by kh...@apache.org.
GEODE-2536: Remove the inappropriate implementation of methods in DiskId for persistent regions.

markForWriting and unmarkForWriting should not be used for persistent region.
needsToBeWritten always return false now for persistent region, as the diskEntry either is being written to disk (sync) or sheduled to be written to disk (async)


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/26700b5d
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/26700b5d
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/26700b5d

Branch: refs/heads/feature/GEODE-2420
Commit: 26700b5d0df5468d58acbde6f82ca331544c760f
Parents: fca17e5
Author: eshu <es...@pivotal.io>
Authored: Thu Mar 16 15:07:38 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:46 2017 -0700

----------------------------------------------------------------------
 .../apache/geode/internal/cache/DiskEntry.java  |  1 +
 .../org/apache/geode/internal/cache/DiskId.java | 26 +++++---------------
 .../geode/internal/cache/DiskStoreImpl.java     |  3 +--
 .../geode/internal/cache/DiskIdJUnitTest.java   | 24 ++++++++++++++++++
 4 files changed, 32 insertions(+), 22 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/26700b5d/geode-core/src/main/java/org/apache/geode/internal/cache/DiskEntry.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/DiskEntry.java b/geode-core/src/main/java/org/apache/geode/internal/cache/DiskEntry.java
index bde6a32..d00f7a0 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/DiskEntry.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/DiskEntry.java
@@ -1690,6 +1690,7 @@ public interface DiskEntry extends RegionEntry {
 
       // Asif: This will convert the -ve OplogKeyId to positive as part of fixing
       // Bug # 39989
+      // GEODE-2535 fix should address the original #39989 negative keyId issue.
       did.unmarkForWriting();
 
       // System.out.println("DEBUG: removeFromDisk doing remove(" + id + ")");

http://git-wip-us.apache.org/repos/asf/geode/blob/26700b5d/geode-core/src/main/java/org/apache/geode/internal/cache/DiskId.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/DiskId.java b/geode-core/src/main/java/org/apache/geode/internal/cache/DiskId.java
index e802ac9..8d2c675 100755
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/DiskId.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/DiskId.java
@@ -558,24 +558,17 @@ public abstract class DiskId {
 
     @Override
     void markForWriting() {
-      if (this.keyId > DiskRegion.INVALID_ID) {
-        // Mark the id as needing to be written
-        // The disk remove that this section used to do caused bug 30961
-        this.setKeyId(-this.keyId);
-      }
+      throw new IllegalStateException("Should not be used for persistent region");
     }
 
     @Override
     void unmarkForWriting() {
-      if (this.keyId < DiskRegion.INVALID_ID) {
-        // Mark the id as NOT needing to be written
-        this.setKeyId(-this.keyId);
-      }
+      // Do nothing
     }
 
     @Override
     boolean needsToBeWritten() {
-      return this.keyId <= DiskRegion.INVALID_ID;
+      return false;
     }
 
     @Override
@@ -643,19 +636,12 @@ public abstract class DiskId {
 
     @Override
     void markForWriting() {
-      if (this.keyId > DiskRegion.INVALID_ID) {
-        // Mark the id as needing to be written
-        // The disk remove that this section used to do caused bug 30961
-        this.setKeyId(-this.keyId);
-      }
+      throw new IllegalStateException("Should not be used for persistent region");
     }
 
     @Override
     void unmarkForWriting() {
-      if (this.keyId < DiskRegion.INVALID_ID) {
-        // Mark the id as NOT needing to be written
-        this.setKeyId(-this.keyId);
-      }
+      // Do nothing
     }
 
     @Override
@@ -670,7 +656,7 @@ public abstract class DiskId {
 
     @Override
     boolean needsToBeWritten() {
-      return this.keyId <= DiskRegion.INVALID_ID;
+      return false;
     }
   }
   final protected static class PersistenceWithLongOffset extends PersistenceWithLongOffsetNoLL {

http://git-wip-us.apache.org/repos/asf/geode/blob/26700b5d/geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java b/geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
index cce8100..642eed3 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/DiskStoreImpl.java
@@ -704,8 +704,7 @@ public class DiskStoreImpl implements DiskStore {
     DiskRegion dr = region.getDiskRegion();
     DiskId id = entry.getDiskId();
     if (dr.isBackup() && id.getKeyId() < 0) {
-      throw new IllegalArgumentException(
-          LocalizedStrings.DiskRegion_CANT_PUT_A_KEYVALUE_PAIR_WITH_ID_0.toLocalizedString(id));
+      id.setKeyId(-id.getKeyId());
     }
     long start = async ? this.stats.startFlush() : this.stats.startWrite();
     if (!async) {

http://git-wip-us.apache.org/repos/asf/geode/blob/26700b5d/geode-core/src/test/java/org/apache/geode/internal/cache/DiskIdJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/internal/cache/DiskIdJUnitTest.java b/geode-core/src/test/java/org/apache/geode/internal/cache/DiskIdJUnitTest.java
index 195c6a2..494ccff 100755
--- a/geode-core/src/test/java/org/apache/geode/internal/cache/DiskIdJUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/internal/cache/DiskIdJUnitTest.java
@@ -14,6 +14,7 @@
  */
 package org.apache.geode.internal.cache;
 
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
 import static org.junit.Assert.*;
 
 import org.junit.Test;
@@ -219,4 +220,27 @@ public class DiskIdJUnitTest {
     return DiskId.createDiskId(1024, true /* is persistence type */, true);
   }
 
+  /**
+   * Tests unmarkForWrite for persistent region does not change keyId
+   */
+  @Test
+  public void testPersistUnmarkForWrite() {
+    DiskId diskId = getDiskId();
+    diskId.setKeyId(11);
+    diskId.unmarkForWriting();
+    long newKeyId = diskId.getKeyId();
+
+    assertEquals(11, newKeyId);
+  }
+
+  /**
+   * Tests markForWrite for persistent region failed
+   */
+  @Test
+  public void testPersistMarkForWrite() {
+    DiskId diskId = getDiskId();
+    diskId.setKeyId(11);
+    assertThatThrownBy(() -> diskId.markForWriting()).isInstanceOf(IllegalStateException.class);
+  }
+
 }


[39/49] geode git commit: GEODE-2267: enhance error output for gfsh.

Posted by kh...@apache.org.
GEODE-2267: enhance error output for gfsh.

* correctly output error message if gfsh execution has an error
* export logs should output correct log message over http connection as well


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/94ac0bc0
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/94ac0bc0
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/94ac0bc0

Branch: refs/heads/feature/GEODE-2420
Commit: 94ac0bc006730313d7f781bc6777b75e4a7452fc
Parents: 189faaa
Author: Jinmei Liao <ji...@pivotal.io>
Authored: Thu Mar 9 22:33:13 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../internal/cli/commands/ExportLogCommand.java |   9 +-
 .../cli/shell/GfshExecutionStrategy.java        |   6 +-
 .../web/controllers/ExportLogController.java    |  44 +++++-
 .../web/shell/AbstractHttpOperationInvoker.java |  19 ++-
 .../cli/commands/ExportLogsIntegrationTest.java |  71 +++++++++
 .../cli/commands/ExportLogsStatsDUnitTest.java  | 150 ++++++++++++++++++
 .../cli/commands/ExportStatsDUnitTest.java      | 153 -------------------
 .../controllers/ExportLogControllerTest.java    |  56 +++++++
 .../dunit/rules/GfshShellConnectionRule.java    |   4 +-
 .../commands/ExportLogsOverHttpDUnitTest.java   |  71 ---------
 .../ExportLogsOverHttpIntegrationTest.java      |  30 ++++
 .../ExportLogsStatsOverHttpDUnitTest.java       |  71 +++++++++
 12 files changed, 440 insertions(+), 244 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogCommand.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogCommand.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogCommand.java
index c293c33..7dfc63d 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogCommand.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/ExportLogCommand.java
@@ -99,6 +99,10 @@ public class ExportLogCommand implements CommandMarker {
       Set<DistributedMember> targetMembers =
           CliUtil.findMembersIncludingLocators(groups, memberIds);
 
+      if (targetMembers.isEmpty()) {
+        return ResultBuilder.createUserErrorResult(CliStrings.NO_MEMBERS_FOUND_MESSAGE);
+      }
+
       Map<String, Path> zipFilesFromMembers = new HashMap<>();
       for (DistributedMember server : targetMembers) {
         Region region = ExportLogsFunction.createOrGetExistingExportLogsRegion(true, cache);
@@ -121,8 +125,11 @@ public class ExportLogCommand implements CommandMarker {
         }
       }
 
-      Path tempDir = Files.createTempDirectory("exportedLogs");
+      if (zipFilesFromMembers.isEmpty()) {
+        return ResultBuilder.createUserErrorResult("No files to be exported.");
+      }
 
+      Path tempDir = Files.createTempDirectory("exportedLogs");
       // make sure the directory is created, so that even if there is no files unzipped to this dir,
       // we can
       // still zip it and send an empty zip file back to the client

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-core/src/main/java/org/apache/geode/management/internal/cli/shell/GfshExecutionStrategy.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/shell/GfshExecutionStrategy.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/shell/GfshExecutionStrategy.java
index ad78efd..d74f5d6 100755
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/shell/GfshExecutionStrategy.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/shell/GfshExecutionStrategy.java
@@ -284,12 +284,10 @@ public class GfshExecutionStrategy implements ExecutionStrategy {
       if (postExecResult != null) {
         if (Status.ERROR.equals(postExecResult.getStatus())) {
           if (logWrapper.infoEnabled()) {
-            logWrapper
-                .info("Post execution Result :: " + ResultBuilder.resultAsString(postExecResult));
+            logWrapper.info("Post execution Result :: " + postExecResult);
           }
         } else if (logWrapper.fineEnabled()) {
-          logWrapper
-              .fine("Post execution Result :: " + ResultBuilder.resultAsString(postExecResult));
+          logWrapper.fine("Post execution Result :: " + postExecResult);
         }
         commandResult = postExecResult;
       }

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java b/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java
index 0351573..527e059 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/web/controllers/ExportLogController.java
@@ -16,13 +16,16 @@
 package org.apache.geode.management.internal.web.controllers;
 
 import org.apache.commons.io.FileUtils;
+import org.apache.commons.io.IOUtils;
 import org.apache.geode.internal.lang.StringUtils;
+import org.apache.geode.management.cli.Result;
 import org.apache.geode.management.internal.cli.i18n.CliStrings;
 import org.apache.geode.management.internal.cli.result.ResultBuilder;
 import org.apache.geode.management.internal.cli.util.CommandStringBuilder;
 import org.springframework.core.io.InputStreamResource;
 import org.springframework.http.HttpHeaders;
 import org.springframework.http.HttpStatus;
+import org.springframework.http.MediaType;
 import org.springframework.http.ResponseEntity;
 import org.springframework.stereotype.Controller;
 import org.springframework.web.bind.annotation.RequestMapping;
@@ -85,21 +88,50 @@ public class ExportLogController extends AbstractCommandsController {
       command.addOption(CliStrings.EXPORT_LOGS__ENDTIME, endTime);
     }
 
-    // the result is json string from CommandResult
     String result = processCommand(command.toString());
+    return getResponse(result);
 
-    // parse the result to get the file path. This file Path should always exist in the file system
-    String filePath = ResultBuilder.fromJson(result).nextLine().trim();
+  }
+
+  ResponseEntity<InputStreamResource> getResponse(String result) {
+    // the result is json string from CommandResul
+    Result commandResult = ResultBuilder.fromJson(result);
+    if (commandResult.getStatus().equals(Result.Status.OK)) {
+      return getOKResponse(commandResult);
+
+    } else {
+      return getErrorResponse(result);
+    }
+  }
+
+  private ResponseEntity<InputStreamResource> getErrorResponse(String result) {
+    HttpHeaders respHeaders = new HttpHeaders();
+    InputStreamResource isr;// if the command is successful, the output is the filepath,
+    // else we need to send the orignal result back so that the receiver will know to turn it
+    // into a Result object
+    try {
+      isr = new InputStreamResource(IOUtils.toInputStream(result, "UTF-8"));
+      respHeaders.set(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
+      return new ResponseEntity<InputStreamResource>(isr, respHeaders, HttpStatus.OK);
+    } catch (Exception e) {
+      throw new RuntimeException("IO Error writing file to output stream", e);
+    }
+  }
 
+  private ResponseEntity<InputStreamResource> getOKResponse(Result commandResult) {
     HttpHeaders respHeaders = new HttpHeaders();
+    InputStreamResource isr;// if the command is successful, the output is the filepath,
+    String filePath = commandResult.nextLine().trim();
     File zipFile = new File(filePath);
     try {
-      InputStreamResource isr = new InputStreamResource(new FileInputStream(zipFile));
+      isr = new InputStreamResource(new FileInputStream(zipFile));
+      respHeaders.set(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_OCTET_STREAM_VALUE);
       return new ResponseEntity<InputStreamResource>(isr, respHeaders, HttpStatus.OK);
-    } catch (Exception ex) {
-      throw new RuntimeException("IOError writing file to output stream", ex);
+    } catch (Exception e) {
+      throw new RuntimeException("IO Error writing file to output stream", e);
     } finally {
       FileUtils.deleteQuietly(zipFile);
     }
   }
+
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-core/src/main/java/org/apache/geode/management/internal/web/shell/AbstractHttpOperationInvoker.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/web/shell/AbstractHttpOperationInvoker.java b/geode-core/src/main/java/org/apache/geode/management/internal/web/shell/AbstractHttpOperationInvoker.java
index f60aabc..25f972e 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/web/shell/AbstractHttpOperationInvoker.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/web/shell/AbstractHttpOperationInvoker.java
@@ -633,7 +633,7 @@ public abstract class AbstractHttpOperationInvoker implements HttpOperationInvok
     return response.getBody();
   }
 
-  protected Path downloadResponseToTempFile(ClientHttpRequest request,
+  protected Object downloadResponseToTempFile(ClientHttpRequest request,
       Map<String, ?> uriVariables) {
     final URI url = request.getURL(uriVariables);
 
@@ -645,13 +645,18 @@ public abstract class AbstractHttpOperationInvoker implements HttpOperationInvok
     };
 
     // Streams the response instead of loading it all in memory
-    ResponseExtractor<Path> responseExtractor = resp -> {
-      Path tempFile = Files.createTempFile("fileDownload", "");
-      if (tempFile.toFile().exists()) {
-        FileUtils.deleteQuietly(tempFile.toFile());
+    ResponseExtractor<Object> responseExtractor = resp -> {
+      MediaType mediaType = resp.getHeaders().getContentType();
+      if (mediaType.equals(MediaType.APPLICATION_JSON)) {
+        return org.apache.commons.io.IOUtils.toString(resp.getBody(), "UTF-8");
+      } else {
+        Path tempFile = Files.createTempFile("fileDownload", "");
+        if (tempFile.toFile().exists()) {
+          FileUtils.deleteQuietly(tempFile.toFile());
+        }
+        Files.copy(resp.getBody(), tempFile);
+        return tempFile;
       }
-      Files.copy(resp.getBody(), tempFile);
-      return tempFile;
     };
     return getRestTemplate().execute(url, org.springframework.http.HttpMethod.GET, requestCallback,
         responseExtractor);

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsIntegrationTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsIntegrationTest.java
new file mode 100644
index 0000000..46a07ad
--- /dev/null
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsIntegrationTest.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.geode.management.internal.cli.commands;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import org.apache.geode.distributed.ConfigurationProperties;
+import org.apache.geode.internal.AvailablePortHelper;
+import org.apache.geode.test.dunit.rules.GfshShellConnectionRule;
+import org.apache.geode.test.dunit.rules.LocatorStarterRule;
+import org.apache.geode.test.junit.categories.IntegrationTest;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import java.util.Properties;
+
+@Category(IntegrationTest.class)
+public class ExportLogsIntegrationTest {
+
+  @ClassRule
+  public static LocatorStarterRule locator = new LocatorStarterRule();
+
+  @Rule
+  public GfshShellConnectionRule gfsh = new GfshShellConnectionRule();
+
+  private static int[] ports = AvailablePortHelper.getRandomAvailableTCPPorts(2);
+  protected static int httpPort = ports[0];
+  protected static int jmxPort = ports[1];
+
+  @BeforeClass
+  public static void before() throws Exception {
+    Properties properties = new Properties();
+    properties.setProperty(ConfigurationProperties.HTTP_SERVICE_PORT, httpPort + "");
+    properties.setProperty(ConfigurationProperties.JMX_MANAGER_PORT, jmxPort + "");
+    locator.startLocator(properties);
+  }
+
+  protected void connect() throws Exception {
+    gfsh.connectAndVerify(locator);
+  }
+
+  @Test
+  public void testInvalidMember() throws Exception {
+    connect();
+    gfsh.executeCommand("export logs --member=member1,member2");
+    assertThat(gfsh.getGfshOutput()).contains("No Members Found");
+  }
+
+  @Test
+  public void testNothingToExport() throws Exception {
+    connect();
+    gfsh.executeCommand("export logs --stats-only");
+    assertThat(gfsh.getGfshOutput()).contains("No files to be exported.");
+  }
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsDUnitTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsDUnitTest.java
new file mode 100644
index 0000000..de2ea64
--- /dev/null
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsDUnitTest.java
@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.geode.management.internal.cli.commands;
+
+import static org.apache.geode.distributed.ConfigurationProperties.HTTP_SERVICE_BIND_ADDRESS;
+import static org.apache.geode.distributed.ConfigurationProperties.HTTP_SERVICE_PORT;
+import static org.apache.geode.distributed.ConfigurationProperties.JMX_MANAGER_PORT;
+import static org.apache.geode.distributed.ConfigurationProperties.STATISTIC_ARCHIVE_FILE;
+import static org.apache.geode.management.internal.cli.commands.ExportLogCommand.ONLY_DATE_FORMAT;
+import static org.assertj.core.api.Assertions.assertThat;
+
+import com.google.common.collect.Sets;
+
+import org.apache.geode.distributed.ConfigurationProperties;
+import org.apache.geode.internal.AvailablePortHelper;
+import org.apache.geode.management.internal.cli.util.CommandStringBuilder;
+import org.apache.geode.test.dunit.rules.GfshShellConnectionRule;
+import org.apache.geode.test.dunit.rules.LocatorServerStartupRule;
+import org.apache.geode.test.dunit.rules.MemberVM;
+import org.apache.geode.test.junit.categories.DistributedTest;
+import org.junit.BeforeClass;
+import org.junit.ClassRule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import java.io.IOException;
+import java.time.LocalDateTime;
+import java.time.ZoneId;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.HashSet;
+import java.util.Properties;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.zip.ZipEntry;
+import java.util.zip.ZipFile;
+
+@Category(DistributedTest.class)
+public class ExportLogsStatsDUnitTest {
+  @ClassRule
+  public static LocatorServerStartupRule lsRule = new LocatorServerStartupRule();
+
+  @ClassRule
+  public static GfshShellConnectionRule connector = new GfshShellConnectionRule();
+
+  protected static int jmxPort, httpPort;
+  protected static Set<String> expectedZipEntries = new HashSet<>();
+  protected static MemberVM locator;
+
+  @BeforeClass
+  public static void beforeClass() throws Exception {
+    int[] ports = AvailablePortHelper.getRandomAvailableTCPPorts(2);
+    httpPort = ports[0];
+    jmxPort = ports[1];
+    Properties locatorProperties = new Properties();
+    locatorProperties.setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
+    locatorProperties.setProperty(HTTP_SERVICE_PORT, httpPort + "");
+    locatorProperties.setProperty(JMX_MANAGER_PORT, jmxPort + "");
+
+    // start the locator in vm0 and then connect to it over http
+    locator = lsRule.startLocatorVM(0, locatorProperties);
+
+    Properties serverProperties = new Properties();
+    serverProperties.setProperty(ConfigurationProperties.STATISTIC_SAMPLING_ENABLED, "true");
+    serverProperties.setProperty(STATISTIC_ARCHIVE_FILE, "statistics.gfs");
+    lsRule.startServerVM(1, serverProperties, locator.getPort());
+
+    expectedZipEntries = Sets.newHashSet("locator-0/locator-0.log", "server-1/server-1.log",
+        "server-1/statistics.gfs");
+  }
+
+  protected void connectIfNeeded() throws Exception {
+    if (!connector.isConnected()) {
+      connector.connect(locator);
+    }
+  }
+
+  @Test
+  public void testExportLogsAndStats() throws Exception {
+    connectIfNeeded();
+    connector.executeAndVerifyCommand("export logs");
+    String zipPath = getZipPathFromCommandResult(connector.getGfshOutput());
+    Set<String> actualZipEnries = getZipEntries(zipPath);
+
+    Set<String> expectedFiles = Sets.newHashSet("locator-0/locator-0.log", "server-1/server-1.log",
+        "server-1/statistics.gfs");
+    assertThat(actualZipEnries).isEqualTo(expectedFiles);
+  }
+
+  @Test
+  public void testExportLogsOnly() throws Exception {
+    connectIfNeeded();
+    connector.executeAndVerifyCommand("export logs --logs-only");
+    String zipPath = getZipPathFromCommandResult(connector.getGfshOutput());
+    Set<String> actualZipEnries = getZipEntries(zipPath);
+
+    Set<String> expectedFiles = Sets.newHashSet("locator-0/locator-0.log", "server-1/server-1.log");
+    assertThat(actualZipEnries).isEqualTo(expectedFiles);
+  }
+
+  @Test
+  public void testExportStatsOnly() throws Exception {
+    connectIfNeeded();
+    connector.executeAndVerifyCommand("export logs --stats-only");
+    String zipPath = getZipPathFromCommandResult(connector.getGfshOutput());
+    Set<String> actualZipEnries = getZipEntries(zipPath);
+
+    Set<String> expectedFiles = Sets.newHashSet("server-1/statistics.gfs");
+    assertThat(actualZipEnries).isEqualTo(expectedFiles);
+  }
+
+  @Test
+  public void startAndEndDateCanExcludeLogs() throws Exception {
+    connectIfNeeded();
+    ZonedDateTime now = LocalDateTime.now().atZone(ZoneId.systemDefault());
+    ZonedDateTime yesterday = now.minusDays(1);
+    ZonedDateTime twoDaysAgo = now.minusDays(2);
+
+    DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern(ONLY_DATE_FORMAT);
+
+    CommandStringBuilder commandStringBuilder = new CommandStringBuilder("export logs");
+    commandStringBuilder.addOption("start-time", dateTimeFormatter.format(twoDaysAgo));
+    commandStringBuilder.addOption("end-time", dateTimeFormatter.format(yesterday));
+    commandStringBuilder.addOption("log-level", "debug");
+
+    String output = connector.execute(commandStringBuilder.toString());
+    assertThat(output).contains("No files to be exported");
+  }
+
+  protected String getZipPathFromCommandResult(String message) {
+    return message.replaceAll("Logs exported to the connected member's file system: ", "").trim();
+  }
+
+  private static Set<String> getZipEntries(String zipFilePath) throws IOException {
+    return new ZipFile(zipFilePath).stream().map(ZipEntry::getName).collect(Collectors.toSet());
+  }
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportStatsDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportStatsDUnitTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportStatsDUnitTest.java
deleted file mode 100644
index b7f8c2a..0000000
--- a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/ExportStatsDUnitTest.java
+++ /dev/null
@@ -1,153 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-
-package org.apache.geode.management.internal.cli.commands;
-
-import static org.apache.geode.distributed.ConfigurationProperties.HTTP_SERVICE_BIND_ADDRESS;
-import static org.apache.geode.distributed.ConfigurationProperties.HTTP_SERVICE_PORT;
-import static org.apache.geode.distributed.ConfigurationProperties.JMX_MANAGER_PORT;
-import static org.apache.geode.distributed.ConfigurationProperties.STATISTIC_ARCHIVE_FILE;
-import static org.apache.geode.management.internal.cli.commands.ExportLogCommand.ONLY_DATE_FORMAT;
-import static org.assertj.core.api.Assertions.assertThat;
-
-import com.google.common.collect.Sets;
-
-import org.apache.geode.distributed.ConfigurationProperties;
-import org.apache.geode.internal.AvailablePortHelper;
-import org.apache.geode.management.internal.cli.util.CommandStringBuilder;
-import org.apache.geode.test.dunit.rules.GfshShellConnectionRule;
-import org.apache.geode.test.dunit.rules.LocatorServerStartupRule;
-import org.apache.geode.test.dunit.rules.MemberVM;
-import org.apache.geode.test.junit.categories.DistributedTest;
-import org.junit.BeforeClass;
-import org.junit.ClassRule;
-import org.junit.Test;
-import org.junit.experimental.categories.Category;
-
-import java.io.IOException;
-import java.time.LocalDateTime;
-import java.time.ZoneId;
-import java.time.ZonedDateTime;
-import java.time.format.DateTimeFormatter;
-import java.util.HashSet;
-import java.util.Properties;
-import java.util.Set;
-import java.util.stream.Collectors;
-import java.util.zip.ZipEntry;
-import java.util.zip.ZipFile;
-
-@Category(DistributedTest.class)
-public class ExportStatsDUnitTest {
-  @ClassRule
-  public static LocatorServerStartupRule lsRule = new LocatorServerStartupRule();
-
-  @ClassRule
-  public static GfshShellConnectionRule connector = new GfshShellConnectionRule();
-
-  protected static int jmxPort, httpPort;
-  protected static Set<String> expectedZipEntries = new HashSet<>();
-  protected static MemberVM locator;
-
-  @BeforeClass
-  public static void beforeClass() throws Exception {
-    int[] ports = AvailablePortHelper.getRandomAvailableTCPPorts(2);
-    httpPort = ports[0];
-    jmxPort = ports[1];
-    Properties locatorProperties = new Properties();
-    locatorProperties.setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
-    locatorProperties.setProperty(HTTP_SERVICE_PORT, httpPort + "");
-    locatorProperties.setProperty(JMX_MANAGER_PORT, jmxPort + "");
-
-    // start the locator in vm0 and then connect to it over http
-    locator = lsRule.startLocatorVM(0, locatorProperties);
-
-    Properties serverProperties = new Properties();
-    serverProperties.setProperty(ConfigurationProperties.STATISTIC_SAMPLING_ENABLED, "true");
-    serverProperties.setProperty(STATISTIC_ARCHIVE_FILE, "statistics.gfs");
-    lsRule.startServerVM(1, serverProperties, locator.getPort());
-
-    expectedZipEntries = Sets.newHashSet("locator-0/locator-0.log", "server-1/server-1.log",
-        "server-1/statistics.gfs");
-  }
-
-  protected void connectIfNeeded() throws Exception {
-    if (!connector.isConnected()) {
-      connector.connect(locator);
-    }
-  }
-
-  @Test
-  public void testExportLogsAndStats() throws Exception {
-    connectIfNeeded();
-    connector.executeAndVerifyCommand("export logs");
-    String zipPath = getZipPathFromCommandResult(connector.getGfshOutput());
-    Set<String> actualZipEnries = getZipEntries(zipPath);
-
-    Set<String> expectedFiles = Sets.newHashSet("locator-0/locator-0.log", "server-1/server-1.log",
-        "server-1/statistics.gfs");
-    assertThat(actualZipEnries).isEqualTo(expectedFiles);
-  }
-
-  @Test
-  public void testExportLogsOnly() throws Exception {
-    connectIfNeeded();
-    connector.executeAndVerifyCommand("export logs --logs-only");
-    String zipPath = getZipPathFromCommandResult(connector.getGfshOutput());
-    Set<String> actualZipEnries = getZipEntries(zipPath);
-
-    Set<String> expectedFiles = Sets.newHashSet("locator-0/locator-0.log", "server-1/server-1.log");
-    assertThat(actualZipEnries).isEqualTo(expectedFiles);
-  }
-
-  @Test
-  public void testExportStatsOnly() throws Exception {
-    connectIfNeeded();
-    connector.executeAndVerifyCommand("export logs --stats-only");
-    String zipPath = getZipPathFromCommandResult(connector.getGfshOutput());
-    Set<String> actualZipEnries = getZipEntries(zipPath);
-
-    Set<String> expectedFiles = Sets.newHashSet("server-1/statistics.gfs");
-    assertThat(actualZipEnries).isEqualTo(expectedFiles);
-  }
-
-  @Test
-  public void startAndEndDateCanExcludeLogs() throws Exception {
-    connectIfNeeded();
-    ZonedDateTime now = LocalDateTime.now().atZone(ZoneId.systemDefault());
-    ZonedDateTime yesterday = now.minusDays(1);
-    ZonedDateTime twoDaysAgo = now.minusDays(2);
-
-    DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern(ONLY_DATE_FORMAT);
-
-    CommandStringBuilder commandStringBuilder = new CommandStringBuilder("export logs");
-    commandStringBuilder.addOption("start-time", dateTimeFormatter.format(twoDaysAgo));
-    commandStringBuilder.addOption("end-time", dateTimeFormatter.format(yesterday));
-    commandStringBuilder.addOption("log-level", "debug");
-
-    connector.executeAndVerifyCommand(commandStringBuilder.toString());
-    String zipPath = getZipPathFromCommandResult(connector.getGfshOutput());
-
-    Set<String> actualZipEnries = getZipEntries(zipPath);
-    assertThat(actualZipEnries).hasSize(0);
-  }
-
-  protected String getZipPathFromCommandResult(String message) {
-    return message.replaceAll("Logs exported to the connected member's file system: ", "").trim();
-  }
-
-  private static Set<String> getZipEntries(String zipFilePath) throws IOException {
-    return new ZipFile(zipFilePath).stream().map(ZipEntry::getName).collect(Collectors.toSet());
-  }
-}

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-core/src/test/java/org/apache/geode/management/internal/web/controllers/ExportLogControllerTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/web/controllers/ExportLogControllerTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/web/controllers/ExportLogControllerTest.java
new file mode 100644
index 0000000..bee7db2
--- /dev/null
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/web/controllers/ExportLogControllerTest.java
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.geode.management.internal.web.controllers;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.geode.management.internal.cli.CommandResponseBuilder;
+import org.apache.geode.management.internal.cli.result.CommandResult;
+import org.apache.geode.management.internal.cli.result.ResultBuilder;
+import org.apache.geode.test.junit.categories.UnitTest;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.springframework.core.io.InputStreamResource;
+import org.springframework.http.HttpHeaders;
+import org.springframework.http.MediaType;
+import org.springframework.http.ResponseEntity;
+
+@Category(UnitTest.class)
+public class ExportLogControllerTest {
+  private ExportLogController controller;
+
+  @Before
+  public void before() throws Exception {
+    controller = new ExportLogController();
+  }
+
+  @Test
+  public void testErrorResponse() throws Exception {
+    String message = "No Members Found";
+    CommandResult result = (CommandResult) ResultBuilder.createUserErrorResult(message);
+    String responseJson = CommandResponseBuilder.createCommandResponseJson("memberName", result);
+
+    ResponseEntity<InputStreamResource> resp = controller.getResponse(responseJson);
+    HttpHeaders headers = resp.getHeaders();
+    assertThat(headers.get(HttpHeaders.CONTENT_TYPE).get(0))
+        .isEqualTo(MediaType.APPLICATION_JSON_VALUE);
+
+    InputStreamResource body = resp.getBody();
+    assertThat(IOUtils.toString(body.getInputStream(), "utf-8")).isEqualTo(responseJson);
+  }
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-core/src/test/java/org/apache/geode/test/dunit/rules/GfshShellConnectionRule.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/test/dunit/rules/GfshShellConnectionRule.java b/geode-core/src/test/java/org/apache/geode/test/dunit/rules/GfshShellConnectionRule.java
index f367458..3b1c99a 100644
--- a/geode-core/src/test/java/org/apache/geode/test/dunit/rules/GfshShellConnectionRule.java
+++ b/geode-core/src/test/java/org/apache/geode/test/dunit/rules/GfshShellConnectionRule.java
@@ -86,11 +86,11 @@ public class GfshShellConnectionRule extends DescribedExternalResource {
 
   }
 
-  public void connect(MemberVM locator, String... options) throws Exception {
+  public void connect(Member locator, String... options) throws Exception {
     connect(locator.getPort(), PortType.locator, options);
   }
 
-  public void connectAndVerify(MemberVM locator, String... options) throws Exception {
+  public void connectAndVerify(Member locator, String... options) throws Exception {
     connect(locator.getPort(), PortType.locator, options);
     assertThat(this.connected).isTrue();
   }

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpDUnitTest.java b/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpDUnitTest.java
deleted file mode 100644
index cc4ae28..0000000
--- a/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpDUnitTest.java
+++ /dev/null
@@ -1,71 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-
-package org.apache.geode.management.internal.cli.commands;
-
-import static org.assertj.core.api.Assertions.assertThat;
-
-import org.apache.geode.test.dunit.rules.GfshShellConnectionRule;
-import org.apache.geode.test.junit.categories.DistributedTest;
-import org.junit.Rule;
-import org.junit.Test;
-import org.junit.experimental.categories.Category;
-import org.junit.rules.TemporaryFolder;
-
-import java.io.File;
-import java.util.Arrays;
-import java.util.Set;
-import java.util.stream.Collectors;
-import java.util.zip.ZipEntry;
-import java.util.zip.ZipFile;
-
-@Category(DistributedTest.class)
-public class ExportLogsOverHttpDUnitTest extends ExportStatsDUnitTest {
-
-  @Rule
-  public TemporaryFolder temporaryFolder = new TemporaryFolder();
-
-  @Override
-  public void connectIfNeeded() throws Exception {
-    if (!connector.isConnected())
-      connector.connect(httpPort, GfshShellConnectionRule.PortType.http);
-  }
-
-  @Test
-  public void testExportWithDir() throws Exception {
-    connectIfNeeded();
-    File dir = temporaryFolder.newFolder();
-    // export the logs
-    connector.executeCommand("export logs --dir=" + dir.getAbsolutePath());
-    // verify that the message contains a path to the user.dir
-    String message = connector.getGfshOutput();
-    assertThat(message).contains("Logs exported to: ");
-    assertThat(message).contains(dir.getAbsolutePath());
-
-    String zipPath = getZipPathFromCommandResult(message);
-    Set<String> actualZipEntries =
-        new ZipFile(zipPath).stream().map(ZipEntry::getName).collect(Collectors.toSet());
-
-    assertThat(actualZipEntries).isEqualTo(expectedZipEntries);
-
-    // also verify that the zip file on locator is deleted
-    assertThat(Arrays.stream(locator.getWorkingDir().listFiles())
-        .filter(file -> file.getName().endsWith(".zip")).collect(Collectors.toSet())).isEmpty();
-  }
-
-  protected String getZipPathFromCommandResult(String message) {
-    return message.replaceAll("Logs exported to: ", "").trim();
-  }
-}

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpIntegrationTest.java b/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpIntegrationTest.java
new file mode 100644
index 0000000..420f2dd
--- /dev/null
+++ b/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsOverHttpIntegrationTest.java
@@ -0,0 +1,30 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.geode.management.internal.cli.commands;
+
+import org.apache.geode.test.dunit.rules.GfshShellConnectionRule;
+import org.apache.geode.test.junit.categories.IntegrationTest;
+import org.junit.experimental.categories.Category;
+
+@Category(IntegrationTest.class)
+public class ExportLogsOverHttpIntegrationTest extends ExportLogsIntegrationTest {
+
+  @Override
+  protected void connect() throws Exception {
+    gfsh.connectAndVerify(httpPort, GfshShellConnectionRule.PortType.http);
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/94ac0bc0/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsOverHttpDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsOverHttpDUnitTest.java b/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsOverHttpDUnitTest.java
new file mode 100644
index 0000000..c0b08cf
--- /dev/null
+++ b/geode-web/src/test/java/org/apache/geode/management/internal/cli/commands/ExportLogsStatsOverHttpDUnitTest.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+
+package org.apache.geode.management.internal.cli.commands;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import org.apache.geode.test.dunit.rules.GfshShellConnectionRule;
+import org.apache.geode.test.junit.categories.DistributedTest;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TemporaryFolder;
+
+import java.io.File;
+import java.util.Arrays;
+import java.util.Set;
+import java.util.stream.Collectors;
+import java.util.zip.ZipEntry;
+import java.util.zip.ZipFile;
+
+@Category(DistributedTest.class)
+public class ExportLogsStatsOverHttpDUnitTest extends ExportLogsStatsDUnitTest {
+
+  @Rule
+  public TemporaryFolder temporaryFolder = new TemporaryFolder();
+
+  @Override
+  public void connectIfNeeded() throws Exception {
+    if (!connector.isConnected())
+      connector.connect(httpPort, GfshShellConnectionRule.PortType.http);
+  }
+
+  @Test
+  public void testExportWithDir() throws Exception {
+    connectIfNeeded();
+    File dir = temporaryFolder.newFolder();
+    // export the logs
+    connector.executeCommand("export logs --dir=" + dir.getAbsolutePath());
+    // verify that the message contains a path to the user.dir
+    String message = connector.getGfshOutput();
+    assertThat(message).contains("Logs exported to: ");
+    assertThat(message).contains(dir.getAbsolutePath());
+
+    String zipPath = getZipPathFromCommandResult(message);
+    Set<String> actualZipEntries =
+        new ZipFile(zipPath).stream().map(ZipEntry::getName).collect(Collectors.toSet());
+
+    assertThat(actualZipEntries).isEqualTo(expectedZipEntries);
+
+    // also verify that the zip file on locator is deleted
+    assertThat(Arrays.stream(locator.getWorkingDir().listFiles())
+        .filter(file -> file.getName().endsWith(".zip")).collect(Collectors.toSet())).isEmpty();
+  }
+
+  protected String getZipPathFromCommandResult(String message) {
+    return message.replaceAll("Logs exported to: ", "").trim();
+  }
+}


[27/49] geode git commit: GEODE_1793 spotless fixes and removal of dead code

Posted by kh...@apache.org.
GEODE_1793 spotless fixes and removal of dead code


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/f7aad2e4
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/f7aad2e4
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/f7aad2e4

Branch: refs/heads/feature/GEODE-2420
Commit: f7aad2e406f32a4c9171cc61937aaf9274e4d0bc
Parents: c47761d
Author: Bruce Schuchardt <bs...@pivotal.io>
Authored: Fri Mar 3 14:50:33 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../internal/tcpserver/TcpServer.java            |  6 +++---
 .../geode/distributed/LocatorDUnitTest.java      | 19 +------------------
 2 files changed, 4 insertions(+), 21 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/f7aad2e4/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java b/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java
index 8f1b537..9845bf1 100755
--- a/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java
+++ b/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java
@@ -89,7 +89,7 @@ public class TcpServer {
   // This GOSSIPVERSION is used in _getVersionForAddress request for getting GemFire version of a
   // GossipServer.
   public final static int OLDGOSSIPVERSION = 1001;
-  
+
   private static/* GemStoneAddition */ final Map GOSSIP_TO_GEMFIRE_VERSION_MAP = new HashMap();
 
   // For test purpose only
@@ -364,8 +364,8 @@ public class TcpServer {
             sock.getOutputStream().write("unknown protocol version".getBytes());
             sock.getOutputStream().flush();
           } catch (IOException e) {
-            log.debug("exception in sending reply to process using unknown protocol "
-                + gossipVersion, e);
+            log.debug(
+                "exception in sending reply to process using unknown protocol " + gossipVersion, e);
           }
           sock.close();
           return;

http://git-wip-us.apache.org/repos/asf/geode/blob/f7aad2e4/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java b/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java
index 5812e74..6f01f90 100644
--- a/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java
@@ -308,14 +308,11 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
   private Boolean startLocatorWithPortAndProperties(final int port, final Properties properties)
       throws IOException {
     try {
-//      System.setProperty("p2p.joinTimeout", "5000"); // set a short join timeout. default is 17000ms
       Locator.startLocatorAndDS(port, new File(""), properties);
     } catch (SystemConnectException e) {
       return Boolean.FALSE;
     } catch (GemFireConfigException e) {
       return Boolean.FALSE;
-    } finally {
-//      System.getProperties().remove("p2p.joinTimeout");
     }
     return Boolean.TRUE;
   }
@@ -635,12 +632,7 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
   private Locator startLocatorInternal(final int port, final Properties properties)
       throws IOException {
     Locator locator;
-    try {
-//      System.setProperty("p2p.joinTimeout", "5000"); // set a short join timeout. default is 17000ms
-      locator = Locator.startLocatorAndDS(port, new File(""), properties);
-    } finally {
-//      System.getProperties().remove("p2p.joinTimeout");
-    }
+    locator = Locator.startLocatorAndDS(port, new File(""), properties);
     return locator;
   }
 
@@ -1305,7 +1297,6 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
     bgexecLogger.info(addExpected);
 
     boolean exceptionOccurred = true;
-//    String oldValue = (String) System.getProperties().put("p2p.joinTimeout", "15000");
     try {
       DistributedSystem.connect(props);
       exceptionOccurred = false;
@@ -1322,11 +1313,6 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
       // if expected then add in a catch block for it above this catch
       org.apache.geode.test.dunit.Assert.fail("Failed with unexpected exception", ex);
     } finally {
-//      if (oldValue == null) {
-//        System.getProperties().remove("p2p.joinTimeout");
-//      } else {
-//        System.getProperties().put("p2p.joinTimeout", oldValue);
-//      }
       bgexecLogger.info(removeExpected);
     }
 
@@ -1360,7 +1346,6 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
 
       SerializableRunnable connect = new SerializableRunnable("Connect to " + locators) {
         public void run() {
-          // System.setProperty("p2p.joinTimeout", "5000");
           Properties props = new Properties();
           props.setProperty(MCAST_PORT, "0");
           props.setProperty(LOCATORS, locators);
@@ -1962,7 +1947,6 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
     File logFile = new File("");
     try {
       System.setProperty(InternalLocator.LOCATORS_PREFERRED_AS_COORDINATORS, "true");
-//      System.setProperty("p2p.joinTimeout", "1000");
       Properties locProps = new Properties();
       locProps.put(MCAST_PORT, "0");
       locProps.put(LOG_LEVEL, LogWriterUtils.getDUnitLogLevel());
@@ -1972,7 +1956,6 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
       org.apache.geode.test.dunit.Assert.fail("While starting locator on port " + port, ex);
     } finally {
       System.getProperties().remove(InternalLocator.LOCATORS_PREFERRED_AS_COORDINATORS);
-//      System.getProperties().remove("p2p.joinTimeout");
     }
   }
 


[08/49] geode git commit: GEODE-1969: fix IOException on force flush

Posted by kh...@apache.org.
GEODE-1969: fix IOException on force flush

An oplog closed while writing to oplog with gemfire.syncWrites=true
whill no longer cause the force flush to fail with an
IOException.

The channel force call is now done inside the sync block
which makes sure that the write is flushed before another
thread can close the channel.

This closes #320


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/019ee40d
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/019ee40d
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/019ee40d

Branch: refs/heads/feature/GEODE-2420
Commit: 019ee40d676d65e7474581dd0bd357364094de2f
Parents: 3f2de08
Author: shankar <sh...@ampool.io>
Authored: Fri Dec 16 16:31:54 2016 +0530
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../main/java/org/apache/geode/internal/cache/Oplog.java  | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/019ee40d/geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java b/geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
index 8b0bba5..27f41a2 100755
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/Oplog.java
@@ -5275,11 +5275,11 @@ public final class Oplog implements CompactableOplog, Flushable {
           olf.bytesFlushed += flushed;
           bb.clear();
         }
-      }
-      if (doSync) {
-        if (SYNC_WRITES) {
-          // Synch Meta Data as well as content
-          olf.channel.force(true);
+        if (doSync) {
+          if (SYNC_WRITES) {
+            // Synch Meta Data as well as content
+            olf.channel.force(true);
+          }
         }
       }
     } catch (ClosedChannelException ignore) {


[43/49] geode git commit: GEODE-2539: Upgrading Jetty causes RestSecurityIntegrationTest to fail (#3)

Posted by kh...@apache.org.
GEODE-2539: Upgrading Jetty causes RestSecurityIntegrationTest to fail (#3)


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/04526476
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/04526476
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/04526476

Branch: refs/heads/feature/GEODE-2420
Commit: 04526476b5cf5e73bcf6d30dcc5f168b9c51df78
Parents: c02970b
Author: Kevin J. Duling <kd...@pivotal.io>
Authored: Mon Mar 13 13:49:08 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 gradle/dependency-versions.properties | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/04526476/gradle/dependency-versions.properties
----------------------------------------------------------------------
diff --git a/gradle/dependency-versions.properties b/gradle/dependency-versions.properties
index 7613dba..da8bdf2 100644
--- a/gradle/dependency-versions.properties
+++ b/gradle/dependency-versions.properties
@@ -52,7 +52,7 @@ javax.resource-api.version = 1.7
 javax.servlet-api.version = 3.1.0
 javax.transaction-api.version = 1.2
 jedis.version = 2.9.0
-jetty.version = 9.3.11.v20160721
+jetty.version = 9.3.10.v20160621
 jgroups.version = 3.6.10.Final
 jmock.version = 2.8.2
 jna.version = 4.0.0


[14/49] geode git commit: GEODE-2614: fix javadoc warnings

Posted by kh...@apache.org.
GEODE-2614: fix javadoc warnings


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/04f9350e
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/04f9350e
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/04f9350e

Branch: refs/heads/feature/GEODE-2420
Commit: 04f9350e70b64b38e6de3b74c68f7d9495514982
Parents: caf0767
Author: Jinmei Liao <ji...@pivotal.io>
Authored: Wed Mar 8 09:50:06 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../apache/geode/management/internal/cli/CliAroundInterceptor.java  | 1 -
 1 file changed, 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/04f9350e/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
index 7a95306..f552460 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/CliAroundInterceptor.java
@@ -38,7 +38,6 @@ public interface CliAroundInterceptor {
 
   /**
    * called by the OperationInvoker after the command is executed
-   * 
    * @param tempFile if the command's isFileDownloadOverHttp is true, the is the File downloaded
    *        after the http response is processed.
    */


[05/49] geode git commit: GEODE-1198: add todo comment to dubious test

Posted by kh...@apache.org.
GEODE-1198: add todo comment to dubious test


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/54eab9a0
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/54eab9a0
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/54eab9a0

Branch: refs/heads/feature/GEODE-2420
Commit: 54eab9a0cd8cc0adde84e144b6dbd035d63a3192
Parents: 8afd0d6
Author: Kirk Lund <kl...@apache.org>
Authored: Tue Mar 7 14:34:47 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:08:56 2017 -0700

----------------------------------------------------------------------
 .../org/apache/geode/distributed/DistributedSystemDUnitTest.java   | 2 ++
 1 file changed, 2 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/54eab9a0/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java b/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
index dfe4588..9a64f53 100644
--- a/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/distributed/DistributedSystemDUnitTest.java
@@ -124,6 +124,8 @@ public class DistributedSystemDUnitTest extends JUnit4DistributedTestCase {
       }
     });
 
+    // TODO: assert that queue was created or this test is just broken
+
     assertThat(distributionManager.getMembershipManager().waitForDeparture(member))
         .as("expected the serial queue to be flushed").isTrue();
   }


[35/49] geode git commit: GEODE-2655: Added DUnit tests for lucene indexes on mixed objects.

Posted by kh...@apache.org.
GEODE-2655: Added DUnit tests for lucene indexes on mixed objects.

	* Lucene must be able to index on the common field name of mixed objects
	* Lucene indexes must also work when there are mixed objects with no common field names.
	* Lucene indexes work with different objects, different data type, but same field name.

	This closes #424


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/fca17e56
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/fca17e56
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/fca17e56

Branch: refs/heads/feature/GEODE-2420
Commit: fca17e56f5f8b13ef1665b5211622089a55c29e4
Parents: d5a3635
Author: nabarun <nn...@pivotal.io>
Authored: Tue Mar 14 00:02:51 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../lucene/LuceneQueriesIntegrationTest.java    |  56 ---
 .../cache/lucene/MixedObjectIndexDUnitTest.java | 458 +++++++++++++++++++
 .../cache/lucene/test/LuceneTestUtilities.java  |  51 +++
 3 files changed, 509 insertions(+), 56 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/fca17e56/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java
index 5b85c19..9db0a5e 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesIntegrationTest.java
@@ -17,9 +17,6 @@ package org.apache.geode.cache.lucene;
 import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.*;
 import static org.junit.Assert.*;
 
-import java.io.DataInput;
-import java.io.DataOutput;
-import java.io.IOException;
 import java.util.ArrayList;
 import java.util.HashMap;
 import java.util.HashSet;
@@ -39,20 +36,15 @@ import org.apache.lucene.analysis.core.KeywordAnalyzer;
 import org.apache.lucene.analysis.core.LowerCaseFilter;
 import org.apache.lucene.analysis.standard.StandardAnalyzer;
 import org.apache.lucene.analysis.util.CharTokenizer;
-import org.apache.lucene.document.IntPoint;
-import org.apache.lucene.search.Query;
 import org.junit.Ignore;
 import org.junit.Rule;
 import org.junit.Test;
 import org.junit.experimental.categories.Category;
 import org.junit.rules.ExpectedException;
 
-import org.apache.geode.DataSerializer;
 import org.apache.geode.cache.Region;
 import org.apache.geode.cache.RegionShortcut;
 import org.apache.geode.cache.lucene.test.TestObject;
-import org.apache.geode.internal.DataSerializableFixedID;
-import org.apache.geode.internal.Version;
 import org.apache.geode.pdx.JSONFormatter;
 import org.apache.geode.pdx.PdxInstance;
 import org.apache.geode.test.junit.categories.IntegrationTest;
@@ -515,52 +507,4 @@ public class LuceneQueriesIntegrationTest extends LuceneIntegrationTest {
       return new TokenStreamComponents(tokenizer, filter);
     }
   }
-
-  public static class IntRangeQueryProvider
-      implements LuceneQueryProvider, DataSerializableFixedID {
-    public static final short LUCENE_INT_RANGE_QUERY_PROVIDER = 2177;
-    String fieldName;
-    int lowerValue;
-    int upperValue;
-
-    private transient Query luceneQuery;
-
-    public IntRangeQueryProvider(String fieldName, int lowerValue, int upperValue) {
-      this.fieldName = fieldName;
-      this.lowerValue = lowerValue;
-      this.upperValue = upperValue;
-    }
-
-    @Override
-    public Version[] getSerializationVersions() {
-      return null;
-    }
-
-    @Override
-    public int getDSFID() {
-      return LUCENE_INT_RANGE_QUERY_PROVIDER;
-    }
-
-    @Override
-    public void toData(DataOutput out) throws IOException {
-      DataSerializer.writeString(fieldName, out);
-      out.writeInt(lowerValue);
-      out.writeInt(upperValue);
-    }
-
-    @Override
-    public void fromData(DataInput in) throws IOException, ClassNotFoundException {
-      fieldName = DataSerializer.readString(in);
-      lowerValue = in.readInt();
-      upperValue = in.readInt();
-    }
-
-    @Override
-    public Query getQuery(LuceneIndex index) throws LuceneQueryException {
-      if (luceneQuery == null) {
-        luceneQuery = IntPoint.newRangeQuery(fieldName, lowerValue, upperValue);
-      }
-      return luceneQuery;
-    }
-  }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/fca17e56/geode-lucene/src/test/java/org/apache/geode/cache/lucene/MixedObjectIndexDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/MixedObjectIndexDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/MixedObjectIndexDUnitTest.java
new file mode 100644
index 0000000..affccde
--- /dev/null
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/MixedObjectIndexDUnitTest.java
@@ -0,0 +1,458 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.lucene;
+
+
+import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.*;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+import org.apache.geode.cache.Region;
+import org.apache.geode.test.dunit.SerializableRunnableIF;
+import org.apache.geode.test.junit.categories.DistributedTest;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+
+import java.io.Serializable;
+import java.util.List;
+import java.util.stream.IntStream;
+import junitparams.JUnitParamsRunner;
+import junitparams.Parameters;
+
+@Category(DistributedTest.class)
+@RunWith(JUnitParamsRunner.class)
+public class MixedObjectIndexDUnitTest extends LuceneQueriesAccessorBase {
+
+  protected RegionTestableType[] getPartitionRegionTypes() {
+    return new RegionTestableType[] {RegionTestableType.PARTITION,
+        RegionTestableType.PARTITION_PERSISTENT, RegionTestableType.PARTITION_REDUNDANT,
+        RegionTestableType.PARTITION_REDUNDANT_PERSISTENT};
+
+  }
+
+  @Test
+  @Parameters(method = "getPartitionRegionTypes")
+  public void luceneCanIndexFieldsWithSameNameButInDifferentObjects(
+      RegionTestableType regionTestType) {
+    SerializableRunnableIF createIndex = getSerializableRunnableIFCreateIndexOnFieldText();
+
+    dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
+    dataStore2.invoke(() -> initDataStore(createIndex, regionTestType));
+
+    accessor.invoke(() -> initDataStore(createIndex, regionTestType));
+
+    accessor.invoke(() -> {
+      Region region = getCache().getRegion(REGION_NAME);
+      IntStream.range(0, NUM_BUCKETS).forEach(i -> region.put(i, new TestObject("hello world")));
+      IntStream.range(NUM_BUCKETS, (2 * NUM_BUCKETS))
+          .forEach(i -> region.put(i, new TestObjectWithSameFieldName("hello world 2")));
+    });
+
+    waitForFlushBeforeExecuteTextSearch(accessor, 60000);
+
+    accessor.invoke(() -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+      LuceneQuery luceneQuery = luceneService.createLuceneQueryFactory().setResultLimit(100)
+          .create(INDEX_NAME, REGION_NAME, "world", "text");
+      List resultList = luceneQuery.findResults();
+      int objectType_1_count = 0;
+      int objectType_2_count = 0;
+      for (Object luceneResultStruct : resultList) {
+        if (((LuceneResultStruct) luceneResultStruct).getValue() instanceof TestObject) {
+          objectType_1_count++;
+        } else if (((LuceneResultStruct) luceneResultStruct)
+            .getValue() instanceof TestObjectWithSameFieldName) {
+          objectType_2_count++;
+        }
+      }
+      assertEquals(NUM_BUCKETS, objectType_1_count);
+      assertEquals(NUM_BUCKETS, objectType_2_count);
+    });
+  }
+
+  @Test
+  @Parameters(method = "getPartitionRegionTypes")
+  public void luceneMustIndexFieldsWithMixedObjects(RegionTestableType regionTestableType) {
+    SerializableRunnableIF createIndexOnTextAndDataField =
+        getSerializableRunnableIFCreateIndexOnFieldData();
+
+    dataStore1.invoke(() -> initDataStore(createIndexOnTextAndDataField, regionTestableType));
+    dataStore2.invoke(() -> initDataStore(createIndexOnTextAndDataField, regionTestableType));
+
+    accessor.invoke(() -> initDataStore(createIndexOnTextAndDataField, regionTestableType));
+
+    accessor.invoke(() -> {
+      Region region = getCache().getRegion(REGION_NAME);
+      IntStream.range(0, NUM_BUCKETS).forEach(i -> region.put(i, new TestObject("hello world")));
+      IntStream.range(NUM_BUCKETS, 2 * NUM_BUCKETS)
+          .forEach(i -> region.put(i, new TestObjectWithSameFieldName("hello world")));
+      IntStream.range(2 * NUM_BUCKETS, 3 * NUM_BUCKETS)
+          .forEach(i -> region.put(i, new TestObjectWithNoCommonField("hello world")));
+    });
+
+
+    waitForFlushBeforeExecuteTextSearch(accessor, 60000);
+
+    accessor.invoke(() -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+      LuceneQuery luceneQueryForTextField = luceneService.createLuceneQueryFactory()
+          .setResultLimit(100).create(INDEX_NAME, REGION_NAME, "world", "text");
+      List luceneResults = luceneQueryForTextField.findResults();
+      validateObjectResultCounts(luceneResults, TestObject.class, NUM_BUCKETS,
+          TestObjectWithSameFieldName.class, NUM_BUCKETS, TestObjectWithNoCommonField.class, 0);
+
+      luceneQueryForTextField = luceneService.createLuceneQueryFactory().setResultLimit(100)
+          .create(INDEX_NAME, REGION_NAME, "world", "data");
+      luceneResults = luceneQueryForTextField.findResults();
+      validateObjectResultCounts(luceneResults, TestObject.class, 0,
+          TestObjectWithSameFieldName.class, 0, TestObjectWithNoCommonField.class, NUM_BUCKETS);
+    });
+  }
+
+
+
+  @Test
+  @Parameters(method = "getPartitionRegionTypes")
+  public void luceneMustIndexFieldsWithTheSameNameInARegionWithMixedObjects(
+      RegionTestableType regionTestableType) {
+    SerializableRunnableIF createIndexOnTextField =
+        getSerializableRunnableIFCreateIndexOnFieldText();
+
+    dataStore1.invoke(() -> initDataStore(createIndexOnTextField, regionTestableType));
+    dataStore2.invoke(() -> initDataStore(createIndexOnTextField, regionTestableType));
+
+    accessor.invoke(() -> initDataStore(createIndexOnTextField, regionTestableType));
+
+    accessor.invoke(() -> {
+      Region region = getCache().getRegion(REGION_NAME);
+      IntStream.range(0, NUM_BUCKETS).forEach(i -> region.put(i, new TestObject("hello world")));
+      IntStream.range(NUM_BUCKETS, 2 * NUM_BUCKETS)
+          .forEach(i -> region.put(i, new TestObjectWithSameFieldName("hello world")));
+      IntStream.range(2 * NUM_BUCKETS, 3 * NUM_BUCKETS)
+          .forEach(i -> region.put(i, new TestObjectWithNoCommonField("hello world")));
+    });
+
+    waitForFlushBeforeExecuteTextSearch(accessor, 60000);
+
+    accessor.invoke(() -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+      LuceneQuery luceneQueryForTextField = luceneService.createLuceneQueryFactory()
+          .setResultLimit(100).create(INDEX_NAME, REGION_NAME, "world", "text");
+      List luceneResults = luceneQueryForTextField.findResults();
+      validateObjectResultCounts(luceneResults, TestObject.class, NUM_BUCKETS,
+          TestObjectWithSameFieldName.class, NUM_BUCKETS, TestObjectWithNoCommonField.class, 0);
+    });
+
+  }
+
+  @Test
+  @Parameters(method = "getPartitionRegionTypes")
+  public void luceneMustIndexFieldsWithTheSameNameDifferentDataTypeInARegionWithMixedObjects(
+      RegionTestableType regionTestableType) {
+    SerializableRunnableIF createIndexOnTextField =
+        getSerializableRunnableIFCreateIndexOnFieldText();
+
+    dataStore1.invoke(() -> initDataStore(createIndexOnTextField, regionTestableType));
+    dataStore2.invoke(() -> initDataStore(createIndexOnTextField, regionTestableType));
+
+    accessor.invoke(() -> initDataStore(createIndexOnTextField, regionTestableType));
+
+    accessor.invoke(() -> {
+      Region region = getCache().getRegion(REGION_NAME);
+      IntStream.range(2 * NUM_BUCKETS, 3 * NUM_BUCKETS).forEach(i -> region.put(i,
+          new TestObjectSameFieldNameButDifferentDataTypeInteger(new Integer(1000))));
+      IntStream.range(0, NUM_BUCKETS).forEach(i -> region.put(i, new TestObject("hello world")));
+      IntStream.range(NUM_BUCKETS, 2 * NUM_BUCKETS).forEach(i -> region.put(i,
+          new TestObjectSameFieldNameButDifferentDataTypeFloat(new Float(999.1f))));
+    });
+
+    assertTrue(waitForFlushBeforeExecuteTextSearch(accessor, 60000));
+
+    accessor.invoke(() -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+
+      LuceneQuery luceneQueryForTextField = luceneService.createLuceneQueryFactory()
+          .setResultLimit(100).create(INDEX_NAME, REGION_NAME, "world", "text");
+
+      List luceneResults = luceneQueryForTextField.findResults();
+      validateObjectResultCounts(luceneResults, TestObject.class, NUM_BUCKETS,
+          TestObjectSameFieldNameButDifferentDataTypeFloat.class, 0,
+          TestObjectSameFieldNameButDifferentDataTypeInteger.class, 0);
+
+      FloatRangeQueryProvider floatRangeQueryProvider =
+          new FloatRangeQueryProvider("text", 999.0f, 999.2f);
+      luceneQueryForTextField = luceneService.createLuceneQueryFactory().setResultLimit(100)
+          .create(INDEX_NAME, REGION_NAME, floatRangeQueryProvider);
+
+      luceneResults = luceneQueryForTextField.findResults();
+      validateObjectResultCounts(luceneResults, TestObject.class, 0,
+          TestObjectSameFieldNameButDifferentDataTypeFloat.class, NUM_BUCKETS,
+          TestObjectSameFieldNameButDifferentDataTypeInteger.class, 0);
+
+      IntRangeQueryProvider intRangeQueryProvider = new IntRangeQueryProvider("text", 1000, 1000);
+      luceneQueryForTextField = luceneService.createLuceneQueryFactory().setResultLimit(100)
+          .create(INDEX_NAME, REGION_NAME, intRangeQueryProvider);
+
+      luceneResults = luceneQueryForTextField.findResults();
+      validateObjectResultCounts(luceneResults, TestObject.class, 0,
+          TestObjectSameFieldNameButDifferentDataTypeFloat.class, 0,
+          TestObjectSameFieldNameButDifferentDataTypeInteger.class, NUM_BUCKETS);
+    });
+  }
+
+  private void validateObjectResultCounts(List luceneResults, Class objectType_1,
+      int expectedObjectType_1_count, Class objectType_2, int expectedObjectType_2_count,
+      Class objectType_3, int expectedObjectType_3_count) {
+    int actualObjectType_1_count = 0;
+    int actualObjectType_2_count = 0;
+    int actualObjectType_3_count = 0;
+
+    for (Object luceneResult : luceneResults) {
+      Object resultValue = ((LuceneResultStruct) luceneResult).getValue();
+      if (objectType_1.isInstance(resultValue)) {
+        actualObjectType_1_count++;
+      } else if (objectType_2.isInstance(resultValue)) {
+        actualObjectType_2_count++;
+      } else if (objectType_3.isInstance(resultValue)) {
+        actualObjectType_3_count++;
+      }
+    }
+
+    assertEquals(expectedObjectType_1_count, actualObjectType_1_count);
+    assertEquals(expectedObjectType_2_count, actualObjectType_2_count);
+    assertEquals(expectedObjectType_3_count, actualObjectType_3_count);
+  }
+
+  private SerializableRunnableIF getSerializableRunnableIFCreateIndexOnFieldText() {
+    return () -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
+    };
+  }
+
+  private SerializableRunnableIF getSerializableRunnableIFCreateIndexOnFieldData() {
+    return () -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+      luceneService.createIndexFactory().setFields("data", "text").create(INDEX_NAME, REGION_NAME);
+    };
+  }
+
+
+
+  protected static class TestObject implements Serializable {
+    private static final long serialVersionUID = 1L;
+    private String text;
+
+    public TestObject(String text) {
+      this.text = text;
+    }
+
+    @Override
+    public int hashCode() {
+      final int prime = 31;
+      int result = 1;
+      result = prime * result + ((text == null) ? 0 : text.hashCode());
+      return result;
+    }
+
+    @Override
+    public boolean equals(Object obj) {
+      if (this == obj)
+        return true;
+      if (obj == null)
+        return false;
+      if (getClass() != obj.getClass())
+        return false;
+      MixedObjectIndexDUnitTest.TestObject other = (MixedObjectIndexDUnitTest.TestObject) obj;
+      if (text == null) {
+        if (other.text != null)
+          return false;
+      } else if (!text.equals(other.text))
+        return false;
+      return true;
+    }
+
+    @Override
+    public String toString() {
+      return "TestObject[" + text + "]";
+    }
+
+
+  }
+
+  protected static class TestObjectSameFieldNameButDifferentDataTypeFloat implements Serializable {
+    private static final long serialVersionUID = 1L;
+    private Float text;
+
+    public TestObjectSameFieldNameButDifferentDataTypeFloat(Float text) {
+      this.text = text;
+    }
+
+    @Override
+    public int hashCode() {
+      final int prime = 31;
+      int result = 1;
+      result = prime * result + ((text == null) ? 0 : text.hashCode());
+      return result;
+    }
+
+    @Override
+    public boolean equals(Object obj) {
+      if (this == obj)
+        return true;
+      if (obj == null)
+        return false;
+      if (getClass() != obj.getClass())
+        return false;
+      TestObjectSameFieldNameButDifferentDataTypeFloat other =
+          (TestObjectSameFieldNameButDifferentDataTypeFloat) obj;
+      if (text == null) {
+        if (other.text != null)
+          return false;
+      } else if (!text.equals(other.text))
+        return false;
+      return true;
+    }
+
+    @Override
+    public String toString() {
+      return "TestObjectSameFieldNameButDifferentDataTypeFloat[" + text + "]";
+    }
+  }
+
+  protected static class TestObjectSameFieldNameButDifferentDataTypeInteger
+      implements Serializable {
+    private static final long serialVersionUID = 1L;
+    private Integer text;
+
+    public TestObjectSameFieldNameButDifferentDataTypeInteger(Integer text) {
+      this.text = text;
+    }
+
+    @Override
+    public int hashCode() {
+      final int prime = 31;
+      int result = 1;
+      result = prime * result + ((text == null) ? 0 : text.hashCode());
+      return result;
+    }
+
+    @Override
+    public boolean equals(Object obj) {
+      if (this == obj)
+        return true;
+      if (obj == null)
+        return false;
+      if (getClass() != obj.getClass())
+        return false;
+      MixedObjectIndexDUnitTest.TestObjectSameFieldNameButDifferentDataTypeInteger other =
+          (MixedObjectIndexDUnitTest.TestObjectSameFieldNameButDifferentDataTypeInteger) obj;
+      if (text == null) {
+        if (other.text != null)
+          return false;
+      } else if (!text.equals(other.text))
+        return false;
+      return true;
+    }
+
+    @Override
+    public String toString() {
+      return "TestObjectSameFieldNameButDifferentDataTypeInteger[" + text + "]";
+    }
+  }
+
+  protected static class TestObjectWithSameFieldName implements Serializable {
+    private static final long serialVersionUID = 1L;
+    private String text;
+
+    public TestObjectWithSameFieldName(String text) {
+      this.text = text;
+    }
+
+    @Override
+    public int hashCode() {
+      final int prime = 31;
+      int result = 1;
+      result = prime * result + ((text == null) ? 0 : text.hashCode());
+      return result;
+    }
+
+    @Override
+    public boolean equals(Object obj) {
+      if (this == obj)
+        return true;
+      if (obj == null)
+        return false;
+      if (getClass() != obj.getClass())
+        return false;
+      TestObjectWithSameFieldName other = (TestObjectWithSameFieldName) obj;
+      if (text == null) {
+        if (other.text != null)
+          return false;
+      } else if (!text.equals(other.text))
+        return false;
+      return true;
+    }
+
+    @Override
+    public String toString() {
+      return "TestObjectWithSameFieldName[" + text + "]";
+    }
+
+
+  }
+
+  protected static class TestObjectWithNoCommonField implements Serializable {
+    private static final long serialVersionUID = 1L;
+    private String data;
+
+    public TestObjectWithNoCommonField(String data) {
+      this.data = data;
+    }
+
+    @Override
+    public int hashCode() {
+      final int prime = 31;
+      int result = 1;
+      result = prime * result + ((data == null) ? 0 : data.hashCode());
+      return result;
+    }
+
+    @Override
+    public boolean equals(Object obj) {
+      if (this == obj)
+        return true;
+      if (obj == null)
+        return false;
+      if (getClass() != obj.getClass())
+        return false;
+      TestObjectWithNoCommonField other = (TestObjectWithNoCommonField) obj;
+      if (data == null) {
+        if (other.data != null)
+          return false;
+      } else if (!data.equals(other.data))
+        return false;
+      return true;
+    }
+
+    @Override
+    public String toString() {
+      return "TestObjectWithNoCommonField[" + data + "]";
+    }
+
+
+  }
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/fca17e56/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
index 670e6a8..e9b9369 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
@@ -38,6 +38,7 @@ import org.apache.geode.cache.lucene.LuceneIndex;
 import org.apache.geode.cache.lucene.LuceneIndexFactory;
 import org.apache.geode.cache.lucene.LuceneQuery;
 import org.apache.geode.cache.lucene.LuceneQueryException;
+import org.apache.geode.cache.lucene.LuceneQueryProvider;
 import org.apache.geode.cache.lucene.PageableLuceneQueryResults;
 import org.apache.geode.cache.lucene.LuceneService;
 import org.apache.geode.cache.lucene.LuceneServiceProvider;
@@ -46,6 +47,9 @@ import org.apache.geode.cache.lucene.internal.LuceneServiceImpl;
 import org.apache.geode.internal.cache.LocalRegion;
 import org.apache.geode.internal.cache.wan.AbstractGatewaySender;
 import org.apache.geode.test.dunit.VM;
+import org.apache.lucene.document.FloatPoint;
+import org.apache.lucene.document.IntPoint;
+import org.apache.lucene.search.Query;
 
 public class LuceneTestUtilities {
   public static final String INDEX_NAME = "index";
@@ -78,6 +82,53 @@ public class LuceneTestUtilities {
   public static String Quarter3 = "Q3";
   public static String Quarter4 = "Q4";
 
+  public static class IntRangeQueryProvider implements LuceneQueryProvider {
+    String fieldName;
+    int lowerValue;
+    int upperValue;
+
+    private transient Query luceneQuery;
+
+    public IntRangeQueryProvider(String fieldName, int lowerValue, int upperValue) {
+      this.fieldName = fieldName;
+      this.lowerValue = lowerValue;
+      this.upperValue = upperValue;
+    }
+
+    @Override
+    public Query getQuery(LuceneIndex index) throws LuceneQueryException {
+      if (luceneQuery == null) {
+        luceneQuery = IntPoint.newRangeQuery(fieldName, lowerValue, upperValue);
+      }
+      System.out.println("IntRangeQueryProvider, using java serializable");
+      return luceneQuery;
+    }
+  }
+
+  public static class FloatRangeQueryProvider implements LuceneQueryProvider {
+    String fieldName;
+    float lowerValue;
+    float upperValue;
+
+    private transient Query luceneQuery;
+
+    public FloatRangeQueryProvider(String fieldName, float lowerValue, float upperValue) {
+      this.fieldName = fieldName;
+      this.lowerValue = lowerValue;
+      this.upperValue = upperValue;
+    }
+
+    @Override
+    public Query getQuery(LuceneIndex index) throws LuceneQueryException {
+      if (luceneQuery == null) {
+        luceneQuery = FloatPoint.newRangeQuery(fieldName, lowerValue, upperValue);
+        // luceneQuery = DoublePoint.newRangeQuery(fieldName, lowerValue, upperValue);
+      }
+      System.out.println("IntRangeQueryProvider, using java serializable");
+      return luceneQuery;
+    }
+  }
+
   public static Region initDataStoreForFixedPR(final Cache cache) throws Exception {
     List<FixedPartitionAttributes> fpaList = new ArrayList<FixedPartitionAttributes>();
     int vmNum = VM.getCurrentVMNum();


[21/49] geode git commit: GEODE-2594: do not use Attach API or tools.jar by default

Posted by kh...@apache.org.
GEODE-2594: do not use Attach API or tools.jar by default

* remove tools.jar from classpath in gfsh and gfsh.bat
* remove use of Attach API from start commands
* deprecate --pid option in help of status and stop commands

If a user still wants to use the --pid option in status or
stop commands then they can manually add the tools.jar to
their classpath before launching gfsh.


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/72e152f3
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/72e152f3
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/72e152f3

Branch: refs/heads/feature/GEODE-2420
Commit: 72e152f34097d67ed6b3d25ee8434299a9f2c440
Parents: 8c371be
Author: Kirk Lund <kl...@apache.org>
Authored: Wed Mar 8 14:13:14 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 geode-assembly/src/main/dist/bin/gfsh           |  14 -
 geode-assembly/src/main/dist/bin/gfsh.bat       |   8 -
 ...auncherLifecycleCommandsIntegrationTest.java |   4 +-
 .../LauncherLifecycleCommandsJUnitTest.java     | 647 -------------------
 .../commands/LauncherLifecycleCommandsTest.java | 647 +++++++++++++++++++
 .../geode/distributed/AbstractLauncher.java     |  62 +-
 .../cli/commands/LauncherLifecycleCommands.java |  58 +-
 .../internal/cli/i18n/CliStrings.java           |   9 +-
 .../geode/distributed/AbstractLauncherTest.java |  33 +-
 .../commands/HelpCommandsIntegrationTest.java   |   3 +-
 .../cli/commands/golden-help-offline.properties |  12 +-
 11 files changed, 672 insertions(+), 825 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-assembly/src/main/dist/bin/gfsh
----------------------------------------------------------------------
diff --git a/geode-assembly/src/main/dist/bin/gfsh b/geode-assembly/src/main/dist/bin/gfsh
index 309723e..51533c3 100755
--- a/geode-assembly/src/main/dist/bin/gfsh
+++ b/geode-assembly/src/main/dist/bin/gfsh
@@ -110,20 +110,6 @@ if [ "x$GF_JAVA_PATH" == "x" ]; then
   echo "ERROR: Could not find java executable in the path. Please set JAVA_HOME to point to the JDK directory or point GF_JAVA to java executable from JDK."
   exit 1
 fi
-GF_JAVA_PARENT=`dirname "$GF_JAVA_PATH"`
-GF_JAVA_PARENT=`dirname "$GF_JAVA_PARENT"`
-TOOLS_JAR="$GF_JAVA_PARENT/lib/tools.jar"
-if [ ! -f "$TOOLS_JAR" ]; then
-    # Now consider java is from JRE in JDK 
-    TOOLS_JAR="$GF_JAVA_PARENT/../lib/tools.jar"
-    if [ ! -f "$TOOLS_JAR" ]; then
-        TOOLS_JAR=
-    fi
-fi
-
-if [ "x$TOOLS_JAR" != "x" ]; then
-  CLASSPATH="$CLASSPATH:$TOOLS_JAR"
-fi
 
 if [ "${UNAME}" = "cygwin" ]; then
   CLASSPATH=$( cygpath -w -p "$CLASSPATH" )

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-assembly/src/main/dist/bin/gfsh.bat
----------------------------------------------------------------------
diff --git a/geode-assembly/src/main/dist/bin/gfsh.bat b/geode-assembly/src/main/dist/bin/gfsh.bat
index 5a40141..b55ffd7 100755
--- a/geode-assembly/src/main/dist/bin/gfsh.bat
+++ b/geode-assembly/src/main/dist/bin/gfsh.bat
@@ -68,14 +68,6 @@ REM @if not exist "%USERPROFILE%\.gemfire" (
 REM @mkdir "%USERPROFILE%\.gemfire"
 REM )
 
-REM  Expect to find the tools.jar from the JDK
-@set TOOLS_JAR=%JAVA_HOME%\lib\tools.jar
-@IF EXIST "%TOOLS_JAR%" (
-    @set DEPENDENCIES=%DEPENDENCIES%;%TOOLS_JAR%
-) ELSE (
-    set TOOLS_JAR=
-)
-
 @set LAUNCHER=org.apache.geode.management.internal.cli.Launcher
 @if defined JAVA_ARGS (
     @set JAVA_ARGS="%JAVA_ARGS%"

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsIntegrationTest.java b/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsIntegrationTest.java
index 1872a84..b51505a 100644
--- a/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsIntegrationTest.java
+++ b/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsIntegrationTest.java
@@ -40,8 +40,8 @@ import org.apache.geode.internal.util.IOUtils;
 import org.apache.geode.test.junit.categories.IntegrationTest;
 
 /**
- * The LauncherLifecycleCommandsJUnitTest class is a test suite of test cases testing the contract
- * and functionality of the lifecycle launcher GemFire shell (Gfsh) commands.
+ * The LauncherLifecycleCommandsIntegrationTest class is a test suite of test cases testing the
+ * contract and functionality of the lifecycle launcher GemFire shell (Gfsh) commands.
  *
  * @see org.apache.geode.management.internal.cli.commands.LauncherLifecycleCommands
  * @see org.junit.Assert

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsJUnitTest.java b/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsJUnitTest.java
deleted file mode 100755
index 947da42..0000000
--- a/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsJUnitTest.java
+++ /dev/null
@@ -1,647 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-package org.apache.geode.management.internal.cli.commands;
-
-import static org.apache.geode.distributed.ConfigurationProperties.*;
-import static org.junit.Assert.*;
-
-import java.io.File;
-import java.util.ArrayList;
-import java.util.HashSet;
-import java.util.List;
-import java.util.Properties;
-import java.util.Set;
-import java.util.Stack;
-
-import org.apache.geode.distributed.LocatorLauncher;
-import org.junit.After;
-import org.junit.Before;
-import org.junit.Test;
-import org.junit.experimental.categories.Category;
-
-import org.apache.geode.GemFireException;
-import org.apache.geode.cache.server.CacheServer;
-import org.apache.geode.distributed.ServerLauncher;
-import org.apache.geode.distributed.internal.DistributionConfig;
-import org.apache.geode.internal.DistributionLocator;
-import org.apache.geode.internal.lang.StringUtils;
-import org.apache.geode.internal.lang.SystemUtils;
-import org.apache.geode.internal.util.IOUtils;
-import org.apache.geode.management.internal.cli.i18n.CliStrings;
-import org.apache.geode.test.junit.categories.UnitTest;
-
-/**
- * The LauncherLifecycleCommandsJUnitTest class is a test suite of test cases testing the contract
- * and functionality of the lifecycle launcher GemFire shell (Gfsh) commands.
- *
- * @see org.apache.geode.management.internal.cli.commands.LauncherLifecycleCommands
- * @see org.junit.Assert
- * @see org.junit.Test
- * @since GemFire 7.0
- */
-@Category(UnitTest.class)
-@SuppressWarnings("unused")
-public class LauncherLifecycleCommandsJUnitTest {
-
-  private LauncherLifecycleCommands launcherCommands;
-
-  @Before
-  public void setup() {
-    launcherCommands = new LauncherLifecycleCommands();
-  }
-
-  @After
-  public void tearDown() {
-    launcherCommands = null;
-  }
-
-  @Test
-  public void testAddGemFirePropertyFileToCommandLine() {
-    final List<String> commandLine = new ArrayList<>();
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addGemFirePropertyFile(commandLine, null);
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addGemFirePropertyFile(commandLine, StringUtils.EMPTY_STRING);
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addGemFirePropertyFile(commandLine, " ");
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addGemFirePropertyFile(commandLine,
-        "/path/to/gemfire.properties");
-
-    assertFalse(commandLine.isEmpty());
-    assertTrue(commandLine.contains("-DgemfirePropertyFile=/path/to/gemfire.properties"));
-  }
-
-  @Test
-  public void testAddGemFireSystemPropertiesToCommandLine() {
-    final List<String> commandLine = new ArrayList<>();
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addGemFireSystemProperties(commandLine, new Properties());
-
-    assertTrue(commandLine.isEmpty());
-
-    final Properties gemfireProperties = new Properties();
-
-    gemfireProperties.setProperty(LOCATORS, "localhost[11235]");
-    gemfireProperties.setProperty(LOG_LEVEL, "config");
-    gemfireProperties.setProperty(LOG_FILE, StringUtils.EMPTY_STRING);
-    gemfireProperties.setProperty(MCAST_PORT, "0");
-    gemfireProperties.setProperty(NAME, "machine");
-
-    getLauncherLifecycleCommands().addGemFireSystemProperties(commandLine, gemfireProperties);
-
-    assertFalse(commandLine.isEmpty());
-    assertEquals(4, commandLine.size());
-
-    for (final String propertyName : gemfireProperties.stringPropertyNames()) {
-      final String propertyValue = gemfireProperties.getProperty(propertyName);
-      if (StringUtils.isBlank(propertyValue)) {
-        for (final String systemProperty : commandLine) {
-          assertFalse(systemProperty.startsWith(
-              "-D" + DistributionConfig.GEMFIRE_PREFIX + "".concat(propertyName).concat("=")));
-        }
-      } else {
-        assertTrue(commandLine.contains("-D" + DistributionConfig.GEMFIRE_PREFIX
-            + "".concat(propertyName).concat("=").concat(propertyValue)));
-      }
-    }
-  }
-
-
-  @Test
-  public void testAddGemFireSystemPropertiesToCommandLineWithRestAPI() {
-    final List<String> commandLine = new ArrayList<>();
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addGemFireSystemProperties(commandLine, new Properties());
-
-    assertTrue(commandLine.isEmpty());
-
-    final Properties gemfireProperties = new Properties();
-
-    gemfireProperties.setProperty(LOCATORS, "localhost[11235]");
-    gemfireProperties.setProperty(LOG_LEVEL, "config");
-    gemfireProperties.setProperty(LOG_FILE, StringUtils.EMPTY_STRING);
-    gemfireProperties.setProperty(MCAST_PORT, "0");
-    gemfireProperties.setProperty(NAME, "machine");
-
-    gemfireProperties.setProperty(START_DEV_REST_API, "true");
-    gemfireProperties.setProperty(HTTP_SERVICE_PORT, "8080");
-    gemfireProperties.setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
-
-
-    getLauncherLifecycleCommands().addGemFireSystemProperties(commandLine, gemfireProperties);
-
-    assertFalse(commandLine.isEmpty());
-    assertEquals(7, commandLine.size());
-
-    for (final String propertyName : gemfireProperties.stringPropertyNames()) {
-      final String propertyValue = gemfireProperties.getProperty(propertyName);
-      if (StringUtils.isBlank(propertyValue)) {
-        for (final String systemProperty : commandLine) {
-          assertFalse(systemProperty.startsWith(
-              "-D" + DistributionConfig.GEMFIRE_PREFIX + "".concat(propertyName).concat("=")));
-        }
-      } else {
-        assertTrue(commandLine.contains("-D" + DistributionConfig.GEMFIRE_PREFIX
-            + "".concat(propertyName).concat("=").concat(propertyValue)));
-      }
-    }
-  }
-
-  @Test
-  public void testAddInitialHeapToCommandLine() {
-    final List<String> commandLine = new ArrayList<>();
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addInitialHeap(commandLine, null);
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addInitialHeap(commandLine, StringUtils.EMPTY_STRING);
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addInitialHeap(commandLine, " ");
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addInitialHeap(commandLine, "512M");
-
-    assertFalse(commandLine.isEmpty());
-    assertEquals("-Xms512M", commandLine.get(0));
-  }
-
-  @Test
-  public void testAddJvmArgumentsAndOptionsToCommandLine() {
-    final List<String> commandLine = new ArrayList<>();
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addJvmArgumentsAndOptions(commandLine, null);
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addJvmArgumentsAndOptions(commandLine, new String[] {});
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addJvmArgumentsAndOptions(commandLine,
-        new String[] {"-DmyProp=myVal", "-d64", "-server", "-Xprof"});
-
-    assertFalse(commandLine.isEmpty());
-    assertEquals(4, commandLine.size());
-    assertEquals("-DmyProp=myVal", commandLine.get(0));
-    assertEquals("-d64", commandLine.get(1));
-    assertEquals("-server", commandLine.get(2));
-    assertEquals("-Xprof", commandLine.get(3));
-  }
-
-  // Fix for Bug #47192 - "Making GemFire (JVM) to exit in case of OutOfMemory"
-  @Test
-  public void testAddJvmOptionsForOutOfMemoryErrors() {
-    final List<String> jvmOptions = new ArrayList<>(1);
-
-    getLauncherLifecycleCommands().addJvmOptionsForOutOfMemoryErrors(jvmOptions);
-
-    if (SystemUtils.isHotSpotVM()) {
-      if (SystemUtils.isWindows()) {
-        assertTrue(jvmOptions.contains("-XX:OnOutOfMemoryError=taskkill /F /PID %p"));
-      } else {
-        assertTrue(jvmOptions.contains("-XX:OnOutOfMemoryError=kill -KILL %p"));
-      }
-    } else if (SystemUtils.isJ9VM()) {
-      assertEquals(1, jvmOptions.size());
-      assertTrue(jvmOptions.contains("-Xcheck:memory"));
-    } else if (SystemUtils.isJRockitVM()) {
-      assertEquals(1, jvmOptions.size());
-      assertTrue(jvmOptions.contains("-XXexitOnOutOfMemory"));
-    } else {
-      assertTrue(jvmOptions.isEmpty());
-    }
-  }
-
-  @Test
-  public void testAddMaxHeapToCommandLine() {
-    final List<String> commandLine = new ArrayList<>();
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addMaxHeap(commandLine, null);
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addMaxHeap(commandLine, StringUtils.EMPTY_STRING);
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addMaxHeap(commandLine, "  ");
-
-    assertTrue(commandLine.isEmpty());
-
-    getLauncherLifecycleCommands().addMaxHeap(commandLine, "1024M");
-
-    assertFalse(commandLine.isEmpty());
-    assertEquals(3, commandLine.size());
-    assertEquals("-Xmx1024M", commandLine.get(0));
-    assertEquals("-XX:+UseConcMarkSweepGC", commandLine.get(1));
-    assertEquals("-XX:CMSInitiatingOccupancyFraction="
-        + LauncherLifecycleCommands.CMS_INITIAL_OCCUPANCY_FRACTION, commandLine.get(2));
-  }
-
-  @Test(expected = AssertionError.class)
-  public void testReadPidWithNull() {
-    try {
-      getLauncherLifecycleCommands().readPid(null);
-    } catch (AssertionError expected) {
-      assertEquals("The file from which to read the process ID (pid) cannot be null!",
-          expected.getMessage());
-      throw expected;
-    }
-  }
-
-  @Test
-  @SuppressWarnings("deprecation")
-  public void testGetClasspath() {
-    assertEquals(System.getProperty("java.class.path"),
-        getLauncherLifecycleCommands().getClasspath(null));
-  }
-
-  @Test
-  @SuppressWarnings("deprecation")
-  public void testGetClasspathWithUserDefinedClasspath() {
-    assertEquals(
-        System.getProperty("java.class.path") + File.pathSeparator + "/path/to/user/classes",
-        getLauncherLifecycleCommands().getClasspath("/path/to/user/classes"));
-  }
-
-  @Test
-  public void testGetSystemClasspath() {
-    assertEquals(System.getProperty("java.class.path"),
-        getLauncherLifecycleCommands().getSystemClasspath());
-  }
-
-  @Test
-  public void testLocatorClasspathOrder() {
-    String userClasspath = "/path/to/user/lib/app.jar:/path/to/user/classes";
-
-    String expectedClasspath = launcherCommands.getGemFireJarPath().concat(File.pathSeparator)
-        .concat(userClasspath).concat(File.pathSeparator)
-        .concat(System.getProperty("java.class.path")).concat(File.pathSeparator)
-        .concat(LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME);
-
-    String actualClasspath = launcherCommands.getLocatorClasspath(true, userClasspath);
-
-    assertEquals(expectedClasspath, actualClasspath);
-  }
-
-  @Test
-  public void testServerClasspathOrder() {
-    String userClasspath = "/path/to/user/lib/app.jar:/path/to/user/classes";
-
-    String expectedClasspath = launcherCommands.getGemFireJarPath().concat(File.pathSeparator)
-        .concat(userClasspath).concat(File.pathSeparator)
-        .concat(LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME);
-
-    String actualClasspath = launcherCommands.getServerClasspath(false, userClasspath);
-
-    assertEquals(expectedClasspath, actualClasspath);
-  }
-
-  @Test
-  public void testToClasspath() {
-    final boolean EXCLUDE_SYSTEM_CLASSPATH = false;
-    final boolean INCLUDE_SYSTEM_CLASSPATH = true;
-
-    String[] jarFilePathnames =
-        {"/path/to/user/libs/A.jar", "/path/to/user/libs/B.jar", "/path/to/user/libs/C.jar"};
-
-    String[] userClasspaths = {"/path/to/classes:/path/to/libs/1.jar:/path/to/libs/2.jar",
-        "/path/to/ext/libs/1.jar:/path/to/ext/classes:/path/to/ext/lib/10.jar"};
-
-    String expectedClasspath = LauncherLifecycleCommands.GEODE_JAR_PATHNAME
-        .concat(File.pathSeparator).concat(toClasspath(userClasspaths)).concat(File.pathSeparator)
-        .concat(toClasspath(jarFilePathnames));
-
-    assertEquals(expectedClasspath, getLauncherLifecycleCommands()
-        .toClasspath(EXCLUDE_SYSTEM_CLASSPATH, jarFilePathnames, userClasspaths));
-
-    expectedClasspath = LauncherLifecycleCommands.GEODE_JAR_PATHNAME.concat(File.pathSeparator)
-        .concat(toClasspath(userClasspaths)).concat(File.pathSeparator)
-        .concat(System.getProperty("java.class.path")).concat(File.pathSeparator)
-        .concat(toClasspath(jarFilePathnames));
-
-    assertEquals(expectedClasspath, getLauncherLifecycleCommands()
-        .toClasspath(INCLUDE_SYSTEM_CLASSPATH, jarFilePathnames, userClasspaths));
-
-    expectedClasspath = LauncherLifecycleCommands.GEODE_JAR_PATHNAME.concat(File.pathSeparator)
-        .concat(System.getProperty("java.class.path"));
-
-    assertEquals(expectedClasspath, getLauncherLifecycleCommands()
-        .toClasspath(INCLUDE_SYSTEM_CLASSPATH, null, (String[]) null));
-
-    assertEquals(LauncherLifecycleCommands.GEODE_JAR_PATHNAME, getLauncherLifecycleCommands()
-        .toClasspath(EXCLUDE_SYSTEM_CLASSPATH, null, (String[]) null));
-
-    assertEquals(LauncherLifecycleCommands.GEODE_JAR_PATHNAME,
-        getLauncherLifecycleCommands().toClasspath(EXCLUDE_SYSTEM_CLASSPATH, new String[0], ""));
-  }
-
-  @Test
-  public void testToClassPathOrder() {
-    String userClasspathOne = "/path/to/user/lib/a.jar:/path/to/user/classes";
-    String userClasspathTwo =
-        "/path/to/user/lib/x.jar:/path/to/user/lib/y.jar:/path/to/user/lib/z.jar";
-
-    String expectedClasspath = launcherCommands.getGemFireJarPath().concat(File.pathSeparator)
-        .concat(userClasspathOne).concat(File.pathSeparator).concat(userClasspathTwo)
-        .concat(File.pathSeparator).concat(System.getProperty("java.class.path"))
-        .concat(File.pathSeparator).concat(LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME)
-        .concat(File.pathSeparator)
-        .concat(LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME);
-
-    String actualClasspath = launcherCommands.toClasspath(true,
-        new String[] {LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME,
-            LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME},
-        userClasspathOne, userClasspathTwo);
-
-    assertEquals(expectedClasspath, actualClasspath);
-  }
-
-  @Test
-  public void testGetJavaPathname() {
-    assertEquals(
-        IOUtils.appendToPath(System.getProperty("java.home"), "bin",
-            "java" + LauncherLifecycleCommands.getExecutableSuffix()),
-        getLauncherLifecycleCommands().getJdkToolPathname(
-            "java" + LauncherLifecycleCommands.getExecutableSuffix(), new GemFireException() {}));
-  }
-
-  @Test(expected = NullPointerException.class)
-  public void testGetJdkToolPathnameWithNullPathnames() {
-    try {
-      getLauncherLifecycleCommands().getJdkToolPathname((Stack<String>) null,
-          new GemFireException() {});
-    } catch (NullPointerException expected) {
-      assertEquals("The JDK tool executable pathnames cannot be null!", expected.getMessage());
-      throw expected;
-    }
-  }
-
-  @Test(expected = NullPointerException.class)
-  public void testGetJdkToolPathnameWithNullGemFireException() {
-    try {
-      getLauncherLifecycleCommands().getJdkToolPathname(new Stack<String>(), null);
-    } catch (NullPointerException expected) {
-      assertEquals("The GemFireException cannot be null!", expected.getMessage());
-      throw expected;
-    }
-  }
-
-  @Test
-  public void testGetJdkToolPathnameForNonExistingTool() {
-    try {
-      final GemFireException expected = new GemFireException() {
-        @Override
-        public String getMessage() {
-          return "expected";
-        }
-      };
-
-      getLauncherLifecycleCommands().getJdkToolPathname("nonExistingTool.exe", expected);
-    } catch (GemFireException expected) {
-      assertEquals("expected", expected.getMessage());
-    }
-  }
-
-  @Test
-  public void testGetLocatorId() {
-    assertEquals("machine[11235]", getLauncherLifecycleCommands().getLocatorId("machine", 11235));
-    assertEquals("machine.domain.org[11235]",
-        getLauncherLifecycleCommands().getLocatorId("machine.domain.org", 11235));
-    assertEquals("machine[" + DistributionLocator.DEFAULT_LOCATOR_PORT + "]",
-        getLauncherLifecycleCommands().getLocatorId("machine", null));
-  }
-
-  @Test
-  public void testGetServerId() {
-    assertEquals("machine[12480]", getLauncherLifecycleCommands().getServerId("machine", 12480));
-    assertEquals("machine.domain.org[12480]",
-        getLauncherLifecycleCommands().getServerId("machine.domain.org", 12480));
-    assertEquals("machine[" + CacheServer.DEFAULT_PORT + "]",
-        getLauncherLifecycleCommands().getServerId("machine", null));
-  }
-
-  @Test
-  public void testCreateJmxServerUrlWithMemberName() {
-    assertEquals("service:jmx:rmi://localhost:8192/jndi/rmi://localhost:8192/jmxrmi",
-        getLauncherLifecycleCommands().getJmxServiceUrlAsString("localhost[8192]"));
-  }
-
-  @Test(expected = IllegalArgumentException.class)
-  public void testCreateJmxServiceUrlWithInvalidMemberName() {
-    try {
-      System.err.println(getLauncherLifecycleCommands().getJmxServiceUrlAsString("memberOne[]"));
-    } catch (IllegalArgumentException expected) {
-      assertEquals(CliStrings.START_JCONSOLE__CONNECT_BY_MEMBER_NAME_ID_ERROR_MESSAGE,
-          expected.getMessage());
-      throw expected;
-    }
-  }
-
-  @Test
-  public void testCreateServerCommandLine() throws Exception {
-    ServerLauncher serverLauncher = new ServerLauncher.Builder()
-        .setCommand(ServerLauncher.Command.START).setDisableDefaultServer(true)
-        .setMemberName("testCreateServerCommandLine").setRebalance(true)
-        // .setServerBindAddress("localhost")
-        .setServerPort(41214).setCriticalHeapPercentage(95.5f).setEvictionHeapPercentage(85.0f)
-        .setSocketBufferSize(1024 * 1024).setMessageTimeToLive(93).build();
-
-    String[] commandLineElements = launcherCommands.createStartServerCommandLine(serverLauncher,
-        null, null, new Properties(), null, false, new String[0], false, null, null);
-
-    assertNotNull(commandLineElements);
-    assertTrue(commandLineElements.length > 0);
-
-    Set<String> expectedCommandLineElements = new HashSet<>(6);
-
-    expectedCommandLineElements.add(serverLauncher.getCommand().getName());
-    expectedCommandLineElements.add("--disable-default-server");
-    expectedCommandLineElements.add(serverLauncher.getMemberName().toLowerCase());
-    expectedCommandLineElements.add("--rebalance");
-    // expectedCommandLineElements.add(String.format("--server-bind-address=%1$s",
-    // serverLauncher.getServerBindAddress().getHostName()));
-    expectedCommandLineElements
-        .add(String.format("--server-port=%1$d", serverLauncher.getServerPort()));
-    expectedCommandLineElements.add(String.format("--critical-heap-percentage=%1$s",
-        serverLauncher.getCriticalHeapPercentage()));
-    expectedCommandLineElements.add(String.format("--eviction-heap-percentage=%1$s",
-        serverLauncher.getEvictionHeapPercentage()));
-    expectedCommandLineElements
-        .add(String.format("--socket-buffer-size=%1$d", serverLauncher.getSocketBufferSize()));
-    expectedCommandLineElements
-        .add(String.format("--message-time-to-live=%1$d", serverLauncher.getMessageTimeToLive()));
-
-    for (String commandLineElement : commandLineElements) {
-      expectedCommandLineElements.remove(commandLineElement.toLowerCase());
-    }
-
-    assertTrue(String.format("Expected ([]); but was (%1$s)", expectedCommandLineElements),
-        expectedCommandLineElements.isEmpty());
-  }
-
-  @Test
-  public void testCreateServerCommandLineWithRestAPI() throws Exception {
-    ServerLauncher serverLauncher = new ServerLauncher.Builder()
-        .setCommand(ServerLauncher.Command.START).setDisableDefaultServer(true)
-        .setMemberName("testCreateServerCommandLine").setRebalance(true)
-        // .setServerBindAddress("localhost")
-        .setServerPort(41214).setCriticalHeapPercentage(95.5f).setEvictionHeapPercentage(85.0f)
-        .build();
-
-    Properties gemfireProperties = new Properties();
-    gemfireProperties.setProperty(START_DEV_REST_API, "true");
-    gemfireProperties.setProperty(HTTP_SERVICE_PORT, "8080");
-    gemfireProperties.setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
-
-
-    String[] commandLineElements = launcherCommands.createStartServerCommandLine(serverLauncher,
-        null, null, gemfireProperties, null, false, new String[0], false, null, null);
-
-    assertNotNull(commandLineElements);
-    assertTrue(commandLineElements.length > 0);
-
-    Set<String> expectedCommandLineElements = new HashSet<>(6);
-
-    expectedCommandLineElements.add(serverLauncher.getCommand().getName());
-    expectedCommandLineElements.add("--disable-default-server");
-    expectedCommandLineElements.add(serverLauncher.getMemberName().toLowerCase());
-    expectedCommandLineElements.add("--rebalance");
-    // expectedCommandLineElements.add(String.format("--server-bind-address=%1$s",
-    // serverLauncher.getServerBindAddress().getHostName()));
-    expectedCommandLineElements
-        .add(String.format("--server-port=%1$d", serverLauncher.getServerPort()));
-    expectedCommandLineElements.add(String.format("--critical-heap-percentage=%1$s",
-        serverLauncher.getCriticalHeapPercentage()));
-    expectedCommandLineElements.add(String.format("--eviction-heap-percentage=%1$s",
-        serverLauncher.getEvictionHeapPercentage()));
-
-    expectedCommandLineElements
-        .add("-d" + DistributionConfig.GEMFIRE_PREFIX + "" + START_DEV_REST_API + "=" + "true");
-    expectedCommandLineElements
-        .add("-d" + DistributionConfig.GEMFIRE_PREFIX + "" + HTTP_SERVICE_PORT + "=" + "8080");
-    expectedCommandLineElements.add("-d" + DistributionConfig.GEMFIRE_PREFIX + ""
-        + HTTP_SERVICE_BIND_ADDRESS + "=" + "localhost");
-
-
-    for (String commandLineElement : commandLineElements) {
-      expectedCommandLineElements.remove(commandLineElement.toLowerCase());
-    }
-
-    assertTrue(String.format("Expected ([]); but was (%1$s)", expectedCommandLineElements),
-        expectedCommandLineElements.isEmpty());
-  }
-
-  /**
-   * Verify commandline parameters passed for starting locator
-   * 
-   * @throws Exception
-   */
-  @Test
-  public void testLocatorCommandLineWithRestAPI() throws Exception {
-    LocatorLauncher locatorLauncher =
-        new LocatorLauncher.Builder().setCommand(LocatorLauncher.Command.START)
-            .setMemberName("testLocatorCommandLineWithRestAPI").setBindAddress("localhost")
-            .setPort(11111).build();
-
-    Properties gemfireProperties = new Properties();
-    gemfireProperties.setProperty(HTTP_SERVICE_PORT, "8089");
-    gemfireProperties.setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
-
-
-    String[] commandLineElements = launcherCommands.createStartLocatorCommandLine(locatorLauncher,
-        null, null, gemfireProperties, null, false, new String[0], null, null);
-
-    assertNotNull(commandLineElements);
-    assertTrue(commandLineElements.length > 0);
-
-    Set<String> expectedCommandLineElements = new HashSet<>(6);
-
-    expectedCommandLineElements.add(locatorLauncher.getCommand().getName());
-    expectedCommandLineElements.add(locatorLauncher.getMemberName().toLowerCase());
-    expectedCommandLineElements.add(String.format("--port=%1$d", locatorLauncher.getPort()));
-    expectedCommandLineElements
-        .add("-d" + DistributionConfig.GEMFIRE_PREFIX + "" + HTTP_SERVICE_PORT + "=" + "8089");
-    expectedCommandLineElements.add("-d" + DistributionConfig.GEMFIRE_PREFIX + ""
-        + HTTP_SERVICE_BIND_ADDRESS + "=" + "localhost");
-
-
-    for (String commandLineElement : commandLineElements) {
-      expectedCommandLineElements.remove(commandLineElement.toLowerCase());
-    }
-
-    assertTrue(String.format("Expected ([]); but was (%1$s)", expectedCommandLineElements),
-        expectedCommandLineElements.isEmpty());
-  }
-
-
-  @Test
-  public void testReadPidWithNonExistingFile() {
-    assertEquals(LauncherLifecycleCommands.INVALID_PID,
-        getLauncherLifecycleCommands().readPid(new File("/path/to/non_existing/pid.file")));
-  }
-
-  private LauncherLifecycleCommands getLauncherLifecycleCommands() {
-    return launcherCommands;
-  }
-
-  private String toClasspath(final String... jarFilePathnames) {
-    String classpath = StringUtils.EMPTY_STRING;
-
-    if (jarFilePathnames != null) {
-      for (final String jarFilePathname : jarFilePathnames) {
-        classpath += (classpath.isEmpty() ? StringUtils.EMPTY_STRING : File.pathSeparator);
-        classpath += jarFilePathname;
-      }
-    }
-
-    return classpath;
-  }
-
-  private String toPath(Object... pathElements) {
-    String path = "";
-
-    for (Object pathElement : pathElements) {
-      path += (path.isEmpty() ? StringUtils.EMPTY_STRING : File.pathSeparator);
-      path += pathElement;
-    }
-
-    return path;
-  }
-
-}

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsTest.java
----------------------------------------------------------------------
diff --git a/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsTest.java b/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsTest.java
new file mode 100755
index 0000000..0554f69
--- /dev/null
+++ b/geode-assembly/src/test/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommandsTest.java
@@ -0,0 +1,647 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.management.internal.cli.commands;
+
+import static org.apache.geode.distributed.ConfigurationProperties.*;
+import static org.junit.Assert.*;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Properties;
+import java.util.Set;
+import java.util.Stack;
+
+import org.apache.geode.distributed.LocatorLauncher;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import org.apache.geode.GemFireException;
+import org.apache.geode.cache.server.CacheServer;
+import org.apache.geode.distributed.ServerLauncher;
+import org.apache.geode.distributed.internal.DistributionConfig;
+import org.apache.geode.internal.DistributionLocator;
+import org.apache.geode.internal.lang.StringUtils;
+import org.apache.geode.internal.lang.SystemUtils;
+import org.apache.geode.internal.util.IOUtils;
+import org.apache.geode.management.internal.cli.i18n.CliStrings;
+import org.apache.geode.test.junit.categories.UnitTest;
+
+/**
+ * The LauncherLifecycleCommandsTest class is a test suite of test cases testing the contract and
+ * functionality of the lifecycle launcher GemFire shell (Gfsh) commands.
+ *
+ * @see org.apache.geode.management.internal.cli.commands.LauncherLifecycleCommands
+ * @see org.junit.Assert
+ * @see org.junit.Test
+ * @since GemFire 7.0
+ */
+@Category(UnitTest.class)
+@SuppressWarnings("unused")
+public class LauncherLifecycleCommandsTest {
+
+  private LauncherLifecycleCommands launcherCommands;
+
+  @Before
+  public void setup() {
+    launcherCommands = new LauncherLifecycleCommands();
+  }
+
+  @After
+  public void tearDown() {
+    launcherCommands = null;
+  }
+
+  @Test
+  public void testAddGemFirePropertyFileToCommandLine() {
+    final List<String> commandLine = new ArrayList<>();
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addGemFirePropertyFile(commandLine, null);
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addGemFirePropertyFile(commandLine, StringUtils.EMPTY_STRING);
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addGemFirePropertyFile(commandLine, " ");
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addGemFirePropertyFile(commandLine,
+        "/path/to/gemfire.properties");
+
+    assertFalse(commandLine.isEmpty());
+    assertTrue(commandLine.contains("-DgemfirePropertyFile=/path/to/gemfire.properties"));
+  }
+
+  @Test
+  public void testAddGemFireSystemPropertiesToCommandLine() {
+    final List<String> commandLine = new ArrayList<>();
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addGemFireSystemProperties(commandLine, new Properties());
+
+    assertTrue(commandLine.isEmpty());
+
+    final Properties gemfireProperties = new Properties();
+
+    gemfireProperties.setProperty(LOCATORS, "localhost[11235]");
+    gemfireProperties.setProperty(LOG_LEVEL, "config");
+    gemfireProperties.setProperty(LOG_FILE, StringUtils.EMPTY_STRING);
+    gemfireProperties.setProperty(MCAST_PORT, "0");
+    gemfireProperties.setProperty(NAME, "machine");
+
+    getLauncherLifecycleCommands().addGemFireSystemProperties(commandLine, gemfireProperties);
+
+    assertFalse(commandLine.isEmpty());
+    assertEquals(4, commandLine.size());
+
+    for (final String propertyName : gemfireProperties.stringPropertyNames()) {
+      final String propertyValue = gemfireProperties.getProperty(propertyName);
+      if (StringUtils.isBlank(propertyValue)) {
+        for (final String systemProperty : commandLine) {
+          assertFalse(systemProperty.startsWith(
+              "-D" + DistributionConfig.GEMFIRE_PREFIX + "".concat(propertyName).concat("=")));
+        }
+      } else {
+        assertTrue(commandLine.contains("-D" + DistributionConfig.GEMFIRE_PREFIX
+            + "".concat(propertyName).concat("=").concat(propertyValue)));
+      }
+    }
+  }
+
+
+  @Test
+  public void testAddGemFireSystemPropertiesToCommandLineWithRestAPI() {
+    final List<String> commandLine = new ArrayList<>();
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addGemFireSystemProperties(commandLine, new Properties());
+
+    assertTrue(commandLine.isEmpty());
+
+    final Properties gemfireProperties = new Properties();
+
+    gemfireProperties.setProperty(LOCATORS, "localhost[11235]");
+    gemfireProperties.setProperty(LOG_LEVEL, "config");
+    gemfireProperties.setProperty(LOG_FILE, StringUtils.EMPTY_STRING);
+    gemfireProperties.setProperty(MCAST_PORT, "0");
+    gemfireProperties.setProperty(NAME, "machine");
+
+    gemfireProperties.setProperty(START_DEV_REST_API, "true");
+    gemfireProperties.setProperty(HTTP_SERVICE_PORT, "8080");
+    gemfireProperties.setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
+
+
+    getLauncherLifecycleCommands().addGemFireSystemProperties(commandLine, gemfireProperties);
+
+    assertFalse(commandLine.isEmpty());
+    assertEquals(7, commandLine.size());
+
+    for (final String propertyName : gemfireProperties.stringPropertyNames()) {
+      final String propertyValue = gemfireProperties.getProperty(propertyName);
+      if (StringUtils.isBlank(propertyValue)) {
+        for (final String systemProperty : commandLine) {
+          assertFalse(systemProperty.startsWith(
+              "-D" + DistributionConfig.GEMFIRE_PREFIX + "".concat(propertyName).concat("=")));
+        }
+      } else {
+        assertTrue(commandLine.contains("-D" + DistributionConfig.GEMFIRE_PREFIX
+            + "".concat(propertyName).concat("=").concat(propertyValue)));
+      }
+    }
+  }
+
+  @Test
+  public void testAddInitialHeapToCommandLine() {
+    final List<String> commandLine = new ArrayList<>();
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addInitialHeap(commandLine, null);
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addInitialHeap(commandLine, StringUtils.EMPTY_STRING);
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addInitialHeap(commandLine, " ");
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addInitialHeap(commandLine, "512M");
+
+    assertFalse(commandLine.isEmpty());
+    assertEquals("-Xms512M", commandLine.get(0));
+  }
+
+  @Test
+  public void testAddJvmArgumentsAndOptionsToCommandLine() {
+    final List<String> commandLine = new ArrayList<>();
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addJvmArgumentsAndOptions(commandLine, null);
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addJvmArgumentsAndOptions(commandLine, new String[] {});
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addJvmArgumentsAndOptions(commandLine,
+        new String[] {"-DmyProp=myVal", "-d64", "-server", "-Xprof"});
+
+    assertFalse(commandLine.isEmpty());
+    assertEquals(4, commandLine.size());
+    assertEquals("-DmyProp=myVal", commandLine.get(0));
+    assertEquals("-d64", commandLine.get(1));
+    assertEquals("-server", commandLine.get(2));
+    assertEquals("-Xprof", commandLine.get(3));
+  }
+
+  // Fix for Bug #47192 - "Making GemFire (JVM) to exit in case of OutOfMemory"
+  @Test
+  public void testAddJvmOptionsForOutOfMemoryErrors() {
+    final List<String> jvmOptions = new ArrayList<>(1);
+
+    getLauncherLifecycleCommands().addJvmOptionsForOutOfMemoryErrors(jvmOptions);
+
+    if (SystemUtils.isHotSpotVM()) {
+      if (SystemUtils.isWindows()) {
+        assertTrue(jvmOptions.contains("-XX:OnOutOfMemoryError=taskkill /F /PID %p"));
+      } else {
+        assertTrue(jvmOptions.contains("-XX:OnOutOfMemoryError=kill -KILL %p"));
+      }
+    } else if (SystemUtils.isJ9VM()) {
+      assertEquals(1, jvmOptions.size());
+      assertTrue(jvmOptions.contains("-Xcheck:memory"));
+    } else if (SystemUtils.isJRockitVM()) {
+      assertEquals(1, jvmOptions.size());
+      assertTrue(jvmOptions.contains("-XXexitOnOutOfMemory"));
+    } else {
+      assertTrue(jvmOptions.isEmpty());
+    }
+  }
+
+  @Test
+  public void testAddMaxHeapToCommandLine() {
+    final List<String> commandLine = new ArrayList<>();
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addMaxHeap(commandLine, null);
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addMaxHeap(commandLine, StringUtils.EMPTY_STRING);
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addMaxHeap(commandLine, "  ");
+
+    assertTrue(commandLine.isEmpty());
+
+    getLauncherLifecycleCommands().addMaxHeap(commandLine, "1024M");
+
+    assertFalse(commandLine.isEmpty());
+    assertEquals(3, commandLine.size());
+    assertEquals("-Xmx1024M", commandLine.get(0));
+    assertEquals("-XX:+UseConcMarkSweepGC", commandLine.get(1));
+    assertEquals("-XX:CMSInitiatingOccupancyFraction="
+        + LauncherLifecycleCommands.CMS_INITIAL_OCCUPANCY_FRACTION, commandLine.get(2));
+  }
+
+  @Test(expected = AssertionError.class)
+  public void testReadPidWithNull() {
+    try {
+      getLauncherLifecycleCommands().readPid(null);
+    } catch (AssertionError expected) {
+      assertEquals("The file from which to read the process ID (pid) cannot be null!",
+          expected.getMessage());
+      throw expected;
+    }
+  }
+
+  @Test
+  @SuppressWarnings("deprecation")
+  public void testGetClasspath() {
+    assertEquals(System.getProperty("java.class.path"),
+        getLauncherLifecycleCommands().getClasspath(null));
+  }
+
+  @Test
+  @SuppressWarnings("deprecation")
+  public void testGetClasspathWithUserDefinedClasspath() {
+    assertEquals(
+        System.getProperty("java.class.path") + File.pathSeparator + "/path/to/user/classes",
+        getLauncherLifecycleCommands().getClasspath("/path/to/user/classes"));
+  }
+
+  @Test
+  public void testGetSystemClasspath() {
+    assertEquals(System.getProperty("java.class.path"),
+        getLauncherLifecycleCommands().getSystemClasspath());
+  }
+
+  @Test
+  public void testLocatorClasspathOrder() {
+    String userClasspath = "/path/to/user/lib/app.jar:/path/to/user/classes";
+
+    String expectedClasspath = launcherCommands.getGemFireJarPath().concat(File.pathSeparator)
+        .concat(userClasspath).concat(File.pathSeparator)
+        .concat(System.getProperty("java.class.path")).concat(File.pathSeparator)
+        .concat(LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME);
+
+    String actualClasspath = launcherCommands.getLocatorClasspath(true, userClasspath);
+
+    assertEquals(expectedClasspath, actualClasspath);
+  }
+
+  @Test
+  public void testServerClasspathOrder() {
+    String userClasspath = "/path/to/user/lib/app.jar:/path/to/user/classes";
+
+    String expectedClasspath = launcherCommands.getGemFireJarPath().concat(File.pathSeparator)
+        .concat(userClasspath).concat(File.pathSeparator)
+        .concat(LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME);
+
+    String actualClasspath = launcherCommands.getServerClasspath(false, userClasspath);
+
+    assertEquals(expectedClasspath, actualClasspath);
+  }
+
+  @Test
+  public void testToClasspath() {
+    final boolean EXCLUDE_SYSTEM_CLASSPATH = false;
+    final boolean INCLUDE_SYSTEM_CLASSPATH = true;
+
+    String[] jarFilePathnames =
+        {"/path/to/user/libs/A.jar", "/path/to/user/libs/B.jar", "/path/to/user/libs/C.jar"};
+
+    String[] userClasspaths = {"/path/to/classes:/path/to/libs/1.jar:/path/to/libs/2.jar",
+        "/path/to/ext/libs/1.jar:/path/to/ext/classes:/path/to/ext/lib/10.jar"};
+
+    String expectedClasspath = LauncherLifecycleCommands.GEODE_JAR_PATHNAME
+        .concat(File.pathSeparator).concat(toClasspath(userClasspaths)).concat(File.pathSeparator)
+        .concat(toClasspath(jarFilePathnames));
+
+    assertEquals(expectedClasspath, getLauncherLifecycleCommands()
+        .toClasspath(EXCLUDE_SYSTEM_CLASSPATH, jarFilePathnames, userClasspaths));
+
+    expectedClasspath = LauncherLifecycleCommands.GEODE_JAR_PATHNAME.concat(File.pathSeparator)
+        .concat(toClasspath(userClasspaths)).concat(File.pathSeparator)
+        .concat(System.getProperty("java.class.path")).concat(File.pathSeparator)
+        .concat(toClasspath(jarFilePathnames));
+
+    assertEquals(expectedClasspath, getLauncherLifecycleCommands()
+        .toClasspath(INCLUDE_SYSTEM_CLASSPATH, jarFilePathnames, userClasspaths));
+
+    expectedClasspath = LauncherLifecycleCommands.GEODE_JAR_PATHNAME.concat(File.pathSeparator)
+        .concat(System.getProperty("java.class.path"));
+
+    assertEquals(expectedClasspath, getLauncherLifecycleCommands()
+        .toClasspath(INCLUDE_SYSTEM_CLASSPATH, null, (String[]) null));
+
+    assertEquals(LauncherLifecycleCommands.GEODE_JAR_PATHNAME, getLauncherLifecycleCommands()
+        .toClasspath(EXCLUDE_SYSTEM_CLASSPATH, null, (String[]) null));
+
+    assertEquals(LauncherLifecycleCommands.GEODE_JAR_PATHNAME,
+        getLauncherLifecycleCommands().toClasspath(EXCLUDE_SYSTEM_CLASSPATH, new String[0], ""));
+  }
+
+  @Test
+  public void testToClassPathOrder() {
+    String userClasspathOne = "/path/to/user/lib/a.jar:/path/to/user/classes";
+    String userClasspathTwo =
+        "/path/to/user/lib/x.jar:/path/to/user/lib/y.jar:/path/to/user/lib/z.jar";
+
+    String expectedClasspath = launcherCommands.getGemFireJarPath().concat(File.pathSeparator)
+        .concat(userClasspathOne).concat(File.pathSeparator).concat(userClasspathTwo)
+        .concat(File.pathSeparator).concat(System.getProperty("java.class.path"))
+        .concat(File.pathSeparator).concat(LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME)
+        .concat(File.pathSeparator)
+        .concat(LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME);
+
+    String actualClasspath = launcherCommands.toClasspath(true,
+        new String[] {LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME,
+            LauncherLifecycleCommands.CORE_DEPENDENCIES_JAR_PATHNAME},
+        userClasspathOne, userClasspathTwo);
+
+    assertEquals(expectedClasspath, actualClasspath);
+  }
+
+  @Test
+  public void testGetJavaPathname() {
+    assertEquals(
+        IOUtils.appendToPath(System.getProperty("java.home"), "bin",
+            "java" + LauncherLifecycleCommands.getExecutableSuffix()),
+        getLauncherLifecycleCommands().getJdkToolPathname(
+            "java" + LauncherLifecycleCommands.getExecutableSuffix(), new GemFireException() {}));
+  }
+
+  @Test(expected = NullPointerException.class)
+  public void testGetJdkToolPathnameWithNullPathnames() {
+    try {
+      getLauncherLifecycleCommands().getJdkToolPathname((Stack<String>) null,
+          new GemFireException() {});
+    } catch (NullPointerException expected) {
+      assertEquals("The JDK tool executable pathnames cannot be null!", expected.getMessage());
+      throw expected;
+    }
+  }
+
+  @Test(expected = NullPointerException.class)
+  public void testGetJdkToolPathnameWithNullGemFireException() {
+    try {
+      getLauncherLifecycleCommands().getJdkToolPathname(new Stack<String>(), null);
+    } catch (NullPointerException expected) {
+      assertEquals("The GemFireException cannot be null!", expected.getMessage());
+      throw expected;
+    }
+  }
+
+  @Test
+  public void testGetJdkToolPathnameForNonExistingTool() {
+    try {
+      final GemFireException expected = new GemFireException() {
+        @Override
+        public String getMessage() {
+          return "expected";
+        }
+      };
+
+      getLauncherLifecycleCommands().getJdkToolPathname("nonExistingTool.exe", expected);
+    } catch (GemFireException expected) {
+      assertEquals("expected", expected.getMessage());
+    }
+  }
+
+  @Test
+  public void testGetLocatorId() {
+    assertEquals("machine[11235]", getLauncherLifecycleCommands().getLocatorId("machine", 11235));
+    assertEquals("machine.domain.org[11235]",
+        getLauncherLifecycleCommands().getLocatorId("machine.domain.org", 11235));
+    assertEquals("machine[" + DistributionLocator.DEFAULT_LOCATOR_PORT + "]",
+        getLauncherLifecycleCommands().getLocatorId("machine", null));
+  }
+
+  @Test
+  public void testGetServerId() {
+    assertEquals("machine[12480]", getLauncherLifecycleCommands().getServerId("machine", 12480));
+    assertEquals("machine.domain.org[12480]",
+        getLauncherLifecycleCommands().getServerId("machine.domain.org", 12480));
+    assertEquals("machine[" + CacheServer.DEFAULT_PORT + "]",
+        getLauncherLifecycleCommands().getServerId("machine", null));
+  }
+
+  @Test
+  public void testCreateJmxServerUrlWithMemberName() {
+    assertEquals("service:jmx:rmi://localhost:8192/jndi/rmi://localhost:8192/jmxrmi",
+        getLauncherLifecycleCommands().getJmxServiceUrlAsString("localhost[8192]"));
+  }
+
+  @Test(expected = IllegalArgumentException.class)
+  public void testCreateJmxServiceUrlWithInvalidMemberName() {
+    try {
+      System.err.println(getLauncherLifecycleCommands().getJmxServiceUrlAsString("memberOne[]"));
+    } catch (IllegalArgumentException expected) {
+      assertEquals(CliStrings.START_JCONSOLE__CONNECT_BY_MEMBER_NAME_ID_ERROR_MESSAGE,
+          expected.getMessage());
+      throw expected;
+    }
+  }
+
+  @Test
+  public void testCreateServerCommandLine() throws Exception {
+    ServerLauncher serverLauncher = new ServerLauncher.Builder()
+        .setCommand(ServerLauncher.Command.START).setDisableDefaultServer(true)
+        .setMemberName("testCreateServerCommandLine").setRebalance(true)
+        // .setServerBindAddress("localhost")
+        .setServerPort(41214).setCriticalHeapPercentage(95.5f).setEvictionHeapPercentage(85.0f)
+        .setSocketBufferSize(1024 * 1024).setMessageTimeToLive(93).build();
+
+    String[] commandLineElements = launcherCommands.createStartServerCommandLine(serverLauncher,
+        null, null, new Properties(), null, false, new String[0], false, null, null);
+
+    assertNotNull(commandLineElements);
+    assertTrue(commandLineElements.length > 0);
+
+    Set<String> expectedCommandLineElements = new HashSet<>(6);
+
+    expectedCommandLineElements.add(serverLauncher.getCommand().getName());
+    expectedCommandLineElements.add("--disable-default-server");
+    expectedCommandLineElements.add(serverLauncher.getMemberName().toLowerCase());
+    expectedCommandLineElements.add("--rebalance");
+    // expectedCommandLineElements.add(String.format("--server-bind-address=%1$s",
+    // serverLauncher.getServerBindAddress().getHostName()));
+    expectedCommandLineElements
+        .add(String.format("--server-port=%1$d", serverLauncher.getServerPort()));
+    expectedCommandLineElements.add(String.format("--critical-heap-percentage=%1$s",
+        serverLauncher.getCriticalHeapPercentage()));
+    expectedCommandLineElements.add(String.format("--eviction-heap-percentage=%1$s",
+        serverLauncher.getEvictionHeapPercentage()));
+    expectedCommandLineElements
+        .add(String.format("--socket-buffer-size=%1$d", serverLauncher.getSocketBufferSize()));
+    expectedCommandLineElements
+        .add(String.format("--message-time-to-live=%1$d", serverLauncher.getMessageTimeToLive()));
+
+    for (String commandLineElement : commandLineElements) {
+      expectedCommandLineElements.remove(commandLineElement.toLowerCase());
+    }
+
+    assertTrue(String.format("Expected ([]); but was (%1$s)", expectedCommandLineElements),
+        expectedCommandLineElements.isEmpty());
+  }
+
+  @Test
+  public void testCreateServerCommandLineWithRestAPI() throws Exception {
+    ServerLauncher serverLauncher = new ServerLauncher.Builder()
+        .setCommand(ServerLauncher.Command.START).setDisableDefaultServer(true)
+        .setMemberName("testCreateServerCommandLine").setRebalance(true)
+        // .setServerBindAddress("localhost")
+        .setServerPort(41214).setCriticalHeapPercentage(95.5f).setEvictionHeapPercentage(85.0f)
+        .build();
+
+    Properties gemfireProperties = new Properties();
+    gemfireProperties.setProperty(START_DEV_REST_API, "true");
+    gemfireProperties.setProperty(HTTP_SERVICE_PORT, "8080");
+    gemfireProperties.setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
+
+
+    String[] commandLineElements = launcherCommands.createStartServerCommandLine(serverLauncher,
+        null, null, gemfireProperties, null, false, new String[0], false, null, null);
+
+    assertNotNull(commandLineElements);
+    assertTrue(commandLineElements.length > 0);
+
+    Set<String> expectedCommandLineElements = new HashSet<>(6);
+
+    expectedCommandLineElements.add(serverLauncher.getCommand().getName());
+    expectedCommandLineElements.add("--disable-default-server");
+    expectedCommandLineElements.add(serverLauncher.getMemberName().toLowerCase());
+    expectedCommandLineElements.add("--rebalance");
+    // expectedCommandLineElements.add(String.format("--server-bind-address=%1$s",
+    // serverLauncher.getServerBindAddress().getHostName()));
+    expectedCommandLineElements
+        .add(String.format("--server-port=%1$d", serverLauncher.getServerPort()));
+    expectedCommandLineElements.add(String.format("--critical-heap-percentage=%1$s",
+        serverLauncher.getCriticalHeapPercentage()));
+    expectedCommandLineElements.add(String.format("--eviction-heap-percentage=%1$s",
+        serverLauncher.getEvictionHeapPercentage()));
+
+    expectedCommandLineElements
+        .add("-d" + DistributionConfig.GEMFIRE_PREFIX + "" + START_DEV_REST_API + "=" + "true");
+    expectedCommandLineElements
+        .add("-d" + DistributionConfig.GEMFIRE_PREFIX + "" + HTTP_SERVICE_PORT + "=" + "8080");
+    expectedCommandLineElements.add("-d" + DistributionConfig.GEMFIRE_PREFIX + ""
+        + HTTP_SERVICE_BIND_ADDRESS + "=" + "localhost");
+
+
+    for (String commandLineElement : commandLineElements) {
+      expectedCommandLineElements.remove(commandLineElement.toLowerCase());
+    }
+
+    assertTrue(String.format("Expected ([]); but was (%1$s)", expectedCommandLineElements),
+        expectedCommandLineElements.isEmpty());
+  }
+
+  /**
+   * Verify commandline parameters passed for starting locator
+   * 
+   * @throws Exception
+   */
+  @Test
+  public void testLocatorCommandLineWithRestAPI() throws Exception {
+    LocatorLauncher locatorLauncher =
+        new LocatorLauncher.Builder().setCommand(LocatorLauncher.Command.START)
+            .setMemberName("testLocatorCommandLineWithRestAPI").setBindAddress("localhost")
+            .setPort(11111).build();
+
+    Properties gemfireProperties = new Properties();
+    gemfireProperties.setProperty(HTTP_SERVICE_PORT, "8089");
+    gemfireProperties.setProperty(HTTP_SERVICE_BIND_ADDRESS, "localhost");
+
+
+    String[] commandLineElements = launcherCommands.createStartLocatorCommandLine(locatorLauncher,
+        null, null, gemfireProperties, null, false, new String[0], null, null);
+
+    assertNotNull(commandLineElements);
+    assertTrue(commandLineElements.length > 0);
+
+    Set<String> expectedCommandLineElements = new HashSet<>(6);
+
+    expectedCommandLineElements.add(locatorLauncher.getCommand().getName());
+    expectedCommandLineElements.add(locatorLauncher.getMemberName().toLowerCase());
+    expectedCommandLineElements.add(String.format("--port=%1$d", locatorLauncher.getPort()));
+    expectedCommandLineElements
+        .add("-d" + DistributionConfig.GEMFIRE_PREFIX + "" + HTTP_SERVICE_PORT + "=" + "8089");
+    expectedCommandLineElements.add("-d" + DistributionConfig.GEMFIRE_PREFIX + ""
+        + HTTP_SERVICE_BIND_ADDRESS + "=" + "localhost");
+
+
+    for (String commandLineElement : commandLineElements) {
+      expectedCommandLineElements.remove(commandLineElement.toLowerCase());
+    }
+
+    assertTrue(String.format("Expected ([]); but was (%1$s)", expectedCommandLineElements),
+        expectedCommandLineElements.isEmpty());
+  }
+
+
+  @Test
+  public void testReadPidWithNonExistingFile() {
+    assertEquals(LauncherLifecycleCommands.INVALID_PID,
+        getLauncherLifecycleCommands().readPid(new File("/path/to/non_existing/pid.file")));
+  }
+
+  private LauncherLifecycleCommands getLauncherLifecycleCommands() {
+    return launcherCommands;
+  }
+
+  private String toClasspath(final String... jarFilePathnames) {
+    String classpath = StringUtils.EMPTY_STRING;
+
+    if (jarFilePathnames != null) {
+      for (final String jarFilePathname : jarFilePathnames) {
+        classpath += (classpath.isEmpty() ? StringUtils.EMPTY_STRING : File.pathSeparator);
+        classpath += jarFilePathname;
+      }
+    }
+
+    return classpath;
+  }
+
+  private String toPath(Object... pathElements) {
+    String path = "";
+
+    for (Object pathElement : pathElements) {
+      path += (path.isEmpty() ? StringUtils.EMPTY_STRING : File.pathSeparator);
+      path += pathElement;
+    }
+
+    return path;
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-core/src/main/java/org/apache/geode/distributed/AbstractLauncher.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/distributed/AbstractLauncher.java b/geode-core/src/main/java/org/apache/geode/distributed/AbstractLauncher.java
index 64e20f3..feba893 100644
--- a/geode-core/src/main/java/org/apache/geode/distributed/AbstractLauncher.java
+++ b/geode-core/src/main/java/org/apache/geode/distributed/AbstractLauncher.java
@@ -72,31 +72,21 @@ public abstract class AbstractLauncher<T extends Comparable<T>> implements Runna
 
   protected static final long READ_PID_FILE_TIMEOUT_MILLIS = 2 * 1000;
 
-  // @see
-  // http://publib.boulder.ibm.com/infocenter/javasdk/v6r0/index.jsp?topic=%2Fcom.ibm.java.doc.user.lnx.60%2Fuser%2Fattachapi.html
-  // @see
-  // http://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/geninfo/diagnos/aboutjrockit.html#wp1083571
-  private static final List<String> ATTACH_API_PACKAGES = Arrays.asList("com.sun.tools.attach",
-      "com/sun/tools/attach", "com.ibm.tools.attach", "com/ibm/tools/attach");
-
   public static final String DEFAULT_WORKING_DIRECTORY = SystemUtils.CURRENT_DIRECTORY;
+
   public static final String SIGNAL_HANDLER_REGISTRATION_SYSTEM_PROPERTY =
       DistributionConfig.GEMFIRE_PREFIX + "launcher.registerSignalHandlers";
 
   protected static final String OPTION_PREFIX = "-";
 
-  private static final String IBM_ATTACH_API_CLASS_NAME =
-      "com.ibm.tools.attach.AgentNotSupportedException";
-  private static final String SUN_ATTACH_API_CLASS_NAME =
-      "com.sun.tools.attach.AttachNotSupportedException";
   private static final String SUN_SIGNAL_API_CLASS_NAME = "sun.misc.Signal";
 
   private volatile boolean debug;
 
   protected final transient AtomicBoolean running = new AtomicBoolean(false);
 
-  protected Logger logger = Logger.getLogger(getClass().getName()); // TODO:KIRK: does this need
-  // log4j2?
+  // TODO: use log4j logger instead of JUL
+  protected Logger logger = Logger.getLogger(getClass().getName());
 
   public AbstractLauncher() {
     try {
@@ -197,52 +187,6 @@ public abstract class AbstractLauncher<T extends Comparable<T>> implements Runna
   }
 
   /**
-   * This method attempts to make a best effort determination for whether the Attach API classes are
-   * on the classpath.
-   *
-   * @param t the Throwable being evaluated for missing Attach API classes.
-   * @return a boolean indicating whether the Exception or Error condition is a result of the Attach
-   *         API missing from the classpath.
-   */
-  protected boolean isAttachAPINotFound(final Throwable t) {
-    boolean missing = false;
-
-    // NOTE prerequisite, Throwable must be a ClassNotFoundException or NoClassDefFoundError
-    if (t instanceof ClassNotFoundException || t instanceof NoClassDefFoundError) {
-      // NOTE the use of the 'testing' class member variable, yuck!
-      if (!isAttachAPIOnClasspath()) {
-        // NOTE ok, the Attach API is not available, however we still do not know whether an user
-        // application class
-        // caused the ClassNotFoundException or NoClassDefFoundError.
-        final StringWriter stackTraceWriter = new StringWriter();
-
-        // NOTE the full stack trace includes the Throwable message, which typically indicates the
-        // Exception/Error
-        // thrown and the reason (as in which class was not found).
-        t.printStackTrace(new PrintWriter(stackTraceWriter));
-
-        final String stackTrace = stackTraceWriter.toString();
-
-        for (String attachApiPackage : ATTACH_API_PACKAGES) {
-          missing |= stackTrace.contains(attachApiPackage);
-        }
-      }
-    }
-
-    return missing;
-  }
-
-  /**
-   * Determines if the Attach API is on the classpath.
-   *
-   * @return a boolean value indicating if the Attach API is on the classpath.
-   */
-  boolean isAttachAPIOnClasspath() {
-    return (ClassUtils.isClassAvailable(SUN_ATTACH_API_CLASS_NAME)
-        || ClassUtils.isClassAvailable(IBM_ATTACH_API_CLASS_NAME));
-  }
-
-  /**
    * Determines whether the Locator launcher is in debug mode.
    *
    * @return a boolean to indicate whether the Locator launcher is in debug mode.

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java
index e677ba3..d42d75e 100755
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java
@@ -74,7 +74,6 @@ import org.apache.geode.internal.process.signal.SignalEvent;
 import org.apache.geode.internal.process.signal.SignalListener;
 import org.apache.geode.internal.util.IOUtils;
 import org.apache.geode.internal.util.StopWatch;
-import org.apache.geode.lang.AttachAPINotFoundException;
 import org.apache.geode.management.DistributedSystemMXBean;
 import org.apache.geode.management.MemberMXBean;
 import org.apache.geode.management.cli.CliMetaData;
@@ -169,11 +168,6 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
   protected static final int MINIMUM_HEAP_FREE_RATIO = 10;
   protected static final int NUM_ATTEMPTS_FOR_SHARED_CONFIGURATION_STATUS = 3;
 
-  protected static final AtomicReference<Boolean> ATTACH_API_AVAILABLE =
-      new AtomicReference<>(null);
-
-  protected static final String ATTACH_API_CLASS_NAME =
-      "com.sun.tools.attach.AttachNotSupportedException";
   protected static final String GEODE_HOME = System.getenv("GEODE_HOME");
   protected static final String JAVA_HOME = System.getProperty("java.home");
   protected static final String LOCALHOST = "localhost";
@@ -185,19 +179,6 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
   protected static final String CORE_DEPENDENCIES_JAR_PATHNAME =
       IOUtils.appendToPath(GEODE_HOME, "lib", "geode-dependencies.jar");
 
-  protected static boolean isAttachApiAvailable() {
-    if (ATTACH_API_AVAILABLE.get() == null) {
-      try {
-        ClassUtils.forName(ATTACH_API_CLASS_NAME, new AttachAPINotFoundException());
-        ATTACH_API_AVAILABLE.set(Boolean.TRUE);
-      } catch (AttachAPINotFoundException ignore) {
-        ATTACH_API_AVAILABLE.set(Boolean.FALSE);
-      }
-    }
-
-    return ATTACH_API_AVAILABLE.get();
-  }
-
   private final ThreePhraseGenerator nameGenerator;
 
   public LauncherLifecycleCommands() {
@@ -424,9 +405,7 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
               TimeUnit.MILLISECONDS.timedWait(this, 500);
             }
 
-            locatorState = (ProcessUtils.isAttachApiAvailable()
-                ? locatorStatus(locatorPidFile, oldPid, memberName)
-                : locatorStatus(workingDirectory, memberName));
+            locatorState = locatorStatus(workingDirectory, memberName);
 
             String currentLocatorStatusMessage = locatorState.getStatusMessage();
 
@@ -1222,29 +1201,6 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
     return new File(new File(JAVA_HOME, "bin"), "java").getPath();
   }
 
-  @Deprecated
-  protected String getToolsJarPath() throws AttachAPINotFoundException {
-    String toolsJarPathname = null;
-
-    if (!SystemUtils.isMacOSX()) {
-      toolsJarPathname = IOUtils.appendToPath(JAVA_HOME, "lib", "tools.jar");
-
-      if (!IOUtils.isExistingPathname(toolsJarPathname)) {
-        // perhaps the java.home System property refers to the JRE ($JAVA_HOME/jre)...
-        String JDK_HOME = new File(JAVA_HOME).getParentFile().getPath();
-        toolsJarPathname = IOUtils.appendToPath(JDK_HOME, "lib", "tools.jar");
-      }
-
-      try {
-        IOUtils.verifyPathnameExists(toolsJarPathname);
-      } catch (IOException e) {
-        throw new AttachAPINotFoundException(getAttachAPINotFoundMessage());
-      }
-    }
-
-    return toolsJarPathname;
-  }
-
   // TODO refactor the following method into a common base class or utility class
   protected String getLocalHost() {
     try {
@@ -1254,11 +1210,6 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
     }
   }
 
-  protected String getAttachAPINotFoundMessage() {
-    return CliStrings.format(CliStrings.ATTACH_API_IN_0_NOT_FOUND_ERROR_MESSAGE,
-        ((SystemUtils.isMacOSX() && SystemUtils.isAppleJVM()) ? "classes.jar" : "tools.jar"));
-  }
-
   protected String getLocatorId(final String host, final Integer port) {
     final String locatorHost = (host != null ? host : getLocalHost());
     final String locatorPort =
@@ -1347,8 +1298,7 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
   }
 
   protected boolean isVmWithProcessIdRunning(final Integer pid) {
-    // note: this will use JNA if available or Attach if available or return false if neither is
-    // available
+    // note: this will use JNA if available or return false
     return ProcessUtils.isProcessAlive(pid);
   }
 
@@ -1717,9 +1667,7 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
               TimeUnit.MILLISECONDS.timedWait(this, 500);
             }
 
-            serverState = (ProcessUtils.isAttachApiAvailable()
-                ? serverStatus(serverPidFile, oldPid, memberName)
-                : serverStatus(workingDirectory, memberName));
+            serverState = serverStatus(workingDirectory, memberName);
 
             String currentServerStatusMessage = serverState.getStatusMessage();
 

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
index 7696aa8..51d39d6 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
@@ -2721,7 +2721,8 @@ public class CliStrings {
   public static final String STATUS_LOCATOR__MEMBER__HELP =
       "Member name or ID of the Locator in the Geode cluster.";
   public static final String STATUS_LOCATOR__PID = "pid";
-  public static final String STATUS_LOCATOR__PID__HELP = "Process ID (PID) of the running Locator.";
+  public static final String STATUS_LOCATOR__PID__HELP =
+      "Process ID (PID) of the running Locator. Deprecated: Since Geode1.2. Requires the JDK tools.jar which is not included on the classpath by default. Use --dir instead.";
   public static final String STATUS_LOCATOR__PORT = "port";
   public static final String STATUS_LOCATOR__PORT__HELP =
       "Port on which the Locator is listening. The default is 10334.";
@@ -2745,7 +2746,7 @@ public class CliStrings {
       "No Geode Cache Server with member name or ID {0} could be found.";
   public static final String STATUS_SERVER__PID = "pid";
   public static final String STATUS_SERVER__PID__HELP =
-      "Process ID (PID) of the running Geode Cache Server.";
+      "Process ID (PID) of the running Geode Cache Server. Deprecated: Since Geode1.2. Requires the JDK tools.jar which is not included on the classpath by default. Use --dir instead.";
 
   /* stop gateway-reciver */
   public static final String STOP_GATEWAYRECEIVER = "stop gateway-receiver";
@@ -2782,7 +2783,7 @@ public class CliStrings {
       "Member name or ID of the Locator in the Geode cluster.";
   public static final String STOP_LOCATOR__PID = "pid";
   public static final String STOP_LOCATOR__PID__HELP =
-      "The process id (PID) of the running Locator.";
+      "The process id (PID) of the running Locator. Deprecated: Since Geode1.2. Requires the JDK tools.jar which is not included on the classpath by default. Use --dir instead.";
   public static final String STOP_LOCATOR__GENERAL_ERROR_MESSAGE =
       "An error occurred while attempting to stop a Locator: %1$s";
   public static final String STOP_LOCATOR__LOCATOR_IS_CACHE_SERVER_ERROR_MESSAGE =
@@ -2813,7 +2814,7 @@ public class CliStrings {
       "No Cache Server with member name or ID {0} could be found.";
   public static final String STOP_SERVER__PID = "pid";
   public static final String STOP_SERVER__PID__HELP =
-      "Process ID (PID) of the running Geode Cache Server.";
+      "Process ID (PID) of the running Geode Cache Server. Deprecated: Since Geode1.2. Requires the JDK tools.jar which is not included on the classpath by default. Use --dir instead.";
   public static final String STOP_SERVER__SHUTDOWN_MEMBER_MESSAGE =
       "Cache Server {0} has been requested to stop.";
   public static final String STOP_SERVER__STOPPING_SERVER_MESSAGE =

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-core/src/test/java/org/apache/geode/distributed/AbstractLauncherTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/distributed/AbstractLauncherTest.java b/geode-core/src/test/java/org/apache/geode/distributed/AbstractLauncherTest.java
index f487df9..feba601 100644
--- a/geode-core/src/test/java/org/apache/geode/distributed/AbstractLauncherTest.java
+++ b/geode-core/src/test/java/org/apache/geode/distributed/AbstractLauncherTest.java
@@ -14,6 +14,9 @@
  */
 package org.apache.geode.distributed;
 
+import static org.apache.geode.distributed.ConfigurationProperties.*;
+import static org.junit.Assert.*;
+
 import org.apache.geode.internal.lang.StringUtils;
 import org.apache.geode.test.junit.categories.UnitTest;
 import org.junit.Test;
@@ -24,9 +27,6 @@ import java.net.URL;
 import java.util.Properties;
 import java.util.concurrent.TimeUnit;
 
-import static org.junit.Assert.*;
-import static org.apache.geode.distributed.ConfigurationProperties.*;
-
 /**
  * The AbstractLauncherTest class is a test suite of unit tests testing the contract and
  * functionality of the AbstractLauncher class.
@@ -46,28 +46,6 @@ public class AbstractLauncherTest {
   }
 
   @Test
-  public void testIsAttachAPINotFound() {
-    final AbstractLauncher<?> launcher = createAbstractLauncher("012", "TestMember");
-
-    assertTrue(launcher.isAttachAPINotFound(new NoClassDefFoundError(
-        "Exception in thread \"main\" java.lang.NoClassDefFoundError: com/sun/tools/attach/AttachNotSupportedException")));
-    assertTrue(launcher.isAttachAPINotFound(new ClassNotFoundException(
-        "Caused by: java.lang.ClassNotFoundException: com.sun.tools.attach.AttachNotSupportedException")));
-    assertTrue(launcher.isAttachAPINotFound(new NoClassDefFoundError(
-        "Exception in thread \"main\" java.lang.NoClassDefFoundError: com/ibm/tools/attach/AgentNotSupportedException")));
-    assertTrue(launcher.isAttachAPINotFound(new ClassNotFoundException(
-        "Caused by: java.lang.ClassNotFoundException: com.ibm.tools.attach.AgentNotSupportedException")));
-    assertFalse(launcher.isAttachAPINotFound(new IllegalArgumentException(
-        "Caused by: java.lang.ClassNotFoundException: com.sun.tools.attach.AttachNotSupportedException")));
-    assertFalse(launcher.isAttachAPINotFound(new IllegalStateException(
-        "Caused by: java.lang.ClassNotFoundException: com.ibm.tools.attach.AgentNotSupportedException")));
-    assertFalse(launcher.isAttachAPINotFound(new NoClassDefFoundError(
-        "Exception in thread \"main\" java.lang.NoClassDefFoundError: com/companyx/app/service/MyServiceClass")));
-    assertFalse(launcher.isAttachAPINotFound(new ClassNotFoundException(
-        "Caused by: java.lang.ClassNotFoundException: com.companyx.app.attach.NutsNotAttachedException")));
-  }
-
-  @Test
   public void testIsSet() {
     final Properties properties = new Properties();
 
@@ -282,11 +260,6 @@ public class AbstractLauncherTest {
     }
 
     @Override
-    boolean isAttachAPIOnClasspath() {
-      return false;
-    }
-
-    @Override
     public String getLogFileName() {
       throw new UnsupportedOperationException("Not Implemented!");
     }

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/HelpCommandsIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/HelpCommandsIntegrationTest.java b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/HelpCommandsIntegrationTest.java
index 1483aad..b91a1f3 100644
--- a/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/HelpCommandsIntegrationTest.java
+++ b/geode-core/src/test/java/org/apache/geode/management/internal/cli/commands/HelpCommandsIntegrationTest.java
@@ -73,8 +73,7 @@ public class HelpCommandsIntegrationTest {
   }
 
   /**
-   * TODO:GEODE-1466: update golden file to geode.properties TODO:GEODE-1566: update golden file to
-   * GeodeRedisServer
+   * TODO:GEODE-1466: update golden file to geode.properties
    */
   @Test
   public void testOfflineHelp() throws Exception {

http://git-wip-us.apache.org/repos/asf/geode/blob/72e152f3/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
----------------------------------------------------------------------
diff --git a/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties b/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
index 8e21a3a..17b7515 100644
--- a/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
+++ b/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
@@ -2874,7 +2874,8 @@ PARAMETERS\n\
 \ \ \ \ \ \ \ \ Port on which the Locator is listening. The default is 10334.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ pid\n\
-\ \ \ \ \ \ \ \ Process ID (PID) of the running Locator.\n\
+\ \ \ \ \ \ \ \ Process ID (PID) of the running Locator. Deprecated: Since Geode1.2. Requires the JDK\n\
+\ \ \ \ \ \ \ \ tools.jar which is not included on the classpath by default. Use --dir instead.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ dir\n\
 \ \ \ \ \ \ \ \ Working directory in which the Locator is running. The default is the current directory.\n\
@@ -2894,7 +2895,8 @@ PARAMETERS\n\
 \ \ \ \ \ \ \ \ Member name or ID of the Cache Server in the Geode cluster.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ pid\n\
-\ \ \ \ \ \ \ \ Process ID (PID) of the running Geode Cache Server.\n\
+\ \ \ \ \ \ \ \ Process ID (PID) of the running Geode Cache Server. Deprecated: Since Geode1.2. Requires\n\
+\ \ \ \ \ \ \ \ the JDK tools.jar which is not included on the classpath by default. Use --dir instead.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ dir\n\
 \ \ \ \ \ \ \ \ Working directory in which the Cache Server is running. The default is the current\n\
@@ -2952,7 +2954,8 @@ PARAMETERS\n\
 \ \ \ \ \ \ \ \ Member name or ID of the Locator in the Geode cluster.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ pid\n\
-\ \ \ \ \ \ \ \ The process id (PID) of the running Locator.\n\
+\ \ \ \ \ \ \ \ The process id (PID) of the running Locator. Deprecated: Since Geode1.2. Requires the JDK\n\
+\ \ \ \ \ \ \ \ tools.jar which is not included on the classpath by default. Use --dir instead.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ dir\n\
 \ \ \ \ \ \ \ \ Working directory in which the Locator is running. The default is the current directory.\n\
@@ -2981,7 +2984,8 @@ PARAMETERS\n\
 \ \ \ \ \ \ \ \ Member name or ID of the Cache Server in the Geode cluster.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ pid\n\
-\ \ \ \ \ \ \ \ Process ID (PID) of the running Geode Cache Server.\n\
+\ \ \ \ \ \ \ \ Process ID (PID) of the running Geode Cache Server. Deprecated: Since Geode1.2. Requires\n\
+\ \ \ \ \ \ \ \ the JDK tools.jar which is not included on the classpath by default. Use --dir instead.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ dir\n\
 \ \ \ \ \ \ \ \ Working directory in which the Cache Server is running. The default is the current\n\


[25/49] geode git commit: GEODE-2643: Combine chunk and file region into a single region

Posted by kh...@apache.org.
GEODE-2643: Combine chunk and file region into a single region

* removed file and chunk count from stat
* removed tests that were doing checks against chunk bucketRegions


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/5f9da297
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/5f9da297
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/5f9da297

Branch: refs/heads/feature/GEODE-2420
Commit: 5f9da297a8e9bbfd41bbd02c831e56d05d0d75e0
Parents: 81794ec
Author: Jason Huynh <hu...@gmail.com>
Authored: Mon Mar 13 16:35:56 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../lucene/internal/IndexRepositoryFactory.java |  28 +++--
 .../LuceneIndexForPartitionedRegion.java        |  76 ++++---------
 .../internal/directory/RegionDirectory.java     |   5 +-
 .../lucene/internal/filesystem/FileSystem.java  |  70 +++++-------
 .../internal/filesystem/FileSystemStats.java    |  20 ----
 .../lucene/LuceneIndexDestroyDUnitTest.java     |   5 -
 .../LuceneIndexMaintenanceIntegrationTest.java  |   2 -
 ...LuceneQueriesPersistenceIntegrationTest.java |   2 -
 .../LuceneIndexForPartitionedRegionTest.java    | 102 ++---------------
 .../LuceneIndexRecoveryHAIntegrationTest.java   |   6 +-
 .../PartitionedRepositoryManagerJUnitTest.java  |  56 ++++------
 .../directory/RegionDirectoryJUnitTest.java     |   3 +-
 .../DistributedScoringJUnitTest.java            |   5 +-
 .../filesystem/FileSystemJUnitTest.java         | 112 ++++---------------
 .../IndexRepositoryImplJUnitTest.java           |   5 +-
 .../IndexRepositoryImplPerformanceTest.java     |  13 +--
 .../cache/lucene/test/LuceneTestUtilities.java  |   3 -
 17 files changed, 131 insertions(+), 382 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/IndexRepositoryFactory.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/IndexRepositoryFactory.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/IndexRepositoryFactory.java
index 7e685b7..475311d 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/IndexRepositoryFactory.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/IndexRepositoryFactory.java
@@ -43,20 +43,18 @@ public class IndexRepositoryFactory {
       LuceneIndexImpl index, PartitionedRegion userRegion, final IndexRepository oldRepository)
       throws IOException {
     LuceneIndexForPartitionedRegion indexForPR = (LuceneIndexForPartitionedRegion) index;
-    final PartitionedRegion fileRegion = indexForPR.getFileRegion();
-    final PartitionedRegion chunkRegion = indexForPR.getChunkRegion();
+    final PartitionedRegion fileRegion = indexForPR.getFileAndChunkRegion();
 
-    BucketRegion fileBucket = getMatchingBucket(fileRegion, bucketId);
-    BucketRegion chunkBucket = getMatchingBucket(chunkRegion, bucketId);
+    BucketRegion fileAndChunkBucket = getMatchingBucket(fileRegion, bucketId);
     BucketRegion dataBucket = getMatchingBucket(userRegion, bucketId);
     boolean success = false;
-    if (fileBucket == null || chunkBucket == null) {
+    if (fileAndChunkBucket == null) {
       if (oldRepository != null) {
         oldRepository.cleanup();
       }
       return null;
     }
-    if (!chunkBucket.getBucketAdvisor().isPrimary()) {
+    if (!fileAndChunkBucket.getBucketAdvisor().isPrimary()) {
       if (oldRepository != null) {
         oldRepository.cleanup();
       }
@@ -71,26 +69,26 @@ public class IndexRepositoryFactory {
       oldRepository.cleanup();
     }
     DistributedLockService lockService = getLockService();
-    String lockName = getLockName(bucketId, fileBucket);
+    String lockName = getLockName(fileAndChunkBucket);
     while (!lockService.lock(lockName, 100, -1)) {
-      if (!chunkBucket.getBucketAdvisor().isPrimary()) {
+      if (!fileAndChunkBucket.getBucketAdvisor().isPrimary()) {
         return null;
       }
     }
 
     final IndexRepository repo;
     try {
-      RegionDirectory dir = new RegionDirectory(getBucketTargetingMap(fileBucket, bucketId),
-          getBucketTargetingMap(chunkBucket, bucketId), indexForPR.getFileSystemStats());
+      RegionDirectory dir = new RegionDirectory(getBucketTargetingMap(fileAndChunkBucket, bucketId),
+          indexForPR.getFileSystemStats());
       IndexWriterConfig config = new IndexWriterConfig(indexForPR.getAnalyzer());
       IndexWriter writer = new IndexWriter(dir, config);
-      repo = new IndexRepositoryImpl(fileBucket, writer, serializer, indexForPR.getIndexStats(),
-          dataBucket, lockService, lockName);
+      repo = new IndexRepositoryImpl(fileAndChunkBucket, writer, serializer,
+          indexForPR.getIndexStats(), dataBucket, lockService, lockName);
       success = true;
       return repo;
     } catch (IOException e) {
       logger.info("Exception thrown while constructing Lucene Index for bucket:" + bucketId
-          + " for file region:" + fileBucket.getFullPath());
+          + " for file region:" + fileAndChunkBucket.getFullPath());
       throw e;
     } finally {
       if (!success) {
@@ -104,8 +102,8 @@ public class IndexRepositoryFactory {
     return new BucketTargetingMap(region, bucketId);
   }
 
-  private String getLockName(final Integer bucketId, final BucketRegion fileBucket) {
-    return FILE_REGION_LOCK_FOR_BUCKET_ID + fileBucket.getFullPath();
+  private String getLockName(final BucketRegion fileAndChunkBucket) {
+    return FILE_REGION_LOCK_FOR_BUCKET_ID + fileAndChunkBucket.getFullPath();
   }
 
   private DistributedLockService getLockService() {

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
index 7274d6a..80e0c44 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
@@ -29,7 +29,6 @@ import org.apache.geode.cache.asyncqueue.internal.AsyncEventQueueImpl;
 import org.apache.geode.cache.execute.FunctionService;
 import org.apache.geode.cache.execute.ResultCollector;
 import org.apache.geode.cache.lucene.internal.directory.DumpDirectoryFiles;
-import org.apache.geode.cache.lucene.internal.filesystem.ChunkKey;
 import org.apache.geode.cache.lucene.internal.filesystem.File;
 import org.apache.geode.cache.lucene.internal.filesystem.FileSystemStats;
 import org.apache.geode.cache.lucene.internal.partition.BucketTargetingFixedResolver;
@@ -48,12 +47,10 @@ import java.util.Set;
 
 /* wrapper of IndexWriter */
 public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
-  protected Region<String, File> fileRegion;
-  protected Region<ChunkKey, byte[]> chunkRegion;
+  protected Region fileAndChunkRegion;
   protected final FileSystemStats fileSystemStats;
 
   public static final String FILES_REGION_SUFFIX = ".files";
-  public static final String CHUNKS_REGION_SUFFIX = ".chunks";
 
   public LuceneIndexForPartitionedRegion(String indexName, String regionPath, Cache cache) {
     super(indexName, regionPath, cache);
@@ -81,16 +78,12 @@ public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
       regionShortCut = RegionShortcut.PARTITION;
     }
 
-    // create PR fileRegion, but not to create its buckets for now
+    // create PR fileAndChunkRegion, but not to create its buckets for now
     final String fileRegionName = createFileRegionName();
     PartitionAttributes partitionAttributes = dataRegion.getPartitionAttributes();
-    if (!fileRegionExists(fileRegionName)) {
-      fileRegion =
-          createFileRegion(regionShortCut, fileRegionName, partitionAttributes, regionAttributes);
-    }
+
 
     // create PR chunkRegion, but not to create its buckets for now
-    final String chunkRegionName = createChunkRegionName();
 
     // we will create RegionDirectories on the fly when data comes in
     HeterogeneousLuceneSerializer mapper = new HeterogeneousLuceneSerializer(getFieldNames());
@@ -99,23 +92,20 @@ public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
     DM dm = ((GemFireCacheImpl) getCache()).getDistributedSystem().getDistributionManager();
     LuceneBucketListener lucenePrimaryBucketListener =
         new LuceneBucketListener(partitionedRepositoryManager, dm);
-    if (!chunkRegionExists(chunkRegionName)) {
-      chunkRegion = createChunkRegion(regionShortCut, fileRegionName, partitionAttributes,
-          chunkRegionName, regionAttributes, lucenePrimaryBucketListener);
+
+    if (!fileRegionExists(fileRegionName)) {
+      fileAndChunkRegion = createFileRegion(regionShortCut, fileRegionName, partitionAttributes,
+          regionAttributes, lucenePrimaryBucketListener);
     }
-    fileSystemStats.setFileSupplier(() -> (int) getFileRegion().getLocalSize());
-    fileSystemStats.setChunkSupplier(() -> (int) getChunkRegion().getLocalSize());
-    fileSystemStats.setBytesSupplier(() -> getChunkRegion().getPrStats().getDataStoreBytesInUse());
 
-    return partitionedRepositoryManager;
-  }
+    fileSystemStats
+        .setBytesSupplier(() -> getFileAndChunkRegion().getPrStats().getDataStoreBytesInUse());
 
-  public PartitionedRegion getFileRegion() {
-    return (PartitionedRegion) fileRegion;
+    return partitionedRepositoryManager;
   }
 
-  public PartitionedRegion getChunkRegion() {
-    return (PartitionedRegion) chunkRegion;
+  public PartitionedRegion getFileAndChunkRegion() {
+    return (PartitionedRegion) fileAndChunkRegion;
   }
 
   public FileSystemStats getFileSystemStats() {
@@ -123,35 +113,20 @@ public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
   }
 
   boolean fileRegionExists(String fileRegionName) {
-    return cache.<String, File>getRegion(fileRegionName) != null;
+    return cache.getRegion(fileRegionName) != null;
   }
 
   Region createFileRegion(final RegionShortcut regionShortCut, final String fileRegionName,
-      final PartitionAttributes partitionAttributes, final RegionAttributes regionAttributes) {
+      final PartitionAttributes partitionAttributes, final RegionAttributes regionAttributes,
+      PartitionListener listener) {
     return createRegion(fileRegionName, regionShortCut, this.regionPath, partitionAttributes,
-        regionAttributes, null);
+        regionAttributes, listener);
   }
 
   public String createFileRegionName() {
     return LuceneServiceImpl.getUniqueIndexRegionName(indexName, regionPath, FILES_REGION_SUFFIX);
   }
 
-  boolean chunkRegionExists(String chunkRegionName) {
-    return cache.<ChunkKey, byte[]>getRegion(chunkRegionName) != null;
-  }
-
-  Region<ChunkKey, byte[]> createChunkRegion(final RegionShortcut regionShortCut,
-      final String fileRegionName, final PartitionAttributes partitionAttributes,
-      final String chunkRegionName, final RegionAttributes regionAttributes,
-      final PartitionListener lucenePrimaryBucketListener) {
-    return createRegion(chunkRegionName, regionShortCut, fileRegionName, partitionAttributes,
-        regionAttributes, lucenePrimaryBucketListener);
-  }
-
-  public String createChunkRegionName() {
-    return LuceneServiceImpl.getUniqueIndexRegionName(indexName, regionPath, CHUNKS_REGION_SUFFIX);
-  }
-
   private PartitionAttributesFactory configureLuceneRegionAttributesFactory(
       PartitionAttributesFactory attributesFactory,
       PartitionAttributes<?, ?> dataRegionAttributes) {
@@ -175,8 +150,7 @@ public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
       final RegionShortcut regionShortCut, final String colocatedWithRegionName,
       final PartitionAttributes partitionAttributes, final RegionAttributes regionAttributes,
       PartitionListener lucenePrimaryBucketListener) {
-    PartitionAttributesFactory partitionAttributesFactory =
-        new PartitionAttributesFactory<String, File>();
+    PartitionAttributesFactory partitionAttributesFactory = new PartitionAttributesFactory();
     if (lucenePrimaryBucketListener != null) {
       partitionAttributesFactory.addPartitionListener(lucenePrimaryBucketListener);
     }
@@ -217,23 +191,13 @@ public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
     // Invoke super destroy to remove the extension
     super.destroy(initiator);
 
-    // Destroy the chunk region (colocated with the file region)
-    // localDestroyRegion can't be used because locally destroying regions is not supported on
-    // colocated regions
-    if (!chunkRegion.isDestroyed()) {
-      chunkRegion.destroyRegion();
-      if (logger.isDebugEnabled()) {
-        logger.debug("Destroyed chunkRegion=" + chunkRegion.getName());
-      }
-    }
-
     // Destroy the file region (colocated with the application region)
     // localDestroyRegion can't be used because locally destroying regions is not supported on
     // colocated regions
-    if (!fileRegion.isDestroyed()) {
-      fileRegion.destroyRegion();
+    if (!fileAndChunkRegion.isDestroyed()) {
+      fileAndChunkRegion.destroyRegion();
       if (logger.isDebugEnabled()) {
-        logger.debug("Destroyed fileRegion=" + fileRegion.getName());
+        logger.debug("Destroyed fileAndChunkRegion=" + fileAndChunkRegion.getName());
       }
     }
 

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectory.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectory.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectory.java
index 18428ec..9cdf94b 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectory.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectory.java
@@ -47,10 +47,9 @@ public class RegionDirectory extends BaseDirectory {
    * Create a region directory with a given file and chunk region. These regions may be bucket
    * regions or they may be replicated regions.
    */
-  public RegionDirectory(Map<String, File> fileRegion, Map<ChunkKey, byte[]> chunkRegion,
-      FileSystemStats stats) {
+  public RegionDirectory(Map fileAndChunkRegion, FileSystemStats stats) {
     super(new SingleInstanceLockFactory());
-    fs = new FileSystem(fileRegion, chunkRegion, stats);
+    fs = new FileSystem(fileAndChunkRegion, stats);
   }
 
   @Override

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystem.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystem.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystem.java
index 660816d..164955f 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystem.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystem.java
@@ -4,9 +4,9 @@
  * copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the
  * "License"); you may not use this file except in compliance with the License. You may obtain a
  * copy of the License at
- * 
+ *
  * http://www.apache.org/licenses/LICENSE-2.0
- * 
+ *
  * Unless required by applicable law or agreed to in writing, software distributed under the License
  * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
  * or implied. See the License for the specific language governing permissions and limitations under
@@ -15,7 +15,6 @@
 
 package org.apache.geode.cache.lucene.internal.filesystem;
 
-import org.apache.geode.cache.Region;
 import org.apache.geode.internal.logging.LogService;
 import org.apache.logging.log4j.Logger;
 
@@ -23,15 +22,15 @@ import java.io.FileNotFoundException;
 import java.io.IOException;
 import java.util.Collection;
 import java.util.Map;
-import java.util.concurrent.ConcurrentMap;
+import java.util.stream.Collectors;
 
 /**
  * A Filesystem like interface that stores file data in geode regions.
- * 
+ *
  * This filesystem is safe for use with multiple threads if the threads are not modifying the same
  * files. A single file is not safe to modify by multiple threads, even between different members of
  * the distributed system.
- * 
+ *
  * Changes to a file may not be visible to other members of the system until the FileOutputStream is
  * closed.
  *
@@ -39,37 +38,34 @@ import java.util.concurrent.ConcurrentMap;
 public class FileSystem {
   private static final Logger logger = LogService.getLogger();
 
-  private final Map<String, File> fileRegion;
-  private final Map<ChunkKey, byte[]> chunkRegion;
+  private final Map fileAndChunkRegion;
 
   static final int CHUNK_SIZE = 1024 * 1024; // 1 MB
   private final FileSystemStats stats;
 
   /**
-   * Create filesystem that will store data in the two provided regions. The fileRegion contains
-   * metadata about the files, and the chunkRegion contains the actual data. If data from either
-   * region is missing or inconsistent, no guarantees are made about what this class will do, so
-   * it's best if these regions are colocated and in the same disk store to ensure the data remains
-   * together.
-   * 
-   * @param fileRegion the region to store metadata about the files
-   * @param chunkRegion the region to store actual file data.
+   * Create filesystem that will store data in the two provided regions. The fileAndChunkRegion
+   * contains metadata about the files, and the chunkRegion contains the actual data. If data from
+   * either region is missing or inconsistent, no guarantees are made about what this class will do,
+   * so it's best if these regions are colocated and in the same disk store to ensure the data
+   * remains together.
+   *
+   * @param fileAndChunkRegion the region to store metadata about the files
    */
-  public FileSystem(Map<String, File> fileRegion, Map<ChunkKey, byte[]> chunkRegion,
-      FileSystemStats stats) {
-    this.fileRegion = fileRegion;
-    this.chunkRegion = chunkRegion;
+  public FileSystem(Map fileAndChunkRegion, FileSystemStats stats) {
+    this.fileAndChunkRegion = fileAndChunkRegion;
     this.stats = stats;
   }
 
   public Collection<String> listFileNames() {
-    return fileRegion.keySet();
+    return (Collection<String>) fileAndChunkRegion.keySet().stream()
+        .filter(entry -> (entry instanceof String)).collect(Collectors.toList());
   }
 
   public File createFile(final String name) throws IOException {
     // TODO lock region ?
     final File file = new File(this, name);
-    if (null != fileRegion.putIfAbsent(name, file)) {
+    if (null != fileAndChunkRegion.putIfAbsent(name, file)) {
       throw new IOException("File exists.");
     }
     stats.incFileCreates(1);
@@ -79,7 +75,7 @@ public class FileSystem {
 
   public File putIfAbsentFile(String name, File file) throws IOException {
     // TODO lock region ?
-    if (null != fileRegion.putIfAbsent(name, file)) {
+    if (null != fileAndChunkRegion.putIfAbsent(name, file)) {
       throw new IOException("File exists.");
     }
     stats.incFileCreates(1);
@@ -94,7 +90,7 @@ public class FileSystem {
   }
 
   public File getFile(final String name) throws FileNotFoundException {
-    final File file = fileRegion.get(name);
+    final File file = (File) fileAndChunkRegion.get(name);
 
     if (null == file) {
       throw new FileNotFoundException(name);
@@ -111,7 +107,7 @@ public class FileSystem {
     // things crash in the middle of removing this file?
     // Seems like a file will be left with some
     // dangling chunks at the end of the file
-    File file = fileRegion.remove(name);
+    File file = (File) fileAndChunkRegion.remove(name);
     if (file == null) {
       throw new FileNotFoundException(name);
     }
@@ -121,7 +117,7 @@ public class FileSystem {
       final ChunkKey key = new ChunkKey(file.id, 0);
       while (true) {
         // TODO consider mutable ChunkKey
-        if (null == chunkRegion.remove(key)) {
+        if (null == fileAndChunkRegion.remove(key)) {
           // no more chunks
           break;
         }
@@ -133,7 +129,7 @@ public class FileSystem {
 
   public void renameFile(String source, String dest) throws IOException {
 
-    final File sourceFile = fileRegion.get(source);
+    final File sourceFile = (File) fileAndChunkRegion.get(source);
     if (null == sourceFile) {
       throw new FileNotFoundException(source);
     }
@@ -152,7 +148,7 @@ public class FileSystem {
     updateFile(sourceFile);
     putIfAbsentFile(dest, destFile);
 
-    fileRegion.remove(source);
+    fileAndChunkRegion.remove(source);
     stats.incFileRenames(1);
   }
 
@@ -162,14 +158,14 @@ public class FileSystem {
     // The file's metadata indicates that this chunk shouldn't
     // exist. Purge all of the chunks that are larger than the file metadata
     if (id >= file.chunks) {
-      while (chunkRegion.containsKey(key)) {
-        chunkRegion.remove(key);
+      while (fileAndChunkRegion.containsKey(key)) {
+        fileAndChunkRegion.remove(key);
         key.chunkId++;
       }
       return null;
     }
 
-    final byte[] chunk = chunkRegion.get(key);
+    final byte[] chunk = (byte[]) fileAndChunkRegion.get(key);
     if (chunk != null) {
       stats.incReadBytes(chunk.length);
     } else {
@@ -181,21 +177,18 @@ public class FileSystem {
 
   public void putChunk(final File file, final int id, final byte[] chunk) {
     final ChunkKey key = new ChunkKey(file.id, id);
-    chunkRegion.put(key, chunk);
+    fileAndChunkRegion.put(key, chunk);
     stats.incWrittenBytes(chunk.length);
   }
 
   void updateFile(File file) {
-    fileRegion.put(file.getName(), file);
+    fileAndChunkRegion.put(file.getName(), file);
   }
 
-  public Map<String, File> getFileRegion() {
-    return fileRegion;
+  public Map getFileAndChunkRegion() {
+    return fileAndChunkRegion;
   }
 
-  public Map<ChunkKey, byte[]> getChunkRegion() {
-    return chunkRegion;
-  }
 
   /**
    * Export all of the files in the filesystem to the provided directory
@@ -208,7 +201,6 @@ public class FileSystem {
       } catch (FileNotFoundException e) {
         // ignore this, it was concurrently removed
       }
-
     });
   }
 

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemStats.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemStats.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemStats.java
index 85ae6d7..09bb989 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemStats.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemStats.java
@@ -38,8 +38,6 @@ public class FileSystemStats {
   private static final int temporaryFileCreatesId;
   private static final int fileDeletesId;
   private static final int fileRenamesId;
-  private static final int filesId;
-  private static final int chunksId;
   private static final int bytesId;
 
   static {
@@ -63,8 +61,6 @@ public class FileSystemStats {
     temporaryFileCreatesId = statsType.nameToId("temporaryFileCreates");
     fileDeletesId = statsType.nameToId("fileDeletes");
     fileRenamesId = statsType.nameToId("fileRenames");
-    filesId = statsType.nameToId("files");
-    chunksId = statsType.nameToId("chunks");
     bytesId = statsType.nameToId("bytes");
   }
 
@@ -96,22 +92,6 @@ public class FileSystemStats {
     stats.incInt(fileRenamesId, delta);
   }
 
-  public void setFileSupplier(IntSupplier supplier) {
-    stats.setIntSupplier(filesId, supplier);
-  }
-
-  public int getFiles() {
-    return stats.getInt(filesId);
-  }
-
-  public void setChunkSupplier(IntSupplier supplier) {
-    stats.setIntSupplier(chunksId, supplier);
-  }
-
-  public int getChunks() {
-    return stats.getInt(chunksId);
-  }
-
   public void setBytesSupplier(LongSupplier supplier) {
     stats.setLongSupplier(bytesId, supplier);
   }

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
index 037c99f..c878fb7 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
@@ -439,11 +439,6 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
         LuceneIndexForPartitionedRegion.FILES_REGION_SUFFIX);
     assertNull(getCache().getRegion(filesRegionName));
 
-    // Verify the underlying chunks region no longer exists
-    String chunksRegionName = LuceneServiceImpl.getUniqueIndexRegionName(indexName, REGION_NAME,
-        LuceneIndexForPartitionedRegion.CHUNKS_REGION_SUFFIX);
-    assertNull(getCache().getRegion(chunksRegionName));
-
     // Verify the underlying AsyncEventQueue no longer exists
     String aeqId = LuceneServiceImpl.getUniqueIndexName(indexName, REGION_NAME);
     assertNull(getCache().getAsyncEventQueue(aeqId));

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java
index 00abb99..f5bd83f 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexMaintenanceIntegrationTest.java
@@ -143,8 +143,6 @@ public class LuceneIndexMaintenanceIntegrationTest extends LuceneIntegrationTest
     FileSystemStats fileSystemStats = index.getFileSystemStats();
     LuceneIndexStats indexStats = index.getIndexStats();
     await(() -> assertEquals(4, indexStats.getDocuments()));
-    await(() -> assertTrue(fileSystemStats.getFiles() > 0));
-    await(() -> assertTrue(fileSystemStats.getChunks() > 0));
     await(() -> assertTrue(fileSystemStats.getBytes() > 0));
   }
 

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java
index f7bb5ab..65b250c 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneQueriesPersistenceIntegrationTest.java
@@ -88,8 +88,6 @@ public class LuceneQueriesPersistenceIntegrationTest extends LuceneIntegrationTe
 
     PartitionedRegion fileRegion = (PartitionedRegion) cache.getRegion(aeqId + ".files");
     assertNotNull(fileRegion);
-    PartitionedRegion chunkRegion = (PartitionedRegion) cache.getRegion(aeqId + ".chunks");
-    assertNotNull(chunkRegion);
     Assert.assertTrue(0 < userRegion.getDiskRegionStats().getNumOverflowOnDisk());
 
     LuceneQuery<Integer, Type1> query = service.createLuceneQueryFactory().create(INDEX_NAME,

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegionTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegionTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegionTest.java
index 93cc0a8..f2bfdd4 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegionTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegionTest.java
@@ -99,33 +99,6 @@ public class LuceneIndexForPartitionedRegionTest {
   }
 
   @Test
-  public void chunkRegionExistsWhenChunkRegionExistsShouldReturnTrue() {
-    String name = "indexName";
-    String regionPath = "regionName";
-    Cache cache = Fakes.cache();
-    PartitionedRegion region = mock(PartitionedRegion.class);
-    LuceneIndexForPartitionedRegion index =
-        new LuceneIndexForPartitionedRegion(name, regionPath, cache);
-    String chunkRegionName = index.createChunkRegionName();
-    when(cache.getRegion(chunkRegionName)).thenReturn(region);
-
-    assertTrue(index.chunkRegionExists(chunkRegionName));
-  }
-
-  @Test
-  public void chunkRegionExistsWhenChunkRegionDoesNotExistShouldReturnFalse() {
-    String name = "indexName";
-    String regionPath = "regionName";
-    Cache cache = Fakes.cache();
-    LuceneIndexForPartitionedRegion index =
-        new LuceneIndexForPartitionedRegion(name, regionPath, cache);
-    String chunkRegionName = index.createChunkRegionName();
-    when(cache.getRegion(chunkRegionName)).thenReturn(null);
-
-    assertFalse(index.chunkRegionExists(chunkRegionName));
-  }
-
-  @Test
   public void createAEQWithPersistenceCallsCreateOnAEQFactory() {
     String name = "indexName";
     String regionPath = "regionName";
@@ -244,30 +217,13 @@ public class LuceneIndexForPartitionedRegionTest {
       final LuceneIndexForPartitionedRegion index) {
     index.setSearchableFields(new String[] {"field"});
     LuceneIndexForPartitionedRegion spy = spy(index);
-    doReturn(null).when(spy).createFileRegion(any(), any(), any(), any());
-    doReturn(null).when(spy).createChunkRegion(any(), any(), any(), any(), any(), any());
+    doReturn(null).when(spy).createFileRegion(any(), any(), any(), any(), any());
     doReturn(null).when(spy).createAEQ(eq(region));
     spy.initialize();
     return spy;
   }
 
   @Test
-  public void initializeShouldCreatePartitionChunkRegion() {
-    boolean withPersistence = false;
-    String name = "indexName";
-    String regionPath = "regionName";
-    Cache cache = Fakes.cache();
-    Region region = initializeScenario(withPersistence, regionPath, cache);
-
-    LuceneIndexForPartitionedRegion index =
-        new LuceneIndexForPartitionedRegion(name, regionPath, cache);
-    LuceneIndexForPartitionedRegion spy = setupSpy(region, index);
-
-    verify(spy).createChunkRegion(eq(RegionShortcut.PARTITION), eq(index.createFileRegionName()),
-        any(), eq(index.createChunkRegionName()), any(), any());
-  }
-
-  @Test
   public void initializeShouldCreatePartitionFileRegion() {
     boolean withPersistence = false;
     String name = "indexName";
@@ -280,7 +236,7 @@ public class LuceneIndexForPartitionedRegionTest {
     LuceneIndexForPartitionedRegion spy = setupSpy(region, index);
 
     verify(spy).createFileRegion(eq(RegionShortcut.PARTITION), eq(index.createFileRegionName()),
-        any(), any());
+        any(), any(), any());
   }
 
   @Test
@@ -296,7 +252,7 @@ public class LuceneIndexForPartitionedRegionTest {
         new LuceneIndexForPartitionedRegion(name, regionPath, cache);
     LuceneIndexForPartitionedRegion indexSpy = spy(index);
     indexSpy.createFileRegion(RegionShortcut.PARTITION, index.createFileRegionName(),
-        partitionAttributes, regionAttributes);
+        partitionAttributes, regionAttributes, null);
     String fileRegionName = index.createFileRegionName();
     verify(indexSpy).createRegion(fileRegionName, RegionShortcut.PARTITION, regionPath,
         partitionAttributes, regionAttributes, null);
@@ -304,48 +260,6 @@ public class LuceneIndexForPartitionedRegionTest {
   }
 
   @Test
-  public void createChunkRegionWithPartitionShortcutCreatesRegionUsingCreateVMRegion()
-      throws Exception {
-    String name = "indexName";
-    String regionPath = "regionName";
-    GemFireCacheImpl cache = Fakes.cache();
-    PartitionAttributes partitionAttributes = initializeAttributes(cache);
-    RegionAttributes regionAttributes = mock(RegionAttributes.class);
-    when(regionAttributes.getDataPolicy()).thenReturn(DataPolicy.PARTITION);
-    LuceneIndexForPartitionedRegion index =
-        new LuceneIndexForPartitionedRegion(name, regionPath, cache);
-    LuceneIndexForPartitionedRegion indexSpy = spy(index);
-    String chunkRegionName = index.createChunkRegionName();
-    String fileRegionName = index.createFileRegionName();
-    indexSpy.createChunkRegion(RegionShortcut.PARTITION, fileRegionName, partitionAttributes,
-        chunkRegionName, regionAttributes, null);
-    verify(indexSpy).createRegion(chunkRegionName, RegionShortcut.PARTITION, fileRegionName,
-        partitionAttributes, regionAttributes, null);
-    verify(cache).createVMRegion(eq(chunkRegionName), any(), any());
-  }
-
-  @Test
-  public void initializeShouldCreatePartitionPersistentChunkRegion() {
-    boolean withPersistence = true;
-    String name = "indexName";
-    String regionPath = "regionName";
-    Cache cache = Fakes.cache();
-    initializeScenario(withPersistence, regionPath, cache);
-
-    LuceneIndexForPartitionedRegion index =
-        new LuceneIndexForPartitionedRegion(name, regionPath, cache);
-    index.setSearchableFields(new String[] {"field"});
-    LuceneIndexForPartitionedRegion spy = spy(index);
-    doReturn(null).when(spy).createFileRegion(any(), any(), any(), any());
-    doReturn(null).when(spy).createChunkRegion(any(), any(), any(), any(), any(), any());
-    doReturn(null).when(spy).createAEQ(any());
-    spy.initialize();
-
-    verify(spy).createChunkRegion(eq(RegionShortcut.PARTITION_PERSISTENT),
-        eq(index.createFileRegionName()), any(), eq(index.createChunkRegionName()), any(), any());
-  }
-
-  @Test
   public void initializeShouldCreatePartitionPersistentFileRegion() {
     boolean withPersistence = true;
     String name = "indexName";
@@ -357,13 +271,12 @@ public class LuceneIndexForPartitionedRegionTest {
         new LuceneIndexForPartitionedRegion(name, regionPath, cache);
     index.setSearchableFields(new String[] {"field"});
     LuceneIndexForPartitionedRegion spy = spy(index);
-    doReturn(null).when(spy).createFileRegion(any(), any(), any(), any());
-    doReturn(null).when(spy).createChunkRegion(any(), any(), any(), any(), any(), any());
+    doReturn(null).when(spy).createFileRegion(any(), any(), any(), any(), any());
     doReturn(null).when(spy).createAEQ(any());
     spy.initialize();
 
     verify(spy).createFileRegion(eq(RegionShortcut.PARTITION_PERSISTENT),
-        eq(index.createFileRegionName()), any(), any());
+        eq(index.createFileRegionName()), any(), any(), any());
   }
 
   @Test
@@ -378,14 +291,13 @@ public class LuceneIndexForPartitionedRegionTest {
         new LuceneIndexForPartitionedRegion(name, regionPath, cache);
     index.setSearchableFields(new String[] {"field"});
     LuceneIndexForPartitionedRegion spy = spy(index);
-    doReturn(null).when(spy).createFileRegion(any(), any(), any(), any());
-    doReturn(null).when(spy).createChunkRegion(any(), any(), any(), any(), any(), any());
+    doReturn(null).when(spy).createFileRegion(any(), any(), any(), any(), any());
     doReturn(null).when(spy).createAEQ(any());
     spy.initialize();
     spy.initialize();
 
     verify(spy).createFileRegion(eq(RegionShortcut.PARTITION_PERSISTENT),
-        eq(index.createFileRegionName()), any(), any());
+        eq(index.createFileRegionName()), any(), any(), any());
   }
 
   @Test

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java
index f853baf..b6b1d23 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/LuceneIndexRecoveryHAIntegrationTest.java
@@ -69,8 +69,8 @@ public class LuceneIndexRecoveryHAIntegrationTest {
   }
 
   /**
-   * On rebalance, new repository manager will be created. It will try to read fileRegion and
-   * construct index. This test simulates the same.
+   * On rebalance, new repository manager will be created. It will try to read fileAndChunkRegion
+   * and construct index. This test simulates the same.
    */
   // @Test
   public void recoverRepoInANewNode()
@@ -99,7 +99,7 @@ public class LuceneIndexRecoveryHAIntegrationTest {
 
     // close the region to simulate bucket movement. New node will create repo using data persisted
     // by old region
-    // ((PartitionedRegion)index.fileRegion).close();
+    // ((PartitionedRegion)index.fileAndChunkRegion).close();
     // ((PartitionedRegion)index.chunkRegion).close();
     userRegion.close();
 

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/PartitionedRepositoryManagerJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/PartitionedRepositoryManagerJUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/PartitionedRepositoryManagerJUnitTest.java
index 9c603c7..87317cc 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/PartitionedRepositoryManagerJUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/PartitionedRepositoryManagerJUnitTest.java
@@ -29,7 +29,6 @@ import java.util.Map;
 import java.util.Set;
 
 import org.apache.geode.cache.lucene.internal.partition.BucketTargetingMap;
-import org.apache.geode.distributed.DistributedLockService;
 import org.apache.geode.distributed.internal.locks.DLockService;
 import org.apache.geode.internal.cache.BucketAdvisor;
 import org.apache.geode.internal.cache.PartitionedRegionHelper;
@@ -63,15 +62,12 @@ import org.apache.geode.test.junit.categories.UnitTest;
 public class PartitionedRepositoryManagerJUnitTest {
 
   protected PartitionedRegion userRegion;
-  protected PartitionedRegion fileRegion;
-  protected PartitionedRegion chunkRegion;
+  protected PartitionedRegion fileAndChunkRegion;
   protected LuceneSerializer serializer;
   protected PartitionedRegionDataStore userDataStore;
   protected PartitionedRegionDataStore fileDataStore;
-  protected PartitionedRegionDataStore chunkDataStore;
 
-  protected Map<Integer, BucketRegion> fileBuckets = new HashMap<Integer, BucketRegion>();
-  protected Map<Integer, BucketRegion> chunkBuckets = new HashMap<Integer, BucketRegion>();
+  protected Map<Integer, BucketRegion> fileAndChunkBuckets = new HashMap<Integer, BucketRegion>();
   protected Map<Integer, BucketRegion> dataBuckets = new HashMap<Integer, BucketRegion>();
   protected LuceneIndexStats indexStats;
   protected FileSystemStats fileSystemStats;
@@ -100,20 +96,16 @@ public class PartitionedRepositoryManagerJUnitTest {
   }
 
   protected void createIndexAndRepoManager() {
-    fileRegion = Mockito.mock(PartitionedRegion.class);
+    fileAndChunkRegion = Mockito.mock(PartitionedRegion.class);
     fileDataStore = Mockito.mock(PartitionedRegionDataStore.class);
-    when(fileRegion.getDataStore()).thenReturn(fileDataStore);
-    when(fileRegion.getTotalNumberOfBuckets()).thenReturn(113);
-    when(fileRegion.getFullPath()).thenReturn("FileRegion");
-    chunkRegion = Mockito.mock(PartitionedRegion.class);
-    chunkDataStore = Mockito.mock(PartitionedRegionDataStore.class);
-    when(chunkRegion.getFullPath()).thenReturn("ChunkRegion");
-    when(chunkRegion.getDataStore()).thenReturn(chunkDataStore);
+    when(fileAndChunkRegion.getDataStore()).thenReturn(fileDataStore);
+    when(fileAndChunkRegion.getTotalNumberOfBuckets()).thenReturn(113);
+    when(fileAndChunkRegion.getFullPath()).thenReturn("FileRegion");
     indexStats = Mockito.mock(LuceneIndexStats.class);
     fileSystemStats = Mockito.mock(FileSystemStats.class);
     indexForPR = Mockito.mock(LuceneIndexForPartitionedRegion.class);
-    when(((LuceneIndexForPartitionedRegion) indexForPR).getFileRegion()).thenReturn(fileRegion);
-    when(((LuceneIndexForPartitionedRegion) indexForPR).getChunkRegion()).thenReturn(chunkRegion);
+    when(((LuceneIndexForPartitionedRegion) indexForPR).getFileAndChunkRegion())
+        .thenReturn(fileAndChunkRegion);
     when(((LuceneIndexForPartitionedRegion) indexForPR).getFileSystemStats())
         .thenReturn(fileSystemStats);
     when(indexForPR.getIndexStats()).thenReturn(indexStats);
@@ -159,7 +151,7 @@ public class PartitionedRepositoryManagerJUnitTest {
     assertNotNull(repo0);
     checkRepository(repo0, 0);
 
-    BucketRegion fileBucket0 = fileBuckets.get(0);
+    BucketRegion fileBucket0 = fileAndChunkBuckets.get(0);
     BucketRegion dataBucket0 = dataBuckets.get(0);
 
     // Simulate rebalancing of a bucket by marking the old bucket is destroyed
@@ -189,11 +181,11 @@ public class PartitionedRepositoryManagerJUnitTest {
 
     when(fileDataStore.getLocalBucketById(eq(0))).thenReturn(null);
 
-    when(fileRegion.getOrCreateNodeForBucketWrite(eq(0), (RetryTimeKeeper) any()))
+    when(fileAndChunkRegion.getOrCreateNodeForBucketWrite(eq(0), (RetryTimeKeeper) any()))
         .then(new Answer() {
           @Override
           public Object answer(InvocationOnMock invocation) throws Throwable {
-            when(fileDataStore.getLocalBucketById(eq(0))).thenReturn(fileBuckets.get(0));
+            when(fileDataStore.getLocalBucketById(eq(0))).thenReturn(fileAndChunkBuckets.get(0));
             return null;
           }
         });
@@ -241,36 +233,30 @@ public class PartitionedRepositoryManagerJUnitTest {
   protected void checkRepository(IndexRepositoryImpl repo0, int bucketId) {
     IndexWriter writer0 = repo0.getWriter();
     RegionDirectory dir0 = (RegionDirectory) writer0.getDirectory();
-    assertEquals(new BucketTargetingMap(fileBuckets.get(bucketId), bucketId),
-        dir0.getFileSystem().getFileRegion());
-    assertEquals(new BucketTargetingMap(chunkBuckets.get(bucketId), bucketId),
-        dir0.getFileSystem().getChunkRegion());
+    assertEquals(new BucketTargetingMap(fileAndChunkBuckets.get(bucketId), bucketId),
+        dir0.getFileSystem().getFileAndChunkRegion());
     assertEquals(serializer, repo0.getSerializer());
   }
 
   protected BucketRegion setUpMockBucket(int id) throws BucketNotFoundException {
     BucketRegion mockBucket = Mockito.mock(BucketRegion.class);
-    BucketRegion fileBucket = Mockito.mock(BucketRegion.class);
-    // Allowing the fileBucket to behave like a map so that the IndexWriter operations don't fail
-    Fakes.addMapBehavior(fileBucket);
-    when(fileBucket.getFullPath()).thenReturn("File" + id);
-    BucketRegion chunkBucket = Mockito.mock(BucketRegion.class);
-    when(chunkBucket.getFullPath()).thenReturn("Chunk" + id);
+    BucketRegion fileAndChunkBucket = Mockito.mock(BucketRegion.class);
+    // Allowing the fileAndChunkBucket to behave like a map so that the IndexWriter operations don't
+    // fail
+    Fakes.addMapBehavior(fileAndChunkBucket);
+    when(fileAndChunkBucket.getFullPath()).thenReturn("File" + id);
     when(mockBucket.getId()).thenReturn(id);
     when(userRegion.getBucketRegion(eq(id), eq(null))).thenReturn(mockBucket);
     when(userDataStore.getLocalBucketById(eq(id))).thenReturn(mockBucket);
     when(userRegion.getBucketRegion(eq(id + 113), eq(null))).thenReturn(mockBucket);
     when(userDataStore.getLocalBucketById(eq(id + 113))).thenReturn(mockBucket);
-    when(fileDataStore.getLocalBucketById(eq(id))).thenReturn(fileBucket);
-    when(chunkDataStore.getLocalBucketById(eq(id))).thenReturn(chunkBucket);
+    when(fileDataStore.getLocalBucketById(eq(id))).thenReturn(fileAndChunkBucket);
 
-    fileBuckets.put(id, fileBucket);
-    chunkBuckets.put(id, chunkBucket);
+    fileAndChunkBuckets.put(id, fileAndChunkBucket);
     dataBuckets.put(id, mockBucket);
 
     BucketAdvisor mockBucketAdvisor = Mockito.mock(BucketAdvisor.class);
-    when(fileBucket.getBucketAdvisor()).thenReturn(mockBucketAdvisor);
-    when(chunkBucket.getBucketAdvisor()).thenReturn(mockBucketAdvisor);
+    when(fileAndChunkBucket.getBucketAdvisor()).thenReturn(mockBucketAdvisor);
     when(mockBucketAdvisor.isPrimary()).thenReturn(true);
     return mockBucket;
   }

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectoryJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectoryJUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectoryJUnitTest.java
index 32249e4..3f468f4 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectoryJUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/directory/RegionDirectoryJUnitTest.java
@@ -58,7 +58,6 @@ public class RegionDirectoryJUnitTest extends BaseDirectoryTestCase {
     // test asserts that no system properties have changed. Unfortunately, there is no
     // way to control the order of rules, so we can't clear this property with a rule
     // or @After method. Instead, do it in the close method of the directory.
-    return new RegionDirectory(new ConcurrentHashMap<String, File>(),
-        new ConcurrentHashMap<ChunkKey, byte[]>(), stats);
+    return new RegionDirectory(new ConcurrentHashMap(), stats);
   }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/distributed/DistributedScoringJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/distributed/DistributedScoringJUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/distributed/DistributedScoringJUnitTest.java
index 6062904..41d90ef 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/distributed/DistributedScoringJUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/distributed/DistributedScoringJUnitTest.java
@@ -132,9 +132,8 @@ public class DistributedScoringJUnitTest {
   }
 
   private IndexRepositoryImpl createIndexRepo() throws IOException {
-    ConcurrentHashMap<String, File> fileRegion = new ConcurrentHashMap<String, File>();
-    ConcurrentHashMap<ChunkKey, byte[]> chunkRegion = new ConcurrentHashMap<ChunkKey, byte[]>();
-    RegionDirectory dir = new RegionDirectory(fileRegion, chunkRegion, fileSystemStats);
+    ConcurrentHashMap fileAndChunkRegion = new ConcurrentHashMap();
+    RegionDirectory dir = new RegionDirectory(fileAndChunkRegion, fileSystemStats);
 
     IndexWriterConfig config = new IndexWriterConfig(analyzer);
     IndexWriter writer = new IndexWriter(dir, config);

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemJUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemJUnitTest.java
index ee41e40..2008458 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemJUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/filesystem/FileSystemJUnitTest.java
@@ -23,9 +23,11 @@ import java.io.InputStream;
 import java.io.OutputStream;
 import java.lang.reflect.Method;
 import java.nio.file.Files;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collections;
 import java.util.HashSet;
+import java.util.Map;
 import java.util.Random;
 import java.util.concurrent.ConcurrentHashMap;
 
@@ -35,7 +37,6 @@ import org.junit.Test;
 import org.junit.experimental.categories.Category;
 import org.mockito.ArgumentCaptor;
 import org.mockito.Mockito;
-import org.mockito.Spy;
 import org.mockito.invocation.InvocationOnMock;
 import org.mockito.stubbing.Answer;
 
@@ -51,8 +52,7 @@ public class FileSystemJUnitTest {
 
   private FileSystem system;
   private Random rand = new Random();
-  private ConcurrentHashMap<String, File> fileRegion;
-  private ConcurrentHashMap<ChunkKey, byte[]> chunkRegion;
+  private ConcurrentHashMap fileAndChunkRegion;
 
   @Rule
   public DiskDirRule dirRule = new DiskDirRule();
@@ -60,10 +60,9 @@ public class FileSystemJUnitTest {
 
   @Before
   public void setUp() {
-    fileRegion = new ConcurrentHashMap<String, File>();
-    chunkRegion = new ConcurrentHashMap<ChunkKey, byte[]>();
+    fileAndChunkRegion = new ConcurrentHashMap();
     fileSystemStats = mock(FileSystemStats.class);
-    system = new FileSystem(fileRegion, chunkRegion, fileSystemStats);
+    system = new FileSystem(fileAndChunkRegion, fileSystemStats);
   }
 
   /**
@@ -201,7 +200,7 @@ public class FileSystemJUnitTest {
 
     // Assert that there are only 2 chunks in the system, since we wrote just
     // past the end of the first chunk.
-    assertEquals(2, chunkRegion.size());
+    assertEquals(2, numberOfChunks(fileAndChunkRegion));
 
     SeekableInputStream in = file.getInputStream();
 
@@ -254,7 +253,7 @@ public class FileSystemJUnitTest {
     file1 = system.getFile(name1);
     file2 = system.getFile(name2);
 
-    assertEquals(new HashSet(Arrays.asList(name1, name2)), system.listFileNames());
+    assertEquals(Arrays.asList(name1, name2), system.listFileNames());
     assertContents(file1Data, file1);
     assertContents(file2Data, file2);
 
@@ -265,7 +264,7 @@ public class FileSystemJUnitTest {
     } catch (IOException expected) {
 
     }
-    assertEquals(new HashSet(Arrays.asList(name1, name2)), system.listFileNames());
+    assertEquals(Arrays.asList(name1, name2), system.listFileNames());
     assertContents(file1Data, file1);
     assertContents(file2Data, file2);
 
@@ -275,17 +274,17 @@ public class FileSystemJUnitTest {
 
     File file3 = system.getFile(name3);
 
-    assertEquals(new HashSet(Arrays.asList(name3, name2)), system.listFileNames());
+    assertEquals(Arrays.asList(name3, name2), system.listFileNames());
     assertContents(file1Data, file3);
     assertContents(file2Data, file2);
 
     system.deleteFile(name2);
 
-    assertEquals(new HashSet(Arrays.asList(name3)), system.listFileNames());
+    assertEquals(Arrays.asList(name3), system.listFileNames());
 
     system.renameFile(name3, name2);
 
-    assertEquals(new HashSet(Arrays.asList(name2)), system.listFileNames());
+    assertEquals(Arrays.asList(name2), system.listFileNames());
 
     file2 = system.getFile(name2);
     assertContents(file1Data, file2);
@@ -313,7 +312,7 @@ public class FileSystemJUnitTest {
     byte[] bytes = getRandomBytes(size);
     file1.getOutputStream().write(bytes);
 
-    FileSystem system2 = new FileSystem(fileRegion, chunkRegion, fileSystemStats);
+    FileSystem system2 = new FileSystem(fileAndChunkRegion, fileSystemStats);
     File file = system2.getFile(name1);
 
     assertTrue(file.getLength() <= bytes.length);
@@ -325,7 +324,7 @@ public class FileSystemJUnitTest {
 
     if (length == 0) {
       assertEquals(-1, file.getInputStream().read(results));
-      assertTrue(chunkRegion.isEmpty());
+      assertEquals(0, numberOfChunks(fileAndChunkRegion));
     } else {
       // Make sure the amount of data we can read matches the length
       assertEquals(length, file.getInputStream().read(results));
@@ -349,12 +348,11 @@ public class FileSystemJUnitTest {
     // Create a couple of mock regions where we count the operations
     // that happen to them. We will then use this to abort the rename
     // in the middle.
-    ConcurrentHashMap<String, File> spyFileRegion =
-        mock(ConcurrentHashMap.class, new SpyWrapper(countOperations, fileRegion));
-    ConcurrentHashMap<ChunkKey, byte[]> spyChunkRegion =
-        mock(ConcurrentHashMap.class, new SpyWrapper(countOperations, chunkRegion));
+    ConcurrentHashMap spyFileAndChunkRegion =
+        mock(ConcurrentHashMap.class, new SpyWrapper(countOperations, fileAndChunkRegion));
 
-    system = new FileSystem(spyFileRegion, spyChunkRegion, fileSystemStats);
+
+    system = new FileSystem(spyFileAndChunkRegion, fileSystemStats);
 
     String name = "file";
     File file = system.createFile(name);
@@ -394,7 +392,7 @@ public class FileSystemJUnitTest {
 
     }
 
-    system = new FileSystem(fileRegion, chunkRegion, fileSystemStats);
+    system = new FileSystem(fileAndChunkRegion, fileSystemStats);
 
     // This is not the ideal behavior. We are left
     // with two duplicate files. However, we will still
@@ -409,72 +407,6 @@ public class FileSystemJUnitTest {
     assertContents(expectedBytes, destFile);
   }
 
-  /**
-   * Test what happens a file delete is aborted in the middle due to the a cache closed exception.
-   * The next member that uses those files should be able to clean up after the partial rename.
-   */
-  @Test
-  public void testPartialDelete() throws Exception {
-
-    final CountOperations countOperations = new CountOperations();
-    // Create a couple of mock regions where we count the operations
-    // that happen to them. We will then use this to abort the rename
-    // in the middle.
-    ConcurrentHashMap<String, File> spyFileRegion =
-        mock(ConcurrentHashMap.class, new SpyWrapper(countOperations, fileRegion));
-    ConcurrentHashMap<ChunkKey, byte[]> spyChunkRegion =
-        mock(ConcurrentHashMap.class, new SpyWrapper(countOperations, chunkRegion));
-
-    system = new FileSystem(spyFileRegion, spyChunkRegion, fileSystemStats);
-
-    String name1 = "file1";
-    String name2 = "file2";
-    File file1 = system.createFile(name1);
-    File file2 = system.createFile(name2);
-
-    ByteArrayOutputStream expected = new ByteArrayOutputStream();
-
-    // Make sure the file has a lot of chunks
-    for (int i = 0; i < 10; i++) {
-      byte[] bytes = writeRandomBytes(file1);
-      writeBytes(file2, bytes);
-      expected.write(bytes);
-    }
-
-    countOperations.reset();
-
-    system.deleteFile(name1);
-
-    assertTrue(2 <= countOperations.count);
-
-    countOperations.after(countOperations.count / 2, new Runnable() {
-
-      @Override
-      public void run() {
-        throw new CacheClosedException();
-      }
-    });
-    countOperations.reset();
-
-    try {
-      system.deleteFile(name2);
-      fail("should have seen an error");
-    } catch (CacheClosedException expectedException) {
-    }
-
-    system = new FileSystem(fileRegion, chunkRegion, fileSystemStats);
-
-    if (system.listFileNames().size() == 0) {
-      // File was deleted, but shouldn't have any dangling chunks at this point
-      assertEquals(Collections.EMPTY_SET, fileRegion.keySet());
-      // TODO - need to purge chunks of deleted files somehow.
-      // assertIndexDetailsEquals(Collections.EMPTY_SET, chunkRegion.keySet());
-    } else {
-      file2 = system.getFile(name2);
-      assertContents(expected.toByteArray(), file2);
-    }
-  }
-
   @Test
   public void testExport() throws IOException {
     String name1 = "testFile1";
@@ -544,8 +476,8 @@ public class FileSystemJUnitTest {
 
   @Test
   public void testDeletePossiblyRenamedFileDoesNotDestroyChunks() throws Exception {
-    ConcurrentHashMap<String, File> spyFileRegion = Mockito.spy(fileRegion);
-    system = new FileSystem(spyFileRegion, chunkRegion, fileSystemStats);
+    ConcurrentHashMap spyFileRegion = Mockito.spy(fileAndChunkRegion);
+    system = new FileSystem(spyFileRegion, fileSystemStats);
 
     String sourceFileName = "sourceFile";
     File file1 = system.createFile(sourceFileName);
@@ -610,6 +542,10 @@ public class FileSystemJUnitTest {
     return data;
   }
 
+  private long numberOfChunks(Map map) {
+    return map.keySet().parallelStream().filter(k -> (k instanceof ChunkKey)).count();
+  }
+
   /**
    * A wrapper around an object that will also invoke a callback before applying an operation.
    *

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplJUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplJUnitTest.java
index 42cc2bc..78fc657 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplJUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplJUnitTest.java
@@ -69,10 +69,9 @@ public class IndexRepositoryImplJUnitTest {
 
   @Before
   public void setUp() throws IOException {
-    ConcurrentHashMap<String, File> fileRegion = new ConcurrentHashMap<String, File>();
-    ConcurrentHashMap<ChunkKey, byte[]> chunkRegion = new ConcurrentHashMap<ChunkKey, byte[]>();
+    ConcurrentHashMap fileAndChunkRegion = new ConcurrentHashMap();
     fileSystemStats = mock(FileSystemStats.class);
-    RegionDirectory dir = new RegionDirectory(fileRegion, chunkRegion, fileSystemStats);
+    RegionDirectory dir = new RegionDirectory(fileAndChunkRegion, fileSystemStats);
     IndexWriterConfig config = new IndexWriterConfig(analyzer);
     writer = new IndexWriter(dir, config);
     String[] indexedFields = new String[] {"s", "i", "l", "d", "f", "s2", "missing"};

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java
index 9067376..46425b9 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/internal/repository/IndexRepositoryImplPerformanceTest.java
@@ -93,12 +93,10 @@ public class IndexRepositoryImplPerformanceTest {
       @Override
       public void init() throws Exception {
         cache = new CacheFactory().set(MCAST_PORT, "0").set(LOG_LEVEL, "error").create();
-        Region<String, File> fileRegion =
-            cache.<String, File>createRegionFactory(RegionShortcut.REPLICATE).create("files");
-        Region<ChunkKey, byte[]> chunkRegion =
-            cache.<ChunkKey, byte[]>createRegionFactory(RegionShortcut.REPLICATE).create("chunks");
+        Region fileAndChunkRegion =
+            cache.createRegionFactory(RegionShortcut.REPLICATE).create("files");
 
-        RegionDirectory dir = new RegionDirectory(fileRegion, chunkRegion,
+        RegionDirectory dir = new RegionDirectory(fileAndChunkRegion,
             new FileSystemStats(cache.getDistributedSystem(), "region-index"));
         final LuceneIndexStats stats =
             new LuceneIndexStats(cache.getDistributedSystem(), "region-index");
@@ -108,7 +106,7 @@ public class IndexRepositoryImplPerformanceTest {
         writer = new IndexWriter(dir, config);
         String[] indexedFields = new String[] {"text"};
         HeterogeneousLuceneSerializer mapper = new HeterogeneousLuceneSerializer(indexedFields);
-        repo = new IndexRepositoryImpl(fileRegion, writer, mapper, stats, null);
+        repo = new IndexRepositoryImpl(fileAndChunkRegion, writer, mapper, stats, null);
       }
 
       @Override
@@ -213,8 +211,7 @@ public class IndexRepositoryImplPerformanceTest {
       public void init() throws Exception {
         cache = new CacheFactory().set(MCAST_PORT, "0").set(LOG_LEVEL, "warning").create();
         final FileSystemStats stats = new FileSystemStats(cache.getDistributedSystem(), "stats");
-        RegionDirectory dir = new RegionDirectory(new ConcurrentHashMap<String, File>(),
-            new ConcurrentHashMap<ChunkKey, byte[]>(), stats);
+        RegionDirectory dir = new RegionDirectory(new ConcurrentHashMap(), stats);
         IndexWriterConfig config = new IndexWriterConfig(analyzer);
         writer = new IndexWriter(dir, config);
         searcherManager = new SearcherManager(writer, true, true, null);

http://git-wip-us.apache.org/repos/asf/geode/blob/5f9da297/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
index e2d3abc..670e6a8 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/test/LuceneTestUtilities.java
@@ -168,10 +168,7 @@ public class LuceneTestUtilities {
     LuceneIndexForPartitionedRegion index =
         (LuceneIndexForPartitionedRegion) luceneService.getIndex(INDEX_NAME, REGION_NAME);
 
-    // Verify the meta regions exist and are internal
-    LocalRegion chunkRegion = (LocalRegion) cache.getRegion(index.createChunkRegionName());
     LocalRegion fileRegion = (LocalRegion) cache.getRegion(index.createFileRegionName());
-    verify.accept(chunkRegion);
     verify.accept(fileRegion);
   }
 


[41/49] geode git commit: GEODE-2645: rewrite test to fix flakiness and improve readability

Posted by kh...@apache.org.
GEODE-2645: rewrite test to fix flakiness and improve readability


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/d5a36354
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/d5a36354
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/d5a36354

Branch: refs/heads/feature/GEODE-2420
Commit: d5a36354e25e7b624f7f2d52dffd59eee74ee1eb
Parents: 94eaca7
Author: Kirk Lund <kl...@apache.org>
Authored: Mon Mar 13 14:51:36 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../geode/cache30/CacheLogRollDUnitTest.java    | 500 -------------------
 .../logging/CacheLogRollingIntegrationTest.java | 356 +++++++++++++
 2 files changed, 356 insertions(+), 500 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/d5a36354/geode-core/src/test/java/org/apache/geode/cache30/CacheLogRollDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/cache30/CacheLogRollDUnitTest.java b/geode-core/src/test/java/org/apache/geode/cache30/CacheLogRollDUnitTest.java
deleted file mode 100644
index 3a0becf..0000000
--- a/geode-core/src/test/java/org/apache/geode/cache30/CacheLogRollDUnitTest.java
+++ /dev/null
@@ -1,500 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
- * agreements. See the NOTICE file distributed with this work for additional information regarding
- * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance with the License. You may obtain a
- * copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software distributed under the License
- * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
- * or implied. See the License for the specific language governing permissions and limitations under
- * the License.
- */
-package org.apache.geode.cache30;
-
-import static org.apache.geode.distributed.ConfigurationProperties.*;
-import static org.junit.Assert.*;
-
-import java.io.BufferedReader;
-import java.io.File;
-import java.io.FileNotFoundException;
-import java.io.FileReader;
-import java.io.FilenameFilter;
-import java.io.IOException;
-import java.util.Properties;
-import java.util.regex.Pattern;
-
-import org.junit.Test;
-import org.junit.experimental.categories.Category;
-
-import org.apache.geode.distributed.DistributedSystem;
-import org.apache.geode.distributed.internal.InternalDistributedSystem;
-import org.apache.geode.internal.logging.InternalLogWriter;
-import org.apache.geode.test.dunit.cache.internal.JUnit4CacheTestCase;
-import org.apache.geode.test.junit.categories.DistributedTest;
-import org.apache.geode.test.junit.categories.FlakyTest;
-
-/**
- * Test to make sure cache close is working.
- *
- * @since GemFire 6.5
- */
-@Category(DistributedTest.class)
-public class CacheLogRollDUnitTest extends JUnit4CacheTestCase {
-
-  private void logAndRollAndVerify(String baseLogName, DistributedSystem ds, String mainId)
-      throws FileNotFoundException, IOException {
-    String logfile = baseLogName + ".log";
-    String metaLogFile = "meta-" + baseLogName + "-" + mainId + ".log";
-    String rolledLogFile1 = baseLogName + "-" + mainId + "-01.log";
-    String rolledLogFile2 = baseLogName + "-" + mainId + "-02.log";
-    String META_MARKER_1 = "Switching to log " + baseLogName + ".log";
-    String META_MARKER_2 = "Rolling current log to " + baseLogName + "-" + mainId + "-01.log";
-    String META_MARKER_3 = "Rolling current log to " + baseLogName + "-" + mainId + "-02.log";
-
-    String FIRST_CHILD_MARKER = "hey guys im the first child whatsup";
-    String LOG_NONSENSE =
-        "what is your story what are you doing hey whatsup i can't believe it wow";
-    System.out.println("LOGNAME:" + logfile);
-    /*
-     * 1. Lets assert that the logfile exists and that it is a proper normal logfile 2. Asser that
-     * the meta logfile exists and has good stuff in it 3. Let's log a bunch and show that we
-     * rolled, 4. Show that old file has right old stuff in it 5. Show that new file has right new
-     * stuff in it 6. Show that meta has right stuff in it
-     */
-
-    ds.getLogWriter().info(FIRST_CHILD_MARKER);
-
-    File f = new File(logfile);
-    assertTrue("log-file :" + logfile + " should exist", f.exists());
-
-    File fm = new File(metaLogFile);
-    assertTrue("meta-log-file :" + metaLogFile + " should exist", fm.exists());
-
-    File f1 = new File(rolledLogFile1);
-    assertTrue("child-log-file :" + rolledLogFile1 + " should'nt exist", !f1.exists());
-
-    File f2 = new File(rolledLogFile2);
-    assertTrue("child-log-file2 :" + rolledLogFile2 + " should'nt exist yet", !f2.exists());
-
-
-    String metalog = getLogContents(metaLogFile);
-    assertTrue("metalog(" + metaLogFile + ") should have " + META_MARKER_1 + " in it:\n" + metalog,
-        metalog.indexOf(META_MARKER_1) != -1);
-
-    String mainlog = getLogContents(logfile);
-    assertTrue("log(" + logfile + ") should have " + FIRST_CHILD_MARKER + " in it:\n" + mainlog,
-        mainlog.indexOf(FIRST_CHILD_MARKER) != -1);
-
-    int i = 0;
-    while (i < 100000 && !f2.exists()) {
-      i++;
-      ds.getLogWriter().info(LOG_NONSENSE);
-    }
-
-    assertTrue("child-log-file1 :" + rolledLogFile1 + " should exist now", f1.exists());
-    assertTrue("child-log-file2 :" + rolledLogFile2 + " should exist now", f2.exists());
-
-    metalog = getLogContents(metaLogFile);
-
-    assertTrue("log(" + metaLogFile + ") should have " + META_MARKER_2 + " in it:\n" + metalog,
-        metalog.indexOf(META_MARKER_2) != -1);
-    assertTrue("log(" + metaLogFile + ") should have " + META_MARKER_3 + " in it:\n" + metalog,
-        metalog.indexOf(META_MARKER_3) != -1);
-    assertTrue("log(" + metaLogFile + ") should'nt have " + LOG_NONSENSE + " in it:\n" + metalog,
-        metalog.indexOf(LOG_NONSENSE) == -1);
-
-    {
-      String logChild2 = getLogContents(logfile);
-      assertTrue("log(" + logfile + ") should have " + LOG_NONSENSE + " in it:\n" + logChild2,
-          logChild2.indexOf(LOG_NONSENSE) != -1);
-    }
-
-    {
-      String logChild2 = getLogContents(rolledLogFile1);
-      assertTrue(
-          "log(" + rolledLogFile1 + ") should have " + LOG_NONSENSE + " in it:\n" + logChild2,
-          logChild2.indexOf(LOG_NONSENSE) != -1);
-    }
-
-    {
-      String logChild2 = getLogContents(rolledLogFile2);
-      assertTrue(
-          "log(" + rolledLogFile2 + ") should have " + LOG_NONSENSE + " in it:\n" + logChild2,
-          logChild2.indexOf(LOG_NONSENSE) != -1);
-    }
-  }
-
-  private void SecurityLogAndRollAndVerify(String baseLogName, DistributedSystem ds, String mainId)
-      throws FileNotFoundException, IOException {
-    String logfile = baseLogName + ".log";
-    String sec_logfile = "sec_" + logfile;
-    String metaLogFile = "meta-" + baseLogName + "-" + mainId + ".log";
-    String rolledLogFile1 = baseLogName + "-" + mainId + "-01.log";
-    String rolledLogFile2 = baseLogName + "-" + mainId + "-02.log";
-    String rolledSecLogFile1 = "sec_" + baseLogName + "-" + mainId + "-01.log";
-    String rolledSecLogFile2 = "sec_" + baseLogName + "-" + mainId + "-02.log";
-    String META_MARKER_1 = "Switching to log " + baseLogName + ".log";
-    String META_MARKER_2 = "Rolling current log to " + baseLogName + "-" + mainId + "-01.log";
-    String META_MARKER_3 = "Rolling current log to " + baseLogName + "-" + mainId + "-02.log";
-
-    String FIRST_CHILD_MARKER = "hey guys im the first child whatsup";
-    String LOG_NONSENSE =
-        "what is your story what are you doing hey whatsup i can't believe it wow";
-    System.out.println("LOGNAME:" + logfile + ", SECLOGNAME:" + sec_logfile);
-    /*
-     * 1. Lets assert that the logfile exists and that it is a proper normal logfile 2. Asser that
-     * the meta logfile exists and has good stuff in it 3. Let's log a bunch and show that we
-     * rolled, 4. Show that old file has right old stuff in it 5. Show that new file has right new
-     * stuff in it 6. Show that meta has right stuff in it
-     */
-
-    ds.getLogWriter().info(FIRST_CHILD_MARKER);
-    ds.getSecurityLogWriter().info(FIRST_CHILD_MARKER);
-
-    File f = new File(logfile);
-    File sec_f = new File(sec_logfile);
-    assertTrue("log-file :" + logfile + " should exist", f.exists());
-    assertTrue("security-log-file :" + sec_logfile + " should exist", sec_f.exists());
-
-    File fm = new File(metaLogFile);
-    assertTrue("meta-log-file :" + metaLogFile + " should exist", fm.exists());
-
-    File f1 = new File(rolledLogFile1);
-    File sec_f1 = new File(rolledSecLogFile1);
-    assertTrue("child-log-file :" + rolledLogFile1 + " should'nt exist", !f1.exists());
-    assertTrue("security-child-log-file :" + rolledLogFile1 + " should'nt exist", !sec_f1.exists());
-
-    File f2 = new File(rolledLogFile2);
-    File sec_f2 = new File(rolledSecLogFile2);
-    assertTrue("child-log-file2 :" + rolledLogFile2 + " should'nt exist yet", !f2.exists());
-    assertTrue("security-child-log-file2 :" + rolledSecLogFile2 + " should'nt exist yet",
-        !sec_f2.exists());
-
-
-    String metalog = getLogContents(metaLogFile);
-    assertTrue("metalog(" + metaLogFile + ") should have " + META_MARKER_1 + " in it:\n" + metalog,
-        metalog.indexOf(META_MARKER_1) != -1);
-
-    String mainlog = getLogContents(logfile);
-    assertTrue("log(" + logfile + ") should have " + FIRST_CHILD_MARKER + " in it:\n" + mainlog,
-        mainlog.indexOf(FIRST_CHILD_MARKER) != -1);
-    String sec_mainlog = getLogContents(sec_logfile);
-    assertTrue(
-        "log(" + sec_logfile + ") should have " + FIRST_CHILD_MARKER + " in it:\n" + sec_mainlog,
-        sec_mainlog.indexOf(FIRST_CHILD_MARKER) != -1);
-
-    int i = 0;
-    while (i < 100000 && !f2.exists()) {
-      i++;
-      ds.getLogWriter().info(LOG_NONSENSE);
-    }
-
-    int j = 0;
-    while (j < 100000 && !sec_f2.exists()) {
-      j++;
-      ds.getSecurityLogWriter().info(LOG_NONSENSE);
-    }
-
-    assertTrue("child-log-file1 :" + rolledLogFile1 + " should exist now", f1.exists());
-    assertTrue("child-log-file2 :" + rolledLogFile2 + " should exist now", f2.exists());
-    assertTrue("security-child-log-file1 :" + rolledSecLogFile1 + " should exist now",
-        sec_f1.exists());
-    assertTrue("security-child-log-file2 :" + rolledSecLogFile2 + " should exist now",
-        sec_f2.exists());
-
-    metalog = getLogContents(metaLogFile);
-
-    assertTrue("log(" + metaLogFile + ") should have " + META_MARKER_2 + " in it:\n" + metalog,
-        metalog.indexOf(META_MARKER_2) != -1);
-    assertTrue("log(" + metaLogFile + ") should have " + META_MARKER_3 + " in it:\n" + metalog,
-        metalog.indexOf(META_MARKER_3) != -1);
-    assertTrue("log(" + metaLogFile + ") should'nt have " + LOG_NONSENSE + " in it:\n" + metalog,
-        metalog.indexOf(LOG_NONSENSE) == -1);
-
-    {
-      String logChild2 = getLogContents(logfile);
-      assertTrue("log(" + logfile + ") should have " + LOG_NONSENSE + " in it:\n" + logChild2,
-          logChild2.indexOf(LOG_NONSENSE) != -1);
-      String sec_logChild2 = getLogContents(sec_logfile);
-      assertTrue(
-          "log(" + sec_logfile + ") should have " + LOG_NONSENSE + " in it:\n" + sec_logChild2,
-          sec_logChild2.indexOf(LOG_NONSENSE) != -1);
-    }
-
-    {
-      String logChild2 = getLogContents(rolledLogFile1);
-      assertTrue(
-          "log(" + rolledLogFile1 + ") should have " + LOG_NONSENSE + " in it:\n" + logChild2,
-          logChild2.indexOf(LOG_NONSENSE) != -1);
-      String sec_logChild2 = getLogContents(rolledSecLogFile1);
-      assertTrue("log(" + rolledSecLogFile1 + ") should have " + LOG_NONSENSE + " in it:\n"
-          + sec_logChild2, sec_logChild2.indexOf(LOG_NONSENSE) != -1);
-    }
-
-    {
-      String logChild2 = getLogContents(rolledLogFile2);
-      assertTrue(
-          "log(" + rolledLogFile2 + ") should have " + LOG_NONSENSE + " in it:\n" + logChild2,
-          logChild2.indexOf(LOG_NONSENSE) != -1);
-      String sec_logChild2 = getLogContents(rolledSecLogFile2);
-      assertTrue("log(" + rolledSecLogFile2 + ") should have " + LOG_NONSENSE + " in it:\n"
-          + sec_logChild2, sec_logChild2.indexOf(LOG_NONSENSE) != -1);
-    }
-  }
-
-  @Test
-  public void testDiskSpace() throws Exception {
-    Properties props = new Properties();
-    String baseLogName = "diskarito";
-    String logfile = baseLogName + ".log";
-    props.put(LOG_FILE, logfile);
-    props.put(LOG_FILE_SIZE_LIMIT, "1");
-    DistributedSystem ds = this.getSystem(props);
-    props.put(LOG_DISK_SPACE_LIMIT, "200");
-    for (int i = 0; i < 10; i++) {
-      ds = this.getSystem(props);
-      ds.disconnect();
-    }
-
-    /*
-     * This was throwing NPEs until my fix...
-     */
-  }
-
-  @Category(FlakyTest.class) // GEODE-674: possible disk pollution, file size sensitive
-  @Test
-  public void testSimpleStartRestartWithRolling() throws Exception {
-    Properties props = new Properties();
-    String baseLogName = "restarto";
-    String logfile = baseLogName + ".log";
-    props.put(LOG_FILE, logfile);
-    props.put(LOG_FILE_SIZE_LIMIT, "1");
-    props.put(LOG_DISK_SPACE_LIMIT, "200");
-    props.put(LOG_LEVEL, "config");
-    InternalDistributedSystem ids = getSystem(props);
-    assertEquals(InternalLogWriter.INFO_LEVEL,
-        ((InternalLogWriter) ids.getLogWriter()).getLogWriterLevel());
-    ids.disconnect();
-    String mainId;
-    {
-      final Pattern mainIdPattern = Pattern.compile("meta-" + baseLogName + "-\\d\\d.log");
-      File[] metaLogs = new File(".").listFiles(new FilenameFilter() {
-        public boolean accept(File d, String name) {
-          return mainIdPattern.matcher(name).matches();
-        }
-      });
-      assertEquals(1, metaLogs.length);
-      String f = metaLogs[0].getName();
-      int idx = f.lastIndexOf("-");
-      int idx2 = f.lastIndexOf(".");
-      mainId = f.substring(idx + 1, idx2);
-    }
-    String metaName = "meta-" + baseLogName + "-" + mainId + ".log";
-    File fm = new File(metaName);
-    assertTrue("Ok so metalog:" + metaName + " better exist:", fm.exists());
-    for (int i = 1; i < 10; i++) {
-      int mainInt = Integer.parseInt(mainId) + (i);
-      String myid;
-      if (mainInt < 10) {
-        myid = "0" + mainInt;
-      } else {
-        myid = "" + mainInt;
-      }
-      String oldMain;
-      if (mainInt < 11) {
-        oldMain = "0" + (mainInt - 1);
-      } else {
-        oldMain = "" + (mainInt - 1);
-      }
-      String lfold = "meta-" + baseLogName + "-" + (oldMain) + ".log";
-      File fold = new File(lfold);
-      assertTrue(
-          "before we even get going here[" + i + "] mainInt:" + mainInt + " myid:" + myid + " "
-              + lfold + " should exist the metaname was :" + metaName + " and it should match that",
-          fold.exists());
-      String lf = "meta-" + baseLogName + "-" + myid + ".log";
-      String lfl = baseLogName + "-" + (oldMain) + "-01.log";
-      File f1m = new File(lf);
-      File f1l = new File(lfl);
-      assertTrue(!f1m.exists());
-      assertTrue(!f1l.exists());
-      DistributedSystem ds = this.getSystem(props);
-      assertTrue("We are hoping that:" + lf + " exists", f1m.exists());
-      assertTrue("We are hoping that:" + lfl + " exists", f1l.exists());
-      ds.disconnect();
-    }
-  }
-
-  @Category(FlakyTest.class) // GEODE-677: possible disk pollution, file size sensitive
-  @Test
-  public void testStartWithRollingThenRestartWithRolling() throws Exception {
-    Properties props = new Properties();
-    String baseLogName = "biscuits";
-    String logfile = baseLogName + ".log";
-    props.put(LOG_FILE, logfile);
-    props.put(LOG_FILE_SIZE_LIMIT, "1");
-    props.put(LOG_LEVEL, "config");
-    DistributedSystem ds = getSystem(props);
-    InternalDistributedSystem ids = (InternalDistributedSystem) ds;
-    assertEquals(InternalLogWriter.INFO_LEVEL,
-        ((InternalLogWriter) ids.getLogWriter()).getLogWriterLevel());
-
-    // Lets figure out the mainId we start with
-    String mainId;
-    {
-      final Pattern mainIdPattern = Pattern.compile("meta-" + baseLogName + "-\\d\\d\\d*.log");
-      File[] metaLogs = new File(".").listFiles(new FilenameFilter() {
-        public boolean accept(File d, String name) {
-          return mainIdPattern.matcher(name).matches();
-        }
-      });
-      assertEquals(1, metaLogs.length);
-      String f = metaLogs[0].getName();
-      int idx = f.lastIndexOf("-");
-      int idx2 = f.lastIndexOf(".");
-      mainId = f.substring(idx + 1, idx2);
-    }
-    logAndRollAndVerify(baseLogName, ds, mainId);
-    /*
-     * Ok now we have rolled and yada yada. Let's disconnect and reconnect with same name!
-     */
-    int dsId = System.identityHashCode(ds);
-    props.put(LOG_DISK_SPACE_LIMIT, "200");
-
-    File f1m = new File(logfile);
-    assertTrue(f1m.exists());
-    File f1c1 = new File(baseLogName + "-" + mainId + "-01.log");
-    assertTrue(f1c1.exists());
-    File f1c2 = new File(baseLogName + "-" + mainId + "-02.log");
-    assertTrue(f1c2.exists());
-
-    File f1c3 = new File(baseLogName + "-" + mainId + "-03.log");
-    assertTrue(!f1c3.exists());
-
-    String nextMainId;
-    {
-      int mId = Integer.parseInt(mainId);
-      mId++;
-      StringBuffer sb = new StringBuffer();
-      if (mId < 10) {
-        sb.append('0');
-      }
-      sb.append(mId);
-      nextMainId = sb.toString();
-    }
-    File f2c1 = new File(baseLogName + "-" + nextMainId + "-01.log");
-    assertTrue(!f2c1.exists());
-
-
-    /*
-     * Lets just make sure all the proper files exist
-     */
-    ds = this.getSystem(props);
-    int dsId2 = System.identityHashCode(ds);
-    assertTrue("This should be a new ds!", dsId != dsId2);
-    /*
-     * creating the new system should have rolled the old rolling log
-     * (biscuits.log->biscuits-02-01.log)
-     * 
-     */
-    // The following assert does not work on Windows because
-    // we can't rename the last biscuits.log because it is still open
-    // The DistributedSystem disconnect is not closing the logger enough
-    // so that it can be renamed.
-    // Reenable this assertion once this issue (bug 42176) is fixed.
-    assertTrue(f1c3.exists());
-  }
-
-  @Category(FlakyTest.class) // GEODE-676: possible disk pollution, file size sensitive
-  @Test
-  public void testLogFileLayoutAndRolling() throws Exception {
-    String baseLogName = "tacos";
-    Properties props = new Properties();
-
-    String logfile = baseLogName + ".log";
-    props.put(LOG_FILE, logfile);
-    props.put(LOG_FILE_SIZE_LIMIT, "1");
-    props.put(LOG_LEVEL, "config");
-
-    DistributedSystem ds = getSystem(props);
-    InternalDistributedSystem ids = (InternalDistributedSystem) ds;
-    assertEquals(InternalLogWriter.INFO_LEVEL,
-        ((InternalLogWriter) ids.getLogWriter()).getLogWriterLevel());
-
-    // Lets figure out the mainId we start with
-    String mainId;
-    {
-      final Pattern mainIdPattern = Pattern.compile("meta-" + baseLogName + "-\\d+.log");
-      File[] metaLogs = new File(".").listFiles(new FilenameFilter() {
-        public boolean accept(File d, String name) {
-          return mainIdPattern.matcher(name).matches();
-        }
-      });
-      assertEquals(1, metaLogs.length);
-      String f = metaLogs[0].getName();
-      int idx = f.lastIndexOf("-");
-      int idx2 = f.lastIndexOf(".");
-      mainId = f.substring(idx + 1, idx2);
-    }
-    ds.getProperties();
-    logAndRollAndVerify(baseLogName, ds, mainId);
-  }
-
-  @Category(FlakyTest.class) // GEODE-675: possible disk pollution, file size sensitive
-  @Test
-  public void testSecurityLogFileLayoutAndRolling() throws Exception {
-    String baseLogName = "securitytacos";
-    Properties props = new Properties();
-
-    String logfile = baseLogName + ".log";
-    String sec_logfile = "sec_" + baseLogName + ".log";
-    props.put(LOG_FILE, logfile);
-    props.put(LOG_FILE_SIZE_LIMIT, "1");
-    props.put(LOG_LEVEL, "config");
-    props.put(SECURITY_LOG_FILE, sec_logfile);
-    props.put(SECURITY_LOG_LEVEL, "config");
-
-    DistributedSystem ds = getSystem(props);
-    InternalDistributedSystem ids = (InternalDistributedSystem) ds;
-    assertEquals(InternalLogWriter.INFO_LEVEL,
-        ((InternalLogWriter) ids.getLogWriter()).getLogWriterLevel());
-    assertEquals(InternalLogWriter.INFO_LEVEL,
-        ((InternalLogWriter) ids.getSecurityLogWriter()).getLogWriterLevel());
-
-    // Lets figure out the mainId we start with
-    String mainId;
-    {
-      final Pattern mainIdPattern = Pattern.compile("meta-" + baseLogName + "-\\d+.log");
-      File[] metaLogs = new File(".").listFiles(new FilenameFilter() {
-        public boolean accept(File d, String name) {
-          return mainIdPattern.matcher(name).matches();
-        }
-      });
-      assertEquals(1, metaLogs.length);
-      String f = metaLogs[0].getName();
-      int idx = f.lastIndexOf("-");
-      int idx2 = f.lastIndexOf(".");
-      mainId = f.substring(idx + 1, idx2);
-    }
-    ds.getProperties();
-    SecurityLogAndRollAndVerify(baseLogName, ds, mainId);
-  }
-
-  String getLogContents(String logfile) throws FileNotFoundException, IOException {
-    File f = new File(logfile);
-    BufferedReader reader = new BufferedReader(new FileReader(f));
-    StringBuffer fileData = new StringBuffer();
-    int numRead = 0;
-    char[] buf = new char[1024];
-    while ((numRead = reader.read(buf)) != -1) {
-      String readData = String.valueOf(buf, 0, numRead);
-      fileData.append(readData);
-      buf = new char[1024];
-    }
-    return fileData.toString();
-  }
-
-}

http://git-wip-us.apache.org/repos/asf/geode/blob/d5a36354/geode-core/src/test/java/org/apache/geode/internal/logging/CacheLogRollingIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/internal/logging/CacheLogRollingIntegrationTest.java b/geode-core/src/test/java/org/apache/geode/internal/logging/CacheLogRollingIntegrationTest.java
new file mode 100644
index 0000000..2444f98
--- /dev/null
+++ b/geode-core/src/test/java/org/apache/geode/internal/logging/CacheLogRollingIntegrationTest.java
@@ -0,0 +1,356 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.internal.logging;
+
+import static org.apache.geode.distributed.ConfigurationProperties.*;
+import static org.assertj.core.api.Assertions.*;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileReader;
+import java.io.IOException;
+import java.util.Properties;
+import java.util.regex.Pattern;
+
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.rules.TemporaryFolder;
+import org.junit.rules.TestName;
+
+import org.apache.geode.LogWriter;
+import org.apache.geode.distributed.DistributedSystem;
+import org.apache.geode.test.junit.categories.IntegrationTest;
+
+/**
+ * Integration tests for log rolling with cache lifecycle.
+ *
+ * @since GemFire 6.5
+ */
+@Category(IntegrationTest.class)
+public class CacheLogRollingIntegrationTest {
+
+  private static final int MAX_LOG_STATEMENTS = 100000;
+  private static final String SECURITY_PREFIX = "security_";
+
+  private String baseName;
+  private File dir;
+  private File logFile;
+  private File securityLogFile;
+  private Pattern mainIdPattern;
+  private DistributedSystem system;
+
+  @Rule
+  public TemporaryFolder temporaryFolder = new TemporaryFolder();
+
+  @Rule
+  public TestName testName = new TestName();
+
+  @Before
+  public void before() throws Exception {
+    this.baseName = this.testName.getMethodName();
+    this.dir = this.temporaryFolder.getRoot();
+    this.logFile = new File(this.dir, logFileName());
+    this.securityLogFile = new File(this.dir, securityLogFileName());
+    this.mainIdPattern = Pattern.compile("meta-" + this.baseName + "-\\d\\d.log");
+  }
+
+  @After
+  public void after() throws Exception {
+    if (this.system != null) {
+      this.system.disconnect();
+    }
+  }
+
+  @Test
+  public void testSimpleStartRestartWithRolling() throws Exception {
+    Properties config = createConfig();
+    config.put(LOG_FILE, this.logFile.getAbsolutePath());
+    config.put(LOG_FILE_SIZE_LIMIT, "1");
+    config.put(LOG_DISK_SPACE_LIMIT, "200");
+
+    this.system = DistributedSystem.connect(config);
+    this.system.disconnect();
+
+    for (int mainInt = 2; mainInt <= 4; mainInt++) {
+      assertThat(metaFile(mainInt - 1)).exists();
+
+      File newMetaFile = metaFile(mainInt);
+      File newRolledLogFile = childFile(mainInt - 1, 1);
+
+      assertThat(newMetaFile).doesNotExist();
+      assertThat(newRolledLogFile).doesNotExist();
+
+      this.system = DistributedSystem.connect(config);
+
+      assertThat(newMetaFile).exists();
+      assertThat(newRolledLogFile).exists();
+
+      this.system.disconnect();
+    }
+  }
+
+  @Test
+  public void testStartWithRollingThenRestartWithRolling() throws Exception {
+    Properties config = createConfig();
+    config.put(LOG_FILE, this.logFile.getAbsolutePath());
+    config.put(LOG_FILE_SIZE_LIMIT, "1");
+
+    this.system = DistributedSystem.connect(config);
+
+    logAndRollAndVerify(1);
+
+    DistributedSystem firstSystem = this.system;
+
+    assertThat(this.logFile).exists();
+    assertThat(childFile(1, 1)).exists();
+    assertThat(childFile(1, 2)).exists();
+    assertThat(childFile(1, 3)).doesNotExist();
+    assertThat(childFile(2, 1)).doesNotExist();
+
+    this.system.disconnect();
+
+    config.put(LOG_DISK_SPACE_LIMIT, "200");
+    this.system = DistributedSystem.connect(config);
+
+    assertThat(this.system).isNotSameAs(firstSystem);
+    assertThat(childFile(1, 3)).exists();
+  }
+
+  @Test
+  public void testLogFileLayoutAndRolling() throws Exception {
+    Properties config = createConfig();
+    config.put(LOG_FILE, this.logFile.getAbsolutePath());
+    config.put(LOG_FILE_SIZE_LIMIT, "1");
+
+    this.system = DistributedSystem.connect(config);
+
+    logAndRollAndVerify(1);
+  }
+
+  @Test
+  public void testSecurityLogFileLayoutAndRolling() throws Exception {
+    Properties config = createConfig();
+    config.put(LOG_FILE, this.logFile.getAbsolutePath());
+    config.put(LOG_FILE_SIZE_LIMIT, "1");
+    config.put(SECURITY_LOG_FILE, this.securityLogFile.getAbsolutePath());
+
+    this.system = DistributedSystem.connect(config);
+
+    securityLogAndRollAndVerify(1);
+  }
+
+  @Test
+  public void with_logFileSizeLimit_should_createMetaLogFile() throws Exception {
+    Properties config = createConfig();
+    config.put(LOG_FILE, this.logFile.getAbsolutePath());
+    config.put(LOG_FILE_SIZE_LIMIT, "1");
+
+    this.system = DistributedSystem.connect(config);
+
+    File[] metaLogsMatched =
+        this.dir.listFiles((dir, name) -> mainIdPattern.matcher(name).matches());
+    assertThat(metaLogsMatched).hasSize(1);
+
+    File metaLogFile = metaFile(1);
+    assertThat(metaLogFile).exists();
+  }
+
+  @Test
+  public void without_logFileSizeLimit_shouldNot_createMetaLogFile() throws Exception {
+    Properties config = createConfig();
+    config.put(LOG_FILE, this.logFile.getAbsolutePath());
+
+    this.system = DistributedSystem.connect(config);
+
+    File[] metaLogsMatched =
+        this.dir.listFiles((dir, name) -> mainIdPattern.matcher(name).matches());
+    assertThat(metaLogsMatched).hasSize(0);
+
+    File metaLogFile = metaFile(12);
+    assertThat(metaLogFile).doesNotExist();
+  }
+
+  private Properties createConfig() {
+    Properties config = new Properties();
+    config.setProperty(LOCATORS, "");
+    config.setProperty(MCAST_PORT, "0");
+    return config;
+  }
+
+  private String readContents(final File file) throws IOException {
+    assertThat(file).exists();
+
+    BufferedReader reader = new BufferedReader(new FileReader(file));
+    StringBuffer buffer = new StringBuffer();
+    int numRead;
+    char[] chars = new char[1024];
+
+    while ((numRead = reader.read(chars)) != -1) {
+      String readData = String.valueOf(chars, 0, numRead);
+      buffer.append(readData);
+      chars = new char[1024];
+    }
+
+    return buffer.toString();
+  }
+
+  /**
+   * 1. Lets assert that the logfile exists and that it is a proper normal logfile<br>
+   * 2. Assert that the meta logfile exists and has good stuff in it<br>
+   * 3. Let's log a bunch and show that we rolled<br>
+   * 4. Show that old file has right old stuff in it<br>
+   * 5. Show that new file has right new stuff in it<br>
+   * 6. Show that meta has right stuff in it<br>
+   */
+  private void logAndRollAndVerify(final int mainId) throws IOException {
+    File metaLogFile = metaFile(mainId);
+    File childLogFile01 = childFile(mainId, 1);
+    File childLogFile02 = childFile(mainId, 2);
+
+    String switchingToLog = "Switching to log " + this.logFile;
+    String rollingCurrentLogTo01 = "Rolling current log to " + childLogFile01;
+    String rollingCurrentLogTo02 = "Rolling current log to " + childLogFile02;
+
+    String messageInChild = "hey guys im the first child";
+    String messagePrefix = "hey whatsup i can't believe it wow ";
+
+    this.system.getLogWriter().info(messageInChild);
+
+    assertThat(this.logFile).exists();
+    assertThat(metaLogFile).exists();
+    assertThat(childLogFile01).doesNotExist();
+    assertThat(childLogFile02).doesNotExist();
+    assertThat(readContents(metaLogFile)).contains(switchingToLog);
+    assertThat(readContents(this.logFile)).contains(messageInChild);
+
+    logUntilFileExists(this.system.getLogWriter(), messagePrefix, childLogFile02);
+
+    assertThat(childLogFile01).exists();
+    assertThat(childLogFile02).exists();
+
+    String metaLogContents = readContents(metaLogFile);
+    assertThat(metaLogContents).contains(rollingCurrentLogTo01);
+    assertThat(metaLogContents).contains(rollingCurrentLogTo02);
+    assertThat(metaLogContents).doesNotContain(messagePrefix);
+
+    assertThat(readContents(this.logFile)).contains(messagePrefix);
+    assertThat(readContents(childLogFile01)).contains(messagePrefix);
+    assertThat(readContents(childLogFile02)).contains(messagePrefix);
+  }
+
+  /**
+   * 1. Lets assert that the logfile exists and that it is a proper normal logfile<br>
+   * 2. Assert that the meta logfile exists and has good stuff in it<br>
+   * 3. Let's log a bunch and show that we rolled<br>
+   * 4. Show that old file has right old stuff in it<br>
+   * 5. Show that new file has right new stuff in it<br>
+   * 6. Show that meta has right stuff in it<br>
+   */
+  private void securityLogAndRollAndVerify(final int mainId) throws IOException {
+    File metaLogFile = metaFile(mainId);
+    File childLogFile01 = childFile(mainId, 1);
+    File childLogFile02 = childFile(mainId, 2);
+    File childSecurityLogFile01 = childSecurityFile(mainId, 1);
+    File childSecurityLogFile02 = childSecurityFile(mainId, 2);
+
+    String switchingToLog = "Switching to log " + this.logFile;
+    String rollingCurrentLogTo01 = "Rolling current log to " + childLogFile01;
+    String rollingCurrentLogTo02 = "Rolling current log to " + childLogFile02;
+
+    String messageInChild = "hey guys im the first child";
+    String messageInSecurityChild = "hey guys im the first security child";
+    String messagePrefix = "hey whatsup i can't believe it wow ";
+
+    this.system.getLogWriter().info(messageInChild);
+    this.system.getSecurityLogWriter().info(messageInSecurityChild);
+
+    assertThat(readContents(this.logFile)).contains(messageInChild)
+        .doesNotContain(messageInSecurityChild);
+    assertThat(readContents(this.securityLogFile)).contains(messageInSecurityChild)
+        .doesNotContain(messageInChild);
+
+    assertThat(readContents(metaLogFile)).contains(switchingToLog);
+
+    assertThat(childLogFile01).doesNotExist();
+    assertThat(childSecurityLogFile01).doesNotExist();
+    assertThat(childLogFile02).doesNotExist();
+    assertThat(childSecurityLogFile02).doesNotExist();
+
+    logUntilFileExists(this.system.getLogWriter(), messagePrefix, childLogFile02);
+    logUntilFileExists(this.system.getSecurityLogWriter(), messagePrefix, childSecurityLogFile02);
+
+    assertThat(readContents(this.logFile)).contains(messagePrefix);
+    assertThat(readContents(this.securityLogFile)).contains(messagePrefix);
+
+    String metaLogContents = readContents(metaLogFile);
+    assertThat(metaLogContents).contains(rollingCurrentLogTo01);
+    assertThat(metaLogContents).contains(rollingCurrentLogTo02);
+    assertThat(metaLogContents).doesNotContain(messagePrefix);
+
+    assertThat(readContents(childLogFile01)).contains(messagePrefix);
+    assertThat(readContents(childSecurityLogFile01)).contains(messagePrefix);
+    assertThat(readContents(childLogFile02)).contains(messagePrefix);
+    assertThat(readContents(childSecurityLogFile02)).contains(messagePrefix);
+  }
+
+  private void logUntilFileExists(final LogWriter logWriter, final String message,
+      final File logFile) {
+    for (int i = 0; i < MAX_LOG_STATEMENTS && !logFile.exists(); i++) {
+      logWriter.info(message + "line-" + i);
+    }
+    assertThat(logFile).exists();
+  }
+
+  private String formatId(final int id) {
+    return String.format("%02d", id);
+  }
+
+  private String logFileName() {
+    return this.baseName + ".log";
+  }
+
+  private String securityLogFileName() {
+    return SECURITY_PREFIX + this.baseName + ".log";
+  }
+
+  private String metaFileName(int mainId) {
+    return "meta-" + this.baseName + "-" + formatId(mainId) + ".log";
+  }
+
+  private File metaFile(int mainId) {
+    return new File(this.dir, metaFileName(mainId));
+  }
+
+  private String childFileName(int mainId, int childId) {
+    return this.baseName + "-" + formatId(mainId) + "-" + formatId(childId) + ".log";
+  }
+
+  private File childFile(int mainId, int childId) {
+    return new File(this.dir, childFileName(mainId, childId));
+  }
+
+  private String childSecurityFileName(int mainId, int childId) {
+    return SECURITY_PREFIX + this.baseName + "-" + formatId(mainId) + "-" + formatId(childId)
+        + ".log";
+  }
+
+  private File childSecurityFile(int mainId, int childId) {
+    return new File(this.dir, childSecurityFileName(mainId, childId));
+  }
+
+}


[24/49] geode git commit: GEODE-2633: When turning on fine logging, GEODE logs the keystore password in clear text * spotless fix

Posted by kh...@apache.org.
GEODE-2633: When turning on fine logging, GEODE logs the keystore password in clear text
  * spotless fix


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/5fb264b0
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/5fb264b0
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/5fb264b0

Branch: refs/heads/feature/GEODE-2420
Commit: 5fb264b0a82df1d1623e48cfd1ea1cd4773437c9
Parents: 0452647
Author: Kevin J. Duling <kd...@pivotal.io>
Authored: Mon Mar 13 15:12:31 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../java/org/apache/geode/internal/util/ArgumentRedactor.java    | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/5fb264b0/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java b/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java
index 8873a52..df82dff 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/util/ArgumentRedactor.java
@@ -63,8 +63,8 @@ public class ArgumentRedactor {
 
   /**
    * Parse a string to find key=value pairs and redact the values if necessary. If more than one
-   * key=value pair exists in the input, each pair must be preceded by a hyphen '-' to delineate
-   * the pairs. <br>
+   * key=value pair exists in the input, each pair must be preceded by a hyphen '-' to delineate the
+   * pairs. <br>
    * Example:<br>
    * Single value: "password=secret" or "--password=secret" Multiple values: "-Dflag -Dkey=value
    * --classpath=."


[16/49] geode git commit: GEODE-2539: revert jetty version change for now

Posted by kh...@apache.org.
GEODE-2539: revert jetty version change for now


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/a4036c75
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/a4036c75
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/a4036c75

Branch: refs/heads/feature/GEODE-2420
Commit: a4036c75c8ca66c98dca45acebde8e4d25e14ff2
Parents: 72378a7
Author: Kevin J. Duling <kd...@pivotal.io>
Authored: Wed Mar 8 14:25:12 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 gradle/dependency-versions.properties | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/a4036c75/gradle/dependency-versions.properties
----------------------------------------------------------------------
diff --git a/gradle/dependency-versions.properties b/gradle/dependency-versions.properties
index 90601ac..34e5d67 100644
--- a/gradle/dependency-versions.properties
+++ b/gradle/dependency-versions.properties
@@ -52,7 +52,7 @@ javax.resource-api.version = 1.7
 javax.servlet-api.version = 3.1.0
 javax.transaction-api.version = 1.2
 jedis.version = 2.9.0
-jetty.version = 9.4.2.v20170220
+jetty.version = 9.3.6.v20151106
 jgroups.version = 3.6.10.Final
 jmock.version = 2.8.2
 jna.version = 4.0.0


[17/49] geode git commit: GEODE-2616: fix leak in colocated region removal

Posted by kh...@apache.org.
GEODE-2616: fix leak in colocated region removal

postDestroyRegion now removes itself from colocated parent colocatedByList


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/72378a74
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/72378a74
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/72378a74

Branch: refs/heads/feature/GEODE-2420
Commit: 72378a743471ec53bffeef9cfb92f1c98fdd19bb
Parents: c6c6459
Author: Darrel Schneider <ds...@pivotal.io>
Authored: Fri Jan 20 16:05:41 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../cache/query/internal/DefaultQuery.java      |  4 +-
 .../geode/internal/cache/PartitionedRegion.java | 12 ++---
 .../internal/cache/ColocatedPRJUnitTest.java    | 47 ++++++++++++++++++++
 .../cache/PartitionedRegionTestHelper.java      | 11 ++++-
 4 files changed, 65 insertions(+), 9 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/72378a74/geode-core/src/main/java/org/apache/geode/cache/query/internal/DefaultQuery.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/cache/query/internal/DefaultQuery.java b/geode-core/src/main/java/org/apache/geode/cache/query/internal/DefaultQuery.java
index 8bfd4fa..a721091 100644
--- a/geode-core/src/main/java/org/apache/geode/cache/query/internal/DefaultQuery.java
+++ b/geode-core/src/main/java/org/apache/geode/cache/query/internal/DefaultQuery.java
@@ -650,8 +650,8 @@ public class DefaultQuery implements Query {
             continue;
           }
           other = allPRs;
-          if ((((PartitionedRegion) eachPR).colocatedByList.contains(allPRs)
-              || ((PartitionedRegion) allPRs).colocatedByList.contains(eachPR))) {
+          if ((((PartitionedRegion) eachPR).getColocatedByList().contains(allPRs)
+              || ((PartitionedRegion) allPRs).getColocatedByList().contains(eachPR))) {
             colocated = true;
             break;
           }

http://git-wip-us.apache.org/repos/asf/geode/blob/72378a74/geode-core/src/main/java/org/apache/geode/internal/cache/PartitionedRegion.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/cache/PartitionedRegion.java b/geode-core/src/main/java/org/apache/geode/internal/cache/PartitionedRegion.java
index 9374d4b..173f35c 100755
--- a/geode-core/src/main/java/org/apache/geode/internal/cache/PartitionedRegion.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/cache/PartitionedRegion.java
@@ -600,7 +600,8 @@ public class PartitionedRegion extends LocalRegion
 
   private byte fixedPASet;
 
-  public List<PartitionedRegion> colocatedByList = new CopyOnWriteArrayList<PartitionedRegion>();
+  private final List<PartitionedRegion> colocatedByList =
+      new CopyOnWriteArrayList<PartitionedRegion>();
 
   private final PartitionListener[] partitionListeners;
 
@@ -670,9 +671,7 @@ public class PartitionedRegion extends LocalRegion
     this.colocatedWithRegion = ColocationHelper.getColocatedRegion(this);
 
     if (colocatedWithRegion != null) {
-      synchronized (colocatedWithRegion.colocatedByList) {
-        colocatedWithRegion.colocatedByList.add(this);
-      }
+      colocatedWithRegion.getColocatedByList().add(this);
     }
 
     if (colocatedWithRegion != null && !internalRegionArgs.isUsedForParallelGatewaySenderQueue()) {
@@ -786,7 +785,7 @@ public class PartitionedRegion extends LocalRegion
     return parallelGatewaySenderIds;
   }
 
-  List<PartitionedRegion> getColocatedByList() {
+  public List<PartitionedRegion> getColocatedByList() {
     return this.colocatedByList;
   }
 
@@ -7922,6 +7921,9 @@ public class PartitionedRegion extends LocalRegion
         }
       }
     }
+    if (colocatedWithRegion != null) {
+      colocatedWithRegion.getColocatedByList().remove(this);
+    }
 
     RegionLogger.logDestroy(getName(), cache.getMyId(), null, op.isClose());
   }

http://git-wip-us.apache.org/repos/asf/geode/blob/72378a74/geode-core/src/test/java/org/apache/geode/internal/cache/ColocatedPRJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/internal/cache/ColocatedPRJUnitTest.java b/geode-core/src/test/java/org/apache/geode/internal/cache/ColocatedPRJUnitTest.java
new file mode 100644
index 0000000..66484d9
--- /dev/null
+++ b/geode-core/src/test/java/org/apache/geode/internal/cache/ColocatedPRJUnitTest.java
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.internal.cache;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.assertFalse;
+
+import java.util.List;
+
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+import org.apache.geode.cache.PartitionAttributes;
+import org.apache.geode.cache.PartitionAttributesFactory;
+import org.apache.geode.test.junit.categories.IntegrationTest;
+
+@Category(IntegrationTest.class)
+public class ColocatedPRJUnitTest {
+  @SuppressWarnings("rawtypes")
+  @Test
+  public void destroyColocatedPRCheckForLeak() {
+    PartitionedRegion parent =
+        (PartitionedRegion) PartitionedRegionTestHelper.createPartionedRegion("PARENT");
+    List<PartitionedRegion> colocatedList = parent.getColocatedByList();
+    assertEquals(0, colocatedList.size());
+    PartitionAttributes PRatts =
+        new PartitionAttributesFactory().setColocatedWith("/PARENT").create();
+    PartitionedRegion child =
+        (PartitionedRegion) PartitionedRegionTestHelper.createPartionedRegion("CHILD", PRatts);
+    assertTrue(colocatedList.contains(child));
+    child.destroyRegion();
+    assertFalse(colocatedList.contains(child));
+  }
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/72378a74/geode-core/src/test/java/org/apache/geode/internal/cache/PartitionedRegionTestHelper.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/internal/cache/PartitionedRegionTestHelper.java b/geode-core/src/test/java/org/apache/geode/internal/cache/PartitionedRegionTestHelper.java
index 42c48ac..2824d74 100644
--- a/geode-core/src/test/java/org/apache/geode/internal/cache/PartitionedRegionTestHelper.java
+++ b/geode-core/src/test/java/org/apache/geode/internal/cache/PartitionedRegionTestHelper.java
@@ -49,9 +49,16 @@ public class PartitionedRegionTestHelper
    */
 
   public static Region createPartionedRegion(String regionname) throws RegionExistsException {
+    return createPartionedRegion(regionname, new PartitionAttributesFactory().create());
+  }
+
+  /**
+   * This method creates a partitioned region with the given PR attributes. The cache created is a
+   * loner, so this is only suitable for single VM tests.
+   */
+  public static Region createPartionedRegion(String regionname, PartitionAttributes prattribs)
+      throws RegionExistsException {
     AttributesFactory attribFactory = new AttributesFactory();
-    PartitionAttributesFactory paf = new PartitionAttributesFactory();
-    PartitionAttributes prattribs = paf.create();
     attribFactory.setDataPolicy(DataPolicy.PARTITION);
     attribFactory.setPartitionAttributes(prattribs);
     RegionAttributes regionAttribs = attribFactory.create();


[29/49] geode git commit: GEODE-2635: Creating DUnit tests to test eviction in lucene

Posted by kh...@apache.org.
GEODE-2635: Creating DUnit tests to test eviction in lucene

	* DUnit tests for eviction with local destroy and overflow
	* Refactored the integration tests for eviction

	This closes #420


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/16bf532a
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/16bf532a
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/16bf532a

Branch: refs/heads/feature/GEODE-2420
Commit: 16bf532a7bf92ef2ef666d3c823ab86a69f8cec8
Parents: 6c66e65
Author: nabarunnag <na...@cs.wisc.edu>
Authored: Wed Mar 8 16:16:52 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../geode/cache/lucene/EvictionDUnitTest.java   | 152 +++++++++++++++++++
 .../geode/cache/lucene/LuceneDUnitTest.java     |  22 ++-
 .../LuceneIndexCreationIntegrationTest.java     |  18 ++-
 3 files changed, 189 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/16bf532a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java
new file mode 100644
index 0000000..0fb0d9d
--- /dev/null
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more contributor license
+ * agreements. See the NOTICE file distributed with this work for additional information regarding
+ * copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License. You may obtain a
+ * copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software distributed under the License
+ * is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
+ * or implied. See the License for the specific language governing permissions and limitations under
+ * the License.
+ */
+package org.apache.geode.cache.lucene;
+
+import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.INDEX_NAME;
+import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.REGION_NAME;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+
+import org.apache.geode.cache.Cache;
+import org.apache.geode.cache.Region;
+import org.apache.geode.distributed.internal.DistributionConfig;
+import org.apache.geode.internal.cache.GemFireCacheImpl;
+import org.apache.geode.internal.cache.PartitionedRegion;
+import org.apache.geode.internal.cache.control.HeapMemoryMonitor;
+import org.apache.geode.test.dunit.SerializableRunnableIF;
+import org.apache.geode.test.junit.categories.DistributedTest;
+import org.awaitility.Awaitility;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.IntStream;
+import junitparams.JUnitParamsRunner;
+import junitparams.Parameters;
+
+
+@Category(DistributedTest.class)
+@RunWith(JUnitParamsRunner.class)
+public class EvictionDUnitTest extends LuceneQueriesAccessorBase {
+
+  protected final static float INITIAL_EVICTION_HEAP_PERCENTAGE = 50.9f;
+  protected final static float EVICTION_HEAP_PERCENTAGE_FAKE_NOTIFICATION = 85.0f;
+  protected final static int TEST_MAX_MEMORY = 100;
+  protected final static int MEMORY_USED_FAKE_NOTIFICATION = 90;
+
+  protected RegionTestableType[] getPartitionRedundantOverflowEvictionRegionType() {
+    return new RegionTestableType[] {
+        RegionTestableType.PARTITION_PERSISTENT_REDUNDANT_EVICTION_OVERFLOW};
+  }
+
+  protected RegionTestableType[] getPartitionRedundantLocalDestroyEvictionRegionType() {
+    return new RegionTestableType[] {RegionTestableType.PARTITION_REDUNDANT_EVICTION_LOCAL_DESTROY,
+        RegionTestableType.PARTITION_REDUNDANT_PERSISTENT_EVICTION_LOCAL_DESTROY,
+        RegionTestableType.PARTITION_EVICTION_LOCAL_DESTROY,
+        RegionTestableType.PARTITION_PERSISTENT_EVICTION_LOCAL_DESTROY};
+  }
+
+  @Test
+  @Parameters(method = "getPartitionRedundantLocalDestroyEvictionRegionType")
+  public void regionWithEvictionWithLocalDestroyMustNotbeAbleToCreateLuceneIndexes(
+      RegionTestableType regionTestType) {
+    SerializableRunnableIF createIndex = getSerializableRunnableIFCreateIndex();
+
+    dataStore1.invoke(() -> {
+      try {
+        initDataStore(createIndex, regionTestType);
+      } catch (UnsupportedOperationException e) {
+        assertEquals(
+            "Lucene indexes on regions with eviction and action local destroy are not supported",
+            e.getMessage());
+        assertNull(getCache().getRegion(REGION_NAME));
+      }
+    });
+
+  }
+
+  private SerializableRunnableIF getSerializableRunnableIFCreateIndex() {
+    return () -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
+    };
+  }
+
+  @Test
+  @Parameters(method = "getPartitionRedundantOverflowEvictionRegionType")
+  public void regionsWithEvictionWithOverflowMustBeAbleToCreateLuceneIndexes(
+      RegionTestableType regionTestType) {
+    SerializableRunnableIF createIndex = () -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+      luceneService.createIndexFactory().setFields("text").create(INDEX_NAME, REGION_NAME);
+    };
+
+    dataStore1.invoke(() -> initDataStore(createIndex, regionTestType));
+
+    accessor.invoke(() -> initDataStore(createIndex, regionTestType));
+
+    accessor.invoke(() -> {
+      Cache cache = getCache();
+      Region region = cache.getRegion(REGION_NAME);
+      IntStream.range(0, NUM_BUCKETS).forEach(i -> region.put(i, new TestObject("hello world")));
+    });
+
+    dataStore1.invoke(() -> {
+      try {
+        getCache().getResourceManager().setEvictionHeapPercentage(INITIAL_EVICTION_HEAP_PERCENTAGE);
+        final PartitionedRegion partitionedRegion = (PartitionedRegion) getRootRegion(REGION_NAME);
+        raiseFakeNotification();
+        Awaitility.await().atMost(60, TimeUnit.SECONDS).until(() -> {
+          assertTrue(partitionedRegion.getDiskRegionStats().getNumOverflowOnDisk() > 0);
+        });
+      } finally {
+        cleanUpAfterFakeNotification();
+      }
+    });
+
+    accessor.invoke(() -> {
+      LuceneService luceneService = LuceneServiceProvider.get(getCache());
+      LuceneQuery<Integer, TestObject> query = luceneService.createLuceneQueryFactory()
+          .setResultLimit(100).create(INDEX_NAME, REGION_NAME, "world", "text");
+      List<LuceneResultStruct<Integer, TestObject>> resultList = query.findResults();
+      assertEquals(NUM_BUCKETS, resultList.size());
+    });
+
+  }
+
+  protected void raiseFakeNotification() {
+    ((GemFireCacheImpl) getCache()).getHeapEvictor().testAbortAfterLoopCount = 1;
+    HeapMemoryMonitor.setTestDisableMemoryUpdates(true);
+    System.setProperty(DistributionConfig.GEMFIRE_PREFIX + "memoryEventTolerance", "0");
+
+    getCache().getResourceManager()
+        .setEvictionHeapPercentage(EVICTION_HEAP_PERCENTAGE_FAKE_NOTIFICATION);
+    HeapMemoryMonitor heapMemoryMonitor =
+        ((GemFireCacheImpl) getCache()).getResourceManager().getHeapMonitor();
+    heapMemoryMonitor.setTestMaxMemoryBytes(TEST_MAX_MEMORY);
+
+    heapMemoryMonitor.updateStateAndSendEvent(MEMORY_USED_FAKE_NOTIFICATION);
+  }
+
+  protected void cleanUpAfterFakeNotification() {
+    ((GemFireCacheImpl) getCache()).getHeapEvictor().testAbortAfterLoopCount = Integer.MAX_VALUE;
+    HeapMemoryMonitor.setTestDisableMemoryUpdates(false);
+    System.clearProperty(DistributionConfig.GEMFIRE_PREFIX + "memoryEventTolerance");
+  }
+
+}

http://git-wip-us.apache.org/repos/asf/geode/blob/16bf532a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java
index db48294..fa3c470 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneDUnitTest.java
@@ -36,7 +36,7 @@ public abstract class LuceneDUnitTest extends JUnit4CacheTestCase {
   protected VM dataStore2;
 
   protected static int NUM_BUCKETS = 10;
-
+  protected static int MAX_ENTRIES_FOR_EVICTION = 20;
 
   @Override
   public void postSetUp() throws Exception {
@@ -83,7 +83,25 @@ public abstract class LuceneDUnitTest extends JUnit4CacheTestCase {
     PARTITION_OVERFLOW_TO_DISK(RegionShortcut.PARTITION_PROXY, RegionShortcut.PARTITION_OVERFLOW,
         EvictionAttributes.createLRUEntryAttributes(1, EvictionAction.OVERFLOW_TO_DISK)),
     FIXED_PARTITION(RegionShortcut.PARTITION, RegionShortcut.PARTITION),
-    PARTITION_WITH_CLIENT(RegionShortcut.PARTITION_PROXY, RegionShortcut.PARTITION);
+    PARTITION_WITH_CLIENT(RegionShortcut.PARTITION_PROXY, RegionShortcut.PARTITION),
+    PARTITION_PERSISTENT_REDUNDANT_EVICTION_OVERFLOW(RegionShortcut.PARTITION_PROXY_REDUNDANT,
+        RegionShortcut.PARTITION_REDUNDANT_PERSISTENT,
+        EvictionAttributes.createLRUHeapAttributes(null, EvictionAction.OVERFLOW_TO_DISK)),
+    PARTITION_REDUNDANT_EVICTION_LOCAL_DESTROY(RegionShortcut.PARTITION_PROXY_REDUNDANT,
+        RegionShortcut.PARTITION_REDUNDANT, EvictionAttributes
+            .createLRUEntryAttributes(MAX_ENTRIES_FOR_EVICTION, EvictionAction.LOCAL_DESTROY)),
+    PARTITION_REDUNDANT_PERSISTENT_EVICTION_LOCAL_DESTROY(RegionShortcut.PARTITION_PROXY_REDUNDANT,
+        RegionShortcut.PARTITION_REDUNDANT_PERSISTENT, EvictionAttributes
+            .createLRUEntryAttributes(MAX_ENTRIES_FOR_EVICTION, EvictionAction.LOCAL_DESTROY)),
+    PARTITION_EVICTION_LOCAL_DESTROY(RegionShortcut.PARTITION_PROXY, RegionShortcut.PARTITION,
+        EvictionAttributes.createLRUEntryAttributes(MAX_ENTRIES_FOR_EVICTION,
+            EvictionAction.LOCAL_DESTROY)),
+    PARTITION_PERSISTENT_EVICTION_LOCAL_DESTROY(RegionShortcut.PARTITION_PROXY,
+        RegionShortcut.PARTITION_PERSISTENT, EvictionAttributes
+            .createLRUEntryAttributes(MAX_ENTRIES_FOR_EVICTION, EvictionAction.LOCAL_DESTROY))
+
+
+    ;
 
 
     EvictionAttributes evictionAttributes = null;

http://git-wip-us.apache.org/repos/asf/geode/blob/16bf532a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java
index 80e77e9..5a86ccb 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexCreationIntegrationTest.java
@@ -195,7 +195,8 @@ public class LuceneIndexCreationIntegrationTest extends LuceneIntegrationTest {
   }
 
   @Test
-  public void cannotCreateLuceneIndexForRegionWithEviction() throws IOException, ParseException {
+  public void cannotCreateLuceneIndexForRegionWithEvictionWithLocalDestroy()
+      throws IOException, ParseException {
     try {
       createIndex("field1", "field2", "field3");
       RegionFactory regionFactory = this.cache.createRegionFactory(RegionShortcut.PARTITION);
@@ -211,6 +212,21 @@ public class LuceneIndexCreationIntegrationTest extends LuceneIntegrationTest {
   }
 
   @Test
+  public void canCreateLuceneIndexForRegionWithEvictionWithOverflowToDisk()
+      throws IOException, ParseException {
+    try {
+      createIndex("field1", "field2", "field3");
+      RegionFactory regionFactory = this.cache.createRegionFactory(RegionShortcut.PARTITION);
+      regionFactory.setEvictionAttributes(
+          EvictionAttributes.createLRUHeapAttributes(null, EvictionAction.OVERFLOW_TO_DISK));
+      regionFactory.create(REGION_NAME);
+    } catch (Exception e) {
+      fail("Should have been able to create Lucene Index");
+      e.printStackTrace();
+    }
+  }
+
+  @Test
   public void cannotCreateLuceneIndexWithExistingIndexName() {
     expectedException.expect(IllegalArgumentException.class);
     createIndex("field1", "field2", "field3");


[46/49] geode git commit: GEODE-2539: Upgrading Jetty causes RestSecurityIntegrationTest to fail (#2)

Posted by kh...@apache.org.
GEODE-2539: Upgrading Jetty causes RestSecurityIntegrationTest to fail (#2)


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/755661ad
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/755661ad
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/755661ad

Branch: refs/heads/feature/GEODE-2420
Commit: 755661adaf43ef5e392bc4bc79463e7589a739d8
Parents: 6cc8a11
Author: Kevin J. Duling <kd...@pivotal.io>
Authored: Thu Mar 9 16:23:51 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../SessionReplicationIntegrationJUnitTest.java | 31 ++++++++++++--------
 gradle/dependency-versions.properties           |  2 +-
 2 files changed, 20 insertions(+), 13 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/755661ad/extensions/geode-modules-session/src/test/java/org/apache/geode/modules/session/internal/filter/SessionReplicationIntegrationJUnitTest.java
----------------------------------------------------------------------
diff --git a/extensions/geode-modules-session/src/test/java/org/apache/geode/modules/session/internal/filter/SessionReplicationIntegrationJUnitTest.java b/extensions/geode-modules-session/src/test/java/org/apache/geode/modules/session/internal/filter/SessionReplicationIntegrationJUnitTest.java
index 3bd517c..cc52bde 100644
--- a/extensions/geode-modules-session/src/test/java/org/apache/geode/modules/session/internal/filter/SessionReplicationIntegrationJUnitTest.java
+++ b/extensions/geode-modules-session/src/test/java/org/apache/geode/modules/session/internal/filter/SessionReplicationIntegrationJUnitTest.java
@@ -14,6 +14,16 @@
  */
 package org.apache.geode.modules.session.internal.filter;
 
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+
+import com.meterware.httpunit.GetMethodWebRequest;
+import com.meterware.httpunit.WebConversation;
+import com.meterware.httpunit.WebRequest;
+import com.meterware.httpunit.WebResponse;
 import org.apache.geode.cache.Region;
 import org.apache.geode.distributed.internal.DistributionConfig;
 import org.apache.geode.modules.session.filter.SessionCachingFilter;
@@ -32,6 +42,15 @@ import org.junit.experimental.categories.Category;
 import org.junit.rules.TemporaryFolder;
 import org.junit.runner.RunWith;
 
+import java.io.File;
+import java.io.IOException;
+import java.io.PrintWriter;
+import java.util.ArrayList;
+import java.util.EnumSet;
+import java.util.Enumeration;
+import java.util.List;
+import java.util.StringTokenizer;
+import java.util.concurrent.TimeUnit;
 import javax.servlet.DispatcherType;
 import javax.servlet.RequestDispatcher;
 import javax.servlet.ServletException;
@@ -39,18 +58,6 @@ import javax.servlet.http.Cookie;
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
 import javax.servlet.http.HttpSession;
-import java.io.File;
-import java.io.IOException;
-import java.io.PrintWriter;
-import java.util.*;
-import java.util.concurrent.TimeUnit;
-
-import static org.junit.Assert.*;
-
-import com.meterware.httpunit.GetMethodWebRequest;
-import com.meterware.httpunit.WebConversation;
-import com.meterware.httpunit.WebRequest;
-import com.meterware.httpunit.WebResponse;
 
 /**
  * In-container testing using Jetty. This allows us to test context listener events as well as

http://git-wip-us.apache.org/repos/asf/geode/blob/755661ad/gradle/dependency-versions.properties
----------------------------------------------------------------------
diff --git a/gradle/dependency-versions.properties b/gradle/dependency-versions.properties
index 34e5d67..7613dba 100644
--- a/gradle/dependency-versions.properties
+++ b/gradle/dependency-versions.properties
@@ -52,7 +52,7 @@ javax.resource-api.version = 1.7
 javax.servlet-api.version = 3.1.0
 javax.transaction-api.version = 1.2
 jedis.version = 2.9.0
-jetty.version = 9.3.6.v20151106
+jetty.version = 9.3.11.v20160721
 jgroups.version = 3.6.10.Final
 jmock.version = 2.8.2
 jna.version = 4.0.0


[37/49] geode git commit: GEODE-2639: Added a wait for flush to avoid flakiness in test

Posted by kh...@apache.org.
GEODE-2639: Added a wait for flush to avoid flakiness in test


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/eeb01bbc
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/eeb01bbc
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/eeb01bbc

Branch: refs/heads/feature/GEODE-2420
Commit: eeb01bbc97ca6f3f4f07115d21aba43400191e41
Parents: 5f9da29
Author: Jason Huynh <hu...@gmail.com>
Authored: Tue Mar 14 10:16:38 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/eeb01bbc/geode-lucene/src/test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java
index 0fb0d9d..fc2be39 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/EvictionDUnitTest.java
@@ -105,7 +105,7 @@ public class EvictionDUnitTest extends LuceneQueriesAccessorBase {
       Region region = cache.getRegion(REGION_NAME);
       IntStream.range(0, NUM_BUCKETS).forEach(i -> region.put(i, new TestObject("hello world")));
     });
-
+    waitForFlushBeforeExecuteTextSearch(accessor, 60000);
     dataStore1.invoke(() -> {
       try {
         getCache().getResourceManager().setEvictionHeapPercentage(INITIAL_EVICTION_HEAP_PERCENTAGE);


[36/49] geode git commit: GEODE-2599: fix for "null" in string of dots

Posted by kh...@apache.org.
GEODE-2599: fix for "null" in string of dots

Check for string "null" in server/locator status message

Refactor server and locator launcer wait loops by using Process.isAlive
instead of Process.exitValue


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/2e2992ac
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/2e2992ac
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/2e2992ac

Branch: refs/heads/feature/GEODE-2420
Commit: 2e2992ac1eb2e9539300dcc0d33918e41d7f6a86
Parents: e46400d
Author: Ken Howe <kh...@pivotal.io>
Authored: Fri Mar 10 15:09:04 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../cli/commands/LauncherLifecycleCommands.java | 53 ++++++++------------
 1 file changed, 22 insertions(+), 31 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/2e2992ac/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java
index d42d75e..3ad93ce 100755
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/commands/LauncherLifecycleCommands.java
@@ -384,21 +384,9 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
                 new File(locatorLauncher.getWorkingDirectory()))),
             null);
 
+        locatorState = locatorStatus(workingDirectory, memberName);
         do {
-          try {
-            final int exitValue = locatorProcess.exitValue();
-
-            stderrReader.join(PROCESS_STREAM_READER_JOIN_TIMEOUT_MILLIS); // was Long.MAX_VALUE
-
-            // Gfsh.println(message);
-
-            return ResultBuilder.createShellClientErrorResult(
-                String.format(CliStrings.START_LOCATOR__PROCESS_TERMINATED_ABNORMALLY_ERROR_MESSAGE,
-                    exitValue, locatorLauncher.getWorkingDirectory(), message.toString()));
-          } catch (IllegalThreadStateException ignore) {
-            // the IllegalThreadStateException is expected; it means the Locator's process has not
-            // terminated,
-            // and basically should not
+          if (locatorProcess.isAlive()) {
             Gfsh.print(".");
 
             synchronized (this) {
@@ -410,12 +398,19 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
             String currentLocatorStatusMessage = locatorState.getStatusMessage();
 
             if (isStartingOrNotResponding(locatorState.getStatus())
-                && !(StringUtils.isBlank(currentLocatorStatusMessage) || currentLocatorStatusMessage
-                    .equalsIgnoreCase(previousLocatorStatusMessage))) {
+                && !(StringUtils.isBlank(currentLocatorStatusMessage)
+                    || currentLocatorStatusMessage.equalsIgnoreCase(previousLocatorStatusMessage)
+                    || currentLocatorStatusMessage.trim().toLowerCase().equals("null"))) {
               Gfsh.println();
               Gfsh.println(currentLocatorStatusMessage);
               previousLocatorStatusMessage = currentLocatorStatusMessage;
             }
+          } else {
+            final int exitValue = locatorProcess.exitValue();
+
+            return ResultBuilder.createShellClientErrorResult(
+                String.format(CliStrings.START_LOCATOR__PROCESS_TERMINATED_ABNORMALLY_ERROR_MESSAGE,
+                    exitValue, locatorLauncher.getWorkingDirectory(), message.toString()));
           }
         } while (!(registeredLocatorSignalListener && locatorSignalListener.isSignaled())
             && isStartingOrNotResponding(locatorState.getStatus()));
@@ -1646,21 +1641,9 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
                 new File(serverLauncher.getWorkingDirectory()))),
             null);
 
+        serverState = serverStatus(workingDirectory, memberName);
         do {
-          try {
-            final int exitValue = serverProcess.exitValue();
-
-            stderrReader.join(PROCESS_STREAM_READER_JOIN_TIMEOUT_MILLIS); // was Long.MAX_VALUE
-
-            // Gfsh.println(message);
-
-            return ResultBuilder.createShellClientErrorResult(
-                String.format(CliStrings.START_SERVER__PROCESS_TERMINATED_ABNORMALLY_ERROR_MESSAGE,
-                    exitValue, serverLauncher.getWorkingDirectory(), message.toString()));
-          } catch (IllegalThreadStateException ignore) {
-            // the IllegalThreadStateException is expected; it means the Server's process has not
-            // terminated,
-            // and should not
+          if (serverProcess.isAlive()) {
             Gfsh.print(".");
 
             synchronized (this) {
@@ -1673,11 +1656,19 @@ public class LauncherLifecycleCommands extends AbstractCommandsSupport {
 
             if (isStartingOrNotResponding(serverState.getStatus())
                 && !(StringUtils.isBlank(currentServerStatusMessage)
-                    || currentServerStatusMessage.equalsIgnoreCase(previousServerStatusMessage))) {
+                    || currentServerStatusMessage.equalsIgnoreCase(previousServerStatusMessage)
+                    || currentServerStatusMessage.trim().toLowerCase().equals("null"))) {
               Gfsh.println();
               Gfsh.println(currentServerStatusMessage);
               previousServerStatusMessage = currentServerStatusMessage;
             }
+          } else {
+            final int exitValue = serverProcess.exitValue();
+
+            return ResultBuilder.createShellClientErrorResult(
+                String.format(CliStrings.START_SERVER__PROCESS_TERMINATED_ABNORMALLY_ERROR_MESSAGE,
+                    exitValue, serverLauncher.getWorkingDirectory(), message.toString()));
+
           }
         } while (!(registeredServerSignalListener && serverSignalListener.isSignaled())
             && isStartingOrNotResponding(serverState.getStatus()));


[31/49] geode git commit: GEODE-2653: Add FlakyTest category to test with Thread.sleep

Posted by kh...@apache.org.
GEODE-2653: Add FlakyTest category to test with Thread.sleep


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/189faaac
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/189faaac
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/189faaac

Branch: refs/heads/feature/GEODE-2420
Commit: 189faaac179fe4cb597dfda58af7344ba00510e4
Parents: 5fb264b
Author: Kirk Lund <kl...@apache.org>
Authored: Mon Mar 13 15:39:43 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../internal/membership/gms/membership/GMSJoinLeaveJUnitTest.java  | 2 ++
 1 file changed, 2 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/189faaac/geode-core/src/test/java/org/apache/geode/distributed/internal/membership/gms/membership/GMSJoinLeaveJUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/distributed/internal/membership/gms/membership/GMSJoinLeaveJUnitTest.java b/geode-core/src/test/java/org/apache/geode/distributed/internal/membership/gms/membership/GMSJoinLeaveJUnitTest.java
index b3076be..05ab6f7 100644
--- a/geode-core/src/test/java/org/apache/geode/distributed/internal/membership/gms/membership/GMSJoinLeaveJUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/distributed/internal/membership/gms/membership/GMSJoinLeaveJUnitTest.java
@@ -55,6 +55,7 @@ import org.apache.geode.distributed.internal.membership.gms.messages.RemoveMembe
 import org.apache.geode.distributed.internal.membership.gms.messages.ViewAckMessage;
 import org.apache.geode.internal.Version;
 import org.apache.geode.security.AuthenticationFailedException;
+import org.apache.geode.test.junit.categories.FlakyTest;
 import org.apache.geode.test.junit.categories.IntegrationTest;
 import org.apache.geode.test.junit.categories.MembershipTest;
 import org.junit.After;
@@ -326,6 +327,7 @@ public class GMSJoinLeaveJUnitTest {
     return memberList;
   }
 
+  @Category(FlakyTest.class) // GEODE-2653: flaky due to Thread.sleep
   @Test
   public void testRemoveMember() throws Exception {
     initMocks();


[44/49] geode git commit: GEODE-2641: minor help string changes

Posted by kh...@apache.org.
GEODE-2641: minor help string changes


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/e46400df
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/e46400df
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/e46400df

Branch: refs/heads/feature/GEODE-2420
Commit: e46400df3edc286f355fec39258ea5fdbc665127
Parents: 1e806c6
Author: Jinmei Liao <ji...@pivotal.io>
Authored: Tue Mar 14 09:34:26 2017 -0700
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../geode/management/internal/cli/i18n/CliStrings.java       | 4 ++--
 .../internal/cli/commands/golden-help-offline.properties     | 8 +++++---
 2 files changed, 7 insertions(+), 5 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/e46400df/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java b/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
index b835e8b..5b1f089 100644
--- a/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
+++ b/geode-core/src/main/java/org/apache/geode/management/internal/cli/i18n/CliStrings.java
@@ -1391,7 +1391,7 @@ public class CliStrings {
   public static final String EXPORT_LOGS__HELP = "Export the log files for a member or members.";
   public static final String EXPORT_LOGS__DIR = "dir";
   public static final String EXPORT_LOGS__DIR__HELP =
-      "Local directory to which log files will be written. This is only used when you are exporting logs using http connection. If not specified, user.dir will be used.";
+      "Local directory to which logs will be written. This is used only when you are exporting logs using an http connection. If not specified, logs are written to the location specified by the user.dir system property.";
   public static final String EXPORT_LOGS__MEMBER = "member";
   public static final String EXPORT_LOGS__MEMBER__HELP =
       "Name/Id of the member whose log files will be exported.";
@@ -1413,7 +1413,7 @@ public class CliStrings {
       "Log entries that occurred before this time will be exported. The default is no limit. Format: yyyy/MM/dd/HH/mm/ss/SSS/z OR yyyy/MM/dd";
   public static final String EXPORT_LOGS__MERGELOG = "merge-log";
   public static final String EXPORT_LOGS__MERGELOG__HELP =
-      "Whether to merge logs after exporting to the target directory.";
+      "Whether to merge logs after exporting to the target directory. Deprecated: Since Geode1.2, no longer used.";
   public static final String EXPORT_LOGS__MSG__CANNOT_MERGE =
       "Could not merge the files in target directory";
   public static final String EXPORT_LOGS__MSG__SPECIFY_ONE_OF_OPTION =

http://git-wip-us.apache.org/repos/asf/geode/blob/e46400df/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
----------------------------------------------------------------------
diff --git a/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties b/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
index 8cc8aba..3c56def 100644
--- a/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
+++ b/geode-core/src/test/resources/org/apache/geode/management/internal/cli/commands/golden-help-offline.properties
@@ -1473,8 +1473,9 @@ SYNTAX\n\
 \ \ \ \ [--end-time=value] [--logs-only(=value)?] [--stats-only(=value)?]\n\
 PARAMETERS\n\
 \ \ \ \ dir\n\
-\ \ \ \ \ \ \ \ Local directory to which log files will be written. This is only used when you are\n\
-\ \ \ \ \ \ \ \ exporting logs using http connection. If not specified, user.dir will be used.\n\
+\ \ \ \ \ \ \ \ Local directory to which logs will be written. This is used only when you are exporting\n\
+\ \ \ \ \ \ \ \ logs using an http connection. If not specified, logs are written to the location specified\n\
+\ \ \ \ \ \ \ \ by the user.dir system property.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ group\n\
 \ \ \ \ \ \ \ \ Group of members whose log files will be exported.\n\
@@ -1492,7 +1493,8 @@ PARAMETERS\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ \ \ \ \ Default (if the parameter is not specified): false\n\
 \ \ \ \ merge-log\n\
-\ \ \ \ \ \ \ \ Whether to merge logs after exporting to the target directory.\n\
+\ \ \ \ \ \ \ \ Whether to merge logs after exporting to the target directory. Deprecated: Since Geode1.2,\n\
+\ \ \ \ \ \ \ \ no longer used.\n\
 \ \ \ \ \ \ \ \ Required: false\n\
 \ \ \ \ \ \ \ \ Default (if the parameter is not specified): false\n\
 \ \ \ \ start-time\n\


[38/49] geode git commit: GEODE-2303 Add eccn text to README.md

Posted by kh...@apache.org.
GEODE-2303 Add eccn text to README.md


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/81794ec3
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/81794ec3
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/81794ec3

Branch: refs/heads/feature/GEODE-2420
Commit: 81794ec3dffb0c887567fae4ace46f3c918a92f0
Parents: 94ac0bc
Author: Anthony Baker <ab...@apache.org>
Authored: Mon Mar 6 17:01:39 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 README.md | 32 +++++++++++++++++++++++++++++++-
 1 file changed, 31 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/81794ec3/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 0909364..bc9e519 100644
--- a/README.md
+++ b/README.md
@@ -12,7 +12,7 @@
 5. [Application Development](#development)
 6. [Documentation](http://geode.apache.org/docs/)
 7. [Wiki](https://cwiki.apache.org/confluence/display/GEODE/Index)
-
+8. [Export Control](#export)8. [Export Control](#export)8. [Export Control](#export)8. [Export Control](#export)8. [Export Control](#export)8. [Export Control](#export)8. [Export Control](#export)8. [Export Control](#export)8. [Export Control](#export)
 
 ## <a name="overview"></a>Overview
 
@@ -199,3 +199,33 @@ The following libraries are available external to the Apache Geode project:
 * [Spring Data GemFire](http://projects.spring.io/spring-data-gemfire/)
 * [Spring Cache](http://docs.spring.io/spring/docs/current/spring-framework-reference/html/cache.html)
 * [Python](https://github.com/gemfire/py-gemfire-rest)
+
+## <a name="export"></a>Export Control
+
+This distribution includes cryptographic software.
+The country in which you currently reside may have restrictions
+on the import, possession, use, and/or re-export to another country,
+of encryption software. BEFORE using any encryption software,
+please check your country's laws, regulations and policies
+concerning the import, possession, or use, and re-export of
+encryption software, to see if this is permitted.
+See <http://www.wassenaar.org/> for more information.
+
+The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS),
+has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1,
+which includes information security software using or performing
+cryptographic functions with asymmetric algorithms.
+The form and manner of this Apache Software Foundation distribution makes
+it eligible for export under the License Exception
+ENC Technology Software Unrestricted (TSU) exception
+(see the BIS Export Administration Regulations, Section 740.13)
+for both object code and source code.
+
+The following provides more details on the included cryptographic software:
+
+* Apache Geode is designed to be used with
+  [Java Secure Socket Extension](https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html) (JSSE) and
+  [Java Cryptography Extension](http://docs.oracle.com/javase/8/docs/technotes/guides/security/crypto/CryptoSpec.html) (JCE).
+  The [JCE Unlimited Strength Jurisdiction Policy](http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html)
+  may need to be installed separately to use keystore passwords with 7 or more characters.
+* Apache Geode links to and uses [OpenSSL](https://www.openssl.org/) ciphers.


[40/49] geode git commit: GEODE-2595: Correcting javadoc for createIndexFactory

Posted by kh...@apache.org.
GEODE-2595: Correcting javadoc for createIndexFactory

This closes #421


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/6c66e65b
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/6c66e65b
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/6c66e65b

Branch: refs/heads/feature/GEODE-2420
Commit: 6c66e65bb0c756f8800a52fb6b24534fac5e1711
Parents: 5e200db
Author: Dan Smith <up...@apache.org>
Authored: Fri Mar 10 12:08:14 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../src/main/java/org/apache/geode/cache/lucene/LuceneService.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/6c66e65b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java
index 24ca902..976ab22 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneService.java
@@ -44,7 +44,7 @@ import java.util.concurrent.TimeUnit;
  * <pre>
  * {
  *   &#64;code
-*       luceneService.createCreateIndexFactory()
+*       luceneService.createIndexFactory()
  *        .addField("name")
  *        .addField("zipcode")
  *        .addField("email", new KeywordAnalyzer())


[26/49] geode git commit: GEODE-1793 LocatorDUnitTest.testStartTwoLocatorsOneWithSSLAndTheOtherNonSSL

Posted by kh...@apache.org.
GEODE-1793 LocatorDUnitTest.testStartTwoLocatorsOneWithSSLAndTheOtherNonSSL

This was a product issue.  When the locator using plain-text sockets is
contacted by a TcpClient using SSL the locator often just closes the socket.
On some platforms this causes a SSLHandshakeException but on others it
just causes a "SocketException: connection reset".  Writing some text to
the socket forces the TcpClient to get a SSLException (which is the superclass
of SSLHandshakeException).

The test class is still marked as Flaky due to GEODE-2542.


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/c47761d0
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/c47761d0
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/c47761d0

Branch: refs/heads/feature/GEODE-2420
Commit: c47761d08c33190f89a32dc5a092c439e68c2051
Parents: 5be7003
Author: Bruce Schuchardt <bs...@pivotal.io>
Authored: Fri Mar 3 13:47:42 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../internal/tcpserver/TcpClient.java           |   5 +-
 .../internal/tcpserver/TcpServer.java           |  15 +-
 .../geode/distributed/LocatorDUnitTest.java     | 159 ++-----------------
 3 files changed, 29 insertions(+), 150 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/c47761d0/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpClient.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpClient.java b/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpClient.java
index 47b5703..6b54170 100644
--- a/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpClient.java
+++ b/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpClient.java
@@ -25,6 +25,7 @@ import java.net.Socket;
 import java.util.HashMap;
 import java.util.Map;
 
+import javax.net.ssl.SSLException;
 import javax.net.ssl.SSLHandshakeException;
 
 import org.apache.logging.log4j.Logger;
@@ -264,8 +265,8 @@ public class TcpClient {
     try {
       sock = socketCreator.connect(ipAddr.getAddress(), ipAddr.getPort(), timeout, null, false);
       sock.setSoTimeout(timeout);
-    } catch (SSLHandshakeException e) {
-      throw new LocatorCancelException("Unrecognisable response received", e);
+    } catch (SSLException e) {
+      throw new LocatorCancelException("Unable to form SSL connection", e);
     }
 
     try {

http://git-wip-us.apache.org/repos/asf/geode/blob/c47761d0/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java b/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java
index 4265349..8f1b537 100755
--- a/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java
+++ b/geode-core/src/main/java/org/apache/geode/distributed/internal/tcpserver/TcpServer.java
@@ -77,9 +77,9 @@ public class TcpServer {
    * <p>
    * This should be incremented if the gossip message structures change
    * <p>
-   * 1000 - gemfire 5.5 - using java serialization 1001 - 5.7 - using DataSerializable and
-   * supporting server locator messages. 1002 - 7.1 - sending GemFire version along with
-   * GOSSIP_VERSION in each request.
+   * 1000 - gemfire 5.5 - using java serialization<br>
+   * 1001 - 5.7 - using DataSerializable and supporting server locator messages.<br>
+   * 1002 - 7.1 - sending GemFire version along with GOSSIP_VERSION in each request.
    * <p>
    * with the addition of support for all old versions of clients you can no longer change this
    * version number
@@ -89,7 +89,7 @@ public class TcpServer {
   // This GOSSIPVERSION is used in _getVersionForAddress request for getting GemFire version of a
   // GossipServer.
   public final static int OLDGOSSIPVERSION = 1001;
-
+  
   private static/* GemStoneAddition */ final Map GOSSIP_TO_GEMFIRE_VERSION_MAP = new HashMap();
 
   // For test purpose only
@@ -360,6 +360,13 @@ public class TcpServer {
           versionOrdinal = (short) GOSSIP_TO_GEMFIRE_VERSION_MAP.get(gossipVersion);
         } else {
           // Close the socket. We can not accept requests from a newer version
+          try {
+            sock.getOutputStream().write("unknown protocol version".getBytes());
+            sock.getOutputStream().flush();
+          } catch (IOException e) {
+            log.debug("exception in sending reply to process using unknown protocol "
+                + gossipVersion, e);
+          }
           sock.close();
           return;
         }

http://git-wip-us.apache.org/repos/asf/geode/blob/c47761d0/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java b/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java
index 37e8b5c..5812e74 100644
--- a/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java
+++ b/geode-core/src/test/java/org/apache/geode/distributed/LocatorDUnitTest.java
@@ -20,7 +20,6 @@ import static org.apache.geode.distributed.ConfigurationProperties.ENABLE_NETWOR
 import static org.apache.geode.distributed.ConfigurationProperties.LOCATORS;
 import static org.apache.geode.distributed.ConfigurationProperties.LOG_LEVEL;
 import static org.apache.geode.distributed.ConfigurationProperties.MCAST_PORT;
-import static org.apache.geode.distributed.ConfigurationProperties.MCAST_TTL;
 import static org.apache.geode.distributed.ConfigurationProperties.MEMBER_TIMEOUT;
 import static org.apache.geode.distributed.ConfigurationProperties.SECURITY_PEER_AUTHENTICATOR;
 import static org.apache.geode.distributed.ConfigurationProperties.SECURITY_PEER_AUTH_INIT;
@@ -43,7 +42,6 @@ import static org.junit.Assert.assertNotSame;
 import static org.junit.Assert.assertTrue;
 import static org.junit.Assert.fail;
 
-import org.apache.geode.distributed.internal.membership.gms.Services;
 import org.apache.geode.test.dunit.AsyncInvocation;
 import org.awaitility.Awaitility;
 import org.apache.geode.ForcedDisconnectException;
@@ -86,7 +84,6 @@ import org.apache.geode.test.junit.categories.DistributedTest;
 import org.apache.geode.test.junit.categories.FlakyTest;
 import org.apache.geode.test.junit.categories.MembershipTest;
 import org.apache.geode.util.test.TestUtil;
-import org.junit.Ignore;
 import org.junit.Test;
 import org.junit.experimental.categories.Category;
 
@@ -311,14 +308,14 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
   private Boolean startLocatorWithPortAndProperties(final int port, final Properties properties)
       throws IOException {
     try {
-      System.setProperty("p2p.joinTimeout", "5000"); // set a short join timeout. default is 17000ms
+//      System.setProperty("p2p.joinTimeout", "5000"); // set a short join timeout. default is 17000ms
       Locator.startLocatorAndDS(port, new File(""), properties);
     } catch (SystemConnectException e) {
       return Boolean.FALSE;
     } catch (GemFireConfigException e) {
       return Boolean.FALSE;
     } finally {
-      System.getProperties().remove("p2p.joinTimeout");
+//      System.getProperties().remove("p2p.joinTimeout");
     }
     return Boolean.TRUE;
   }
@@ -507,7 +504,6 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
     }
   }
 
-  @Ignore
   @Test
   public void testSSLEnabledLocatorDiesWhenConnectingToNonSSLLocator() {
     IgnoredException.addIgnoredException("Remote host closed connection during handshake");
@@ -640,10 +636,10 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
       throws IOException {
     Locator locator;
     try {
-      System.setProperty("p2p.joinTimeout", "5000"); // set a short join timeout. default is 17000ms
+//      System.setProperty("p2p.joinTimeout", "5000"); // set a short join timeout. default is 17000ms
       locator = Locator.startLocatorAndDS(port, new File(""), properties);
     } finally {
-      System.getProperties().remove("p2p.joinTimeout");
+//      System.getProperties().remove("p2p.joinTimeout");
     }
     return locator;
   }
@@ -1309,7 +1305,7 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
     bgexecLogger.info(addExpected);
 
     boolean exceptionOccurred = true;
-    String oldValue = (String) System.getProperties().put("p2p.joinTimeout", "15000");
+//    String oldValue = (String) System.getProperties().put("p2p.joinTimeout", "15000");
     try {
       DistributedSystem.connect(props);
       exceptionOccurred = false;
@@ -1326,11 +1322,11 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
       // if expected then add in a catch block for it above this catch
       org.apache.geode.test.dunit.Assert.fail("Failed with unexpected exception", ex);
     } finally {
-      if (oldValue == null) {
-        System.getProperties().remove("p2p.joinTimeout");
-      } else {
-        System.getProperties().put("p2p.joinTimeout", oldValue);
-      }
+//      if (oldValue == null) {
+//        System.getProperties().remove("p2p.joinTimeout");
+//      } else {
+//        System.getProperties().put("p2p.joinTimeout", oldValue);
+//      }
       bgexecLogger.info(removeExpected);
     }
 
@@ -1449,7 +1445,7 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
     DistributedTestUtils.deleteLocatorStateFile(port1);
     final String locators = NetworkUtils.getServerHostName(host) + "[" + port + "]";
 
-    vm0.invoke(getUniqueName() + 1, () -> getStartSBLocatorRunnable(port));
+    vm0.invoke(getUniqueName() + 1, () -> startSBLocator(port));
     try {
 
       final Properties props = new Properties();
@@ -1489,7 +1485,7 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
       // restart the locator to demonstrate reconnection & make disconnects faster
       // it should also regain the role of coordinator, so we check to make sure
       // that the coordinator has changed
-      vm0.invoke(getUniqueName() + "2", () -> getStartSBLocatorRunnable(port));
+      vm0.invoke(getUniqueName() + "2", () -> startSBLocator(port));
 
       final DistributedMember tempCoord = newCoord;
 
@@ -1532,20 +1528,6 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
 
   }
 
-  /**
-   * set a short locator refresh rate
-   */
-  public static void setShortRefreshWait() {
-    System.setProperty("p2p.gossipRefreshRate", "2000");
-  }
-
-  /**
-   * remove shortened locator refresh rate
-   */
-  public static void resetRefreshWait() {
-    System.getProperties().remove("p2p.gossipRefreshRate");
-  }
-
   public static boolean isSystemConnected() {
     DistributedSystem sys = InternalDistributedSystem.getAnyInstance();
     if (sys != null && sys.isConnected()) {
@@ -1817,117 +1799,6 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
   }
 
   /**
-   * Tests starting multiple locators in multiple VMs.
-   */
-  @Test
-  public void testMultipleMcastLocators() throws Exception {
-    disconnectAllFromDS();
-    IgnoredException.addIgnoredException("Could not stop  Distribution Locator"); // shutdown timing
-    // issue in
-    // InternalLocator
-    Host host = Host.getHost(0);
-    VM vm0 = host.getVM(0);
-    VM vm1 = host.getVM(1);
-    VM vm2 = host.getVM(2);
-    VM vm3 = host.getVM(3);
-
-    final int[] freeTCPPorts = AvailablePortHelper.getRandomAvailableTCPPorts(2);
-    final int port1 = freeTCPPorts[0];
-    this.port1 = port1;
-    final int port2 = freeTCPPorts[1];
-    this.port2 = port2;
-    DistributedTestUtils.deleteLocatorStateFile(port1, port2);
-    final int mcastport = AvailablePort.getRandomAvailablePort(AvailablePort.MULTICAST);
-
-    final String host0 = NetworkUtils.getServerHostName(host);
-    final String locators = host0 + "[" + port1 + "]," + host0 + "[" + port2 + "]";
-    final String uniqueName = getUniqueName();
-
-    vm0.invoke(new SerializableRunnable("Start locator on " + port1) {
-      public void run() {
-        File logFile = new File("");
-        try {
-          Properties props = new Properties();
-          props.setProperty(MCAST_PORT, String.valueOf(mcastport));
-          props.setProperty(LOCATORS, locators);
-          props.setProperty(LOG_LEVEL, LogWriterUtils.getDUnitLogLevel());
-          props.setProperty(MCAST_TTL, "0");
-          props.setProperty(ENABLE_NETWORK_PARTITION_DETECTION, "true");
-          props.setProperty(ENABLE_CLUSTER_CONFIGURATION, "false");
-
-          addDSProps(props);
-          Locator.startLocatorAndDS(port1, logFile, null, props);
-        } catch (IOException ex) {
-          org.apache.geode.test.dunit.Assert.fail("While starting locator on port " + port1, ex);
-        }
-      }
-    });
-    vm3.invoke("Start locator on " + port2, () -> {
-      File logFile = new File("");
-      try {
-        Properties props = new Properties();
-        props.setProperty(MCAST_PORT, String.valueOf(mcastport));
-        props.setProperty(LOCATORS, locators);
-        props.setProperty(LOG_LEVEL, LogWriterUtils.getDUnitLogLevel());
-        props.setProperty(MCAST_TTL, "0");
-        props.setProperty(ENABLE_NETWORK_PARTITION_DETECTION, "true");
-        props.setProperty(ENABLE_CLUSTER_CONFIGURATION, "false");
-        addDSProps(props);
-        Locator.startLocatorAndDS(port2, logFile, null, props);
-      } catch (IOException ex) {
-        org.apache.geode.test.dunit.Assert.fail("While starting locator on port " + port2, ex);
-      }
-    });
-
-    SerializableRunnable connect = new SerializableRunnable("Connect to " + locators) {
-      public void run() {
-        Properties props = new Properties();
-        props.setProperty(MCAST_PORT, String.valueOf(mcastport));
-        props.setProperty(LOCATORS, locators);
-        props.setProperty(LOG_LEVEL, LogWriterUtils.getDUnitLogLevel());
-        props.setProperty(MCAST_TTL, "0");
-        props.setProperty(ENABLE_NETWORK_PARTITION_DETECTION, "true");
-        addDSProps(props);
-        DistributedSystem.connect(props);
-      }
-    };
-    try {
-      vm1.invoke(connect);
-      vm2.invoke(connect);
-
-      Properties props = new Properties();
-      props.setProperty(MCAST_PORT, String.valueOf(mcastport));
-      props.setProperty(LOCATORS, locators);
-      props.setProperty(LOG_LEVEL, LogWriterUtils.getDUnitLogLevel());
-      props.setProperty(MCAST_TTL, "0");
-      props.setProperty(ENABLE_NETWORK_PARTITION_DETECTION, "true");
-
-      addDSProps(props);
-      system = (InternalDistributedSystem) DistributedSystem.connect(props);
-
-      Awaitility.waitAtMost(WAIT2_MS, TimeUnit.MILLISECONDS)
-          .pollInterval(200, TimeUnit.MILLISECONDS).until(() -> {
-            try {
-              return system.getDM().getViewMembers().size() == 5;
-            } catch (Exception e) {
-              org.apache.geode.test.dunit.Assert.fail("unexpected exception", e);
-            }
-            return false; // NOTREACHED
-          });
-      system.disconnect();
-
-      vm1.invoke(() -> disconnectDistributedSystem());
-      vm2.invoke(() -> disconnectDistributedSystem());
-    } finally {
-      vm0.invoke(() -> stopLocator());
-      vm3.invoke(() -> stopLocator());
-      if (system != null) {
-        system.disconnect();
-      }
-    }
-  }
-
-  /**
    * Tests that a VM can connect to a locator that is hosted in its own VM.
    */
   @Test
@@ -2087,11 +1958,11 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
     }
   }
 
-  private void getStartSBLocatorRunnable(final int port) {
+  private void startSBLocator(final int port) {
     File logFile = new File("");
     try {
       System.setProperty(InternalLocator.LOCATORS_PREFERRED_AS_COORDINATORS, "true");
-      System.setProperty("p2p.joinTimeout", "1000");
+//      System.setProperty("p2p.joinTimeout", "1000");
       Properties locProps = new Properties();
       locProps.put(MCAST_PORT, "0");
       locProps.put(LOG_LEVEL, LogWriterUtils.getDUnitLogLevel());
@@ -2101,7 +1972,7 @@ public class LocatorDUnitTest extends JUnit4DistributedTestCase {
       org.apache.geode.test.dunit.Assert.fail("While starting locator on port " + port, ex);
     } finally {
       System.getProperties().remove(InternalLocator.LOCATORS_PREFERRED_AS_COORDINATORS);
-      System.getProperties().remove("p2p.joinTimeout");
+//      System.getProperties().remove("p2p.joinTimeout");
     }
   }
 


[15/49] geode git commit: GEODE-2617: use DataSeriazableFixedID to implement

Posted by kh...@apache.org.
GEODE-2617: use DataSeriazableFixedID to implement


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/aa9ced53
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/aa9ced53
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/aa9ced53

Branch: refs/heads/feature/GEODE-2420
Commit: aa9ced53d4f7351cac99e44622a2ee9011927b89
Parents: ce5523f
Author: zhouxh <gz...@pivotal.io>
Authored: Thu Mar 9 09:54:42 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../geode/internal/DataSerializableFixedID.java |  1 +
 .../geode/cache/lucene/LuceneResultStruct.java  |  4 +--
 .../lucene/internal/LuceneResultStructImpl.java | 37 +++++++++++++++++++-
 .../lucene/internal/LuceneServiceImpl.java      |  3 ++
 4 files changed, 41 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/aa9ced53/geode-core/src/main/java/org/apache/geode/internal/DataSerializableFixedID.java
----------------------------------------------------------------------
diff --git a/geode-core/src/main/java/org/apache/geode/internal/DataSerializableFixedID.java b/geode-core/src/main/java/org/apache/geode/internal/DataSerializableFixedID.java
index 63a95a5..18f382b 100644
--- a/geode-core/src/main/java/org/apache/geode/internal/DataSerializableFixedID.java
+++ b/geode-core/src/main/java/org/apache/geode/internal/DataSerializableFixedID.java
@@ -811,6 +811,7 @@ public interface DataSerializableFixedID extends SerializationVersions {
   public static final short WAIT_UNTIL_FLUSHED_FUNCTION_CONTEXT = 2177;
   public static final short DESTROY_LUCENE_INDEX_MESSAGE = 2178;
   public static final short LUCENE_PAGE_RESULTS = 2179;
+  public static final short LUCENE_RESULT_STRUCT = 2180;
 
   // NOTE, codes > 65535 will take 4 bytes to serialize
 

http://git-wip-us.apache.org/repos/asf/geode/blob/aa9ced53/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneResultStruct.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneResultStruct.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneResultStruct.java
index 5d2184e..4a6481f 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneResultStruct.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/LuceneResultStruct.java
@@ -14,8 +14,6 @@
  */
 package org.apache.geode.cache.lucene;
 
-import java.io.Serializable;
-
 import org.apache.geode.annotations.Experimental;
 
 /**
@@ -23,7 +21,7 @@ import org.apache.geode.annotations.Experimental;
  * 
  */
 @Experimental
-public interface LuceneResultStruct<K, V> extends Serializable {
+public interface LuceneResultStruct<K, V> {
 
   /**
    * @return The region key of the entry matching the query

http://git-wip-us.apache.org/repos/asf/geode/blob/aa9ced53/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneResultStructImpl.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneResultStructImpl.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneResultStructImpl.java
index 6c31025..708ac1e 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneResultStructImpl.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneResultStructImpl.java
@@ -15,13 +15,23 @@
 
 package org.apache.geode.cache.lucene.internal;
 
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+
+import org.apache.geode.DataSerializer;
 import org.apache.geode.cache.lucene.LuceneResultStruct;
+import org.apache.geode.internal.DataSerializableFixedID;
+import org.apache.geode.internal.Version;
 
-public class LuceneResultStructImpl<K, V> implements LuceneResultStruct<K, V> {
+public class LuceneResultStructImpl<K, V>
+    implements LuceneResultStruct<K, V>, DataSerializableFixedID {
   K key;
   V value;
   float score;
 
+  public LuceneResultStructImpl() {}
+
   public LuceneResultStructImpl(K key, V value, float score) {
     this.key = key;
     this.value = value;
@@ -81,4 +91,29 @@ public class LuceneResultStructImpl<K, V> implements LuceneResultStruct<K, V> {
   public String toString() {
     return "LuceneResultStructImpl [key=" + key + ", value=" + value + ", score=" + score + "]";
   }
+
+  @Override
+  public Version[] getSerializationVersions() {
+    return null;
+  }
+
+  @Override
+  public int getDSFID() {
+    return LUCENE_RESULT_STRUCT;
+  }
+
+  @Override
+  public void toData(DataOutput out) throws IOException {
+    DataSerializer.writeObject(key, out);
+    DataSerializer.writeObject(value, out);
+    out.writeFloat(score);
+  }
+
+  @Override
+  public void fromData(DataInput in) throws IOException, ClassNotFoundException {
+    key = DataSerializer.readObject(in);
+    value = DataSerializer.readObject(in);
+    score = in.readFloat();
+  }
+
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/aa9ced53/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java
index 5c908f5..a212094 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneServiceImpl.java
@@ -381,6 +381,9 @@ public class LuceneServiceImpl implements InternalLuceneService {
         DestroyLuceneIndexMessage.class);
 
     DSFIDFactory.registerDSFID(DataSerializableFixedID.LUCENE_PAGE_RESULTS, PageResults.class);
+
+    DSFIDFactory.registerDSFID(DataSerializableFixedID.LUCENE_RESULT_STRUCT,
+        LuceneResultStructImpl.class);
   }
 
   public Collection<LuceneIndexCreationProfile> getAllDefinedIndexes() {


[13/49] geode git commit: GEODE-2553: Closed IndexRepositories when deleting an index

Posted by kh...@apache.org.
GEODE-2553: Closed IndexRepositories when deleting an index


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/ce5523f6
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/ce5523f6
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/ce5523f6

Branch: refs/heads/feature/GEODE-2420
Commit: ce5523f68e171abfb16f39d4787c95df9f96de71
Parents: a4036c7
Author: Barry Oglesby <bo...@pivotal.io>
Authored: Wed Mar 1 10:41:03 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:44 2017 -0700

----------------------------------------------------------------------
 .../AbstractPartitionedRepositoryManager.java   | 16 +++++-
 .../lucene/internal/LuceneBucketListener.java   |  7 +--
 .../LuceneIndexForPartitionedRegion.java        | 31 ++---------
 .../cache/lucene/internal/LuceneIndexImpl.java  | 36 ++++++++++---
 .../internal/RawLuceneRepositoryManager.java    |  6 ---
 .../internal/repository/RepositoryManager.java  |  5 ++
 .../lucene/LuceneIndexDestroyDUnitTest.java     | 56 +++++++++++++++++++-
 7 files changed, 107 insertions(+), 50 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/ce5523f6/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/AbstractPartitionedRepositoryManager.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/AbstractPartitionedRepositoryManager.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/AbstractPartitionedRepositoryManager.java
index 97acea1..26179c7 100755
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/AbstractPartitionedRepositoryManager.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/AbstractPartitionedRepositoryManager.java
@@ -49,11 +49,13 @@ public abstract class AbstractPartitionedRepositoryManager implements Repository
   protected final PartitionedRegion userRegion;
   protected final LuceneSerializer serializer;
   protected final LuceneIndexImpl index;
+  protected volatile boolean closed;
 
   public AbstractPartitionedRepositoryManager(LuceneIndexImpl index, LuceneSerializer serializer) {
     this.index = index;
     this.userRegion = (PartitionedRegion) index.getCache().getRegion(index.getRegionPath());
     this.serializer = serializer;
+    this.closed = false;
   }
 
   @Override
@@ -91,9 +93,13 @@ public abstract class AbstractPartitionedRepositoryManager implements Repository
       LuceneSerializer serializer, LuceneIndexImpl index, PartitionedRegion userRegion,
       IndexRepository oldRepository) throws IOException;
 
-  protected IndexRepository computeRepository(Integer bucketId) throws BucketNotFoundException {
+  protected IndexRepository computeRepository(Integer bucketId) {
     IndexRepository repo = indexRepositories.compute(bucketId, (key, oldRepository) -> {
       try {
+        if (closed && oldRepository != null) {
+          oldRepository.cleanup();
+          return null;
+        }
         return computeRepository(bucketId, serializer, index, userRegion, oldRepository);
       } catch (IOException e) {
         throw new InternalGemFireError("Unable to create index repository", e);
@@ -119,4 +125,12 @@ public abstract class AbstractPartitionedRepositoryManager implements Repository
     }
     return repo;
   }
+
+  @Override
+  public void close() {
+    this.closed = true;
+    for (Integer bucketId : indexRepositories.keySet()) {
+      computeRepository(bucketId);
+    }
+  }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/ce5523f6/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneBucketListener.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneBucketListener.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneBucketListener.java
index 9532249..32fb3fc 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneBucketListener.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneBucketListener.java
@@ -40,11 +40,6 @@ public class LuceneBucketListener extends PartitionListenerAdapter {
         lucenePartitionRepositoryManager.computeRepository(bucketId);
       } catch (PrimaryBucketException e) {
         logger.info("Index repository could not be created because we are no longer primary?", e);
-      } catch (BucketNotFoundException e) {
-        logger.info(
-            "Index repository could not be created when index chunk region bucket became primary. "
-                + "Deferring index repository to be created lazily during lucene query execution."
-                + e);
       }
     });
   }
@@ -57,7 +52,7 @@ public class LuceneBucketListener extends PartitionListenerAdapter {
     dm.getWaitingThreadPool().execute(() -> {
       try {
         lucenePartitionRepositoryManager.computeRepository(bucketId);
-      } catch (PrimaryBucketException | BucketNotFoundException | AlreadyClosedException e) {
+      } catch (PrimaryBucketException | AlreadyClosedException e) {
         logger.debug("Exception while cleaning up Lucene Index Repository", e);
       }
     });

http://git-wip-us.apache.org/repos/asf/geode/blob/ce5523f6/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
index 4aa24b5..f24c6d6 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexForPartitionedRegion.java
@@ -217,10 +217,6 @@ public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
     // Invoke super destroy to remove the extension
     super.destroy(initiator);
 
-    // Destroy the AsyncEventQueue
-    PartitionedRegion pr = (PartitionedRegion) getDataRegion();
-    destroyAsyncEventQueue(pr);
-
     // Destroy the chunk region (colocated with the file region)
     // localDestroyRegion can't be used because locally destroying regions is not supported on
     // colocated regions
@@ -243,7 +239,7 @@ public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
 
     // Destroy index on remote members if necessary
     if (initiator) {
-      destroyOnRemoteMembers(pr);
+      destroyOnRemoteMembers();
     }
 
     if (logger.isDebugEnabled()) {
@@ -252,29 +248,8 @@ public class LuceneIndexForPartitionedRegion extends LuceneIndexImpl {
     }
   }
 
-  private void destroyAsyncEventQueue(PartitionedRegion pr) {
-    String aeqId = LuceneServiceImpl.getUniqueIndexName(indexName, regionPath);
-
-    // Get the AsyncEventQueue
-    AsyncEventQueueImpl aeq = (AsyncEventQueueImpl) cache.getAsyncEventQueue(aeqId);
-
-    // Stop the AsyncEventQueue (this stops the AsyncEventQueue's underlying GatewaySender)
-    aeq.stop();
-
-    // Remove the id from the dataRegion's AsyncEventQueue ids
-    // Note: The region may already have been destroyed by a remote member
-    if (!pr.isDestroyed()) {
-      pr.getAttributesMutator().removeAsyncEventQueueId(aeqId);
-    }
-
-    // Destroy the aeq (this also removes it from the GemFireCacheImpl)
-    aeq.destroy();
-    if (logger.isDebugEnabled()) {
-      logger.debug("Destroyed aeqId=" + aeqId);
-    }
-  }
-
-  private void destroyOnRemoteMembers(PartitionedRegion pr) {
+  private void destroyOnRemoteMembers() {
+    PartitionedRegion pr = (PartitionedRegion) getDataRegion();
     DM dm = pr.getDistributionManager();
     Set<InternalDistributedMember> recipients = pr.getRegionAdvisor().adviseDataStore();
     if (!recipients.isEmpty()) {

http://git-wip-us.apache.org/repos/asf/geode/blob/ce5523f6/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
index cf519be..b5b13c1 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/LuceneIndexImpl.java
@@ -16,9 +16,7 @@
 package org.apache.geode.cache.lucene.internal;
 
 import java.util.Collections;
-import java.util.List;
 import java.util.Map;
-import java.util.concurrent.TimeUnit;
 
 import org.apache.geode.internal.cache.extension.Extension;
 import org.apache.logging.log4j.Logger;
@@ -33,11 +31,6 @@ import org.apache.geode.cache.RegionAttributes;
 import org.apache.geode.cache.asyncqueue.AsyncEventQueue;
 import org.apache.geode.cache.asyncqueue.internal.AsyncEventQueueFactoryImpl;
 import org.apache.geode.cache.asyncqueue.internal.AsyncEventQueueImpl;
-import org.apache.geode.cache.execute.Execution;
-import org.apache.geode.cache.execute.FunctionService;
-import org.apache.geode.cache.execute.ResultCollector;
-import org.apache.geode.cache.lucene.internal.distributed.WaitUntilFlushedFunction;
-import org.apache.geode.cache.lucene.internal.distributed.WaitUntilFlushedFunctionContext;
 import org.apache.geode.cache.lucene.internal.repository.RepositoryManager;
 import org.apache.geode.cache.lucene.internal.xml.LuceneIndexCreation;
 import org.apache.geode.internal.cache.GemFireCacheImpl;
@@ -217,6 +210,12 @@ public abstract class LuceneIndexImpl implements InternalLuceneIndex {
     if (extensionToDelete != null) {
       getDataRegion().getExtensionPoint().removeExtension(extensionToDelete);
     }
+
+    // Destroy the async event queue
+    destroyAsyncEventQueue();
+
+    // Close the repository manager
+    repositoryManager.close();
   }
 
   protected <K, V> Region<K, V> createRegion(final String regionName,
@@ -237,4 +236,27 @@ public abstract class LuceneIndexImpl implements InternalLuceneIndex {
       throw ige;
     }
   }
+
+  private void destroyAsyncEventQueue() {
+    String aeqId = LuceneServiceImpl.getUniqueIndexName(indexName, regionPath);
+
+    // Get the AsyncEventQueue
+    AsyncEventQueueImpl aeq = (AsyncEventQueueImpl) cache.getAsyncEventQueue(aeqId);
+
+    // Stop the AsyncEventQueue (this stops the AsyncEventQueue's underlying GatewaySender)
+    aeq.stop();
+
+    // Remove the id from the dataRegion's AsyncEventQueue ids
+    // Note: The region may already have been destroyed by a remote member
+    Region region = getDataRegion();
+    if (!region.isDestroyed()) {
+      region.getAttributesMutator().removeAsyncEventQueueId(aeqId);
+    }
+
+    // Destroy the aeq (this also removes it from the GemFireCacheImpl)
+    aeq.destroy();
+    if (logger.isDebugEnabled()) {
+      logger.debug("Destroyed aeqId=" + aeqId);
+    }
+  }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/ce5523f6/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/RawLuceneRepositoryManager.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/RawLuceneRepositoryManager.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/RawLuceneRepositoryManager.java
index c31f19c..b9f4de8 100755
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/RawLuceneRepositoryManager.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/RawLuceneRepositoryManager.java
@@ -28,12 +28,6 @@ public class RawLuceneRepositoryManager extends AbstractPartitionedRepositoryMan
     super(index, serializer);
   }
 
-  public void close() {
-    for (IndexRepository repo : indexRepositories.values()) {
-      repo.cleanup();
-    }
-  }
-
   @Override
   protected IndexRepository getRepository(Integer bucketId) throws BucketNotFoundException {
     IndexRepository repo = indexRepositories.get(bucketId);

http://git-wip-us.apache.org/repos/asf/geode/blob/ce5523f6/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/repository/RepositoryManager.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/repository/RepositoryManager.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/repository/RepositoryManager.java
index b569d70..7f4a6c6 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/repository/RepositoryManager.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/repository/RepositoryManager.java
@@ -39,4 +39,9 @@ public interface RepositoryManager {
    */
   Collection<IndexRepository> getRepositories(RegionFunctionContext context)
       throws BucketNotFoundException;
+
+  /**
+   * Closes this {@link RepositoryManager}
+   */
+  void close();
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/ce5523f6/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
index 6260075..1afde6a 100644
--- a/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
+++ b/geode-lucene/src/test/java/org/apache/geode/cache/lucene/LuceneIndexDestroyDUnitTest.java
@@ -23,6 +23,7 @@ import org.apache.geode.cache.lucene.test.TestObject;
 import org.apache.geode.internal.cache.LocalRegion;
 import org.apache.geode.test.dunit.AsyncInvocation;
 import org.apache.geode.test.dunit.SerializableRunnableIF;
+import org.apache.geode.test.dunit.ThreadUtils;
 import org.apache.geode.test.junit.categories.DistributedTest;
 import org.awaitility.Awaitility;
 import org.junit.Ignore;
@@ -30,6 +31,7 @@ import org.junit.Test;
 import org.junit.experimental.categories.Category;
 import org.junit.runner.RunWith;
 
+import java.io.InterruptedIOException;
 import java.util.concurrent.TimeUnit;
 
 import static org.apache.geode.cache.lucene.test.LuceneTestUtilities.INDEX_NAME;
@@ -38,6 +40,7 @@ import static org.apache.geode.internal.Assert.fail;
 import static org.junit.Assert.assertFalse;
 import static org.junit.Assert.assertNotNull;
 import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
 
 @Category(DistributedTest.class)
 @RunWith(JUnitParamsRunner.class)
@@ -124,7 +127,7 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
     dataStore2.invoke(() -> verifyIndexCreated());
 
     // Start puts
-    AsyncInvocation putter = dataStore1.invokeAsync(() -> doPuts());
+    AsyncInvocation putter = dataStore1.invokeAsync(() -> doPutsUntilStopped());
 
     // Wait until puts have started
     dataStore1.invoke(() -> waitUntilPutsHaveStarted());
@@ -141,6 +144,42 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
     putter.join();
   }
 
+  @Test
+  @Parameters(method = "getListOfRegionTestTypes")
+  public void verifyDestroyRecreateSingleIndex(RegionTestableType regionType) {
+    // Create index and region
+    dataStore1.invoke(() -> initDataStore(createIndex(), regionType));
+    dataStore2.invoke(() -> initDataStore(createIndex(), regionType));
+
+    // Verify index created
+    dataStore1.invoke(() -> verifyIndexCreated());
+    dataStore2.invoke(() -> verifyIndexCreated());
+
+    // Do puts to cause IndexRepositories to be created
+    dataStore1.invoke(() -> doPuts(10));
+
+    // Destroy indexes (only needs to be done on one member)
+    dataStore1.invoke(() -> destroyIndexes());
+
+    // Verify indexes destroyed
+    dataStore1.invoke(() -> verifyIndexesDestroyed());
+    dataStore2.invoke(() -> verifyIndexesDestroyed());
+
+    // Destroy data region
+    dataStore1.invoke(() -> destroyDataRegion(true));
+
+    // Recreate index and region
+    dataStore1.invoke(() -> initDataStore(createIndex(), regionType));
+    dataStore2.invoke(() -> initDataStore(createIndex(), regionType));
+
+    // Do puts to cause IndexRepositories to be recreated
+    dataStore1.invoke(() -> doPuts(10));
+
+    // Wait until queue is flushed
+    // This verifies there are no deadlocks
+    dataStore1.invoke(() -> waitUntilFlushed(INDEX_NAME));
+    dataStore2.invoke(() -> waitUntilFlushed(INDEX_NAME));
+  }
 
   private SerializableRunnableIF createIndex() {
     return () -> {
@@ -168,7 +207,20 @@ public class LuceneIndexDestroyDUnitTest extends LuceneDUnitTest {
     assertNotNull(luceneService.getIndex(INDEX_NAME + "1", REGION_NAME));
   }
 
-  private void doPuts() throws Exception {
+  private void waitUntilFlushed(String indexName) throws Exception {
+    LuceneService luceneService = LuceneServiceProvider.get(getCache());
+    assertTrue(
+        luceneService.waitUntilFlushed(indexName, REGION_NAME, 30000, TimeUnit.MILLISECONDS));
+  }
+
+  private void doPuts(int numPuts) throws Exception {
+    Region region = getCache().getRegion(REGION_NAME);
+    for (int i = 0; i < numPuts; i++) {
+      region.put(i, new TestObject());
+    }
+  }
+
+  private void doPutsUntilStopped() throws Exception {
     Region region = getCache().getRegion(REGION_NAME);
     int i = 0;
     while (!STOP_PUTS) {


[28/49] geode git commit: GEODE-2620: Rate stats removed from LuceneIndexMetrics

Posted by kh...@apache.org.
GEODE-2620: Rate stats removed from LuceneIndexMetrics

	* Commit Rate, update rate and query rate removed from LuceneIndexMetrics
	* They can be added back again when the getRate method in StatsRate.java is fixed

	This closes #418


Project: http://git-wip-us.apache.org/repos/asf/geode/repo
Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/5e200db2
Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/5e200db2
Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/5e200db2

Branch: refs/heads/feature/GEODE-2420
Commit: 5e200db252352b84a36dce1470ae5f0e7f087379
Parents: 016f07a
Author: nabarun <nn...@pivotal.io>
Authored: Tue Mar 7 15:05:47 2017 -0800
Committer: Ken Howe <kh...@pivotal.io>
Committed: Fri Mar 17 13:09:45 2017 -0700

----------------------------------------------------------------------
 .../management/LuceneIndexStatsMonitor.java     | 22 ++-----
 .../lucene/management/LuceneIndexMetrics.java   | 66 ++++----------------
 2 files changed, 17 insertions(+), 71 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/geode/blob/5e200db2/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/management/LuceneIndexStatsMonitor.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/management/LuceneIndexStatsMonitor.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/management/LuceneIndexStatsMonitor.java
index 8d7d7ca..26eeff6 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/management/LuceneIndexStatsMonitor.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/internal/management/LuceneIndexStatsMonitor.java
@@ -24,16 +24,10 @@ import org.apache.geode.management.internal.beans.stats.StatsRate;
 
 public class LuceneIndexStatsMonitor extends MBeanStatsMonitor {
 
-  private StatsRate updateRate;
-
   private StatsAverageLatency updateRateAverageLatency;
 
-  private StatsRate commitRate;
-
   private StatsAverageLatency commitRateAverageLatency;
 
-  private StatsRate queryRate;
-
   private StatsAverageLatency queryRateAverageLatency;
 
   public static final String LUCENE_SERVICE_MXBEAN_MONITOR_PREFIX = "LuceneServiceMXBeanMonitor_";
@@ -45,11 +39,6 @@ public class LuceneIndexStatsMonitor extends MBeanStatsMonitor {
   }
 
   private void configureMetrics() {
-    this.queryRate = new StatsRate(StatsKey.QUERIES, StatType.INT_TYPE, this);
-
-    this.updateRate = new StatsRate(StatsKey.UPDATES, StatType.INT_TYPE, this);
-
-    this.commitRate = new StatsRate(StatsKey.COMMITS, StatType.INT_TYPE, this);
 
     this.queryRateAverageLatency =
         new StatsAverageLatency(StatsKey.QUERIES, StatType.INT_TYPE, StatsKey.QUERY_TIME, this);
@@ -64,29 +53,26 @@ public class LuceneIndexStatsMonitor extends MBeanStatsMonitor {
   protected LuceneIndexMetrics getIndexMetrics(LuceneIndex index) {
     int queryExecutions = getStatistic(StatsKey.QUERIES).intValue();
     long queryExecutionTime = getStatistic(StatsKey.QUERY_TIME).longValue();
-    float queryRateValue = this.queryRate.getRate();
     long queryRateAverageLatencyValue = this.queryRateAverageLatency.getAverageLatency();
     int queryExecutionsInProgress = getStatistic(StatsKey.QUERIES_IN_PROGRESS).intValue();
     long queryExecutionTotalHits = getStatistic(StatsKey.QUERIES_TOTAL_HITS).longValue();
 
     int updates = getStatistic(StatsKey.UPDATES).intValue();
     long updateTime = getStatistic(StatsKey.UPDATE_TIME).longValue();
-    float updateRateValue = this.updateRate.getRate();
     long updateRateAverageLatencyValue = this.updateRateAverageLatency.getAverageLatency();
     int updatesInProgress = getStatistic(StatsKey.UPDATES_IN_PROGRESS).intValue();
 
     int commits = getStatistic(StatsKey.COMMITS).intValue();
     long commitTime = getStatistic(StatsKey.COMMIT_TIME).longValue();
-    float commitRateValue = this.commitRate.getRate();
     long commitRateAverageLatencyValue = this.commitRateAverageLatency.getAverageLatency();
     int commitsInProgress = getStatistic(StatsKey.COMMITS_IN_PROGRESS).intValue();
 
     int documents = getStatistic(StatsKey.DOCUMENTS).intValue();
 
     return new LuceneIndexMetrics(index.getRegionPath(), index.getName(), queryExecutions,
-        queryExecutionTime, queryRateValue, queryRateAverageLatencyValue, queryExecutionsInProgress,
-        queryExecutionTotalHits, updates, updateTime, updateRateValue,
-        updateRateAverageLatencyValue, updatesInProgress, commits, commitTime, commitRateValue,
-        commitRateAverageLatencyValue, commitsInProgress, documents);
+        queryExecutionTime, queryRateAverageLatencyValue, queryExecutionsInProgress,
+        queryExecutionTotalHits, updates, updateTime, updateRateAverageLatencyValue,
+        updatesInProgress, commits, commitTime, commitRateAverageLatencyValue, commitsInProgress,
+        documents);
   }
 }

http://git-wip-us.apache.org/repos/asf/geode/blob/5e200db2/geode-lucene/src/main/java/org/apache/geode/cache/lucene/management/LuceneIndexMetrics.java
----------------------------------------------------------------------
diff --git a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/management/LuceneIndexMetrics.java b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/management/LuceneIndexMetrics.java
index 24d14c1..1d7984d 100644
--- a/geode-lucene/src/main/java/org/apache/geode/cache/lucene/management/LuceneIndexMetrics.java
+++ b/geode-lucene/src/main/java/org/apache/geode/cache/lucene/management/LuceneIndexMetrics.java
@@ -28,8 +28,6 @@ public class LuceneIndexMetrics {
 
   private final long queryExecutionTime;
 
-  private final float queryRate;
-
   private final long queryRateAverageLatency;
 
   private final int queryExecutionsInProgress;
@@ -40,8 +38,6 @@ public class LuceneIndexMetrics {
 
   private final long updateTime;
 
-  private final float updateRate;
-
   private final long updateRateAverageLatency;
 
   private final int updatesInProgress;
@@ -50,8 +46,6 @@ public class LuceneIndexMetrics {
 
   private final long commitTime;
 
-  private final float commitRate;
-
   private final long commitRateAverageLatency;
 
   private final int commitsInProgress;
@@ -63,32 +57,27 @@ public class LuceneIndexMetrics {
    * an instance of this class.
    */
   @ConstructorProperties({"regionPath", "indexName", "queryExecutions", "queryExecutionTime",
-      "queryRate", "queryRateAverageLatency", "queryExecutionsInProgress",
-      "queryExecutionTotalHits", "updates", "updateTime", "updateRate", "updateRateAverageLatency",
-      "updatesInProgress", "commits", "commitTime", "commitRate", "commitRateAverageLatency",
-      "commitsInProgress", "documents"})
+      "queryRateAverageLatency", "queryExecutionsInProgress", "queryExecutionTotalHits", "updates",
+      "updateTime", "updateRateAverageLatency", "updatesInProgress", "commits", "commitTime",
+      "commitRateAverageLatency", "commitsInProgress", "documents"})
   public LuceneIndexMetrics(String regionPath, String indexName, int queryExecutions,
-      long queryExecutionTime, float queryRate, long queryRateAverageLatency,
-      int queryExecutionsInProgress, long queryExecutionTotalHits, int updates, long updateTime,
-      float updateRate, long updateRateAverageLatency, int updatesInProgress, int commits,
-      long commitTime, float commitRate, long commitRateAverageLatency, int commitsInProgress,
-      int documents) {
+      long queryExecutionTime, long queryRateAverageLatency, int queryExecutionsInProgress,
+      long queryExecutionTotalHits, int updates, long updateTime, long updateRateAverageLatency,
+      int updatesInProgress, int commits, long commitTime, long commitRateAverageLatency,
+      int commitsInProgress, int documents) {
     this.regionPath = regionPath;
     this.indexName = indexName;
     this.queryExecutions = queryExecutions;
     this.queryExecutionTime = queryExecutionTime;
-    this.queryRate = queryRate;
     this.queryRateAverageLatency = queryRateAverageLatency;
     this.queryExecutionsInProgress = queryExecutionsInProgress;
     this.queryExecutionTotalHits = queryExecutionTotalHits;
     this.updates = updates;
     this.updateTime = updateTime;
-    this.updateRate = updateRate;
     this.updateRateAverageLatency = updateRateAverageLatency;
     this.updatesInProgress = updatesInProgress;
     this.commits = commits;
     this.commitTime = commitTime;
-    this.commitRate = commitRate;
     this.commitRateAverageLatency = commitRateAverageLatency;
     this.commitsInProgress = commitsInProgress;
     this.documents = documents;
@@ -132,15 +121,6 @@ public class LuceneIndexMetrics {
   }
 
   /**
-   * Returns the rate of query execution using the {@link LuceneIndex}
-   *
-   * @return the rate of query execution using the Lucene Index
-   */
-  public float getQueryRate() {
-    return this.queryRate;
-  }
-
-  /**
    * Returns the average latency for query executions using the {@link LuceneIndex}
    *
    * @return the average latency for query executions in nanoseconds using the Lucene Index
@@ -186,15 +166,6 @@ public class LuceneIndexMetrics {
   }
 
   /**
-   * Returns the rate at which update operations are executed on the {@link LuceneIndex}
-   *
-   * @return rate at which update operations are executed on the {@link LuceneIndex}
-   */
-  public float getUpdateRate() {
-    return this.updateRate;
-  }
-
-  /**
    * Returns the average latency for the update operations on the {@link LuceneIndex}
    *
    * @return the average latency for the update operations in nanoseconds on the Lucene Index
@@ -231,15 +202,6 @@ public class LuceneIndexMetrics {
   }
 
   /**
-   * Returns the rate of commit operations on the {@link LuceneIndex}
-   *
-   * @return the rate of commit operations on the Lucene Index
-   */
-  public float getCommitRate() {
-    return this.commitRate;
-  }
-
-  /**
    * Returns the average latency for the commit operations using the {@link LuceneIndex}
    *
    * @return Returns the average latency for the commit operations in nanoseconds using the Lucene
@@ -277,16 +239,14 @@ public class LuceneIndexMetrics {
     return new StringBuilder().append(getClass().getSimpleName()).append("[").append("regionPath=")
         .append(this.regionPath).append("; indexName=").append(this.indexName)
         .append("; queryExecutions=").append(this.queryExecutions).append("; queryExecutionTime=")
-        .append(this.queryExecutionTime).append("; queryRate=").append(this.queryRate)
-        .append("; queryRateAverageLatency=").append(this.queryRateAverageLatency)
-        .append("; queryExecutionsInProgress=").append(this.queryExecutionsInProgress)
-        .append("; queryExecutionTotalHits=").append(this.queryExecutionTotalHits)
-        .append("; updates=").append(this.updates).append("; updateTime=").append(this.updateTime)
-        .append("; updateRate=").append(this.updateRate).append("; updateRateAverageLatency=")
+        .append(this.queryExecutionTime).append("; queryRateAverageLatency=")
+        .append(this.queryRateAverageLatency).append("; queryExecutionsInProgress=")
+        .append(this.queryExecutionsInProgress).append("; queryExecutionTotalHits=")
+        .append(this.queryExecutionTotalHits).append("; updates=").append(this.updates)
+        .append("; updateTime=").append(this.updateTime).append("; updateRateAverageLatency=")
         .append(this.updateRateAverageLatency).append("; updatesInProgress=")
         .append(this.updatesInProgress).append("; commits=").append(this.commits)
-        .append("; commitTime=").append(this.commitTime).append("; commitRate=")
-        .append(this.commitRate).append("; commitRateAverageLatency=")
+        .append("; commitTime=").append(this.commitTime).append("; commitRateAverageLatency=")
         .append(this.commitRateAverageLatency).append("; commitsInProgress=")
         .append(this.commitsInProgress).append("; documents=").append(this.documents).append("]")
         .toString();