You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@rya.apache.org by meiercaleb <gi...@git.apache.org> on 2016/12/15 22:10:27 UTC
[GitHub] incubator-rya pull request #127: Rya 226 config builders
GitHub user meiercaleb opened a pull request:
https://github.com/apache/incubator-rya/pull/127
Rya 226 config builders
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
## Description
>What Changed?
Added config builders for Accumulo and Mongo configuration
### Tests
Tests were added for all new builders, and all existing integration tests passed.
Basic tests were added to verify that the correct configuration values are added
through each of the builder objects.
### Links
[Jira](https://issues.apache.org/jira/browse/226)
### Checklist
- [ ] Code Review
- [x] Squash Commits
#### People To Reivew
Aaron Mihalik, Puja Valiyil
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/meiercaleb/incubator-rya RYA-226-ConfigBuilders
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rya/pull/127.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #127
----
commit e5b315f0bcb4a1ec4aba26a258d26c3a42ace960
Author: Caleb Meier <me...@gmail.com>
Date: 2016-11-22T16:39:09Z
RYA-224 Fluo Visibility Join Bug Fix
commit 6bafa7419044cfbfff780aab8fcbbaeca648b9a6
Author: Caleb Meier <ca...@parsons.com>
Date: 2016-11-30T17:07:33Z
RYA-222-Fixed Column Visibility Bug for Results Streamed into Fluo
commit 426e35cf43281225497216de6517aa8592f58a39
Author: Caleb Meier <ca...@parsons.com>
Date: 2016-12-08T16:56:03Z
RYA-226-ConfigBuilders, de-duplication of some config keys, and config documentation
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110947663
--- Diff: extras/indexing/src/main/java/org/apache/rya/indexing/accumulo/ConfigUtils.java ---
@@ -66,21 +67,21 @@
import com.google.common.collect.Lists;
/**
- * A set of configuration utils to read a Hadoop {@link Configuration} object and create Cloudbase/Accumulo objects.
- * Soon will deprecate this class. Use installer for the set methods, use {@link RyaDetails} for the get methods.
- * New code must separate parameters that are set at Rya install time from that which is specific to the client.
- * Also Accumulo index tables are pushed down to the implementation and not configured in conf.
+ * A set of configuration utils to read a Hadoop {@link Configuration} object
+ * and create Cloudbase/Accumulo objects. Soon will deprecate this class. Use
+ * installer for the set methods, use {@link RyaDetails} for the get methods.
+ * New code must separate parameters that are set at Rya install time from that
+ * which is specific to the client. Also Accumulo index tables are pushed down
+ * to the implementation and not configured in conf.
*/
public class ConfigUtils {
private static final Logger logger = Logger.getLogger(ConfigUtils.class);
- public static final String CLOUDBASE_TBL_PREFIX = "sc.cloudbase.tableprefix";
- public static final String CLOUDBASE_AUTHS = "sc.cloudbase.authorizations";
- public static final String CLOUDBASE_INSTANCE = "sc.cloudbase.instancename";
- public static final String CLOUDBASE_ZOOKEEPERS = "sc.cloudbase.zookeepers";
- public static final String CLOUDBASE_USER = "sc.cloudbase.username";
- public static final String CLOUDBASE_PASSWORD = "sc.cloudbase.password";
-
+ public static final String CLOUDBASE_INSTANCE = AccumuloRdfConfiguration.CLOUDBASE_INSTANCE;
--- End diff --
You should mark all of the moved static variables as deprecated and indicate where they have been moved to in the deprecation documentation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110981101
--- Diff: extras/indexing/src/main/java/org/apache/rya/indexing/accumulo/ConfigUtils.java ---
@@ -66,21 +67,21 @@
import com.google.common.collect.Lists;
/**
- * A set of configuration utils to read a Hadoop {@link Configuration} object and create Cloudbase/Accumulo objects.
- * Soon will deprecate this class. Use installer for the set methods, use {@link RyaDetails} for the get methods.
- * New code must separate parameters that are set at Rya install time from that which is specific to the client.
- * Also Accumulo index tables are pushed down to the implementation and not configured in conf.
+ * A set of configuration utils to read a Hadoop {@link Configuration} object
+ * and create Cloudbase/Accumulo objects. Soon will deprecate this class. Use
+ * installer for the set methods, use {@link RyaDetails} for the get methods.
+ * New code must separate parameters that are set at Rya install time from that
+ * which is specific to the client. Also Accumulo index tables are pushed down
+ * to the implementation and not configured in conf.
*/
public class ConfigUtils {
private static final Logger logger = Logger.getLogger(ConfigUtils.class);
- public static final String CLOUDBASE_TBL_PREFIX = "sc.cloudbase.tableprefix";
- public static final String CLOUDBASE_AUTHS = "sc.cloudbase.authorizations";
- public static final String CLOUDBASE_INSTANCE = "sc.cloudbase.instancename";
- public static final String CLOUDBASE_ZOOKEEPERS = "sc.cloudbase.zookeepers";
- public static final String CLOUDBASE_USER = "sc.cloudbase.username";
- public static final String CLOUDBASE_PASSWORD = "sc.cloudbase.password";
-
+ public static final String CLOUDBASE_INSTANCE = AccumuloRdfConfiguration.CLOUDBASE_INSTANCE;
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92824177
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRdfConfiguration.java ---
@@ -19,101 +19,133 @@
* under the License.
*/
-
-
import java.util.List;
-
-import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
-import org.apache.rya.api.persist.index.RyaSecondaryIndexer;
+import java.util.Properties;
import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
import com.google.common.collect.Lists;
import com.mongodb.MongoClient;
public class MongoDBRdfConfiguration extends RdfCloudTripleStoreConfiguration {
- public static final String MONGO_INSTANCE = "mongo.db.instance";
- public static final String MONGO_INSTANCE_PORT = "mongo.db.port";
- public static final String MONGO_GEO_MAXDISTANCE = "mongo.geo.maxdist";
- public static final String MONGO_DB_NAME = "mongo.db.name";
- public static final String MONGO_COLLECTION_PREFIX = "mongo.db.collectionprefix";
- public static final String MONGO_USER = "mongo.db.user";
- public static final String MONGO_USER_PASSWORD = "mongo.db.userpassword";
- public static final String CONF_ADDITIONAL_INDEXERS = "ac.additional.indexers";
+ public static final String MONGO_INSTANCE = "mongo.db.instance";
+ public static final String MONGO_INSTANCE_PORT = "mongo.db.port";
--- End diff --
tabs
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110960719
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/mongo/MongoIndexingConfigurationTest.java ---
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.mongo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.apache.rya.indexing.mongodb.MongoIndexingConfiguration;
+import org.apache.rya.mongodb.MongoDBRdfConfiguration;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoIndexingConfiguration conf = MongoIndexingConfiguration.builder()//
+ .setVisibilities(visibility)//
+ .setUseInference(useInference)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setUseMockMongo(useMock)//
+ .setMongoCollectionPrefix(prefix)//
+ .setMongoDBName("dbname")//
+ .setMongoHost("host")//
+ .setMongoPort("1000")//
+ .setAuths(auth)//
+ .setMongoUser(user)//
+ .setMongoPassword(password)//
+ .setUseMongoEntityIndex(true)//
+ .setUseMongoFreetextIndex(true)//
+ .setUseMongoTemporalIndex(true)//
+ .setMongoFreeTextPredicates("http://pred1", "http://pred2")//
+ .setMongoTemporalPredicates("http://pred3", "http://pred4")//
+ .build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110958786
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110960196
--- Diff: extras/indexingExample/src/main/java/MongoRyaDirectExample.java ---
@@ -268,8 +270,8 @@ private static void testAddAndFreeTextSearchWithPCJ(SailRepositoryConnection con
private static MockMongoFactory mock = null;
private static Configuration getConf() throws IOException {
- MongoDBRdfConfiguration conf = new MongoDBRdfConfiguration();
- conf.set(ConfigUtils.USE_MONGO, "true");
+ MongoDBIndexingConfigBuilder builder = MongoIndexingConfiguration.builder()//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110946839
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/mongo/MongoIndexingConfigurationTest.java ---
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.mongo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.apache.rya.indexing.mongodb.MongoIndexingConfiguration;
+import org.apache.rya.mongodb.MongoDBRdfConfiguration;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoIndexingConfiguration conf = MongoIndexingConfiguration.builder()//
+ .setVisibilities(visibility)//
+ .setUseInference(useInference)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setUseMockMongo(useMock)//
+ .setMongoCollectionPrefix(prefix)//
+ .setMongoDBName("dbname")//
+ .setMongoHost("host")//
+ .setMongoPort("1000")//
+ .setAuths(auth)//
+ .setMongoUser(user)//
+ .setMongoPassword(password)//
+ .setUseMongoEntityIndex(true)//
+ .setUseMongoFreetextIndex(true)//
+ .setUseMongoTemporalIndex(true)//
+ .setMongoFreeTextPredicates("http://pred1", "http://pred2")//
+ .setMongoTemporalPredicates("http://pred3", "http://pred4")//
+ .build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
Nit: The assert methods could be statically imported.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya issue #127: Rya 226 config builders
Posted by pujav65 <gi...@git.apache.org>.
Github user pujav65 commented on the issue:
https://github.com/apache/incubator-rya/pull/127
could you please fix the conflict i'll look at merging this then
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92824142
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/AbstractMongoDBRdfConfigurationBuilder.java ---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Mongo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to a Mongo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractMongoDBRdfConfigurationBuilder<B extends AbstractMongoDBRdfConfigurationBuilder<B, C>, C extends MongoDBRdfConfiguration>
+ extends RdfCloudTripleStoreConfigurationBuilder<B, C> {
+
+ private String user;
+ private String pass;
+ private boolean useMock = false;
+ private String host = "localhost";
+ private String port = DEFAULT_MONGO_PORT;
+ protected static final String DEFAULT_MONGO_PORT = "27017";
+ private String mongoCollectionPrefix = "rya_";
+ private String mongoDBName = "rya";
+
+ protected static final String MONGO_USER = "mongo.user";
+ protected static final String MONGO_PASSWORD = "mongo.password";
+ protected static final String MONGO_DB_NAME = "mongo.db.name";
+ protected static final String MONGO_COLLECTION_PREFIX = "mongo.collection.prefix";
+ protected static final String MONGO_HOST = "mongo.host";
+ protected static final String MONGO_PORT = "mongo.port";
+ protected static final String MONGO_AUTHS = "mongo.auths";
+ protected static final String MONGO_VISIBILITIES = "mongo.visibilities";
+ protected static final String MONGO_RYA_PREFIX = "mongo.rya.prefix";
+ protected static final String USE_INFERENCE = "use.inference";
+ protected static final String USE_DISPLAY_QUERY_PLAN = "use.display.plan";
+ protected static final String USE_MOCK_MONGO = "use.mock";
+
+ /**
+ * Sets Mongo user.
+ *
+ * @param user
+ * @return specified builder
+ */
+ public B setMongoUser(String user) {
+ this.user = user;
+ return confBuilder();
+ }
+
+ /**
+ * Sets password for Mongo user specified by
+ * {@link AbstractMongoDBRdfConfigurationBuilder#setMongoUser(String)}.
+ *
+ * @param password
+ * @return specified builder
+ */
+ public B setMongoPassword(String password) {
+ this.pass = password;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Mongo port. This parameter must be set to connect to an instance of
+ * MongoDB and will default to "27017" if no value is specified.
+ *
+ * @param port
+ * @return specified builder
+ */
+ public B setMongoPort(String port) {
+ Preconditions.checkNotNull(port, "Mongo port cannot be null.");
+ this.port = port;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Mongo host. This parameter must be set to connect to an instance of
+ * MongoDB and will default to "localhost" if no value is specified.
+ *
+ * @param host
+ * @return specified builder
+ */
+ public B setMongoHost(String host) {
+ Preconditions.checkNotNull(host, "Mongo host cannot be null.");
+ this.host = host;
+ return confBuilder();
+ }
+
+ /**
+ * Sets MongoDB name. This parameter must be set to connect to an instance
+ * of MongoDB and will default to "rya_triples" is no value is specified.
+ *
+ * @param name
+ * @return specified builder
+ */
+ public B setMongoDBName(String name) {
+ Preconditions.checkNotNull(name, "Mongo DBName cannot be null.");
+ this.mongoDBName = name;
+ return confBuilder();
+ }
+
+ /**
+ * Sets MongoDB Collection prefix. This parameter must be set to connect to
+ * an instance of MongoDB and will default to "rya_" is no value is
+ * specified.
+ *
+ * @param name
+ * @return specified builder
+ */
+ public B setMongoCollectionPrefix(String prefix) {
+ Preconditions.checkNotNull(prefix, "Mongo Collection Prefix cannot be null.");
+ this.mongoCollectionPrefix = prefix;
+ return confBuilder();
+ }
+
+ /**
+ * Set whether to use instance of embedded Mongo as backend for Rya
+ * instance.
+ *
+ * @param useMock
+ * @return specified builder
+ */
+ public B useMockMongo(boolean useMock) {
+ this.useMock = useMock;
+ return confBuilder();
+ }
+
+ public C build() {
+ return getConf(super.build());
+ }
+
+ private C getConf(C conf) {
--- End diff --
Same comment as the parent
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92824316
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/AbstractMongoDBRdfConfigurationBuilder.java ---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Mongo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to a Mongo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractMongoDBRdfConfigurationBuilder<B extends AbstractMongoDBRdfConfigurationBuilder<B, C>, C extends MongoDBRdfConfiguration>
--- End diff --
Same question about this being abstract instead of concrete
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92822108
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
+ this.visibilities = visibilites;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the prefix for the Rya instance to connect to. This parameter is
+ * required and the default value is "rya_"
+ *
+ * @param prefix
+ * - the prefix for the Rya instance
+ * @return B - concrete builder class
+ */
+ public B setRyaPrefix(String prefix) {
+ Preconditions.checkNotNull(prefix, "Rya prefix cannot be null.");
+ this.prefix = prefix;
+ return confBuilder();
+ }
+
+ /**
+ * Set whether to display query plan during optimization. The default value
+ * is false.
+ *
+ * @param displayPlan
+ * - display the parsed query plan during query evaluation
+ * (useful for debugging)
+ * @return B - concrete builder class
+ */
+ public B displayQueryPlan(boolean displayPlan) {
+ this.displayPlan = displayPlan;
+ return confBuilder();
+ }
+
+ public C build() {
+ return getConf(createConf());
+ }
+
+ private C getConf(C conf) {
+
+ Preconditions.checkNotNull(auths, "Authorizations cannot be null");
+
+ conf.setInfer(useInference);
+ conf.setTablePrefix(prefix);
+ conf.setInt("sc.cloudbase.numPartitions", 3);
+ conf.set(RdfCloudTripleStoreConfiguration.CONF_QUERY_AUTH, auths);
+ if (visibilities != null) {
+ conf.set(RdfCloudTripleStoreConfiguration.CONF_CV, visibilities);
--- End diff --
Same comment for visibilities..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110946929
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/accumulo/AccumuloIndexingConfigurationTest.java ---
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Properties;
+import java.util.Set;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.api.domain.RyaURI;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class AccumuloIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
--- End diff --
Nit: Unnecessary "//" characters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110961749
--- Diff: dao/accumulo.rya/src/test/java/org/apache/rya/accumulo/AccumuloRdfConfigurationTest.java ---
@@ -72,4 +76,85 @@ public void testIterators() {
assertEquals(setting, iteratorSettings[0]);
}
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration().getBuilder()//
+ .setAuths(auth)//
+ .setVisibilities(visibility)//
+ .setRyaPrefix(prefix)//
+ .setUseInference(useInference)//
+ .setUseCompositeCardinality(useComposite)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setAccumuloInstance(instance)//
+ .setAccumuloPassword(password)//
+ .setAccumuloUser(user)//
+ .setAccumuloZooKeepers(zookeeper)//
+ .setUseMockAccumulo(useMock)//
+ .setUseAccumuloPrefixHashing(usePrefixHash)//
+ .build();
+
+ org.junit.Assert.assertEquals(conf.getTablePrefix(), prefix);
+ org.junit.Assert.assertEquals(conf.getCv(), visibility);
+ org.junit.Assert.assertEquals(conf.getAuthorizations(), new Authorizations(auth));
+ org.junit.Assert.assertEquals(conf.isInfer(), useInference);
+ org.junit.Assert.assertEquals(conf.isUseCompositeCardinality(), useComposite);
+ org.junit.Assert.assertEquals(conf.isDisplayQueryPlan(), displayPlan);
+ org.junit.Assert.assertEquals(conf.getAccumuloInstance(), instance);
+ org.junit.Assert.assertEquals(conf.getAccumuloPassword(), password);
+ org.junit.Assert.assertEquals(conf.getAccumuloUser(), user);
+ org.junit.Assert.assertEquals(conf.getAccumuloZookeepers(), zookeeper);
+ org.junit.Assert.assertEquals(conf.getUseMockAccumulo(), useMock);
+ org.junit.Assert.assertEquals(conf.isPrefixRowsWithHash(), usePrefixHash);
+
+ }
+
+ @Test
+ public void testBuilderFromProperties() throws FileNotFoundException, IOException {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ Properties props = new Properties();
+ props.load(new FileInputStream("src/test/resources/properties/rya.properties"));
+
+ AccumuloRdfConfiguration conf = AccumuloRdfConfiguration.fromProperties(props);
+
+ org.junit.Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111020688
--- Diff: extras/indexingExample/src/main/java/RyaDirectExample.java ---
@@ -789,7 +789,7 @@ private static void createPCJ(final Configuration conf)
for(Map.Entry<Key,Value> entry: scanner) {
System.out.println("PCJ entry :" + entry.getKey() );
}
-
+ scanner.close();
--- End diff --
This needs to happen in a finally block to ensure it'll be closed even if an exception is thrown.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92822062
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
+ this.visibilities = visibilites;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the prefix for the Rya instance to connect to. This parameter is
+ * required and the default value is "rya_"
+ *
+ * @param prefix
+ * - the prefix for the Rya instance
+ * @return B - concrete builder class
+ */
+ public B setRyaPrefix(String prefix) {
+ Preconditions.checkNotNull(prefix, "Rya prefix cannot be null.");
+ this.prefix = prefix;
+ return confBuilder();
+ }
+
+ /**
+ * Set whether to display query plan during optimization. The default value
+ * is false.
+ *
+ * @param displayPlan
+ * - display the parsed query plan during query evaluation
+ * (useful for debugging)
+ * @return B - concrete builder class
+ */
+ public B displayQueryPlan(boolean displayPlan) {
+ this.displayPlan = displayPlan;
+ return confBuilder();
+ }
+
+ public C build() {
+ return getConf(createConf());
+ }
+
+ private C getConf(C conf) {
+
+ Preconditions.checkNotNull(auths, "Authorizations cannot be null");
+
+ conf.setInfer(useInference);
+ conf.setTablePrefix(prefix);
+ conf.setInt("sc.cloudbase.numPartitions", 3);
+ conf.set(RdfCloudTripleStoreConfiguration.CONF_QUERY_AUTH, auths);
--- End diff --
RdfCloudTripleStoreConfiguration shoudl implement a method for setting query auths. Within that method, do the preconditions check instead of within this builder.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111402455
--- Diff: extras/rya.pcj.fluo/pcj.fluo.api/src/main/java/org/apache/rya/indexing/pcj/fluo/api/CreatePcj.java ---
@@ -162,6 +162,7 @@ public String withRyaIntegration(final String pcjId, final PrecomputedJoinStora
final ParsedQuery parsedQuery = new SPARQLParser().parseQuery(sparql, null);
final FluoQuery fluoQuery = new SparqlFluoQueryBuilder().make(parsedQuery, nodeIds);
+<<<<<<< HEAD
--- End diff --
I think you have old merge resolution text in this class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92823465
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
+ extends RdfCloudTripleStoreConfigurationBuilder<B, C> {
+
+ private String user;
+ private String pass;
+ private String zoo = "";
+ private String instance;
+ private boolean useMock = false;
+ private boolean usePrefixHashing = false;
+ private boolean useComposite = false;
+ private boolean useSelectivity = false;
+
+ protected static final String ACCUMULO_USER = "accumulo.user";
+ protected static final String ACCUMULO_PASSWORD = "accumulo.password";
+ protected static final String ACCUMULO_INSTANCE = "accumulo.instance";
+ protected static final String ACCUMULO_AUTHS = "accumulo.auths";
+ protected static final String ACCUMULO_VISIBILITIES = "accumulo.visibilities";
+ protected static final String ACCUMULO_ZOOKEEPERS = "accumulo.zookeepers";
+ protected static final String ACCUMULO_RYA_PREFIX = "accumulo.rya.prefix";
+ protected static final String USE_INFERENCE = "use.inference";
+ protected static final String USE_DISPLAY_QUERY_PLAN = "use.display.plan";
+ protected static final String USE_MOCK_ACCUMULO = "use.mock";
+ protected static final String USE_PREFIX_HASHING = "use.prefix.hashing";
+ protected static final String USE_COUNT_STATS = "use.count.stats";
+ protected static final String USE_JOIN_SELECTIVITY = "use.join.selectivity";
+
+ /**
+ * Sets Accumulo user. This is a required parameter to connect to an
+ * Accumulo Rya Instance
+ *
+ * @param user
+ * - Accumulo user
+ */
+ public B setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user, "User cannot be null.");
+ this.user = user;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo user's password. This is a required parameter to connect to
+ * an Accumulo Rya Instance
+ *
+ * @param password
+ * - password for Accumulo user specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloUser(String)}
+ * @return specified builder
+ */
+ public B setAccumuloPassword(String password) {
+ Preconditions.checkNotNull(password, "Password cannot be null.");
+ this.pass = password;
+ return confBuilder();
+ }
+
+ /**
+ * Sets name of Accumulo instance containing Rya tables. This is a required
+ * parameter to connect to an Accumulo Rya instance.
+ *
+ * @param instance
+ * - Accumulo instance name
+ * @return specified builder
+ */
+ public B setAccumuloInstance(String instance) {
+ Preconditions.checkNotNull(instance, "Instance cannot be null.");
+ this.instance = instance;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo Zookeepers for instance specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloInstance(String)}
+ * . This is a required parameter if
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false.
+ *
+ * @param zoo
+ * - Accumuo Zookeepers
+ * @return specified builder
+ */
+ public B setAccumuloZooKeepers(String zoo) {
--- End diff --
Document the default empty string this builder uses.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92959221
--- Diff: extras/indexingExample/src/main/java/RyaClientExample.java ---
@@ -132,18 +133,17 @@ public static void main(final String[] args) throws Exception {
"}";
// Load some statements into the Rya instance.
- final AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();
- conf.setTablePrefix(ryaInstanceName);
- conf.set(ConfigUtils.CLOUDBASE_USER, accumuloUsername);
- conf.set(ConfigUtils.CLOUDBASE_PASSWORD, accumuloPassword);
- conf.set(ConfigUtils.CLOUDBASE_INSTANCE, cluster.getInstanceName());
- conf.set(ConfigUtils.CLOUDBASE_ZOOKEEPERS, cluster.getZooKeepers());
- conf.set(ConfigUtils.CLOUDBASE_AUTHS, "U");
- conf.set(ConfigUtils.USE_PCJ_FLUO_UPDATER, "true");
- conf.set(ConfigUtils.FLUO_APP_NAME, fluoAppName);
- conf.set(ConfigUtils.PCJ_STORAGE_TYPE, PrecomputedJoinIndexerConfig.PrecomputedJoinStorageType.ACCUMULO.toString());
- conf.set(ConfigUtils.PCJ_UPDATER_TYPE, PrecomputedJoinIndexerConfig.PrecomputedJoinUpdaterType.FLUO.toString());
-
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
+ .auths("U")
+ .setAccumuloUser(accumuloUsername)
--- End diff --
Right. Fixed my formatter to shift by spaces, but I didn't want to apply it to this class so that it would be clear what the diff was when you guys looked at my code. I have no problem fixing the tabs now that you've had a look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92821234
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
+ this.visibilities = visibilites;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the prefix for the Rya instance to connect to. This parameter is
+ * required and the default value is "rya_"
+ *
+ * @param prefix
+ * - the prefix for the Rya instance
+ * @return B - concrete builder class
+ */
+ public B setRyaPrefix(String prefix) {
--- End diff --
You should use a consistent method naming paradigm within your class. I think this is the only setter that says "set" on the start. Either have all of them say set, or none of them. This might also be a good opportunity to change "rya prefix" to "rya instance name" since the prefix is Accumulo specific.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92823960
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
+ * <li>"use.mock" - Use a MockAccumulo instance as back-end for Rya instance. Default is false.
+ * <li>"use.prefix.hashing" - Use prefix hashing for triples. Helps avoid hot-spotting. Default is false.
+ * <li>"use.count.stats" - Use triple pattern cardinalities for query optimization. Default is false.
+ * <li>"use.join.selectivity" - Use join selectivity for query optimization. Default is false.
+ * <li>"use.display.plan" - Display query plan during evaluation. Useful for debugging. Default is true.
+ * <li>"use.inference" - Use backward chaining inference during query evaluation. Default is false.
+ * </ul>
+ * <br>
+ * @param props - Properties file containing Accumulo specific configuration parameters
+ * @return AccumumuloRdfConfiguration with properties set
+ */
+
+ public static AccumuloRdfConfiguration fromProperties(Properties props) {
+ return AccumuloRdfConfigurationBuilder.fromProperties(props).build();
+ }
+
@Override
public AccumuloRdfConfiguration clone() {
return new AccumuloRdfConfiguration(this);
}
+
+ public void setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user);
+ set("sc.cloudbase.username", user);
+ }
+
+ public String getAccumuloUser(){
+ return get("sc.cloudbase.username");
--- End diff --
Are these strings not in public constants or an enum anywhere?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92964261
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
+ this.visibilities = visibilites;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the prefix for the Rya instance to connect to. This parameter is
+ * required and the default value is "rya_"
+ *
+ * @param prefix
+ * - the prefix for the Rya instance
+ * @return B - concrete builder class
+ */
+ public B setRyaPrefix(String prefix) {
+ Preconditions.checkNotNull(prefix, "Rya prefix cannot be null.");
+ this.prefix = prefix;
+ return confBuilder();
+ }
+
+ /**
+ * Set whether to display query plan during optimization. The default value
+ * is false.
+ *
+ * @param displayPlan
+ * - display the parsed query plan during query evaluation
+ * (useful for debugging)
+ * @return B - concrete builder class
+ */
+ public B displayQueryPlan(boolean displayPlan) {
+ this.displayPlan = displayPlan;
+ return confBuilder();
+ }
+
+ public C build() {
+ return getConf(createConf());
+ }
+
+ private C getConf(C conf) {
--- End diff --
I would just say it's nice to think about private method names so that the next guy who has to maintain the code can quickly figure out what is going on.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r93288902
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
--- End diff --
This class is extended by the AccumuloIndexingConfigBuilder and new functionality is added there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92819399
--- Diff: dao/accumulo.rya/src/test/java/org/apache/rya/accumulo/AccumuloRdfConfigurationTest.java ---
@@ -72,4 +76,85 @@ public void testIterators() {
assertEquals(setting, iteratorSettings[0]);
}
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = true;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration().getBuilder()//
--- End diff --
You don't need these "//" at the end of these lines.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110978887
--- Diff: extras/indexing/src/main/java/org/apache/rya/indexing/accumulo/AccumuloIndexingConfiguration.java ---
@@ -0,0 +1,620 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo;
+
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Properties;
+import java.util.Set;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.accumulo.AbstractAccumuloRdfConfigurationBuilder;
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.accumulo.AccumuloRdfConfigurationBuilder;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.domain.RyaURI;
+import org.apache.rya.indexing.accumulo.entity.EntityCentricIndex;
+import org.apache.rya.indexing.accumulo.freetext.AccumuloFreeTextIndexer;
+import org.apache.rya.indexing.accumulo.temporal.AccumuloTemporalIndexer;
+import org.apache.rya.indexing.external.PrecomputedJoinIndexer;
+import org.openrdf.sail.Sail;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Lists;
+
+/**
+ * This class is an extension of the AccumuloRdfConfiguration object used to to
+ * create a {@link Sail} connection to an Accumulo backed instance of Rya. This
+ * configuration object is designed to create Accumulo Rya Sail connections
+ * where one or more of the Accumulo Rya Indexes are enabled. These indexes
+ * include the {@link AccumuloFreeTextIndexer}, {@link AccumuloTemporalIndexer},
+ * {@link EntityCentricIndex}, and the {@link PrecomputedJoinIndexer}.
+ *
+ */
+public class AccumuloIndexingConfiguration extends AccumuloRdfConfiguration {
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92819277
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
--- End diff --
White space is kind of weird here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110945034
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRdfConfiguration.java ---
@@ -19,101 +19,166 @@
* under the License.
*/
-
-
import java.util.List;
-
-import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
-import org.apache.rya.api.persist.index.RyaSecondaryIndexer;
+import java.util.Properties;
import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import com.google.common.base.Preconditions;
import com.google.common.collect.Lists;
import com.mongodb.MongoClient;
public class MongoDBRdfConfiguration extends RdfCloudTripleStoreConfiguration {
--- End diff --
Nit: Tab characters used for spacing in this class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r93289381
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
+ extends RdfCloudTripleStoreConfigurationBuilder<B, C> {
+
+ private String user;
+ private String pass;
+ private String zoo = "";
+ private String instance;
+ private boolean useMock = false;
+ private boolean usePrefixHashing = false;
+ private boolean useComposite = false;
+ private boolean useSelectivity = false;
+
+ protected static final String ACCUMULO_USER = "accumulo.user";
+ protected static final String ACCUMULO_PASSWORD = "accumulo.password";
+ protected static final String ACCUMULO_INSTANCE = "accumulo.instance";
+ protected static final String ACCUMULO_AUTHS = "accumulo.auths";
+ protected static final String ACCUMULO_VISIBILITIES = "accumulo.visibilities";
+ protected static final String ACCUMULO_ZOOKEEPERS = "accumulo.zookeepers";
+ protected static final String ACCUMULO_RYA_PREFIX = "accumulo.rya.prefix";
+ protected static final String USE_INFERENCE = "use.inference";
+ protected static final String USE_DISPLAY_QUERY_PLAN = "use.display.plan";
+ protected static final String USE_MOCK_ACCUMULO = "use.mock";
+ protected static final String USE_PREFIX_HASHING = "use.prefix.hashing";
+ protected static final String USE_COUNT_STATS = "use.count.stats";
+ protected static final String USE_JOIN_SELECTIVITY = "use.join.selectivity";
+
+ /**
+ * Sets Accumulo user. This is a required parameter to connect to an
+ * Accumulo Rya Instance
+ *
+ * @param user
+ * - Accumulo user
+ */
+ public B setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user, "User cannot be null.");
+ this.user = user;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo user's password. This is a required parameter to connect to
+ * an Accumulo Rya Instance
+ *
+ * @param password
+ * - password for Accumulo user specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloUser(String)}
+ * @return specified builder
+ */
+ public B setAccumuloPassword(String password) {
+ Preconditions.checkNotNull(password, "Password cannot be null.");
+ this.pass = password;
+ return confBuilder();
+ }
+
+ /**
+ * Sets name of Accumulo instance containing Rya tables. This is a required
+ * parameter to connect to an Accumulo Rya instance.
+ *
+ * @param instance
+ * - Accumulo instance name
+ * @return specified builder
+ */
+ public B setAccumuloInstance(String instance) {
+ Preconditions.checkNotNull(instance, "Instance cannot be null.");
+ this.instance = instance;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo Zookeepers for instance specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloInstance(String)}
+ * . This is a required parameter if
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false.
+ *
+ * @param zoo
+ * - Accumuo Zookeepers
+ * @return specified builder
+ */
+ public B setAccumuloZooKeepers(String zoo) {
--- End diff --
Made it so default string is not empty.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110951217
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110959078
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/AbstractMongoDBRdfConfigurationBuilder.java ---
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+/**
+ * This builder class will set all of the core Mongo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to a Mongo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractMongoDBRdfConfigurationBuilder<B extends AbstractMongoDBRdfConfigurationBuilder<B, C>, C extends MongoDBRdfConfiguration>
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92959417
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
--- End diff --
You mean when it is being built?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92960423
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/accumulo/AccumuloIndexingConfigurationTest.java ---
@@ -0,0 +1,145 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class AccumuloIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = true;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
+ .auths(auth)//
+ .visibilities(visibility)//
+ .setRyaPrefix(prefix)//
+ .useInference(useInference)//
+ .useCompositeCardinality(useComposite)//
+ .displayQueryPlan(displayPlan)//
+ .setAccumuloInstance(instance)//
+ .setAccumuloPassword(password)//
+ .setAccumuloUser(user)//
+ .setAccumuloZooKeepers(zookeeper)//
+ .useMockAccumulo(useMock)//
+ .useAccumuloPrefixHashing(usePrefixHash)//
+ .setAccumuloFreeTextPredicates("http://pred1", "http://pred2")//
+ .setAccumuloTemporalPredicates("http://pred3", "http://pred4")//
+ .useAccumuloTemporalIndex(true)//
+ .useAccumuloEntityIndex(true)//
+ .useAccumuloFreetextIndex(true)//
+ .pcjUpdaterFluoAppName("fluo")//
+ .useOptimalPcj(true)//
+ .pcjTables("table1", "table2").build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
Agreed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111403016
--- Diff: extras/rya.pcj.fluo/pcj.fluo.api/src/main/java/org/apache/rya/indexing/pcj/fluo/api/CreatePcj.java ---
@@ -202,14 +203,56 @@ public String withRyaIntegration(final String pcjId, final PrecomputedJoinStora
triplesBatch.clear();
}
+=======
+ try (Transaction tx = fluo.newTransaction()) {
+ // Write the query's structure to Fluo.
+ new FluoQueryMetadataDAO().write(tx, fluoQuery);
+
+ // The results of the query are eventually exported to an instance
+ // of Rya, so store the Rya ID for the PCJ.
+ final String queryId = fluoQuery.getQueryMetadata().getNodeId();
+ tx.set(queryId, FluoQueryColumns.RYA_PCJ_ID, pcjId);
+ tx.set(pcjId, FluoQueryColumns.PCJ_ID_QUERY_ID, queryId);
+
+ // Flush the changes to Fluo.
+ tx.commit();
+ }
+
+ // Reuse the same set object while performing batch inserts.
+ final Set<RyaStatement> queryBatch = new HashSet<>();
+
+ // Iterate through each of the statement patterns and insert their
+ // historic matches into Fluo.
+ for (final StatementPatternMetadata patternMetadata : fluoQuery.getStatementPatternMetadata()) {
+ // Get an iterator over all of the binding sets that match the
+ // statement pattern.
+ final StatementPattern pattern = FluoStringConverter
+ .toStatementPattern(patternMetadata.getStatementPattern());
+ queryBatch.add(spToRyaStatement(pattern));
+ }
+
+ Iterator<RyaStatement> triples = queryEngine.query(new BatchRyaQuery(queryBatch)).iterator();
+ Set<RyaStatement> triplesBatch = new HashSet<>();
+
+ // Insert batches of the binding sets into Fluo.
+ while (triples.hasNext()) {
+ if (triplesBatch.size() == spInsertBatchSize) {
+ writeBatch(fluo, triplesBatch);
+ triplesBatch.clear();
+ }
+
+>>>>>>> RYA-222-Fixed Column Visibility Bug for Results Streamed into Fluo
--- End diff --
Same here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110957970
--- Diff: dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRdfConfigurationTest.java ---
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoDBRdfConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoDBRdfConfiguration conf = new MongoDBRdfConfiguration().getBuilder()//
--- End diff --
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110948112
--- Diff: dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRdfConfigurationTest.java ---
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoDBRdfConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoDBRdfConfiguration conf = new MongoDBRdfConfiguration().getBuilder()//
--- End diff --
nit: Unnecessary "//" characters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110958691
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92821040
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
+ this.visibilities = visibilites;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the prefix for the Rya instance to connect to. This parameter is
+ * required and the default value is "rya_"
+ *
+ * @param prefix
+ * - the prefix for the Rya instance
+ * @return B - concrete builder class
+ */
+ public B setRyaPrefix(String prefix) {
+ Preconditions.checkNotNull(prefix, "Rya prefix cannot be null.");
+ this.prefix = prefix;
+ return confBuilder();
+ }
+
+ /**
+ * Set whether to display query plan during optimization. The default value
+ * is false.
+ *
+ * @param displayPlan
+ * - display the parsed query plan during query evaluation
+ * (useful for debugging)
+ * @return B - concrete builder class
+ */
+ public B displayQueryPlan(boolean displayPlan) {
+ this.displayPlan = displayPlan;
+ return confBuilder();
+ }
+
+ public C build() {
--- End diff --
Missing documentation.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110947407
--- Diff: extras/indexing/src/main/java/org/apache/rya/indexing/mongodb/MongoIndexingConfiguration.java ---
@@ -0,0 +1,394 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.mongodb;
+
+import java.util.Properties;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.indexing.accumulo.ConfigUtils;
+import org.apache.rya.indexing.accumulo.freetext.AccumuloFreeTextIndexer;
+import org.apache.rya.indexing.mongodb.freetext.MongoFreeTextIndexer;
+import org.apache.rya.indexing.mongodb.temporal.MongoTemporalIndexer;
+import org.apache.rya.mongodb.AbstractMongoDBRdfConfigurationBuilder;
+import org.apache.rya.mongodb.MongoDBRdfConfiguration;
+import org.apache.rya.mongodb.MongoDBRdfConfigurationBuilder;
+import org.openrdf.sail.Sail;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is an extension of the MongoDBRdfConfiguration object used to to
+ * create a {@link Sail} connection to an Mongo backed instance of Rya. This
+ * configuration object is designed to create Mongo Rya Sail connections where
+ * one or more of the Mongo Rya Indexes are enabled. These indexes include the
+ * {@link MongoFreeTextIndexer} and {@link MongoTemporalIndexer}
+ *
+ */
+public class MongoIndexingConfiguration extends MongoDBRdfConfiguration {
--- End diff --
This class has undocumented methods and constructors.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110946456
--- Diff: extras/indexingExample/src/main/java/MongoRyaDirectExample.java ---
@@ -278,26 +280,20 @@ private static Configuration getConf() throws IOException {
String url = address.getHost();
String port = Integer.toString(address.getPort());
c.close();
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE, url);
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, port);
+ builder.setMongoHost(url).setMongoPort(port);
} else {
// User name and password must be filled in:
- conf.set(MongoDBRdfConfiguration.MONGO_USER, "fill this in");
- conf.set(MongoDBRdfConfiguration.MONGO_USER_PASSWORD, "fill this in");
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE, MONGO_INSTANCE_URL);
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, MONGO_INSTANCE_PORT);
+ builder = builder.setMongoUser("fill this in")//
+ .setMongoPassword("fill this in")//
+ .setMongoHost(MONGO_INSTANCE_URL)//
+ .setMongoPort(MONGO_INSTANCE_PORT);
}
- conf.set(MongoDBRdfConfiguration.MONGO_DB_NAME, MONGO_DB);
- conf.set(MongoDBRdfConfiguration.MONGO_COLLECTION_PREFIX, MONGO_COLL_PREFIX);
- conf.set(ConfigUtils.GEO_PREDICATES_LIST, "http://www.opengis.net/ont/geosparql#asWKT");
-// conf.set(ConfigUtils.USE_GEO, "true");
- conf.set(ConfigUtils.USE_FREETEXT, "true");
- conf.setTablePrefix(MONGO_COLL_PREFIX);
- conf.set(ConfigUtils.GEO_PREDICATES_LIST, GeoConstants.GEO_AS_WKT.stringValue());
- conf.set(ConfigUtils.FREETEXT_PREDICATES_LIST, RDFS.LABEL.stringValue());
- conf.set(ConfigUtils.FREETEXT_PREDICATES_LIST, RDFS.LABEL.stringValue());
- conf.set(RdfCloudTripleStoreConfiguration.CONF_INFER, Boolean.toString(USE_INFER));
- return conf;
+
+ return builder.setMongoDBName(MONGO_DB)//
--- End diff --
Nit: Unnecessary "//" characters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92959106
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/accumulo/AccumuloIndexingConfigurationTest.java ---
@@ -0,0 +1,145 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class AccumuloIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = true;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
+ .auths(auth)//
+ .visibilities(visibility)//
+ .setRyaPrefix(prefix)//
+ .useInference(useInference)//
+ .useCompositeCardinality(useComposite)//
+ .displayQueryPlan(displayPlan)//
+ .setAccumuloInstance(instance)//
+ .setAccumuloPassword(password)//
+ .setAccumuloUser(user)//
+ .setAccumuloZooKeepers(zookeeper)//
+ .useMockAccumulo(useMock)//
+ .useAccumuloPrefixHashing(usePrefixHash)//
+ .setAccumuloFreeTextPredicates("http://pred1", "http://pred2")//
+ .setAccumuloTemporalPredicates("http://pred3", "http://pred4")//
+ .useAccumuloTemporalIndex(true)//
+ .useAccumuloEntityIndex(true)//
+ .useAccumuloFreetextIndex(true)//
+ .pcjUpdaterFluoAppName("fluo")//
+ .useOptimalPcj(true)//
+ .pcjTables("table1", "table2").build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
In general this is valid, however I don't see how much value an equality method would have for a configuration object other than testing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya issue #127: Rya 226 config builders
Posted by pujav65 <gi...@git.apache.org>.
Github user pujav65 commented on the issue:
https://github.com/apache/incubator-rya/pull/127
actually looking more closely, i think you need to address kevin's comments before we can merge this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111391695
--- Diff: extras/indexingExample/src/main/java/RyaDirectExample.java ---
@@ -785,6 +784,13 @@ private static void createPCJ(final Configuration conf)
new PcjTables().createAndPopulatePcj(conn, accCon, tablename2,
queryString2, new String[] { "e", "c", "l", "o" },
Optional.<PcjVarOrderFactory> absent());
+
+ Scanner scanner = accCon.createScanner(tablename1, new Authorizations("U"));
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111409181
--- Diff: extras/rya.pcj.fluo/pcj.fluo.api/src/main/java/org/apache/rya/indexing/pcj/fluo/api/CreatePcj.java ---
@@ -202,14 +203,56 @@ public String withRyaIntegration(final String pcjId, final PrecomputedJoinStora
triplesBatch.clear();
}
+=======
+ try (Transaction tx = fluo.newTransaction()) {
+ // Write the query's structure to Fluo.
+ new FluoQueryMetadataDAO().write(tx, fluoQuery);
+
+ // The results of the query are eventually exported to an instance
+ // of Rya, so store the Rya ID for the PCJ.
+ final String queryId = fluoQuery.getQueryMetadata().getNodeId();
+ tx.set(queryId, FluoQueryColumns.RYA_PCJ_ID, pcjId);
+ tx.set(pcjId, FluoQueryColumns.PCJ_ID_QUERY_ID, queryId);
+
+ // Flush the changes to Fluo.
+ tx.commit();
+ }
+
+ // Reuse the same set object while performing batch inserts.
+ final Set<RyaStatement> queryBatch = new HashSet<>();
+
+ // Iterate through each of the statement patterns and insert their
+ // historic matches into Fluo.
+ for (final StatementPatternMetadata patternMetadata : fluoQuery.getStatementPatternMetadata()) {
+ // Get an iterator over all of the binding sets that match the
+ // statement pattern.
+ final StatementPattern pattern = FluoStringConverter
+ .toStatementPattern(patternMetadata.getStatementPattern());
+ queryBatch.add(spToRyaStatement(pattern));
+ }
+
+ Iterator<RyaStatement> triples = queryEngine.query(new BatchRyaQuery(queryBatch)).iterator();
+ Set<RyaStatement> triplesBatch = new HashSet<>();
+
+ // Insert batches of the binding sets into Fluo.
+ while (triples.hasNext()) {
+ if (triplesBatch.size() == spInsertBatchSize) {
+ writeBatch(fluo, triplesBatch);
+ triplesBatch.clear();
+ }
+
+>>>>>>> RYA-222-Fixed Column Visibility Bug for Results Streamed into Fluo
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110958736
--- Diff: dao/accumulo.rya/src/test/java/org/apache/rya/accumulo/AccumuloRdfConfigurationTest.java ---
@@ -72,4 +76,85 @@ public void testIterators() {
assertEquals(setting, iteratorSettings[0]);
}
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = true;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration().getBuilder()//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92820037
--- Diff: extras/indexingExample/src/main/java/RyaClientExample.java ---
@@ -132,18 +133,17 @@ public static void main(final String[] args) throws Exception {
"}";
// Load some statements into the Rya instance.
- final AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();
- conf.setTablePrefix(ryaInstanceName);
- conf.set(ConfigUtils.CLOUDBASE_USER, accumuloUsername);
- conf.set(ConfigUtils.CLOUDBASE_PASSWORD, accumuloPassword);
- conf.set(ConfigUtils.CLOUDBASE_INSTANCE, cluster.getInstanceName());
- conf.set(ConfigUtils.CLOUDBASE_ZOOKEEPERS, cluster.getZooKeepers());
- conf.set(ConfigUtils.CLOUDBASE_AUTHS, "U");
- conf.set(ConfigUtils.USE_PCJ_FLUO_UPDATER, "true");
- conf.set(ConfigUtils.FLUO_APP_NAME, fluoAppName);
- conf.set(ConfigUtils.PCJ_STORAGE_TYPE, PrecomputedJoinIndexerConfig.PrecomputedJoinStorageType.ACCUMULO.toString());
- conf.set(ConfigUtils.PCJ_UPDATER_TYPE, PrecomputedJoinIndexerConfig.PrecomputedJoinUpdaterType.FLUO.toString());
-
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
+ .auths("U")
+ .setAccumuloUser(accumuloUsername)
--- End diff --
Tab characters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92822170
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
--- End diff --
Null check should be in the configuration object, not the builder.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110948170
--- Diff: dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRdfConfigurationTest.java ---
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoDBRdfConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoDBRdfConfiguration conf = new MongoDBRdfConfiguration().getBuilder()//
+ .setVisibilities(visibility)//
+ .setUseInference(useInference)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setUseMockMongo(useMock)//
+ .setMongoCollectionPrefix(prefix)//
+ .setMongoDBName("dbname")//
+ .setMongoHost("host")//
+ .setMongoPort("1000")//
+ .setAuths(auth)//
+ .setMongoUser(user)//
+ .setMongoPassword(password).build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
nit: Could statically import the assertion methods.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92820516
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
--- End diff --
These kind of checks should happen when you build the concrete object rather than when you're setting values in the builder.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110957600
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRdfConfiguration.java ---
@@ -19,101 +19,133 @@
* under the License.
*/
-
-
import java.util.List;
-
-import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
-import org.apache.rya.api.persist.index.RyaSecondaryIndexer;
+import java.util.Properties;
import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
import com.google.common.collect.Lists;
import com.mongodb.MongoClient;
public class MongoDBRdfConfiguration extends RdfCloudTripleStoreConfiguration {
- public static final String MONGO_INSTANCE = "mongo.db.instance";
- public static final String MONGO_INSTANCE_PORT = "mongo.db.port";
- public static final String MONGO_GEO_MAXDISTANCE = "mongo.geo.maxdist";
- public static final String MONGO_DB_NAME = "mongo.db.name";
- public static final String MONGO_COLLECTION_PREFIX = "mongo.db.collectionprefix";
- public static final String MONGO_USER = "mongo.db.user";
- public static final String MONGO_USER_PASSWORD = "mongo.db.userpassword";
- public static final String CONF_ADDITIONAL_INDEXERS = "ac.additional.indexers";
+ public static final String MONGO_INSTANCE = "mongo.db.instance";
+ public static final String MONGO_INSTANCE_PORT = "mongo.db.port";
--- End diff --
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110960280
--- Diff: extras/indexingExample/src/main/java/MongoRyaDirectExample.java ---
@@ -278,26 +280,20 @@ private static Configuration getConf() throws IOException {
String url = address.getHost();
String port = Integer.toString(address.getPort());
c.close();
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE, url);
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, port);
+ builder.setMongoHost(url).setMongoPort(port);
} else {
// User name and password must be filled in:
- conf.set(MongoDBRdfConfiguration.MONGO_USER, "fill this in");
- conf.set(MongoDBRdfConfiguration.MONGO_USER_PASSWORD, "fill this in");
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE, MONGO_INSTANCE_URL);
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, MONGO_INSTANCE_PORT);
+ builder = builder.setMongoUser("fill this in")//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110944882
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/AbstractMongoDBRdfConfigurationBuilder.java ---
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+/**
+ * This builder class will set all of the core Mongo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to a Mongo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractMongoDBRdfConfigurationBuilder<B extends AbstractMongoDBRdfConfigurationBuilder<B, C>, C extends MongoDBRdfConfiguration>
--- End diff --
Nit: Parameters not documented in this class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92822394
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
+ this.visibilities = visibilites;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the prefix for the Rya instance to connect to. This parameter is
+ * required and the default value is "rya_"
+ *
+ * @param prefix
+ * - the prefix for the Rya instance
+ * @return B - concrete builder class
+ */
+ public B setRyaPrefix(String prefix) {
+ Preconditions.checkNotNull(prefix, "Rya prefix cannot be null.");
+ this.prefix = prefix;
+ return confBuilder();
+ }
+
+ /**
+ * Set whether to display query plan during optimization. The default value
+ * is false.
+ *
+ * @param displayPlan
+ * - display the parsed query plan during query evaluation
+ * (useful for debugging)
+ * @return B - concrete builder class
+ */
+ public B displayQueryPlan(boolean displayPlan) {
+ this.displayPlan = displayPlan;
+ return confBuilder();
+ }
+
+ public C build() {
+ return getConf(createConf());
+ }
+
+ private C getConf(C conf) {
+
+ Preconditions.checkNotNull(auths, "Authorizations cannot be null");
+
+ conf.setInfer(useInference);
+ conf.setTablePrefix(prefix);
+ conf.setInt("sc.cloudbase.numPartitions", 3);
+ conf.set(RdfCloudTripleStoreConfiguration.CONF_QUERY_AUTH, auths);
+ if (visibilities != null) {
+ conf.set(RdfCloudTripleStoreConfiguration.CONF_CV, visibilities);
+ }
+ conf.setDisplayQueryPlan(displayPlan);
+
+ return conf;
+ }
+
+ protected static void checkBooleanString(String boolString) {
+ Preconditions.checkNotNull(boolString, "Boolean string cannot be null.");
+ Preconditions.checkArgument(boolString.equalsIgnoreCase("true") || boolString.equalsIgnoreCase("false"),
+ "Invalid boolean string");
+ }
+
+ protected static boolean getBoolean(String boolString) {
--- End diff --
This could just be Boolean.parseBoolean(string);
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111409203
--- Diff: extras/rya.pcj.fluo/pcj.fluo.api/src/main/java/org/apache/rya/indexing/pcj/fluo/api/CreatePcj.java ---
@@ -162,6 +162,7 @@ public String withRyaIntegration(final String pcjId, final PrecomputedJoinStora
final ParsedQuery parsedQuery = new SPARQLParser().parseQuery(sparql, null);
final FluoQuery fluoQuery = new SparqlFluoQueryBuilder().make(parsedQuery, nodeIds);
+<<<<<<< HEAD
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110946769
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/mongo/MongoIndexingConfigurationTest.java ---
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.mongo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.apache.rya.indexing.mongodb.MongoIndexingConfiguration;
+import org.apache.rya.mongodb.MongoDBRdfConfiguration;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoIndexingConfiguration conf = MongoIndexingConfiguration.builder()//
--- End diff --
Nit: Unnecessary "//" characters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92819095
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/instance/ConfigurationFields.java ---
@@ -31,5 +31,7 @@
static final String USE_FREETEXT = "sc.use_freetext";
static final String USE_TEMPORAL = "sc.use_temporal";
static final String USE_ENTITY = "sc.use_entity";
- static final String USE_PCJ = "sc.use_pcj";
--- End diff --
By getting rid of this flag, can you no longer create a client that is able to disable the PCJ secondary index when said index exists?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92958680
--- Diff: dao/accumulo.rya/src/test/java/org/apache/rya/accumulo/AccumuloRdfConfigurationTest.java ---
@@ -72,4 +76,85 @@ public void testIterators() {
assertEquals(setting, iteratorSettings[0]);
}
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = true;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration().getBuilder()//
--- End diff --
Haha, okay.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110958918
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRdfConfiguration.java ---
@@ -19,101 +19,133 @@
* under the License.
*/
-
-
import java.util.List;
-
-import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
-import org.apache.rya.api.persist.index.RyaSecondaryIndexer;
+import java.util.Properties;
import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
import com.google.common.collect.Lists;
import com.mongodb.MongoClient;
public class MongoDBRdfConfiguration extends RdfCloudTripleStoreConfiguration {
- public static final String MONGO_INSTANCE = "mongo.db.instance";
- public static final String MONGO_INSTANCE_PORT = "mongo.db.port";
- public static final String MONGO_GEO_MAXDISTANCE = "mongo.geo.maxdist";
- public static final String MONGO_DB_NAME = "mongo.db.name";
- public static final String MONGO_COLLECTION_PREFIX = "mongo.db.collectionprefix";
- public static final String MONGO_USER = "mongo.db.user";
- public static final String MONGO_USER_PASSWORD = "mongo.db.userpassword";
- public static final String CONF_ADDITIONAL_INDEXERS = "ac.additional.indexers";
+ public static final String MONGO_INSTANCE = "mongo.db.instance";
+ public static final String MONGO_INSTANCE_PORT = "mongo.db.port";
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110960941
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/accumulo/AccumuloIndexingConfigurationTest.java ---
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Properties;
+import java.util.Set;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.api.domain.RyaURI;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class AccumuloIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110959340
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRdfConfiguration.java ---
@@ -19,101 +19,166 @@
* under the License.
*/
-
-
import java.util.List;
-
-import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
-import org.apache.rya.api.persist.index.RyaSecondaryIndexer;
+import java.util.Properties;
import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import com.google.common.base.Preconditions;
import com.google.common.collect.Lists;
import com.mongodb.MongoClient;
public class MongoDBRdfConfiguration extends RdfCloudTripleStoreConfiguration {
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92958503
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/instance/ConfigurationFields.java ---
@@ -31,5 +31,7 @@
static final String USE_FREETEXT = "sc.use_freetext";
static final String USE_TEMPORAL = "sc.use_temporal";
static final String USE_ENTITY = "sc.use_entity";
- static final String USE_PCJ = "sc.use_pcj";
--- End diff --
I removed this because sc.use.pcj is the incorrect flag to use when overwriting user config. This flag corresponds to whether a query client should use pcj, not whether or not to use the PrecomputedJoinIndex (i.e. Fluo). Originally this was the correct flag to use for the Fluo Updater, but additional config parameters were added to separate the config for client PCJ use and the creation of the PrecomputedJoinIndex.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92959441
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
+ this.visibilities = visibilites;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the prefix for the Rya instance to connect to. This parameter is
+ * required and the default value is "rya_"
+ *
+ * @param prefix
+ * - the prefix for the Rya instance
+ * @return B - concrete builder class
+ */
+ public B setRyaPrefix(String prefix) {
--- End diff --
Okay. Good point.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110961354
--- Diff: dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRdfConfigurationTest.java ---
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoDBRdfConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoDBRdfConfiguration conf = new MongoDBRdfConfiguration().getBuilder()//
+ .setVisibilities(visibility)//
+ .setUseInference(useInference)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setUseMockMongo(useMock)//
+ .setMongoCollectionPrefix(prefix)//
+ .setMongoDBName("dbname")//
+ .setMongoHost("host")//
+ .setMongoPort("1000")//
+ .setAuths(auth)//
+ .setMongoUser(user)//
+ .setMongoPassword(password).build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110951816
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/AbstractMongoDBRdfConfigurationBuilder.java ---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Mongo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to a Mongo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractMongoDBRdfConfigurationBuilder<B extends AbstractMongoDBRdfConfigurationBuilder<B, C>, C extends MongoDBRdfConfiguration>
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110961898
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -45,6 +43,12 @@
*/
public class AccumuloRdfConfiguration extends RdfCloudTripleStoreConfiguration {
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92822797
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/instance/ConfigurationFields.java ---
@@ -31,5 +31,7 @@
static final String USE_FREETEXT = "sc.use_freetext";
static final String USE_TEMPORAL = "sc.use_temporal";
static final String USE_ENTITY = "sc.use_entity";
- static final String USE_PCJ = "sc.use_pcj";
--- End diff --
Is that a feature we still want to support?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92819852
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/accumulo/AccumuloIndexingConfigurationTest.java ---
@@ -0,0 +1,145 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class AccumuloIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = true;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
+ .auths(auth)//
+ .visibilities(visibility)//
+ .setRyaPrefix(prefix)//
+ .useInference(useInference)//
+ .useCompositeCardinality(useComposite)//
+ .displayQueryPlan(displayPlan)//
+ .setAccumuloInstance(instance)//
+ .setAccumuloPassword(password)//
+ .setAccumuloUser(user)//
+ .setAccumuloZooKeepers(zookeeper)//
+ .useMockAccumulo(useMock)//
+ .useAccumuloPrefixHashing(usePrefixHash)//
+ .setAccumuloFreeTextPredicates("http://pred1", "http://pred2")//
+ .setAccumuloTemporalPredicates("http://pred3", "http://pred4")//
+ .useAccumuloTemporalIndex(true)//
+ .useAccumuloEntityIndex(true)//
+ .useAccumuloFreetextIndex(true)//
+ .pcjUpdaterFluoAppName("fluo")//
+ .useOptimalPcj(true)//
+ .pcjTables("table1", "table2").build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
This test could be reduced to a single assertion if you were to implement an equals(Object o) method on the built object. Create the expected configuration object, assert it is equal to the built one.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111484675
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfiguration.java ---
@@ -72,59 +72,6 @@
public static final String CONF_PCJ_TABLES = "pcj.index.tables";
public static final String CONF_STATEMENT_METADATA_PROPERTIES = "statement.metadata.properites";
public static final String CONF_USE_STATEMENT_METADATA = "use.statement.metadata";
-
-
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_DISP_QUERYPLAN = CONF_QUERYPLAN_FLAG;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_AUTH = CONF_QUERY_AUTH;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_CV = CONF_CV;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_TTL = CONF_TTL;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_STARTTIME = CONF_STARTTIME;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_PERFORMANT = CONF_PERFORMANT;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_INFER = CONF_INFER;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_USESTATS = CONF_USE_STATS;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_OFFSET = CONF_OFFSET;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
--- End diff --
Breaking change? Allowed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya issue #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on the issue:
https://github.com/apache/incubator-rya/pull/127
Okay. Addressed code comments and resolved merge issues. This branch should be ready to merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110947021
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/accumulo/AccumuloIndexingConfigurationTest.java ---
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Properties;
+import java.util.Set;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.api.domain.RyaURI;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class AccumuloIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
+ .setAuths(auth)//
+ .setVisibilities(visibility)//
+ .setRyaPrefix(prefix)//
+ .setUseInference(useInference)//
+ .setUseCompositeCardinality(useComposite)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setAccumuloInstance(instance)//
+ .setAccumuloPassword(password)//
+ .setAccumuloUser(user)//
+ .setAccumuloZooKeepers(zookeeper)//
+ .setUseMockAccumulo(useMock)//
+ .setUseAccumuloPrefixHashing(usePrefixHash)//
+ .setAccumuloFreeTextPredicates("http://pred1", "http://pred2")//
+ .setAccumuloTemporalPredicates("http://pred3", "http://pred4")//
+ .setUseAccumuloTemporalIndex(true)//
+ .setUseAccumuloEntityIndex(true)//
+ .setUseAccumuloFreetextIndex(true)//
+ .setPcjUpdaterFluoAppName("fluo")//
+ .setUseOptimalPcj(true)//
+ .setPcjTables("table1", "table2").build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
Nit: Could statically import the assertion methods.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92823728
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
+ * <li>"use.mock" - Use a MockAccumulo instance as back-end for Rya instance. Default is false.
+ * <li>"use.prefix.hashing" - Use prefix hashing for triples. Helps avoid hot-spotting. Default is false.
+ * <li>"use.count.stats" - Use triple pattern cardinalities for query optimization. Default is false.
+ * <li>"use.join.selectivity" - Use join selectivity for query optimization. Default is false.
+ * <li>"use.display.plan" - Display query plan during evaluation. Useful for debugging. Default is true.
+ * <li>"use.inference" - Use backward chaining inference during query evaluation. Default is false.
+ * </ul>
+ * <br>
+ * @param props - Properties file containing Accumulo specific configuration parameters
+ * @return AccumumuloRdfConfiguration with properties set
+ */
+
+ public static AccumuloRdfConfiguration fromProperties(Properties props) {
+ return AccumuloRdfConfigurationBuilder.fromProperties(props).build();
+ }
+
@Override
public AccumuloRdfConfiguration clone() {
return new AccumuloRdfConfiguration(this);
}
+
+ public void setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user);
--- End diff --
Maybe use Objects.requireNonNull since this is built into Java 8 instead of using Guava.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110958808
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/instance/ConfigurationFields.java ---
@@ -31,5 +31,7 @@
static final String USE_FREETEXT = "sc.use_freetext";
static final String USE_TEMPORAL = "sc.use_temporal";
static final String USE_ENTITY = "sc.use_entity";
- static final String USE_PCJ = "sc.use_pcj";
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92960387
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
+ * <li>"use.mock" - Use a MockAccumulo instance as back-end for Rya instance. Default is false.
+ * <li>"use.prefix.hashing" - Use prefix hashing for triples. Helps avoid hot-spotting. Default is false.
+ * <li>"use.count.stats" - Use triple pattern cardinalities for query optimization. Default is false.
+ * <li>"use.join.selectivity" - Use join selectivity for query optimization. Default is false.
+ * <li>"use.display.plan" - Display query plan during evaluation. Useful for debugging. Default is true.
+ * <li>"use.inference" - Use backward chaining inference during query evaluation. Default is false.
+ * </ul>
+ * <br>
+ * @param props - Properties file containing Accumulo specific configuration parameters
+ * @return AccumumuloRdfConfiguration with properties set
+ */
+
+ public static AccumuloRdfConfiguration fromProperties(Properties props) {
+ return AccumuloRdfConfigurationBuilder.fromProperties(props).build();
+ }
+
@Override
public AccumuloRdfConfiguration clone() {
return new AccumuloRdfConfiguration(this);
}
+
+ public void setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user);
+ set("sc.cloudbase.username", user);
+ }
+
+ public String getAccumuloUser(){
+ return get("sc.cloudbase.username");
--- End diff --
I like that idea.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110951123
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
+ * <li>"use.mock" - Use a MockAccumulo instance as back-end for Rya instance. Default is false.
+ * <li>"use.prefix.hashing" - Use prefix hashing for triples. Helps avoid hot-spotting. Default is false.
+ * <li>"use.count.stats" - Use triple pattern cardinalities for query optimization. Default is false.
+ * <li>"use.join.selectivity" - Use join selectivity for query optimization. Default is false.
+ * <li>"use.display.plan" - Display query plan during evaluation. Useful for debugging. Default is true.
+ * <li>"use.inference" - Use backward chaining inference during query evaluation. Default is false.
+ * </ul>
+ * <br>
+ * @param props - Properties file containing Accumulo specific configuration parameters
+ * @return AccumumuloRdfConfiguration with properties set
+ */
+
+ public static AccumuloRdfConfiguration fromProperties(Properties props) {
+ return AccumuloRdfConfigurationBuilder.fromProperties(props).build();
+ }
+
@Override
public AccumuloRdfConfiguration clone() {
return new AccumuloRdfConfiguration(this);
}
+
+ public void setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user);
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92823533
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
+ extends RdfCloudTripleStoreConfigurationBuilder<B, C> {
+
+ private String user;
+ private String pass;
+ private String zoo = "";
+ private String instance;
+ private boolean useMock = false;
+ private boolean usePrefixHashing = false;
+ private boolean useComposite = false;
+ private boolean useSelectivity = false;
+
+ protected static final String ACCUMULO_USER = "accumulo.user";
+ protected static final String ACCUMULO_PASSWORD = "accumulo.password";
+ protected static final String ACCUMULO_INSTANCE = "accumulo.instance";
+ protected static final String ACCUMULO_AUTHS = "accumulo.auths";
+ protected static final String ACCUMULO_VISIBILITIES = "accumulo.visibilities";
+ protected static final String ACCUMULO_ZOOKEEPERS = "accumulo.zookeepers";
+ protected static final String ACCUMULO_RYA_PREFIX = "accumulo.rya.prefix";
+ protected static final String USE_INFERENCE = "use.inference";
+ protected static final String USE_DISPLAY_QUERY_PLAN = "use.display.plan";
+ protected static final String USE_MOCK_ACCUMULO = "use.mock";
+ protected static final String USE_PREFIX_HASHING = "use.prefix.hashing";
+ protected static final String USE_COUNT_STATS = "use.count.stats";
+ protected static final String USE_JOIN_SELECTIVITY = "use.join.selectivity";
+
+ /**
+ * Sets Accumulo user. This is a required parameter to connect to an
+ * Accumulo Rya Instance
+ *
+ * @param user
+ * - Accumulo user
+ */
+ public B setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user, "User cannot be null.");
+ this.user = user;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo user's password. This is a required parameter to connect to
+ * an Accumulo Rya Instance
+ *
+ * @param password
+ * - password for Accumulo user specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloUser(String)}
+ * @return specified builder
+ */
+ public B setAccumuloPassword(String password) {
+ Preconditions.checkNotNull(password, "Password cannot be null.");
+ this.pass = password;
+ return confBuilder();
+ }
+
+ /**
+ * Sets name of Accumulo instance containing Rya tables. This is a required
+ * parameter to connect to an Accumulo Rya instance.
+ *
+ * @param instance
+ * - Accumulo instance name
+ * @return specified builder
+ */
+ public B setAccumuloInstance(String instance) {
+ Preconditions.checkNotNull(instance, "Instance cannot be null.");
+ this.instance = instance;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo Zookeepers for instance specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloInstance(String)}
+ * . This is a required parameter if
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false.
+ *
+ * @param zoo
+ * - Accumuo Zookeepers
+ * @return specified builder
+ */
+ public B setAccumuloZooKeepers(String zoo) {
+ Preconditions.checkNotNull(zoo, "Zookeepers cannot be null.");
+ this.zoo = zoo;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use a mock version of Accumulo for the Rya
+ * Instance. The default value is false. If
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false, then
+ * Accumulo Zookeepers must be specified.
+ *
+ * @param useMock
+ * @return specified builder
+ */
+ public B useMockAccumulo(boolean useMock) {
+ this.useMock = useMock;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether to use prefix hashing as a table design optimization to
+ * prevent hot spotting. The default value is false.
+ *
+ * @param useHash
+ * - use prefix hashing to prevent hot spotting
+ * @return specified builder
+ */
+ public B useAccumuloPrefixHashing(boolean useHash) {
+ this.usePrefixHashing = useHash;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use triple cardinalities for query
+ * optimization. This feature can only be used if the prospects table has
+ * been created by the prospector and the default value is false.
+ *
+ * @param useCompositeCardinality
+ * - use prospects statistics table for query optimization
+ * @return specified builder
+ */
+ public B useCompositeCardinality(boolean useCompositeCardinality) {
+ this.useComposite = useCompositeCardinality;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use the join selectivity for query
+ * optimization. This feature can only be used if the selectivity table has
+ * been created and the default value is false.
+ *
+ * @param useJoinSelectivity
+ * - use join selectivity statistics table for query optimization
+ * @return specified builder
+ */
+ public B useJoinSelectivity(boolean useJoinSelectivity) {
+ this.useSelectivity = useJoinSelectivity;
+ return confBuilder();
+ }
+
+ public C build() {
--- End diff --
docs
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110946537
--- Diff: extras/indexingExample/src/main/java/MongoRyaDirectExample.java ---
@@ -268,8 +270,8 @@ private static void testAddAndFreeTextSearchWithPCJ(SailRepositoryConnection con
private static MockMongoFactory mock = null;
private static Configuration getConf() throws IOException {
- MongoDBRdfConfiguration conf = new MongoDBRdfConfiguration();
- conf.set(ConfigUtils.USE_MONGO, "true");
+ MongoDBIndexingConfigBuilder builder = MongoIndexingConfiguration.builder()//
--- End diff --
Nit: Unnecessary "//" characters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110957630
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRdfConfiguration.java ---
@@ -19,101 +19,166 @@
* under the License.
*/
-
-
import java.util.List;
-
-import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
-import org.apache.rya.api.persist.index.RyaSecondaryIndexer;
+import java.util.Properties;
import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import com.google.common.base.Preconditions;
import com.google.common.collect.Lists;
import com.mongodb.MongoClient;
public class MongoDBRdfConfiguration extends RdfCloudTripleStoreConfiguration {
--- End diff --
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya issue #127: Rya 226 config builders
Posted by pujav65 <gi...@git.apache.org>.
Github user pujav65 commented on the issue:
https://github.com/apache/incubator-rya/pull/127
asfbot build
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110959936
--- Diff: extras/indexingExample/src/main/java/RyaClientExample.java ---
@@ -132,18 +130,17 @@ public static void main(final String[] args) throws Exception {
"}";
// Load some statements into the Rya instance.
- final AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();
- conf.setTablePrefix(ryaInstanceName);
- conf.set(ConfigUtils.CLOUDBASE_USER, accumuloUsername);
- conf.set(ConfigUtils.CLOUDBASE_PASSWORD, accumuloPassword);
- conf.set(ConfigUtils.CLOUDBASE_INSTANCE, cluster.getInstanceName());
- conf.set(ConfigUtils.CLOUDBASE_ZOOKEEPERS, cluster.getZooKeepers());
- conf.set(ConfigUtils.CLOUDBASE_AUTHS, "U");
- conf.set(ConfigUtils.USE_PCJ_FLUO_UPDATER, "true");
- conf.set(ConfigUtils.FLUO_APP_NAME, fluoAppName);
- conf.set(ConfigUtils.PCJ_STORAGE_TYPE, PrecomputedJoinIndexerConfig.PrecomputedJoinStorageType.ACCUMULO.toString());
- conf.set(ConfigUtils.PCJ_UPDATER_TYPE, PrecomputedJoinIndexerConfig.PrecomputedJoinUpdaterType.FLUO.toString());
-
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111571245
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfiguration.java ---
@@ -72,59 +72,6 @@
public static final String CONF_PCJ_TABLES = "pcj.index.tables";
public static final String CONF_STATEMENT_METADATA_PROPERTIES = "statement.metadata.properites";
public static final String CONF_USE_STATEMENT_METADATA = "use.statement.metadata";
-
-
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_DISP_QUERYPLAN = CONF_QUERYPLAN_FLAG;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_AUTH = CONF_QUERY_AUTH;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_CV = CONF_CV;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_TTL = CONF_TTL;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_STARTTIME = CONF_STARTTIME;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_PERFORMANT = CONF_PERFORMANT;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_INFER = CONF_INFER;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_USESTATS = CONF_USE_STATS;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
- public static final String BINDING_OFFSET = CONF_OFFSET;
- /**
- * @deprecated use CONF_*
- */
- @Deprecated
--- End diff --
These predate the package switch to apache (which was a breaking change). Now they are just an unused relic.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110960967
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/accumulo/AccumuloIndexingConfigurationTest.java ---
@@ -0,0 +1,153 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashSet;
+import java.util.Properties;
+import java.util.Set;
+
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.api.domain.RyaURI;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class AccumuloIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
+ .setAuths(auth)//
+ .setVisibilities(visibility)//
+ .setRyaPrefix(prefix)//
+ .setUseInference(useInference)//
+ .setUseCompositeCardinality(useComposite)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setAccumuloInstance(instance)//
+ .setAccumuloPassword(password)//
+ .setAccumuloUser(user)//
+ .setAccumuloZooKeepers(zookeeper)//
+ .setUseMockAccumulo(useMock)//
+ .setUseAccumuloPrefixHashing(usePrefixHash)//
+ .setAccumuloFreeTextPredicates("http://pred1", "http://pred2")//
+ .setAccumuloTemporalPredicates("http://pred3", "http://pred4")//
+ .setUseAccumuloTemporalIndex(true)//
+ .setUseAccumuloEntityIndex(true)//
+ .setUseAccumuloFreetextIndex(true)//
+ .setPcjUpdaterFluoAppName("fluo")//
+ .setUseOptimalPcj(true)//
+ .setPcjTables("table1", "table2").build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110948020
--- Diff: extras/indexing/src/main/java/org/apache/rya/indexing/accumulo/AccumuloIndexingConfiguration.java ---
@@ -0,0 +1,620 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.accumulo;
+
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Properties;
+import java.util.Set;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.accumulo.AbstractAccumuloRdfConfigurationBuilder;
+import org.apache.rya.accumulo.AccumuloRdfConfiguration;
+import org.apache.rya.accumulo.AccumuloRdfConfigurationBuilder;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.domain.RyaURI;
+import org.apache.rya.indexing.accumulo.entity.EntityCentricIndex;
+import org.apache.rya.indexing.accumulo.freetext.AccumuloFreeTextIndexer;
+import org.apache.rya.indexing.accumulo.temporal.AccumuloTemporalIndexer;
+import org.apache.rya.indexing.external.PrecomputedJoinIndexer;
+import org.openrdf.sail.Sail;
+
+import com.google.common.base.Preconditions;
+import com.google.common.collect.Lists;
+
+/**
+ * This class is an extension of the AccumuloRdfConfiguration object used to to
+ * create a {@link Sail} connection to an Accumulo backed instance of Rya. This
+ * configuration object is designed to create Accumulo Rya Sail connections
+ * where one or more of the Accumulo Rya Indexes are enabled. These indexes
+ * include the {@link AccumuloFreeTextIndexer}, {@link AccumuloTemporalIndexer},
+ * {@link EntityCentricIndex}, and the {@link PrecomputedJoinIndexer}.
+ *
+ */
+public class AccumuloIndexingConfiguration extends AccumuloRdfConfiguration {
--- End diff --
There are a bunch of undocumented methods and constructors in this class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110982268
--- Diff: extras/indexingExample/src/main/java/EntityDirectExample.java ---
@@ -237,18 +238,15 @@ private static void testAddAndTemporalSearchWithPCJ(final SailRepositoryConnecti
private static Configuration getConf() {
- final AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();
-
- conf.setBoolean(ConfigUtils.USE_MOCK_INSTANCE, USE_MOCK_INSTANCE);
- conf.set(ConfigUtils.USE_ENTITY, "true");
- conf.setTablePrefix(RYA_TABLE_PREFIX);
- conf.set(ConfigUtils.CLOUDBASE_USER, "root");
- conf.set(ConfigUtils.CLOUDBASE_PASSWORD, "");
- conf.set(ConfigUtils.CLOUDBASE_INSTANCE, INSTANCE);
- conf.setInt(ConfigUtils.NUM_PARTITIONS, 3);
- conf.set(ConfigUtils.CLOUDBASE_AUTHS, AUTHS);
-
- return conf;
+ return AccumuloIndexingConfiguration.builder()//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110945177
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/MongoDBRdfConfiguration.java ---
@@ -19,101 +19,166 @@
* under the License.
*/
-
-
import java.util.List;
-
-import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
-import org.apache.rya.api.persist.index.RyaSecondaryIndexer;
+import java.util.Properties;
import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import com.google.common.base.Preconditions;
import com.google.common.collect.Lists;
import com.mongodb.MongoClient;
public class MongoDBRdfConfiguration extends RdfCloudTripleStoreConfiguration {
--- End diff --
Nit: Also a bunch of undocumented methods added.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110951168
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -45,6 +43,12 @@
*/
public class AccumuloRdfConfiguration extends RdfCloudTripleStoreConfiguration {
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110950839
--- Diff: dao/accumulo.rya/src/test/java/org/apache/rya/accumulo/AccumuloRdfConfigurationTest.java ---
@@ -72,4 +76,85 @@ public void testIterators() {
assertEquals(setting, iteratorSettings[0]);
}
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration().getBuilder()//
+ .setAuths(auth)//
+ .setVisibilities(visibility)//
+ .setRyaPrefix(prefix)//
+ .setUseInference(useInference)//
+ .setUseCompositeCardinality(useComposite)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setAccumuloInstance(instance)//
+ .setAccumuloPassword(password)//
+ .setAccumuloUser(user)//
+ .setAccumuloZooKeepers(zookeeper)//
+ .setUseMockAccumulo(useMock)//
+ .setUseAccumuloPrefixHashing(usePrefixHash)//
+ .build();
+
+ org.junit.Assert.assertEquals(conf.getTablePrefix(), prefix);
+ org.junit.Assert.assertEquals(conf.getCv(), visibility);
+ org.junit.Assert.assertEquals(conf.getAuthorizations(), new Authorizations(auth));
+ org.junit.Assert.assertEquals(conf.isInfer(), useInference);
+ org.junit.Assert.assertEquals(conf.isUseCompositeCardinality(), useComposite);
+ org.junit.Assert.assertEquals(conf.isDisplayQueryPlan(), displayPlan);
+ org.junit.Assert.assertEquals(conf.getAccumuloInstance(), instance);
+ org.junit.Assert.assertEquals(conf.getAccumuloPassword(), password);
+ org.junit.Assert.assertEquals(conf.getAccumuloUser(), user);
+ org.junit.Assert.assertEquals(conf.getAccumuloZookeepers(), zookeeper);
+ org.junit.Assert.assertEquals(conf.getUseMockAccumulo(), useMock);
+ org.junit.Assert.assertEquals(conf.isPrefixRowsWithHash(), usePrefixHash);
+
+ }
+
+ @Test
+ public void testBuilderFromProperties() throws FileNotFoundException, IOException {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ Properties props = new Properties();
+ props.load(new FileInputStream("src/test/resources/properties/rya.properties"));
+
+ AccumuloRdfConfiguration conf = AccumuloRdfConfiguration.fromProperties(props);
+
+ org.junit.Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92958657
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
+ * <li>"use.mock" - Use a MockAccumulo instance as back-end for Rya instance. Default is false.
+ * <li>"use.prefix.hashing" - Use prefix hashing for triples. Helps avoid hot-spotting. Default is false.
+ * <li>"use.count.stats" - Use triple pattern cardinalities for query optimization. Default is false.
+ * <li>"use.join.selectivity" - Use join selectivity for query optimization. Default is false.
+ * <li>"use.display.plan" - Display query plan during evaluation. Useful for debugging. Default is true.
+ * <li>"use.inference" - Use backward chaining inference during query evaluation. Default is false.
+ * </ul>
+ * <br>
+ * @param props - Properties file containing Accumulo specific configuration parameters
+ * @return AccumumuloRdfConfiguration with properties set
+ */
+
+ public static AccumuloRdfConfiguration fromProperties(Properties props) {
+ return AccumuloRdfConfigurationBuilder.fromProperties(props).build();
+ }
+
@Override
public AccumuloRdfConfiguration clone() {
return new AccumuloRdfConfiguration(this);
}
+
+ public void setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user);
+ set("sc.cloudbase.username", user);
+ }
+
+ public String getAccumuloUser(){
+ return get("sc.cloudbase.username");
--- End diff --
Yes, but in ConfigUtils, which this cannot depend on. I can declare these as public constants in AccumuloRdfConfiguration and then have ConfigUtils refer to them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r93288995
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
--- End diff --
This builder is nested within the AccumuloIndexingConfiguration class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110946273
--- Diff: extras/indexingExample/src/main/java/RyaClientExample.java ---
@@ -132,18 +130,17 @@ public static void main(final String[] args) throws Exception {
"}";
// Load some statements into the Rya instance.
- final AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();
- conf.setTablePrefix(ryaInstanceName);
- conf.set(ConfigUtils.CLOUDBASE_USER, accumuloUsername);
- conf.set(ConfigUtils.CLOUDBASE_PASSWORD, accumuloPassword);
- conf.set(ConfigUtils.CLOUDBASE_INSTANCE, cluster.getInstanceName());
- conf.set(ConfigUtils.CLOUDBASE_ZOOKEEPERS, cluster.getZooKeepers());
- conf.set(ConfigUtils.CLOUDBASE_AUTHS, "U");
- conf.set(ConfigUtils.USE_PCJ_FLUO_UPDATER, "true");
- conf.set(ConfigUtils.FLUO_APP_NAME, fluoAppName);
- conf.set(ConfigUtils.PCJ_STORAGE_TYPE, PrecomputedJoinIndexerConfig.PrecomputedJoinStorageType.ACCUMULO.toString());
- conf.set(ConfigUtils.PCJ_UPDATER_TYPE, PrecomputedJoinIndexerConfig.PrecomputedJoinUpdaterType.FLUO.toString());
-
+
+ AccumuloIndexingConfiguration conf = AccumuloIndexingConfiguration.builder()//
--- End diff --
Nit: Unnecessary "//" characters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92823026
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
--- End diff --
You never extend this class to add anything extra to it. Does it actually need to be abstract?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110960701
--- Diff: extras/indexing/src/test/java/org/apache/rya/indexing/mongo/MongoIndexingConfigurationTest.java ---
@@ -0,0 +1,117 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.mongo;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.apache.rya.indexing.mongodb.MongoIndexingConfiguration;
+import org.apache.rya.mongodb.MongoDBRdfConfiguration;
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoIndexingConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoIndexingConfiguration conf = MongoIndexingConfiguration.builder()//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110951253
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/instance/ConfigurationFields.java ---
@@ -31,5 +31,7 @@
static final String USE_FREETEXT = "sc.use_freetext";
static final String USE_TEMPORAL = "sc.use_temporal";
static final String USE_ENTITY = "sc.use_entity";
- static final String USE_PCJ = "sc.use_pcj";
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92820797
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
--- End diff --
Maybe indicate the intent of returning the builder class. Something like this:
@return B This builder so that method invocations may be chained.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110948642
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -45,6 +43,12 @@
*/
public class AccumuloRdfConfiguration extends RdfCloudTripleStoreConfiguration {
--- End diff --
There are a bunch of undocumented methods in this class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111392760
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
+ extends RdfCloudTripleStoreConfigurationBuilder<B, C> {
+
+ private String user;
+ private String pass;
+ private String zoo = "";
+ private String instance;
+ private boolean useMock = false;
+ private boolean usePrefixHashing = false;
+ private boolean useComposite = false;
+ private boolean useSelectivity = false;
+
+ protected static final String ACCUMULO_USER = "accumulo.user";
+ protected static final String ACCUMULO_PASSWORD = "accumulo.password";
+ protected static final String ACCUMULO_INSTANCE = "accumulo.instance";
+ protected static final String ACCUMULO_AUTHS = "accumulo.auths";
+ protected static final String ACCUMULO_VISIBILITIES = "accumulo.visibilities";
+ protected static final String ACCUMULO_ZOOKEEPERS = "accumulo.zookeepers";
+ protected static final String ACCUMULO_RYA_PREFIX = "accumulo.rya.prefix";
+ protected static final String USE_INFERENCE = "use.inference";
+ protected static final String USE_DISPLAY_QUERY_PLAN = "use.display.plan";
+ protected static final String USE_MOCK_ACCUMULO = "use.mock";
+ protected static final String USE_PREFIX_HASHING = "use.prefix.hashing";
+ protected static final String USE_COUNT_STATS = "use.count.stats";
+ protected static final String USE_JOIN_SELECTIVITY = "use.join.selectivity";
+
+ /**
+ * Sets Accumulo user. This is a required parameter to connect to an
+ * Accumulo Rya Instance
+ *
+ * @param user
+ * - Accumulo user
+ */
+ public B setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user, "User cannot be null.");
+ this.user = user;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo user's password. This is a required parameter to connect to
+ * an Accumulo Rya Instance
+ *
+ * @param password
+ * - password for Accumulo user specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloUser(String)}
+ * @return specified builder
+ */
+ public B setAccumuloPassword(String password) {
+ Preconditions.checkNotNull(password, "Password cannot be null.");
+ this.pass = password;
+ return confBuilder();
+ }
+
+ /**
+ * Sets name of Accumulo instance containing Rya tables. This is a required
+ * parameter to connect to an Accumulo Rya instance.
+ *
+ * @param instance
+ * - Accumulo instance name
+ * @return specified builder
+ */
+ public B setAccumuloInstance(String instance) {
+ Preconditions.checkNotNull(instance, "Instance cannot be null.");
+ this.instance = instance;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo Zookeepers for instance specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloInstance(String)}
+ * . This is a required parameter if
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false.
+ *
+ * @param zoo
+ * - Accumuo Zookeepers
+ * @return specified builder
+ */
+ public B setAccumuloZooKeepers(String zoo) {
+ Preconditions.checkNotNull(zoo, "Zookeepers cannot be null.");
+ this.zoo = zoo;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use a mock version of Accumulo for the Rya
+ * Instance. The default value is false. If
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false, then
+ * Accumulo Zookeepers must be specified.
+ *
+ * @param useMock
+ * @return specified builder
+ */
+ public B useMockAccumulo(boolean useMock) {
+ this.useMock = useMock;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether to use prefix hashing as a table design optimization to
+ * prevent hot spotting. The default value is false.
+ *
+ * @param useHash
+ * - use prefix hashing to prevent hot spotting
+ * @return specified builder
+ */
+ public B useAccumuloPrefixHashing(boolean useHash) {
+ this.usePrefixHashing = useHash;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use triple cardinalities for query
+ * optimization. This feature can only be used if the prospects table has
+ * been created by the prospector and the default value is false.
+ *
+ * @param useCompositeCardinality
+ * - use prospects statistics table for query optimization
+ * @return specified builder
+ */
+ public B useCompositeCardinality(boolean useCompositeCardinality) {
+ this.useComposite = useCompositeCardinality;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use the join selectivity for query
+ * optimization. This feature can only be used if the selectivity table has
+ * been created and the default value is false.
+ *
+ * @param useJoinSelectivity
+ * - use join selectivity statistics table for query optimization
+ * @return specified builder
+ */
+ public B useJoinSelectivity(boolean useJoinSelectivity) {
+ this.useSelectivity = useJoinSelectivity;
+ return confBuilder();
+ }
+
+ public C build() {
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110946569
--- Diff: extras/indexingExample/src/main/java/MongoRyaDirectExample.java ---
@@ -278,26 +280,20 @@ private static Configuration getConf() throws IOException {
String url = address.getHost();
String port = Integer.toString(address.getPort());
c.close();
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE, url);
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, port);
+ builder.setMongoHost(url).setMongoPort(port);
} else {
// User name and password must be filled in:
- conf.set(MongoDBRdfConfiguration.MONGO_USER, "fill this in");
- conf.set(MongoDBRdfConfiguration.MONGO_USER_PASSWORD, "fill this in");
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE, MONGO_INSTANCE_URL);
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, MONGO_INSTANCE_PORT);
+ builder = builder.setMongoUser("fill this in")//
--- End diff --
Nit: Unnecessary "//" characters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-rya/pull/127
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92821744
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
+ this.visibilities = visibilites;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the prefix for the Rya instance to connect to. This parameter is
+ * required and the default value is "rya_"
+ *
+ * @param prefix
+ * - the prefix for the Rya instance
+ * @return B - concrete builder class
+ */
+ public B setRyaPrefix(String prefix) {
+ Preconditions.checkNotNull(prefix, "Rya prefix cannot be null.");
+ this.prefix = prefix;
+ return confBuilder();
+ }
+
+ /**
+ * Set whether to display query plan during optimization. The default value
+ * is false.
+ *
+ * @param displayPlan
+ * - display the parsed query plan during query evaluation
+ * (useful for debugging)
+ * @return B - concrete builder class
+ */
+ public B displayQueryPlan(boolean displayPlan) {
+ this.displayPlan = displayPlan;
+ return confBuilder();
+ }
+
+ public C build() {
+ return getConf(createConf());
+ }
+
+ private C getConf(C conf) {
--- End diff --
This method name does not reflect what the method does. Maybe call it "applyBuildersValues(conf)" or something.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92823799
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -1,42 +1,22 @@
package org.apache.rya.accumulo;
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing,
- * software distributed under the License is distributed on an
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- * KIND, either express or implied. See the License for the
- * specific language governing permissions and limitations
- * under the License.
- */
-
-
-
-import java.util.Collection;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;
-
-import org.apache.rya.accumulo.experimental.AccumuloIndexer;
-import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import java.util.Properties;
import org.apache.accumulo.core.client.IteratorSetting;
import org.apache.accumulo.core.security.Authorizations;
import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.accumulo.experimental.AccumuloIndexer;
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
import com.google.common.collect.Lists;
+import jline.internal.Preconditions;
--- End diff --
Did you mean to import JLine's Preconditions class?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110951899
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/AbstractMongoDBRdfConfigurationBuilder.java ---
@@ -0,0 +1,177 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+/**
+ * This builder class will set all of the core Mongo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to a Mongo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractMongoDBRdfConfigurationBuilder<B extends AbstractMongoDBRdfConfigurationBuilder<B, C>, C extends MongoDBRdfConfiguration>
--- End diff --
Parameter names are self-explanatory.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110958353
--- Diff: dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRdfConfigurationTest.java ---
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoDBRdfConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoDBRdfConfiguration conf = new MongoDBRdfConfiguration().getBuilder()//
+ .setVisibilities(visibility)//
+ .setUseInference(useInference)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setUseMockMongo(useMock)//
+ .setMongoCollectionPrefix(prefix)//
+ .setMongoDBName("dbname")//
+ .setMongoHost("host")//
+ .setMongoPort("1000")//
+ .setAuths(auth)//
+ .setMongoUser(user)//
+ .setMongoPassword(password).build();
+
+ Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
Done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92960527
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/RdfCloudTripleStoreConfigurationBuilder.java ---
@@ -0,0 +1,158 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This is a base class meant to be extended by Rya configuration builders. Any
+ * class extending this class inherits setter methods that should be common to
+ * all Rya implementations.
+ *
+ * @param <B>
+ * the configuration builder returned by each of the setter methods
+ * @param <C>
+ * the configuration object returned by the builder when build is
+ * called
+ */
+public abstract class RdfCloudTripleStoreConfigurationBuilder<B extends RdfCloudTripleStoreConfigurationBuilder<B, C>, C extends RdfCloudTripleStoreConfiguration> {
+
+ private String prefix = "rya_";
+ private String auths;
+ private String visibilities;
+ private boolean useInference = false;
+ private boolean displayPlan = false;
+
+ /**
+ * Auxiliary method to return specified Builder in a type safe way
+ *
+ * @return specified builder
+ */
+ protected abstract B confBuilder();
+
+ /**
+ * Auxiliary method to return type of configuration object constructed by
+ * builder in a type safe way
+ *
+ * @return specified configuration
+ */
+ protected abstract C createConf();
+
+ /**
+ * Set whether to use backwards chaining inferencing during query
+ * evaluation. The default value is false.
+ *
+ * @param useInference
+ * - turn inferencing on and off in Rya
+ * @return B - concrete builder class
+ */
+ public B useInference(boolean useInference) {
+ this.useInference = useInference;
+ return confBuilder();
+ }
+
+ /**
+ *
+ * Sets the authorization for querying the underlying data store.
+ *
+ * @param auths
+ * - authorizations for querying underlying datastore
+ * @return B - concrete builder class
+ */
+ public B auths(String auths) {
+ Preconditions.checkNotNull(auths, "User auths cannot be null.");
+ this.auths = auths;
+ return confBuilder();
+ }
+
+ /**
+ * Set the column visibities for ingested triples. If no value is set,
+ * triples won't have a visibility.
+ *
+ * @param visibilities
+ * - visibilities assigned to any triples inserted into Rya
+ * @return B - concrete builder class
+ */
+ public B visibilities(String visibilites) {
+ Preconditions.checkNotNull(visibilites, "Visibilities cannot be null.");
--- End diff --
Yep
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92823330
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
+ extends RdfCloudTripleStoreConfigurationBuilder<B, C> {
+
+ private String user;
+ private String pass;
+ private String zoo = "";
+ private String instance;
+ private boolean useMock = false;
+ private boolean usePrefixHashing = false;
+ private boolean useComposite = false;
+ private boolean useSelectivity = false;
+
+ protected static final String ACCUMULO_USER = "accumulo.user";
+ protected static final String ACCUMULO_PASSWORD = "accumulo.password";
+ protected static final String ACCUMULO_INSTANCE = "accumulo.instance";
+ protected static final String ACCUMULO_AUTHS = "accumulo.auths";
+ protected static final String ACCUMULO_VISIBILITIES = "accumulo.visibilities";
+ protected static final String ACCUMULO_ZOOKEEPERS = "accumulo.zookeepers";
+ protected static final String ACCUMULO_RYA_PREFIX = "accumulo.rya.prefix";
+ protected static final String USE_INFERENCE = "use.inference";
+ protected static final String USE_DISPLAY_QUERY_PLAN = "use.display.plan";
+ protected static final String USE_MOCK_ACCUMULO = "use.mock";
+ protected static final String USE_PREFIX_HASHING = "use.prefix.hashing";
+ protected static final String USE_COUNT_STATS = "use.count.stats";
+ protected static final String USE_JOIN_SELECTIVITY = "use.join.selectivity";
+
+ /**
+ * Sets Accumulo user. This is a required parameter to connect to an
+ * Accumulo Rya Instance
+ *
+ * @param user
+ * - Accumulo user
+ */
+ public B setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user, "User cannot be null.");
--- End diff --
These checks should happen within the build function when delegated to the object they are building, not a builder's set function. Don't duplicate the logic.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by pujav65 <gi...@git.apache.org>.
Github user pujav65 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92945311
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
--- End diff --
yea i agree. you might want to avoid abstraction here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92958592
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
+ * <li>"use.mock" - Use a MockAccumulo instance as back-end for Rya instance. Default is false.
+ * <li>"use.prefix.hashing" - Use prefix hashing for triples. Helps avoid hot-spotting. Default is false.
+ * <li>"use.count.stats" - Use triple pattern cardinalities for query optimization. Default is false.
+ * <li>"use.join.selectivity" - Use join selectivity for query optimization. Default is false.
+ * <li>"use.display.plan" - Display query plan during evaluation. Useful for debugging. Default is true.
+ * <li>"use.inference" - Use backward chaining inference during query evaluation. Default is false.
+ * </ul>
+ * <br>
+ * @param props - Properties file containing Accumulo specific configuration parameters
+ * @return AccumumuloRdfConfiguration with properties set
+ */
+
+ public static AccumuloRdfConfiguration fromProperties(Properties props) {
+ return AccumuloRdfConfigurationBuilder.fromProperties(props).build();
+ }
+
@Override
public AccumuloRdfConfiguration clone() {
return new AccumuloRdfConfiguration(this);
}
+
+ public void setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user);
--- End diff --
Nothing about a static import ;) Don't think this is a big deal given that Guava is already among our dependencies. But going forward, your point is definitely valid.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r111392719
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
+ extends RdfCloudTripleStoreConfigurationBuilder<B, C> {
+
+ private String user;
+ private String pass;
+ private String zoo = "";
+ private String instance;
+ private boolean useMock = false;
+ private boolean usePrefixHashing = false;
+ private boolean useComposite = false;
+ private boolean useSelectivity = false;
+
+ protected static final String ACCUMULO_USER = "accumulo.user";
+ protected static final String ACCUMULO_PASSWORD = "accumulo.password";
+ protected static final String ACCUMULO_INSTANCE = "accumulo.instance";
+ protected static final String ACCUMULO_AUTHS = "accumulo.auths";
+ protected static final String ACCUMULO_VISIBILITIES = "accumulo.visibilities";
+ protected static final String ACCUMULO_ZOOKEEPERS = "accumulo.zookeepers";
+ protected static final String ACCUMULO_RYA_PREFIX = "accumulo.rya.prefix";
+ protected static final String USE_INFERENCE = "use.inference";
+ protected static final String USE_DISPLAY_QUERY_PLAN = "use.display.plan";
+ protected static final String USE_MOCK_ACCUMULO = "use.mock";
+ protected static final String USE_PREFIX_HASHING = "use.prefix.hashing";
+ protected static final String USE_COUNT_STATS = "use.count.stats";
+ protected static final String USE_JOIN_SELECTIVITY = "use.join.selectivity";
+
+ /**
+ * Sets Accumulo user. This is a required parameter to connect to an
+ * Accumulo Rya Instance
+ *
+ * @param user
+ * - Accumulo user
+ */
+ public B setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user, "User cannot be null.");
+ this.user = user;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo user's password. This is a required parameter to connect to
+ * an Accumulo Rya Instance
+ *
+ * @param password
+ * - password for Accumulo user specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloUser(String)}
+ * @return specified builder
+ */
+ public B setAccumuloPassword(String password) {
+ Preconditions.checkNotNull(password, "Password cannot be null.");
+ this.pass = password;
+ return confBuilder();
+ }
+
+ /**
+ * Sets name of Accumulo instance containing Rya tables. This is a required
+ * parameter to connect to an Accumulo Rya instance.
+ *
+ * @param instance
+ * - Accumulo instance name
+ * @return specified builder
+ */
+ public B setAccumuloInstance(String instance) {
+ Preconditions.checkNotNull(instance, "Instance cannot be null.");
+ this.instance = instance;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo Zookeepers for instance specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloInstance(String)}
+ * . This is a required parameter if
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false.
+ *
+ * @param zoo
+ * - Accumuo Zookeepers
+ * @return specified builder
+ */
+ public B setAccumuloZooKeepers(String zoo) {
+ Preconditions.checkNotNull(zoo, "Zookeepers cannot be null.");
+ this.zoo = zoo;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use a mock version of Accumulo for the Rya
+ * Instance. The default value is false. If
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false, then
+ * Accumulo Zookeepers must be specified.
+ *
+ * @param useMock
+ * @return specified builder
+ */
+ public B useMockAccumulo(boolean useMock) {
+ this.useMock = useMock;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether to use prefix hashing as a table design optimization to
+ * prevent hot spotting. The default value is false.
+ *
+ * @param useHash
+ * - use prefix hashing to prevent hot spotting
+ * @return specified builder
+ */
+ public B useAccumuloPrefixHashing(boolean useHash) {
+ this.usePrefixHashing = useHash;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use triple cardinalities for query
+ * optimization. This feature can only be used if the prospects table has
+ * been created by the prospector and the default value is false.
+ *
+ * @param useCompositeCardinality
+ * - use prospects statistics table for query optimization
+ * @return specified builder
+ */
+ public B useCompositeCardinality(boolean useCompositeCardinality) {
+ this.useComposite = useCompositeCardinality;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use the join selectivity for query
+ * optimization. This feature can only be used if the selectivity table has
+ * been created and the default value is false.
+ *
+ * @param useJoinSelectivity
+ * - use join selectivity statistics table for query optimization
+ * @return specified builder
+ */
+ public B useJoinSelectivity(boolean useJoinSelectivity) {
+ this.useSelectivity = useJoinSelectivity;
+ return confBuilder();
+ }
+
+ public C build() {
+ return getConf(super.build());
+ }
+
+ private C getConf(C conf) {
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110958657
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/AbstractMongoDBRdfConfigurationBuilder.java ---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Mongo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to a Mongo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractMongoDBRdfConfigurationBuilder<B extends AbstractMongoDBRdfConfigurationBuilder<B, C>, C extends MongoDBRdfConfiguration>
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r92823584
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AbstractAccumuloRdfConfigurationBuilder.java ---
@@ -0,0 +1,209 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.accumulo;
+
+import org.apache.rya.api.RdfCloudTripleStoreConfiguration;
+import org.apache.rya.api.RdfCloudTripleStoreConfigurationBuilder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This builder class will set all of the core Accumulo-backed Rya configuration
+ * parameters. Any builder extending this class will have setter methods for all
+ * of the necessary parameters to connect to an Accumulo backed Rya instance.
+ *
+ * @param <B>
+ * - builder returned by setter methods extending this class
+ * @param <C>
+ * - configuration object constructed by the builder extending this
+ * class
+ */
+public abstract class AbstractAccumuloRdfConfigurationBuilder<B extends AbstractAccumuloRdfConfigurationBuilder<B, C>, C extends AccumuloRdfConfiguration>
+ extends RdfCloudTripleStoreConfigurationBuilder<B, C> {
+
+ private String user;
+ private String pass;
+ private String zoo = "";
+ private String instance;
+ private boolean useMock = false;
+ private boolean usePrefixHashing = false;
+ private boolean useComposite = false;
+ private boolean useSelectivity = false;
+
+ protected static final String ACCUMULO_USER = "accumulo.user";
+ protected static final String ACCUMULO_PASSWORD = "accumulo.password";
+ protected static final String ACCUMULO_INSTANCE = "accumulo.instance";
+ protected static final String ACCUMULO_AUTHS = "accumulo.auths";
+ protected static final String ACCUMULO_VISIBILITIES = "accumulo.visibilities";
+ protected static final String ACCUMULO_ZOOKEEPERS = "accumulo.zookeepers";
+ protected static final String ACCUMULO_RYA_PREFIX = "accumulo.rya.prefix";
+ protected static final String USE_INFERENCE = "use.inference";
+ protected static final String USE_DISPLAY_QUERY_PLAN = "use.display.plan";
+ protected static final String USE_MOCK_ACCUMULO = "use.mock";
+ protected static final String USE_PREFIX_HASHING = "use.prefix.hashing";
+ protected static final String USE_COUNT_STATS = "use.count.stats";
+ protected static final String USE_JOIN_SELECTIVITY = "use.join.selectivity";
+
+ /**
+ * Sets Accumulo user. This is a required parameter to connect to an
+ * Accumulo Rya Instance
+ *
+ * @param user
+ * - Accumulo user
+ */
+ public B setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user, "User cannot be null.");
+ this.user = user;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo user's password. This is a required parameter to connect to
+ * an Accumulo Rya Instance
+ *
+ * @param password
+ * - password for Accumulo user specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloUser(String)}
+ * @return specified builder
+ */
+ public B setAccumuloPassword(String password) {
+ Preconditions.checkNotNull(password, "Password cannot be null.");
+ this.pass = password;
+ return confBuilder();
+ }
+
+ /**
+ * Sets name of Accumulo instance containing Rya tables. This is a required
+ * parameter to connect to an Accumulo Rya instance.
+ *
+ * @param instance
+ * - Accumulo instance name
+ * @return specified builder
+ */
+ public B setAccumuloInstance(String instance) {
+ Preconditions.checkNotNull(instance, "Instance cannot be null.");
+ this.instance = instance;
+ return confBuilder();
+ }
+
+ /**
+ * Sets Accumulo Zookeepers for instance specified by
+ * {@link AbstractAccumuloRdfConfigurationBuilder#setAccumuloInstance(String)}
+ * . This is a required parameter if
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false.
+ *
+ * @param zoo
+ * - Accumuo Zookeepers
+ * @return specified builder
+ */
+ public B setAccumuloZooKeepers(String zoo) {
+ Preconditions.checkNotNull(zoo, "Zookeepers cannot be null.");
+ this.zoo = zoo;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use a mock version of Accumulo for the Rya
+ * Instance. The default value is false. If
+ * {@link AbstractAccumuloRdfConfigurationBuilder#useMock} is false, then
+ * Accumulo Zookeepers must be specified.
+ *
+ * @param useMock
+ * @return specified builder
+ */
+ public B useMockAccumulo(boolean useMock) {
+ this.useMock = useMock;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether to use prefix hashing as a table design optimization to
+ * prevent hot spotting. The default value is false.
+ *
+ * @param useHash
+ * - use prefix hashing to prevent hot spotting
+ * @return specified builder
+ */
+ public B useAccumuloPrefixHashing(boolean useHash) {
+ this.usePrefixHashing = useHash;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use triple cardinalities for query
+ * optimization. This feature can only be used if the prospects table has
+ * been created by the prospector and the default value is false.
+ *
+ * @param useCompositeCardinality
+ * - use prospects statistics table for query optimization
+ * @return specified builder
+ */
+ public B useCompositeCardinality(boolean useCompositeCardinality) {
+ this.useComposite = useCompositeCardinality;
+ return confBuilder();
+ }
+
+ /**
+ * Specifies whether or not to use the join selectivity for query
+ * optimization. This feature can only be used if the selectivity table has
+ * been created and the default value is false.
+ *
+ * @param useJoinSelectivity
+ * - use join selectivity statistics table for query optimization
+ * @return specified builder
+ */
+ public B useJoinSelectivity(boolean useJoinSelectivity) {
+ this.useSelectivity = useJoinSelectivity;
+ return confBuilder();
+ }
+
+ public C build() {
+ return getConf(super.build());
+ }
+
+ private C getConf(C conf) {
--- End diff --
Same comment as the parent.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110976582
--- Diff: extras/indexing/src/main/java/org/apache/rya/indexing/mongodb/MongoIndexingConfiguration.java ---
@@ -0,0 +1,394 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.mongodb;
+
+import java.util.Properties;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.rya.indexing.accumulo.ConfigUtils;
+import org.apache.rya.indexing.accumulo.freetext.AccumuloFreeTextIndexer;
+import org.apache.rya.indexing.mongodb.freetext.MongoFreeTextIndexer;
+import org.apache.rya.indexing.mongodb.temporal.MongoTemporalIndexer;
+import org.apache.rya.mongodb.AbstractMongoDBRdfConfigurationBuilder;
+import org.apache.rya.mongodb.MongoDBRdfConfiguration;
+import org.apache.rya.mongodb.MongoDBRdfConfigurationBuilder;
+import org.openrdf.sail.Sail;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is an extension of the MongoDBRdfConfiguration object used to to
+ * create a {@link Sail} connection to an Mongo backed instance of Rya. This
+ * configuration object is designed to create Mongo Rya Sail connections where
+ * one or more of the Mongo Rya Indexes are enabled. These indexes include the
+ * {@link MongoFreeTextIndexer} and {@link MongoTemporalIndexer}
+ *
+ */
+public class MongoIndexingConfiguration extends MongoDBRdfConfiguration {
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110948508
--- Diff: dao/accumulo.rya/src/test/java/org/apache/rya/accumulo/AccumuloRdfConfigurationTest.java ---
@@ -72,4 +76,85 @@ public void testIterators() {
assertEquals(setting, iteratorSettings[0]);
}
+
+ @Test
+ public void testBuilder() {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration().getBuilder()//
+ .setAuths(auth)//
+ .setVisibilities(visibility)//
+ .setRyaPrefix(prefix)//
+ .setUseInference(useInference)//
+ .setUseCompositeCardinality(useComposite)//
+ .setDisplayQueryPlan(displayPlan)//
+ .setAccumuloInstance(instance)//
+ .setAccumuloPassword(password)//
+ .setAccumuloUser(user)//
+ .setAccumuloZooKeepers(zookeeper)//
+ .setUseMockAccumulo(useMock)//
+ .setUseAccumuloPrefixHashing(usePrefixHash)//
+ .build();
+
+ org.junit.Assert.assertEquals(conf.getTablePrefix(), prefix);
+ org.junit.Assert.assertEquals(conf.getCv(), visibility);
+ org.junit.Assert.assertEquals(conf.getAuthorizations(), new Authorizations(auth));
+ org.junit.Assert.assertEquals(conf.isInfer(), useInference);
+ org.junit.Assert.assertEquals(conf.isUseCompositeCardinality(), useComposite);
+ org.junit.Assert.assertEquals(conf.isDisplayQueryPlan(), displayPlan);
+ org.junit.Assert.assertEquals(conf.getAccumuloInstance(), instance);
+ org.junit.Assert.assertEquals(conf.getAccumuloPassword(), password);
+ org.junit.Assert.assertEquals(conf.getAccumuloUser(), user);
+ org.junit.Assert.assertEquals(conf.getAccumuloZookeepers(), zookeeper);
+ org.junit.Assert.assertEquals(conf.getUseMockAccumulo(), useMock);
+ org.junit.Assert.assertEquals(conf.isPrefixRowsWithHash(), usePrefixHash);
+
+ }
+
+ @Test
+ public void testBuilderFromProperties() throws FileNotFoundException, IOException {
+ String prefix = "rya_";
+ String auth = "U";
+ String visibility = "U";
+ String user = "user";
+ String password = "password";
+ String instance = "instance";
+ String zookeeper = "zookeeper";
+ boolean useMock = false;
+ boolean useComposite = true;
+ boolean usePrefixHash = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ Properties props = new Properties();
+ props.load(new FileInputStream("src/test/resources/properties/rya.properties"));
+
+ AccumuloRdfConfiguration conf = AccumuloRdfConfiguration.fromProperties(props);
+
+ org.junit.Assert.assertEquals(conf.getTablePrefix(), prefix);
--- End diff --
nit: Could statically import these assertion methods.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110951142
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110960333
--- Diff: extras/indexingExample/src/main/java/MongoRyaDirectExample.java ---
@@ -278,26 +280,20 @@ private static Configuration getConf() throws IOException {
String url = address.getHost();
String port = Integer.toString(address.getPort());
c.close();
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE, url);
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, port);
+ builder.setMongoHost(url).setMongoPort(port);
} else {
// User name and password must be filled in:
- conf.set(MongoDBRdfConfiguration.MONGO_USER, "fill this in");
- conf.set(MongoDBRdfConfiguration.MONGO_USER_PASSWORD, "fill this in");
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE, MONGO_INSTANCE_URL);
- conf.set(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT, MONGO_INSTANCE_PORT);
+ builder = builder.setMongoUser("fill this in")//
+ .setMongoPassword("fill this in")//
+ .setMongoHost(MONGO_INSTANCE_URL)//
+ .setMongoPort(MONGO_INSTANCE_PORT);
}
- conf.set(MongoDBRdfConfiguration.MONGO_DB_NAME, MONGO_DB);
- conf.set(MongoDBRdfConfiguration.MONGO_COLLECTION_PREFIX, MONGO_COLL_PREFIX);
- conf.set(ConfigUtils.GEO_PREDICATES_LIST, "http://www.opengis.net/ont/geosparql#asWKT");
-// conf.set(ConfigUtils.USE_GEO, "true");
- conf.set(ConfigUtils.USE_FREETEXT, "true");
- conf.setTablePrefix(MONGO_COLL_PREFIX);
- conf.set(ConfigUtils.GEO_PREDICATES_LIST, GeoConstants.GEO_AS_WKT.stringValue());
- conf.set(ConfigUtils.FREETEXT_PREDICATES_LIST, RDFS.LABEL.stringValue());
- conf.set(ConfigUtils.FREETEXT_PREDICATES_LIST, RDFS.LABEL.stringValue());
- conf.set(RdfCloudTripleStoreConfiguration.CONF_INFER, Boolean.toString(USE_INFER));
- return conf;
+
+ return builder.setMongoDBName(MONGO_DB)//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110961328
--- Diff: dao/mongodb.rya/src/test/java/org/apache/rya/mongodb/MongoDBRdfConfigurationTest.java ---
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.mongodb;
+
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Properties;
+
+import org.junit.Assert;
+import org.junit.Test;
+
+public class MongoDBRdfConfigurationTest {
+
+ @Test
+ public void testBuilder() {
+ String prefix = "prefix_";
+ String auth = "U,V,W";
+ String visibility = "U,W";
+ String user = "user";
+ String password = "password";
+ boolean useMock = true;
+ boolean useInference = true;
+ boolean displayPlan = false;
+
+ MongoDBRdfConfiguration conf = new MongoDBRdfConfiguration().getBuilder()//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110946052
--- Diff: extras/indexingExample/src/main/java/RyaDirectExample.java ---
@@ -149,24 +153,19 @@ private static void closeQuietly(final SailRepositoryConnection conn) {
private static Configuration getConf() {
- final AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();
-
- conf.setBoolean(ConfigUtils.USE_MOCK_INSTANCE, USE_MOCK_INSTANCE);
- conf.set(ConfigUtils.USE_PCJ, "true");
- conf.set(ConfigUtils.USE_FREETEXT, "true");
- conf.set(ConfigUtils.USE_TEMPORAL, "true");
- conf.set(PrecomputedJoinIndexerConfig.PCJ_STORAGE_TYPE, PrecomputedJoinStorageType.ACCUMULO.name());
- conf.set(RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX, RYA_TABLE_PREFIX);
- conf.set(ConfigUtils.CLOUDBASE_USER, "root");
- conf.set(ConfigUtils.CLOUDBASE_PASSWORD, "");
- conf.set(ConfigUtils.CLOUDBASE_INSTANCE, INSTANCE);
- conf.setInt(ConfigUtils.NUM_PARTITIONS, 3);
- conf.set(ConfigUtils.CLOUDBASE_AUTHS, AUTHS);
-
- // only geo index statements with geo:asWKT predicates
- conf.set(ConfigUtils.GEO_PREDICATES_LIST,
- GeoConstants.GEO_AS_WKT.stringValue());
- return conf;
+
+ return AccumuloIndexingConfiguration.builder()//
--- End diff --
Nit: All of the "//" characters on the ends of these lines are unnecessary.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110959602
--- Diff: extras/indexingExample/src/main/java/RyaDirectExample.java ---
@@ -149,24 +153,19 @@ private static void closeQuietly(final SailRepositoryConnection conn) {
private static Configuration getConf() {
- final AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();
-
- conf.setBoolean(ConfigUtils.USE_MOCK_INSTANCE, USE_MOCK_INSTANCE);
- conf.set(ConfigUtils.USE_PCJ, "true");
- conf.set(ConfigUtils.USE_FREETEXT, "true");
- conf.set(ConfigUtils.USE_TEMPORAL, "true");
- conf.set(PrecomputedJoinIndexerConfig.PCJ_STORAGE_TYPE, PrecomputedJoinStorageType.ACCUMULO.name());
- conf.set(RdfCloudTripleStoreConfiguration.CONF_TBL_PREFIX, RYA_TABLE_PREFIX);
- conf.set(ConfigUtils.CLOUDBASE_USER, "root");
- conf.set(ConfigUtils.CLOUDBASE_PASSWORD, "");
- conf.set(ConfigUtils.CLOUDBASE_INSTANCE, INSTANCE);
- conf.setInt(ConfigUtils.NUM_PARTITIONS, 3);
- conf.set(ConfigUtils.CLOUDBASE_AUTHS, AUTHS);
-
- // only geo index statements with geo:asWKT predicates
- conf.set(ConfigUtils.GEO_PREDICATES_LIST,
- GeoConstants.GEO_AS_WKT.stringValue());
- return conf;
+
+ return AccumuloIndexingConfiguration.builder()//
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110946665
--- Diff: extras/indexingExample/src/main/java/EntityDirectExample.java ---
@@ -237,18 +238,15 @@ private static void testAddAndTemporalSearchWithPCJ(final SailRepositoryConnecti
private static Configuration getConf() {
- final AccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();
-
- conf.setBoolean(ConfigUtils.USE_MOCK_INSTANCE, USE_MOCK_INSTANCE);
- conf.set(ConfigUtils.USE_ENTITY, "true");
- conf.setTablePrefix(RYA_TABLE_PREFIX);
- conf.set(ConfigUtils.CLOUDBASE_USER, "root");
- conf.set(ConfigUtils.CLOUDBASE_PASSWORD, "");
- conf.set(ConfigUtils.CLOUDBASE_INSTANCE, INSTANCE);
- conf.setInt(ConfigUtils.NUM_PARTITIONS, 3);
- conf.set(ConfigUtils.CLOUDBASE_AUTHS, AUTHS);
-
- return conf;
+ return AccumuloIndexingConfiguration.builder()//
--- End diff --
Nit: Unnecessary "//" characters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by kchilton2 <gi...@git.apache.org>.
Github user kchilton2 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110945784
--- Diff: extras/indexingExample/src/main/java/RyaDirectExample.java ---
@@ -785,6 +784,13 @@ private static void createPCJ(final Configuration conf)
new PcjTables().createAndPopulatePcj(conn, accCon, tablename2,
queryString2, new String[] { "e", "c", "l", "o" },
Optional.<PcjVarOrderFactory> absent());
+
+ Scanner scanner = accCon.createScanner(tablename1, new Authorizations("U"));
--- End diff --
This is a resource leak. The scanner needs to be closed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #127: Rya 226 config builders
Posted by meiercaleb <gi...@git.apache.org>.
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/127#discussion_r110958765
--- Diff: dao/accumulo.rya/src/main/java/org/apache/rya/accumulo/AccumuloRdfConfiguration.java ---
@@ -68,10 +48,88 @@ public AccumuloRdfConfiguration(Configuration other) {
super(other);
}
+ public AccumuloRdfConfigurationBuilder getBuilder() {
+ return new AccumuloRdfConfigurationBuilder();
+ }
+
+ /**
+ * Creates an AccumuloRdfConfiguration object from a Properties file. This method assumes
+ * that all values in the Properties file are Strings and that the Properties file uses the keys below.
+ * See accumulo/rya/src/test/resources/properties/rya.properties for an example.
+ * <br>
+ * <ul>
+ * <li>"accumulo.auths" - String of Accumulo authorizations. Default is empty String.
+ * <li>"accumulo.visibilities" - String of Accumulo visibilities assigned to ingested triples.
+ * <li>"accumulo.instance" - Accumulo instance name (required)
+ * <li>"accumulo.user" - Accumulo user (required)
+ * <li>"accumulo.password" - Accumulo password (required)
+ * <li>"accumulo.rya.prefix" - Prefix for Accumulo backed Rya instance. Default is "rya_"
+ * <li>"accumulo.zookeepers" - Zookeepers for underlying Accumulo instance (required if not using Mock)
+ * <li>"use.mock" - Use a MockAccumulo instance as back-end for Rya instance. Default is false.
+ * <li>"use.prefix.hashing" - Use prefix hashing for triples. Helps avoid hot-spotting. Default is false.
+ * <li>"use.count.stats" - Use triple pattern cardinalities for query optimization. Default is false.
+ * <li>"use.join.selectivity" - Use join selectivity for query optimization. Default is false.
+ * <li>"use.display.plan" - Display query plan during evaluation. Useful for debugging. Default is true.
+ * <li>"use.inference" - Use backward chaining inference during query evaluation. Default is false.
+ * </ul>
+ * <br>
+ * @param props - Properties file containing Accumulo specific configuration parameters
+ * @return AccumumuloRdfConfiguration with properties set
+ */
+
+ public static AccumuloRdfConfiguration fromProperties(Properties props) {
+ return AccumuloRdfConfigurationBuilder.fromProperties(props).build();
+ }
+
@Override
public AccumuloRdfConfiguration clone() {
return new AccumuloRdfConfiguration(this);
}
+
+ public void setAccumuloUser(String user) {
+ Preconditions.checkNotNull(user);
--- End diff --
Done.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---