You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@rya.apache.org by DLotts <gi...@git.apache.org> on 2016/08/17 19:51:42 UTC
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
GitHub user DLotts opened a pull request:
https://github.com/apache/incubator-rya/pull/67
Rya 101 and 102 Mongo Rya example fails to initialize.
Initial fix. Blocked by RYA-133. More commits to come.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DLotts/incubator-rya RYA-101
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rya/pull/67.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #67
----
commit 328ac9ebd53f096bd31d45dfec1466773738f115
Author: David W. Lotts <da...@parsons.com>
Date: 2016-08-17T19:26:07Z
fixed, but RYA-133 is preventing test.
commit 03967dae06c6ca33381def4c10f308f518d267c8
Author: David W. Lotts <da...@parsons.com>
Date: 2016-08-17T19:36:29Z
Minor formating
commit 940d4240fe69dc543dd10f6439d60236b6bfb0b1
Author: David W. Lotts <da...@parsons.com>
Date: 2016-08-17T19:40:56Z
Minor import
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts closed the pull request at:
https://github.com/apache/incubator-rya/pull/67
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75197952
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoDBRyaDAO.java ---
@@ -73,14 +73,9 @@
*/
public MongoDBRyaDAO(final MongoDBRdfConfiguration conf) throws RyaDAOException, NumberFormatException, UnknownHostException {
this.conf = conf;
- try {
- mongoClient = MongoConnectorFactory.getMongoClient(conf);
- conf.setMongoClient(mongoClient);
- init();
- } catch (NumberFormatException | UnknownHostException e) {
- log.error("Unable to create a connection to mongo.", e);
- throw e;
- }
+ mongoClient = MongoConnectorFactory.getMongoClient(conf);
--- End diff --
why did you remove the log?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75326137
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
+ if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) {
--- End diff --
Private methods called from this method, correct? Great idea.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75197786
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
+ if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) {
+ try {
+ MongodForTestsFactory testsFactory = MongodForTestsFactory.with(Version.Main.PRODUCTION);
+ mongoClient = testsFactory.newMongo();
+ } catch (IOException e) {
+ // Rethrow as an unchecked error. Since we are in a test mode here, just fail fast.
+ throw new MongoException("While creating a factory for a test/mock MongoDB instance.",e);
+ }
} else {
- mongoClient = new MongoClient(server);
+ // Connect to a running Mongo server
+ final String host = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE), "MongoDB host name is required");
+ final int port = requireNonNullInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT),
+ "MongoDB Port number is required.");
+ ServerAddress server;
+ try {
+ server = new ServerAddress(host, port);
+ } catch (UnknownHostException e) {
+ throw new MongoException("Connecting MongoDB instance, cannot find host="+host,e);
+ }
+ // check for authentication credentials
+ if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
+ final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
+ final String dbName = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME),
+ MongoDBRdfConfiguration.MONGO_DB_NAME + " is null but required configuration if "
+ + MongoDBRdfConfiguration.MONGO_USER + " is configured.");
+ final char[] pswd = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD),
+ MongoDBRdfConfiguration.MONGO_USER_PASSWORD + " is null but required configuration if "
+ + MongoDBRdfConfiguration.MONGO_USER + " is configured.").toCharArray();
+ final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
+ mongoClient = new MongoClient(server, Arrays.asList(cred));
+ } else {
+ // No user was configured:
+ mongoClient = new MongoClient(server);
+ }
}
}
return mongoClient;
}
+
+ /**
+ * Throw exception for un-configured required values.
+ *
+ * @param required String to check
+ * @param message throw configuration exception with this description
+ * @return unaltered required string
+ * @throws ConfigurationRuntimeException if required is null
+ */
+ static private String requireNonNull(String required, String message) throws ConfigurationRuntimeException {
--- End diff --
what Aaron said. This is not needed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75197851
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
+ if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) {
+ try {
+ MongodForTestsFactory testsFactory = MongodForTestsFactory.with(Version.Main.PRODUCTION);
+ mongoClient = testsFactory.newMongo();
+ } catch (IOException e) {
+ // Rethrow as an unchecked error. Since we are in a test mode here, just fail fast.
+ throw new MongoException("While creating a factory for a test/mock MongoDB instance.",e);
+ }
} else {
- mongoClient = new MongoClient(server);
+ // Connect to a running Mongo server
+ final String host = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE), "MongoDB host name is required");
+ final int port = requireNonNullInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT),
+ "MongoDB Port number is required.");
+ ServerAddress server;
+ try {
+ server = new ServerAddress(host, port);
+ } catch (UnknownHostException e) {
+ throw new MongoException("Connecting MongoDB instance, cannot find host="+host,e);
+ }
+ // check for authentication credentials
+ if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
+ final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
+ final String dbName = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME),
+ MongoDBRdfConfiguration.MONGO_DB_NAME + " is null but required configuration if "
+ + MongoDBRdfConfiguration.MONGO_USER + " is configured.");
+ final char[] pswd = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD),
+ MongoDBRdfConfiguration.MONGO_USER_PASSWORD + " is null but required configuration if "
+ + MongoDBRdfConfiguration.MONGO_USER + " is configured.").toCharArray();
+ final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
+ mongoClient = new MongoClient(server, Arrays.asList(cred));
+ } else {
+ // No user was configured:
+ mongoClient = new MongoClient(server);
+ }
}
}
return mongoClient;
}
+
+ /**
+ * Throw exception for un-configured required values.
+ *
+ * @param required String to check
+ * @param message throw configuration exception with this description
+ * @return unaltered required string
+ * @throws ConfigurationRuntimeException if required is null
+ */
+ static private String requireNonNull(String required, String message) throws ConfigurationRuntimeException {
+ if (required == null)
+ throw new ConfigurationRuntimeException(message);
+ return required;
+ }
+
+ /*
+ * Same as above, check that it is a integer and return the parsed integer.
+ */
+ static private int requireNonNullInt(String required, String message) throws ConfigurationRuntimeException {
--- End diff --
same as above, as a general note scope is the first keyword, then static.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75219206
--- Diff: extras/indexing/src/main/java/mvm/rya/indexing/FilterFunctionOptimizer.java ---
@@ -99,29 +99,19 @@ public void setConf(final Configuration conf) {
this.conf = conf;
//reset the init.
init = false;
- try {
init();
- } catch (final NumberFormatException | UnknownHostException e) {
- LOG.error("Unable to update to use new config, falling back to the old config.", e);
- init = true;
- }
}
- private synchronized void init() throws NumberFormatException, UnknownHostException {
+ private synchronized void init() {
if (!init) {
if (ConfigUtils.getUseMongo(conf)) {
- try {
final MongoClient mongoClient = MongoConnectorFactory.getMongoClient(conf);
geoIndexer = new MongoGeoIndexer(mongoClient);
geoIndexer.setConf(conf);
freeTextIndexer = new MongoFreeTextIndexer(mongoClient);
freeTextIndexer.setConf(conf);
temporalIndexer = new MongoTemporalIndexer(mongoClient);
temporalIndexer.setConf(conf);
- } catch (NumberFormatException | UnknownHostException e) {
--- End diff --
Same as the first one. Now redundant, and the catch is no longer required.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75197704
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
+ if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) {
+ try {
+ MongodForTestsFactory testsFactory = MongodForTestsFactory.with(Version.Main.PRODUCTION);
+ mongoClient = testsFactory.newMongo();
+ } catch (IOException e) {
+ // Rethrow as an unchecked error. Since we are in a test mode here, just fail fast.
+ throw new MongoException("While creating a factory for a test/mock MongoDB instance.",e);
+ }
} else {
- mongoClient = new MongoClient(server);
+ // Connect to a running Mongo server
+ final String host = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE), "MongoDB host name is required");
+ final int port = requireNonNullInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT),
+ "MongoDB Port number is required.");
+ ServerAddress server;
+ try {
+ server = new ServerAddress(host, port);
+ } catch (UnknownHostException e) {
+ throw new MongoException("Connecting MongoDB instance, cannot find host="+host,e);
+ }
+ // check for authentication credentials
+ if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
+ final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
+ final String dbName = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME),
+ MongoDBRdfConfiguration.MONGO_DB_NAME + " is null but required configuration if "
+ + MongoDBRdfConfiguration.MONGO_USER + " is configured.");
+ final char[] pswd = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD),
--- End diff --
the requireNonNull() should be before this. assignment should only happen after the precondition checks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75218928
--- Diff: extras/indexing/src/main/java/mvm/rya/indexing/FilterFunctionOptimizer.java ---
@@ -99,29 +99,19 @@ public void setConf(final Configuration conf) {
this.conf = conf;
//reset the init.
init = false;
- try {
init();
- } catch (final NumberFormatException | UnknownHostException e) {
- LOG.error("Unable to update to use new config, falling back to the old config.", e);
--- End diff --
Same reason. But in addition, this is just a bug and a fib. It ignores what error happened and any diagnostic information. It does not restore the old config if there was one. It sets init = false; so that it does not attempt to re-initialize. The next thing that happens is a failure in distant code that assumes that the database is correctly connected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75217446
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
+ if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) {
+ try {
+ MongodForTestsFactory testsFactory = MongodForTestsFactory.with(Version.Main.PRODUCTION);
+ mongoClient = testsFactory.newMongo();
+ } catch (IOException e) {
+ // Rethrow as an unchecked error. Since we are in a test mode here, just fail fast.
+ throw new MongoException("While creating a factory for a test/mock MongoDB instance.",e);
+ }
} else {
- mongoClient = new MongoClient(server);
+ // Connect to a running Mongo server
+ final String host = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE), "MongoDB host name is required");
--- End diff --
This seems like an abuse of try/catch. You generally do a check on the config values way before this point. Doing it this way can also be super annoying if multiple values are missing. Run, break, missing a. Run, break, missing c. Run, break, missing e, etc.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75197992
--- Diff: extras/indexing/src/main/java/mvm/rya/indexing/FilterFunctionOptimizer.java ---
@@ -99,29 +99,19 @@ public void setConf(final Configuration conf) {
this.conf = conf;
//reset the init.
init = false;
- try {
init();
- } catch (final NumberFormatException | UnknownHostException e) {
- LOG.error("Unable to update to use new config, falling back to the old config.", e);
--- End diff --
same here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75216775
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoDBRyaDAO.java ---
@@ -73,14 +73,9 @@
*/
public MongoDBRyaDAO(final MongoDBRdfConfiguration conf) throws RyaDAOException, NumberFormatException, UnknownHostException {
this.conf = conf;
- try {
- mongoClient = MongoConnectorFactory.getMongoClient(conf);
- conf.setMongoClient(mongoClient);
- init();
- } catch (NumberFormatException | UnknownHostException e) {
- log.error("Unable to create a connection to mongo.", e);
- throw e;
- }
+ mongoClient = MongoConnectorFactory.getMongoClient(conf);
--- End diff --
the UnknownHostException is a checked exception and must be caught. I removed both of the exceptions and wrapped into better unchecked exceptions. I assume that the default error handler will log the error. Otherwise we should attempt to log every unchecked exception. I made the underlying thrown errors more descriptive, they all mention that the MongoDB connection has failed, so this log entry is redundant.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75211963
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
+ if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) {
+ try {
+ MongodForTestsFactory testsFactory = MongodForTestsFactory.with(Version.Main.PRODUCTION);
+ mongoClient = testsFactory.newMongo();
+ } catch (IOException e) {
+ // Rethrow as an unchecked error. Since we are in a test mode here, just fail fast.
+ throw new MongoException("While creating a factory for a test/mock MongoDB instance.",e);
+ }
} else {
- mongoClient = new MongoClient(server);
+ // Connect to a running Mongo server
+ final String host = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE), "MongoDB host name is required");
--- End diff --
I started with that. (Objects.requireNonNull() is similar) But I wanted to throw a different exception, not NullPointerException since the caller may want to specifically catch it. So then I wrapped the whole thing in a try with a catch that re-throws. But then it catches ANY null pointer, not just the ones I checked and it became more complicated, so I did it this way. What do you think, is it worth it to throw a different unchecked error?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75197387
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
--- End diff --
this doc is misleading. There is only one, but it isn't a singleton.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya issue #67: Rya 101 and 102 Mongo Rya example fails to initiali...
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts commented on the issue:
https://github.com/apache/incubator-rya/pull/67
This is complete and ready to review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya issue #67: Rya 101 and 102 Mongo Rya example fails to initiali...
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts commented on the issue:
https://github.com/apache/incubator-rya/pull/67
This was pulled into apache:develop !
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by amihalik <gi...@git.apache.org>.
Github user amihalik commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75194741
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
+ if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) {
+ try {
+ MongodForTestsFactory testsFactory = MongodForTestsFactory.with(Version.Main.PRODUCTION);
+ mongoClient = testsFactory.newMongo();
+ } catch (IOException e) {
+ // Rethrow as an unchecked error. Since we are in a test mode here, just fail fast.
+ throw new MongoException("While creating a factory for a test/mock MongoDB instance.",e);
+ }
} else {
- mongoClient = new MongoClient(server);
+ // Connect to a running Mongo server
+ final String host = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE), "MongoDB host name is required");
--- End diff --
Use guava preconditions instead (e.g. [here](https://github.com/apache/incubator-rya/blob/7743a42a5febe440107e037aadc3753643708e83/common/rya.api/src/main/java/mvm/rya/api/RdfCloudTripleStoreConfiguration.java#L167))
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75197537
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
+ if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) {
--- End diff --
break these out into their own methods, getTestMongoClient()/getMongoClient()
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by DLotts <gi...@git.apache.org>.
Github user DLotts commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75215675
--- Diff: dao/mongodb.rya/src/main/java/mvm/rya/mongodb/MongoConnectorFactory.java ---
@@ -41,26 +47,77 @@
* @param conf The {@link Configuration} defining how to construct the MongoClient.
* @return A {@link MongoClient}. This client is lazy loaded and the same one
* is used throughout the lifecycle of the application.
- * @throws NumberFormatException - Thrown if the configured port is not a valid number
- * @throws UnknownHostException - The configured host cannot be found.
+ * @throws IOException - if MongodForTestsFactory constructor has an io exception.
+ * @throws ConfigurationRuntimeException - Thrown if the configured server, port, user, or others are missing.
+ * @throws UnknownHostException - if MongoDB host name is not found.
*/
- public static synchronized MongoClient getMongoClient(final Configuration conf) throws NumberFormatException, UnknownHostException {
- if(mongoClient == null) {
- final String host = conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE);
- final int port = Integer.parseInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT));
- final ServerAddress server = new ServerAddress(host, port);
-
- //check for authentication credentials
- if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
- final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
- final String dbName = conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME);
- final char[] pswd = conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD).toCharArray();
- final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
- mongoClient = new MongoClient(server, Arrays.asList(cred));
+ public static synchronized MongoClient getMongoClient(final Configuration conf)
+ throws ConfigurationRuntimeException, MongoException {
+ if (mongoClient == null) {
+ // The singleton client has not yet created, is it a test/mock instance, or a service?
+ if (conf.getBoolean(MongoDBRdfConfiguration.USE_TEST_MONGO, false)) {
+ try {
+ MongodForTestsFactory testsFactory = MongodForTestsFactory.with(Version.Main.PRODUCTION);
+ mongoClient = testsFactory.newMongo();
+ } catch (IOException e) {
+ // Rethrow as an unchecked error. Since we are in a test mode here, just fail fast.
+ throw new MongoException("While creating a factory for a test/mock MongoDB instance.",e);
+ }
} else {
- mongoClient = new MongoClient(server);
+ // Connect to a running Mongo server
+ final String host = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE), "MongoDB host name is required");
+ final int port = requireNonNullInt(conf.get(MongoDBRdfConfiguration.MONGO_INSTANCE_PORT),
+ "MongoDB Port number is required.");
+ ServerAddress server;
+ try {
+ server = new ServerAddress(host, port);
+ } catch (UnknownHostException e) {
+ throw new MongoException("Connecting MongoDB instance, cannot find host="+host,e);
+ }
+ // check for authentication credentials
+ if (conf.get(MongoDBRdfConfiguration.MONGO_USER) != null) {
+ final String username = conf.get(MongoDBRdfConfiguration.MONGO_USER);
+ final String dbName = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_DB_NAME),
+ MongoDBRdfConfiguration.MONGO_DB_NAME + " is null but required configuration if "
+ + MongoDBRdfConfiguration.MONGO_USER + " is configured.");
+ final char[] pswd = requireNonNull(conf.get(MongoDBRdfConfiguration.MONGO_USER_PASSWORD),
+ MongoDBRdfConfiguration.MONGO_USER_PASSWORD + " is null but required configuration if "
+ + MongoDBRdfConfiguration.MONGO_USER + " is configured.").toCharArray();
+ final MongoCredential cred = MongoCredential.createCredential(username, dbName, pswd);
+ mongoClient = new MongoClient(server, Arrays.asList(cred));
+ } else {
+ // No user was configured:
+ mongoClient = new MongoClient(server);
+ }
}
}
return mongoClient;
}
+
+ /**
+ * Throw exception for un-configured required values.
+ *
+ * @param required String to check
+ * @param message throw configuration exception with this description
+ * @return unaltered required string
+ * @throws ConfigurationRuntimeException if required is null
+ */
+ static private String requireNonNull(String required, String message) throws ConfigurationRuntimeException {
--- End diff --
In addition to what I said to Aaron: I am preserving the difference between a missing configuration and a random NullPointerException and NumberFormatException.
I'll switch it back if you all consider this unimportant. The method now throws two unchecked, and IMO not recoverable errors: ConfigurationRuntimeException for configuration parameters that are missing, and MongoException for issues preventing connection, possibly temporary. But those issues can be specifically caught, for example: to retry. A NullPointerException could be thrown for any reason.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #67: Rya 101 and 102 Mongo Rya example fails to i...
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/67#discussion_r75198037
--- Diff: extras/indexing/src/main/java/mvm/rya/indexing/FilterFunctionOptimizer.java ---
@@ -99,29 +99,19 @@ public void setConf(final Configuration conf) {
this.conf = conf;
//reset the init.
init = false;
- try {
init();
- } catch (final NumberFormatException | UnknownHostException e) {
- LOG.error("Unable to update to use new config, falling back to the old config.", e);
- init = true;
- }
}
- private synchronized void init() throws NumberFormatException, UnknownHostException {
+ private synchronized void init() {
if (!init) {
if (ConfigUtils.getUseMongo(conf)) {
- try {
final MongoClient mongoClient = MongoConnectorFactory.getMongoClient(conf);
geoIndexer = new MongoGeoIndexer(mongoClient);
geoIndexer.setConf(conf);
freeTextIndexer = new MongoFreeTextIndexer(mongoClient);
freeTextIndexer.setConf(conf);
temporalIndexer = new MongoTemporalIndexer(mongoClient);
temporalIndexer.setConf(conf);
- } catch (NumberFormatException | UnknownHostException e) {
--- End diff --
same here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---