You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@rya.apache.org by ejwhite922 <gi...@git.apache.org> on 2018/04/27 19:13:34 UTC
[GitHub] incubator-rya pull request #294: RYA-492 Added language support for Literals
GitHub user ejwhite922 opened a pull request:
https://github.com/apache/incubator-rya/pull/294
RYA-492 Added language support for Literals
## Description
File loading wasn't working when a language tag was used with a Literal. RDF4J is stricter than OpenRDF with needing a language when the RDF.LANGSTRING type is used.
### Tests
Unit Tests
New Integration Test for loading files of all RDF formats
### Links
[Jira](https://issues.apache.org/jira/browse/RYA-492)
### Checklist
- [ ] Code Review
- [ ] Squash Commits
#### People To Review
@kchilton2
@isper3at
@pujav65
@DLotts
@jessehatfield
@meiercaleb
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ejwhite922/incubator-rya RYA-492_LanguageLiterals
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rya/pull/294.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #294
----
commit 32f25c62a06e6afdbc4a926f60ec4de48d14088f
Author: eric.white <er...@...>
Date: 2018-04-27T19:01:34Z
RYA-492 Added language support for Literals
----
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/294
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/754/
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/294
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/750/<h2>Failed Tests: <span class='status-failure'>3</span></h2><h3><a name='incubator-rya-master-with-optionals-pull-requests/org.apache.rya:rya.prospector' /><a href='https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/750/org.apache.rya$rya.prospector/testReport'>incubator-rya-master-with-optionals-pull-requests/org.apache.rya:rya.prospector</a>: <span class='status-failure'>3</span></h3><ul><li><a href='https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/750/org.apache.rya$rya.prospector/testReport/org.apache.rya.prospector.mr/ProspectorTest/testCount/'><strong>org.apache.rya.prospector.mr.ProspectorTest.testCount</strong></a></li><li><a href='https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/750/org.apache.rya$rya.prospector/testReport/org.apache.rya.prospector.service/ProspectorServiceEvalStatsDAOTest/testCount
/'><strong>org.apache.rya.prospector.service.ProspectorServiceEvalStatsDAOTest.testCount</strong></a></li><li><a href='https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/750/org.apache.rya$rya.prospector/testReport/org.apache.rya.prospector.service/ProspectorServiceEvalStatsDAOTest/testNoAuthsCount/'><strong>org.apache.rya.prospector.service.ProspectorServiceEvalStatsDAOTest.testNoAuthsCount</strong></a></li></ul>
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/294
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/748/
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on the issue:
https://github.com/apache/incubator-rya/pull/294
asfbot build
---
[GitHub] incubator-rya pull request #294: RYA-492 Added language support for Literals
Posted by pujav65 <gi...@git.apache.org>.
Github user pujav65 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/294#discussion_r185262879
--- Diff: common/rya.api/src/test/resources/ISO-3166-1_Country_Codes.txt ---
@@ -0,0 +1,236 @@
+AF
--- End diff --
what is this for?
---
[GitHub] incubator-rya pull request #294: RYA-492 Added language support for Literals
Posted by isper3at <gi...@git.apache.org>.
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/294#discussion_r186170248
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/dao/SimpleMongoDBStorageStrategy.java ---
@@ -194,6 +204,7 @@ public BasicDBObject serializeInternal(final RyaStatement statement){
.append(OBJECT, statement.getObject().getData())
.append(OBJECT_HASH, hash(statement.getObject().getData()))
.append(OBJECT_TYPE, statement.getObject().getDataType().toString())
+ .append(OBJECT_LANGUAGE, statement.getObject().getLanguage())
--- End diff --
can language be null here?
---
[GitHub] incubator-rya pull request #294: RYA-492 Added language support for Literals
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/294#discussion_r186189650
--- Diff: dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/dao/SimpleMongoDBStorageStrategy.java ---
@@ -194,6 +204,7 @@ public BasicDBObject serializeInternal(final RyaStatement statement){
.append(OBJECT, statement.getObject().getData())
.append(OBJECT_HASH, hash(statement.getObject().getData()))
.append(OBJECT_TYPE, statement.getObject().getDataType().toString())
+ .append(OBJECT_LANGUAGE, statement.getObject().getLanguage())
--- End diff --
Yea. It's fine though, no exception is thrown.
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on the issue:
https://github.com/apache/incubator-rya/pull/294
I'll squash it but it should need another sign off before merging.
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/294
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/753/
---
[GitHub] incubator-rya pull request #294: RYA-492 Added language support for Literals
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/294#discussion_r185265223
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/utils/LiteralLanguageUtils.java ---
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import org.eclipse.rdf4j.model.Literal;
+
+/**
+ * Utility methods and constants for {@link Literal} languages.
+ */
+public final class LiteralLanguageUtils {
+ /**
+ * Undetermined language type. Should be avoided if possible.
+ */
+ public static final String UNDETERMINED_LANGUAGE = "und";
--- End diff --
We do the same (RyaType will use XMLSchema.STRING if only data is used). This is for the case where a RyaType is being created and RDF.LANGSTRING is explicitly used but either no language was specified for whatever reason or that language does not appear valid.
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/294
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/749/
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by pujav65 <gi...@git.apache.org>.
Github user pujav65 commented on the issue:
https://github.com/apache/incubator-rya/pull/294
looks good -- let me know if its ready to merge
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/294
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/747/
---
[GitHub] incubator-rya pull request #294: RYA-492 Added language support for Literals
Posted by ejwhite922 <gi...@git.apache.org>.
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/294#discussion_r185266277
--- Diff: common/rya.api/src/test/resources/ISO-3166-1_Country_Codes.txt ---
@@ -0,0 +1,236 @@
+AF
--- End diff --
Testing. This is used to test some known country codes to make sure they can be used (i.e. if we see a Literal with a languageCode-countryCode of "en-US" we want to confirm it's valid so it'll work). The unit tests will check that all the country codes from this file can be used with all the language codes provided in the other txt files.
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/294
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/746/<h2>Failed Tests: <span class='status-failure'>1</span></h2><h3><a name='incubator-rya-master-with-optionals-pull-requests/org.apache.rya:rya.geo.mongo' /><a href='https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/746/org.apache.rya$rya.geo.mongo/testReport'>incubator-rya-master-with-optionals-pull-requests/org.apache.rya:rya.geo.mongo</a>: <span class='status-failure'>1</span></h3><ul><li><a href='https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/746/org.apache.rya$rya.geo.mongo/testReport/org.apache.rya.indexing.geotemporal.mongo/GeoTemporalMongoDBStorageStrategyTest/serializeTest/'><strong>org.apache.rya.indexing.geotemporal.mongo.GeoTemporalMongoDBStorageStrategyTest.serializeTest</strong></a></li></ul>
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/294
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/752/<h2>Build result: FAILURE</span></h2>[...truncated 633.85 KB...][ERROR] location: class org.apache.rya.sail.config.AccumuloRyaSailFactoryLoadFilesIT[ERROR] /home/jenkins/jenkins-slave/workspace/incubator-rya-master-with-optionals-pull-requests/extras/indexing/src/test/java/org/apache/rya/sail/config/AccumuloRyaSailFactoryLoadFilesIT.java:[79,28] cannot find symbol[ERROR] symbol: method getZookeepers()[ERROR] location: class org.apache.rya.sail.config.AccumuloRyaSailFactoryLoadFilesIT[ERROR] /home/jenkins/jenkins-slave/workspace/incubator-rya-master-with-optionals-pull-requests/extras/indexing/src/test/java/org/apache/rya/sail/config/AccumuloRyaSailFactoryLoadFilesIT.java:[80,29] cannot find symbol[ERROR] symbol: method getRyaInstanceName()[ERROR] location: class org.apache.rya.sail.config.AccumuloRyaSailFactoryLoadFilesIT[ERROR] /home/jenkins/jenkins-slave/workspace/incubator-rya-master-with-o
ptionals-pull-requests/extras/indexing/src/test/java/org/apache/rya/sail/config/AccumuloRyaSailFactoryLoadFilesIT.java:[164,49] cannot find symbol[ERROR] symbol: method getClass()[ERROR] location: class org.apache.rya.sail.config.AccumuloRyaSailFactoryLoadFilesIT[ERROR] -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn <goals> -rf :rya.indexingchannel stoppedSetting status of 4c4dc2bc4fb65b0c31eebff1ed1b4f89a7b8af81 to FAILURE with url https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/752/ and message: 'FAILURE 'Using context: Jenkins:
clean package -Pgeoindexing
---
[GitHub] incubator-rya pull request #294: RYA-492 Added language support for Literals
Posted by pujav65 <gi...@git.apache.org>.
Github user pujav65 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/294#discussion_r185262619
--- Diff: common/rya.api/src/main/java/org/apache/rya/api/utils/LiteralLanguageUtils.java ---
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.api.utils;
+
+import org.eclipse.rdf4j.model.Literal;
+
+/**
+ * Utility methods and constants for {@link Literal} languages.
+ */
+public final class LiteralLanguageUtils {
+ /**
+ * Undetermined language type. Should be avoided if possible.
+ */
+ public static final String UNDETERMINED_LANGUAGE = "und";
--- End diff --
Is that right? You might want to follow the pattern that rdf4j does:
https://github.com/eclipse/rdf4j/blob/7904f29ce3aafad9b38d1139978b9d5ed77b7df3/model/src/main/java/org/eclipse/rdf4j/model/impl/AbstractValueFactory.java#L102
Looks like they add a xmlstring as the language?
---
[GitHub] incubator-rya pull request #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-rya/pull/294
---
[GitHub] incubator-rya issue #294: RYA-492 Added language support for Literals
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/294
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/751/
---