You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by mohammadshahidkhan <gi...@git.apache.org> on 2016/10/27 17:17:00 UTC
[GitHub] incubator-carbondata pull request #264: [CARBONDATA-341] CarbonTableIdentifi...
GitHub user mohammadshahidkhan opened a pull request:
https://github.com/apache/incubator-carbondata/pull/264
[CARBONDATA-341] CarbonTableIdentifier being passed to the query flow\u2026
# why
CarbonTableIdentifier being passed to the query flow has wrong tableid.
While creating the table the CarbonData system assign a uniqueID to the table. In all the places CarbonTableIdentifier should have the same tableId.
But CarbonTableIdentifier is having the currentTimeStamp as tableId which is not correct.
# Solution
Pass the absolutetableidentifier and carbontableidentifier loaded from the carbondata schema file.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mohammadshahidkhan/incubator-carbondata TableID_Fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/264.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #264
----
commit faa88070eeabe37be5db874c4b7908fc2a721da2
Author: mohammadshahidkhan <mo...@gmail.com>
Date: 2016-10-27T17:13:07Z
[CARBONDATA-341] CarbonTableIdentifier being passed to the query flow has wrong tableid
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.5.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/457/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #264: [CARBONDATA-341] CarbonTableIdentifi...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/264#discussion_r95140944
--- Diff: hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonInputFormat.java ---
@@ -188,13 +194,26 @@ public static void setSegmentsToAccess(Configuration configuration, List<String>
.set(CarbonInputFormat.INPUT_SEGMENT_NUMBERS, CarbonUtil.getSegmentString(validSegments));
}
- private static AbsoluteTableIdentifier getAbsoluteTableIdentifier(Configuration configuration) {
- String dirs = configuration.get(INPUT_DIR, "");
- String[] inputPaths = StringUtils.split(dirs);
- if (inputPaths.length == 0) {
- throw new InvalidPathException("No input paths specified in job");
+ private static AbsoluteTableIdentifier getAbsoluteTableIdentifier(Configuration configuration)
+ throws IOException {
+ CarbonTable carbonTable;
+ String carbonTableStr = configuration.get(CARBON_TABLE);
+ if (null == carbonTableStr) {
--- End diff --
divide functions as below.
> populateCarbonTable
> getCarbonTable will get and request populateCarbonTable if it doesnot exist in configuration
> getAbsoluteTableIdentifier gets from getCarbonTable.getAbsoluteTableIdentifier.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
refactored the code and added test case to verify that proper table is loaded with proper tableId.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.5.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/473/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Found you have checked in binary table schema for testcase, you cannot check-in binary file, please modify testcase
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/725/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #264: [CARBONDATA-341] CarbonTableIdentifi...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/264
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #264: [CARBONDATA-341] CarbonTableIdentifi...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/264
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #264: [CARBONDATA-341] CarbonTableIdentifi...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
GitHub user mohammadshahidkhan reopened a pull request:
https://github.com/apache/incubator-carbondata/pull/264
[CARBONDATA-341] CarbonTableIdentifier being passed to the query flow\u2026
# why
CarbonTableIdentifier being passed to the query flow has wrong tableid.
While creating the table the CarbonData system assign a uniqueID to the table. In all the places CarbonTableIdentifier should have the same tableId.
But CarbonTableIdentifier is having the currentTimeStamp as tableId which is not correct.
# Solution
Pass the absolutetableidentifier and carbontableidentifier loaded from the carbondata schema file.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mohammadshahidkhan/incubator-carbondata TableID_Fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/264.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #264
----
commit f56af7ba787bdb731e275aba30e2377b2626cb2a
Author: mohammadshahidkhan <mo...@gmail.com>
Date: 2017-01-05T13:15:51Z
[CARBONDATA-341] CarbonTableIdentifier being passed to the query flow has wrong tableid
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/728/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/726/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.5.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/522/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/714/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #264: [CARBONDATA-341] CarbonTableIdentifi...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/264
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/651/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/664/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #264: [CARBONDATA-341] CarbonTableIdentifi...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
GitHub user mohammadshahidkhan reopened a pull request:
https://github.com/apache/incubator-carbondata/pull/264
[CARBONDATA-341] CarbonTableIdentifier being passed to the query flow\u2026
# why
CarbonTableIdentifier being passed to the query flow has wrong tableid.
While creating the table the CarbonData system assign a uniqueID to the table. In all the places CarbonTableIdentifier should have the same tableId.
But CarbonTableIdentifier is having the currentTimeStamp as tableId which is not correct.
# Solution
Pass the absolutetableidentifier and carbontableidentifier loaded from the carbondata schema file.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mohammadshahidkhan/incubator-carbondata TableID_Fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/264.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #264
----
commit 95768fbec1d99d6ce49e1664641733e37286c504
Author: mohammadshahidkhan <mo...@gmail.com>
Date: 2017-01-05T13:15:51Z
[CARBONDATA-341] CarbonTableIdentifier being passed to the query flow has wrong tableid
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #264: [CARBONDATA-341] CarbonTableIdentifi...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/264#discussion_r95959773
--- Diff: hadoop/src/test/java/org/apache/carbondata/hadoop/CarbonInputFormatTest.java ---
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.carbondata.hadoop;
+
+import java.io.File;
+import java.lang.reflect.Method;
+
+import org.apache.carbondata.core.carbon.AbsoluteTableIdentifier;
+import org.apache.carbondata.core.carbon.metadata.schema.table.CarbonTable;
+
+import junit.framework.TestCase;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.mapred.JobConf;
+import org.apache.hadoop.mapreduce.Job;
+import org.junit.Assert;
+
+/**
+ * Class: To test loading the CarbonTableIdentifier from the schema file has the proper tableId
+ */
+public class CarbonInputFormatTest extends TestCase {
--- End diff --
Already CarbonInputFormat tests are available , please add to the same
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.5.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/523/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/653/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/652/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by gvramana <gi...@git.apache.org>.
Github user gvramana commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
retest this please
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #264: [CARBONDATA-341] CarbonTableIdentifier bein...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/incubator-carbondata/pull/264
Build Failed, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---