You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by manishgupta88 <gi...@git.apache.org> on 2018/07/12 16:09:29 UTC
[GitHub] carbondata pull request #2499: [CARBONDATA-2648] Fixed NPE issue with legacy...
GitHub user manishgupta88 opened a pull request:
https://github.com/apache/carbondata/pull/2499
[CARBONDATA-2648] Fixed NPE issue with legacy store when CACHE_LEVEL is Blocklet
Things done as part of this PR:
1. Fixed Null pointer exception when store is of <= 1.1 version and DataMap is of type BlockletDataMap.
2. Added clearing of SegmentProperties cache holder from executor
**Problem 1:**
Null pointer exception thrown when store is of <= 1.1 version and DataMap is of type BlockletDataMap.
**Analysis:**
In BlcokletDataMap schema is created to consider blockletInfo while adding to unsafe but in case of legacy store we dont weite the blocklet Info. This lead to Null pointer exception while calculating the size of row during adding to unsafe.
**Solution**
For legacy store always call super class methods which is BlockDataMap.
- [ ] Any interfaces changed?
No
- [ ] Any backward compatibility impacted?
NA
- [ ] Document update required?
No
- [ ] Testing done
Manually verified
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/manishgupta88/carbondata configurable_cache_columns
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2499.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2499
----
commit edbff70ed7c650c6aa36ac397c1c86c2bad3fb1d
Author: manishgupta88 <to...@...>
Date: 2018-07-12T16:05:45Z
1. Fixed Null pointer exception when store is of <= 1.1 version and DataMap is of type BlockletDataMap.
2. Added clearing of SegmentProperties cache holder from executor
----
---
[GitHub] carbondata issue #2499: [CARBONDATA-2648] Fixed NPE issue with legacy store ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2499
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5811/
---
[GitHub] carbondata issue #2499: [CARBONDATA-2648] Fixed NPE issue with legacy store ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2499
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7078/
---
[GitHub] carbondata pull request #2499: [CARBONDATA-2648] Fixed NPE issue with legacy...
Posted by manishgupta88 <gi...@git.apache.org>.
Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2499#discussion_r202278145
--- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/TableBlockIndexUniqueIdentifierWrapper.java ---
@@ -35,18 +35,34 @@
// holds the reference to CarbonTable
private CarbonTable carbonTable;
+ /**
+ * flag to specify whether to load table block metadata in unsafe or safe. Default value is true
+ */
+ private boolean addTableBlockToUnsafe = true;
public TableBlockIndexUniqueIdentifierWrapper(
TableBlockIndexUniqueIdentifier tableBlockIndexUniqueIdentifier, CarbonTable carbonTable) {
this.tableBlockIndexUniqueIdentifier = tableBlockIndexUniqueIdentifier;
this.carbonTable = carbonTable;
}
+ public TableBlockIndexUniqueIdentifierWrapper(
--- End diff --
As discussed I have added a Note
---
[GitHub] carbondata pull request #2499: [CARBONDATA-2648] Fixed NPE issue with legacy...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2499#discussion_r202253245
--- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/TableBlockIndexUniqueIdentifierWrapper.java ---
@@ -35,18 +35,34 @@
// holds the reference to CarbonTable
private CarbonTable carbonTable;
+ /**
+ * flag to specify whether to load table block metadata in unsafe or safe. Default value is true
+ */
+ private boolean addTableBlockToUnsafe = true;
public TableBlockIndexUniqueIdentifierWrapper(
TableBlockIndexUniqueIdentifier tableBlockIndexUniqueIdentifier, CarbonTable carbonTable) {
this.tableBlockIndexUniqueIdentifier = tableBlockIndexUniqueIdentifier;
this.carbonTable = carbonTable;
}
+ public TableBlockIndexUniqueIdentifierWrapper(
--- End diff --
What is the use of this constructor when no body uses it?
---
[GitHub] carbondata pull request #2499: [CARBONDATA-2648] Fixed NPE issue with legacy...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2499
---
[GitHub] carbondata issue #2499: [CARBONDATA-2648] Fixed NPE issue with legacy store ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2499
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5855/
---
[GitHub] carbondata issue #2499: [CARBONDATA-2648] Fixed NPE issue with legacy store ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2499
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5848/
---
[GitHub] carbondata issue #2499: [CARBONDATA-2648] Fixed NPE issue with legacy store ...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2499
LGTM
---
[GitHub] carbondata issue #2499: [CARBONDATA-2648] Fixed NPE issue with legacy store ...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2499
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7069/
---