You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by BJangir <gi...@git.apache.org> on 2018/04/28 00:30:34 UTC
[GitHub] carbondata pull request #2245: [CARBONDATA-2411] infinite loop when sdk writ...
GitHub user BJangir opened a pull request:
https://github.com/apache/carbondata/pull/2245
[CARBONDATA-2411] infinite loop when sdk writer throws Exception
Issue & root Cause :- Exception is not handled when Object is cast to String[] in writer method of CarbonWriter so no Exception thrown to caller and data-loading threads is not interrupted which is leading complete sdk application to infinite loop ( Or data loading max timeout 2 Day).
Solution :- Throw Exception to caller and one Exception call writer close method .
- [ ] Any interfaces changed?
NO
- [ ] Any backward compatibility impacted?
NO
- [ ] Document update required?
NO
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
Testcase Addded
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/BJangir/incubator-carbondata CARBONDATA-2411
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2245.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2245
----
commit e9f0ccc581d41414fc3001c183b72c3b20fc4c93
Author: BJangir <ba...@...>
Date: 2018-04-28T00:15:04Z
[CARBONDATA-2411] infinite loop when sdk writer throws Exception
----
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by kumarvishal09 <gi...@git.apache.org>.
Github user kumarvishal09 commented on the issue:
https://github.com/apache/carbondata/pull/2245
LGTM
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2245
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5485/
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2245
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4320/
---
[GitHub] carbondata pull request #2245: [CARBONDATA-2411] infinite loop when sdk writ...
Posted by BJangir <gi...@git.apache.org>.
Github user BJangir commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2245#discussion_r184945715
--- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java ---
@@ -335,10 +335,12 @@ private CarbonTable buildCarbonTable() {
// When dimensions are default set to sort column,
// Inverted index will be supported by default for sort columns.
for (Field field : schema.getFields()) {
- if (field.getDataType() == DataTypes.STRING ||
- field.getDataType() == DataTypes.DATE ||
- field.getDataType() == DataTypes.TIMESTAMP) {
- sortColumnsList.add(field.getFieldName());
+ if (null != field) {
--- End diff --
Fixed. added comment
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2245
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5533/
---
[GitHub] carbondata pull request #2245: [CARBONDATA-2411] infinite loop when sdk writ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2245
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2245
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4319/
---
[GitHub] carbondata pull request #2245: [CARBONDATA-2411] infinite loop when sdk writ...
Posted by kumarvishal09 <gi...@git.apache.org>.
Github user kumarvishal09 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2245#discussion_r184937474
--- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonWriterBuilder.java ---
@@ -335,10 +335,12 @@ private CarbonTable buildCarbonTable() {
// When dimensions are default set to sort column,
// Inverted index will be supported by default for sort columns.
for (Field field : schema.getFields()) {
- if (field.getDataType() == DataTypes.STRING ||
- field.getDataType() == DataTypes.DATE ||
- field.getDataType() == DataTypes.TIMESTAMP) {
- sortColumnsList.add(field.getFieldName());
+ if (null != field) {
--- End diff --
Please add comment when field can be null
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2245
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4588/
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2245
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4368/
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2245
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4587/
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2245
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/5486/
---
[GitHub] carbondata issue #2245: [CARBONDATA-2411] infinite loop when sdk writer thro...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2245
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4633/
---