You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by manishgupta88 <gi...@git.apache.org> on 2017/09/19 07:01:27 UTC
[GitHub] carbondata pull request #1368: [CARBONDATA-1486] Fixed issue of table status...
GitHub user manishgupta88 opened a pull request:
https://github.com/apache/carbondata/pull/1368
[CARBONDATA-1486] Fixed issue of table status updation on insert overwrite failure and exception thrown while deletion of stale folders
Issues Fixed as part of this PR:
1. Fixed issue of HDFS file system throwing exception while deleting a non-existent folder. This happens because on multiple run of insert overwrite operation, same folder is getting added to the stale folder list even though it has been deleted during first time insert overwrite operation run and when the same folder is tried to be deleted again HDFS file system throws an IO exception.
2. Insert or Load operation should throw exception if:
a. If insert overwrite is in progress and any other load or insert operation is triggered
b. If load or insert into operation is in progress and insert overwrite operation is triggered
3. Fixed issue for table status not getting updated after failure of insert/load job. In case of insert overwrite job status in table status file is "Overwrite in Progress". In this case no other load for same table can run in parallel and if this status is not changes on success of failure jobs, the system will still assume that insert overwrite is in progress and fail all next insert/load jobs
4. Removed Unused Code
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/manishgupta88/carbondata carbondata_issues
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1368.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1368
----
commit 9a1baaad8f2ff3ec53c4b546a944bb488aaf0a49
Author: manishgupta88 <to...@gmail.com>
Date: 2017-09-18T12:33:04Z
Fixed issue of HDFS file system throwing exception while deleting a non-existent folder. this happens because on multiple run of insert overwrite operation, same folder is getting added to the stale folder list even though it has been deleted during first time insert overwrite operation run and when the same folder is tried to be deleted again HDFS file system throws an exception.
Insert or Load operation should throw exception if:
1. If insert overwrite is in progress and any other load or insert operation is triggered
2. If load or insert into operation is in progress and insert overwrite operation is triggered
Fixed issue for table status not getting updated after failure of insert/load job. In case of insert overwrite job status in table status file is "Overwrite in Progress". In this case no other load for same table can run in parallel and if this status is not changes on success of failure jobs, the system will still assume that insert overwrite is in progress and fail all next insert/load jobs
Fixed fortify issue for the category "Identical code for different branches"
----
---
[GitHub] carbondata issue #1368: [CARBONDATA-1486] Fixed issue of table status updati...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1368
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/243/
---
[GitHub] carbondata issue #1368: [CARBONDATA-1486] Fixed issue of table status updati...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1368
Build Failed with Spark 1.6, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/104/
---
[GitHub] carbondata issue #1368: [CARBONDATA-1486] Fixed issue of table status updati...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1368
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/228/
---
[GitHub] carbondata pull request #1368: [CARBONDATA-1486] Fixed issue of table status...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/1368
---
[GitHub] carbondata issue #1368: [CARBONDATA-1486] Fixed issue of table status updati...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/1368
LGTM
---
[GitHub] carbondata issue #1368: [CARBONDATA-1486] Fixed issue of table status updati...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1368
Build Success with Spark 1.6, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/119/
---
[GitHub] carbondata issue #1368: [CARBONDATA-1486] Fixed issue of table status updati...
Posted by manishgupta88 <gi...@git.apache.org>.
Github user manishgupta88 commented on the issue:
https://github.com/apache/carbondata/pull/1368
retest this please
---
[GitHub] carbondata issue #1368: [CARBONDATA-1486] Fixed issue of table status updati...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1368
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/874/
---
[GitHub] carbondata issue #1368: [CARBONDATA-1486] Fixed issue of table status updati...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1368
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/859/
---