You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by dhatchayani <gi...@git.apache.org> on 2017/11/27 09:47:13 UTC
[GitHub] carbondata pull request #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION...
GitHub user dhatchayani opened a pull request:
https://github.com/apache/carbondata/pull/1574
[CARBONDATA-1816] Changing BAD_RECORDS_ACTION default action to FAIL
Currently, the default action is FORCE, this may allow the user by default to load bad records also. So, changing the default action of BAD_RECORDS_ACTION to FAIL will alert the user if any bad_records are there in the loading data.
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dhatchayani/incubator-carbondata bad_rec_action
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1574.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1574
----
commit 40c2c3d111e8b100cbf42e38c60649caed461d46
Author: dhatchayani <dh...@gmail.com>
Date: 2017-11-27T09:45:17Z
[CARBONDATA-1816] Changing BAD_RECORDS_ACTION default action to FAIL
----
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by manishgupta88 <gi...@git.apache.org>.
Github user manishgupta88 commented on the issue:
https://github.com/apache/carbondata/pull/1574
LGTM
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by dhatchayani <gi...@git.apache.org>.
Github user dhatchayani commented on the issue:
https://github.com/apache/carbondata/pull/1574
retest sdv please
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1574
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1952/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1574
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1490/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1574
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1987/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1574
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1919/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1574
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1499/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1574
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1554/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1574
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1953/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1574
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1990/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1574
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1492/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by dhatchayani <gi...@git.apache.org>.
Github user dhatchayani commented on the issue:
https://github.com/apache/carbondata/pull/1574
retest sdv please
---
[GitHub] carbondata pull request #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION...
Posted by manishgupta88 <gi...@git.apache.org>.
Github user manishgupta88 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1574#discussion_r153428387
--- Diff: integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/AlterTableTestCase.scala ---
@@ -1067,7 +1067,7 @@ class AlterTableTestCase extends QueryTest with BeforeAndAfterAll {
val p3 = prop.getProperty("carbon.horizontal.delete.compaction.threshold", CarbonCommonConstants.DEFAULT_DELETE_DELTAFILE_COUNT_THRESHOLD_IUD_COMPACTION)
val p4 = prop.getProperty("carbon.compaction.level.threshold", CarbonCommonConstants.DEFAULT_SEGMENT_LEVEL_THRESHOLD)
val p5 = prop.getProperty("carbon.enable.auto.load.merge", CarbonCommonConstants.DEFAULT_ENABLE_AUTO_LOAD_MERGE)
- val p6 = prop.getProperty("carbon.bad.records.action", CarbonCommonConstants.CARBON_BAD_RECORDS_ACTION_DEFAULT)
+ val p6 = prop.getProperty("carbon.bad.records.action", "FORCE")
--- End diff --
instead of hard coding "FORCE" use enum LoggerAction.Force.name
Do it everywhere you have made the code changes
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1574
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1954/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by dhatchayani <gi...@git.apache.org>.
Github user dhatchayani commented on the issue:
https://github.com/apache/carbondata/pull/1574
retest this please
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1574
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1506/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by dhatchayani <gi...@git.apache.org>.
Github user dhatchayani commented on the issue:
https://github.com/apache/carbondata/pull/1574
retest this please
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1574
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1488/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1574
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1509/
---
[GitHub] carbondata pull request #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/1574
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by dhatchayani <gi...@git.apache.org>.
Github user dhatchayani commented on the issue:
https://github.com/apache/carbondata/pull/1574
retest sdv please
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by dhatchayani <gi...@git.apache.org>.
Github user dhatchayani commented on the issue:
https://github.com/apache/carbondata/pull/1574
retest sdv please
---
[GitHub] carbondata pull request #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1574#discussion_r153420051
--- Diff: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java ---
@@ -1276,7 +1276,7 @@
@CarbonProperty
public static final String CARBON_BAD_RECORDS_ACTION = "carbon.bad.records.action";
- public static final String CARBON_BAD_RECORDS_ACTION_DEFAULT = "FORCE";
+ public static final String CARBON_BAD_RECORDS_ACTION_DEFAULT = "FAIL";
--- End diff --
Can you add comment to describe the behavior for 'FAIL'
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1574
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1503/
---
[GitHub] carbondata issue #1574: [CARBONDATA-1816] Changing BAD_RECORDS_ACTION defaul...
Posted by dhatchayani <gi...@git.apache.org>.
Github user dhatchayani commented on the issue:
https://github.com/apache/carbondata/pull/1574
retest sdv please
---