You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by mohammadshahidkhan <gi...@git.apache.org> on 2017/12/15 10:11:34 UTC
[GitHub] carbondata pull request #1666: table status timezone problem fix
GitHub user mohammadshahidkhan opened a pull request:
https://github.com/apache/carbondata/pull/1666
table status timezone problem fix
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mohammadshahidkhan/incubator-carbondata tablestatus_timezone_fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1666.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1666
----
commit dbfd0db3453ca4ee83869ecfd4c83b78373e22c0
Author: mohammadshahidkhan <mo...@gmail.com>
Date: 2017-12-15T09:04:18Z
table status timezone problem fix
----
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan commented on the issue:
https://github.com/apache/carbondata/pull/1666
This PR is ok, no code change after first push only pushed the commit message change
@ravipesala Please review
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1666
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2383/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2029/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1666
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2387/
---
[GitHub] carbondata pull request #1666: [CARBONDATA-1900][Core,processing] Modify loa...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/1666
---
[GitHub] carbondata issue #1666: table status timezone problem fix
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/779/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2117/
---
[GitHub] carbondata pull request #1666: [CARBONDATA-1900][Core,processing] Modify loa...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1666#discussion_r157420301
--- Diff: core/src/main/java/org/apache/carbondata/core/statusmanager/LoadMetadataDetails.java ---
@@ -211,25 +234,32 @@ public long getLoadStartTimeAsLong() {
* @return
*/
private long convertTimeStampToLong(String factTimeStamp) {
--- End diff --
What is the difference between this function and getTimeStamp, seems they are doing the same thing
---
[GitHub] carbondata pull request #1666: [CARBONDATA-1900][Core,processing] Modify loa...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1666#discussion_r157356451
--- Diff: core/src/main/java/org/apache/carbondata/core/statusmanager/LoadMetadataDetails.java ---
@@ -241,26 +248,28 @@ private long convertTimeStampToLong(String factTimeStamp) {
* @return
*/
public Long getTimeStamp(String loadStartTime) {
- Date dateToStr = null;
try {
- dateToStr = parser.parse(loadStartTime);
- return dateToStr.getTime() * 1000;
- } catch (ParseException e) {
- LOGGER.error("Cannot convert" + loadStartTime + " to Time/Long type value" + e.getMessage());
- return null;
+ // for new loads the factTimeStamp will be long string
--- End diff --
In future for maintenance it is not easy to understand what new load is, can you add more background information for better maintenance.
---
[GitHub] carbondata issue #1666: table status timezone problem fix
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2005/
---
[GitHub] carbondata pull request #1666: [CARBONDATA-1900][Core,processing] Modify loa...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1666#discussion_r157428485
--- Diff: core/src/main/java/org/apache/carbondata/core/statusmanager/LoadMetadataDetails.java ---
@@ -211,25 +234,32 @@ public long getLoadStartTimeAsLong() {
* @return
*/
private long convertTimeStampToLong(String factTimeStamp) {
- SimpleDateFormat parser = new SimpleDateFormat(CarbonCommonConstants.CARBON_TIMESTAMP_MILLIS);
- Date dateToStr = null;
try {
- dateToStr = parser.parse(factTimeStamp);
- return dateToStr.getTime();
- } catch (ParseException e) {
- LOGGER.error("Cannot convert" + factTimeStamp + " to Time/Long type value" + e.getMessage());
- parser = new SimpleDateFormat(CarbonCommonConstants.CARBON_TIMESTAMP);
+ // for new loads the factTimeStamp will be long string
--- End diff --
Fixed
---
[GitHub] carbondata pull request #1666: [CARBONDATA-1900][Core,processing] Modify loa...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1666#discussion_r157428609
--- Diff: core/src/main/java/org/apache/carbondata/core/statusmanager/LoadMetadataDetails.java ---
@@ -241,26 +248,28 @@ private long convertTimeStampToLong(String factTimeStamp) {
* @return
*/
public Long getTimeStamp(String loadStartTime) {
- Date dateToStr = null;
try {
- dateToStr = parser.parse(loadStartTime);
- return dateToStr.getTime() * 1000;
- } catch (ParseException e) {
- LOGGER.error("Cannot convert" + loadStartTime + " to Time/Long type value" + e.getMessage());
- return null;
+ // for new loads the factTimeStamp will be long string
--- End diff --
fixed
---
[GitHub] carbondata pull request #1666: [CARBONDATA-1900][Core,processing] Modify loa...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1666#discussion_r157420112
--- Diff: core/src/main/java/org/apache/carbondata/core/statusmanager/LoadMetadataDetails.java ---
@@ -211,25 +234,32 @@ public long getLoadStartTimeAsLong() {
* @return
*/
private long convertTimeStampToLong(String factTimeStamp) {
- SimpleDateFormat parser = new SimpleDateFormat(CarbonCommonConstants.CARBON_TIMESTAMP_MILLIS);
- Date dateToStr = null;
try {
- dateToStr = parser.parse(factTimeStamp);
- return dateToStr.getTime();
- } catch (ParseException e) {
- LOGGER.error("Cannot convert" + factTimeStamp + " to Time/Long type value" + e.getMessage());
- parser = new SimpleDateFormat(CarbonCommonConstants.CARBON_TIMESTAMP);
+ // for new loads the factTimeStamp will be long string
--- End diff --
Do not mention "new loads" in line 275. Add comment in line 279, mentioning it is the processing for existing table before carbon 1.3
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1666
SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2338/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1666
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2325/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/864/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan commented on the issue:
https://github.com/apache/carbondata/pull/1666
retest this please
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2085/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2089/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1666
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2382/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/893/
---
[GitHub] carbondata pull request #1666: [CARBONDATA-1900][Core,processing] Modify loa...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1666#discussion_r157408394
--- Diff: core/src/main/java/org/apache/carbondata/core/statusmanager/LoadMetadataDetails.java ---
@@ -241,26 +248,28 @@ private long convertTimeStampToLong(String factTimeStamp) {
* @return
*/
public Long getTimeStamp(String loadStartTime) {
- Date dateToStr = null;
try {
- dateToStr = parser.parse(loadStartTime);
- return dateToStr.getTime() * 1000;
- } catch (ParseException e) {
- LOGGER.error("Cannot convert" + loadStartTime + " to Time/Long type value" + e.getMessage());
- return null;
+ // for new loads the factTimeStamp will be long string
--- End diff --
Fixed Added the detailed comment at class level
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/796/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan commented on the issue:
https://github.com/apache/carbondata/pull/1666
retest this please
---
[GitHub] carbondata pull request #1666: [CARBONDATA-1900][Core,processing] Modify loa...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1666#discussion_r157427169
--- Diff: core/src/main/java/org/apache/carbondata/core/statusmanager/LoadMetadataDetails.java ---
@@ -211,25 +234,32 @@ public long getLoadStartTimeAsLong() {
* @return
*/
private long convertTimeStampToLong(String factTimeStamp) {
--- End diff --
This function getTimeStamp returns value in nano second and returns in mili second
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by mohammadshahidkhan <gi...@git.apache.org>.
Github user mohammadshahidkhan commented on the issue:
https://github.com/apache/carbondata/pull/1666
Failed Test in SDV build is unrelated, its randomly failing with other PR #1659,
but retested got paased.
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2032/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/1666
LGTM
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/801/
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by manishgupta88 <gi...@git.apache.org>.
Github user manishgupta88 commented on the issue:
https://github.com/apache/carbondata/pull/1666
LGTM
---
[GitHub] carbondata pull request #1666: [CARBONDATA-1900][Core,processing] Modify loa...
Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1666#discussion_r157420051
--- Diff: core/src/main/java/org/apache/carbondata/core/statusmanager/LoadMetadataDetails.java ---
@@ -241,26 +248,28 @@ private long convertTimeStampToLong(String factTimeStamp) {
* @return
*/
public Long getTimeStamp(String loadStartTime) {
- Date dateToStr = null;
try {
- dateToStr = parser.parse(loadStartTime);
- return dateToStr.getTime() * 1000;
- } catch (ParseException e) {
- LOGGER.error("Cannot convert" + loadStartTime + " to Time/Long type value" + e.getMessage());
- return null;
+ // for new loads the factTimeStamp will be long string
--- End diff --
Do not mention "new loads" in line 275. Add comment in line 279, mentioning it is the processing for existing table before carbon 1.3
---
[GitHub] carbondata issue #1666: [CARBONDATA-1900][Core,processing] Modify loadmetada...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1666
Build Success with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/860/
---