You are viewing a plain text version of this content. The canonical link for it is here.
Posted to codereview@trafodion.apache.org by SuJinpei <gi...@git.apache.org> on 2018/01/31 02:02:40 UTC
[GitHub] trafodion pull request #1425: [Trafodion-2937] the actual copied data is les...
GitHub user SuJinpei opened a pull request:
https://github.com/apache/trafodion/pull/1425
[Trafodion-2937] the actual copied data is less than expected when copying data from MySQL to Trafodion
**root cause:** During building extract command, odb tool will try to split the task into multithread if parallel, but the parallel is equal 1 here, so it should still go to single thread branch.
**fixed by:** only if parallel bigger than 1, odb go splitting branch.
**Aditional fix:** make code cleaner and odb support g++ 4.8.5
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/SuJinpei/incubator-trafodion trafodion-2937
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/trafodion/pull/1425.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1425
----
commit 7e05a8251ae81c401bd7845275a82061e99455b5
Author: SuJinpei <87...@...>
Date: 2018-01-31T01:39:59Z
fix Trafodion-2937
----
---
[GitHub] trafodion pull request #1425: [Trafodion-2937] the actual copied data is les...
Posted by SuJinpei <gi...@git.apache.org>.
Github user SuJinpei commented on a diff in the pull request:
https://github.com/apache/trafodion/pull/1425#discussion_r166286068
--- Diff: core/conn/odb/src/odb.c ---
@@ -10867,7 +10874,7 @@ static int Ocopy(int eid)
}
/* Build Extract Command */
- if ( etab[eid].ps ) {
+ if ( etab[eid].ps > 1 ) {
--- End diff --
This change will address the bug because the default ps == 0 in fact means ps == 1, so odb ought to go default branch if ps == 1.
---
[GitHub] trafodion pull request #1425: [Trafodion-2937] the actual copied data is les...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/trafodion/pull/1425
---
[GitHub] trafodion pull request #1425: [Trafodion-2937] the actual copied data is les...
Posted by sureshsubbiah <gi...@git.apache.org>.
Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/trafodion/pull/1425#discussion_r165555736
--- Diff: core/conn/odb/src/odb.c ---
@@ -10867,7 +10874,7 @@ static int Ocopy(int eid)
}
/* Build Extract Command */
- if ( etab[eid].ps ) {
+ if ( etab[eid].ps > 1 ) {
--- End diff --
Could you please explain either here or in the JIRA how this change will address the bug. I am sorry I am familiar enough with the code to understand without comments.
---
[GitHub] trafodion pull request #1425: [Trafodion-2937] the actual copied data is les...
Posted by SuJinpei <gi...@git.apache.org>.
Github user SuJinpei commented on a diff in the pull request:
https://github.com/apache/trafodion/pull/1425#discussion_r166265982
--- Diff: core/conn/odb/src/odb.c ---
@@ -10867,7 +10874,7 @@ static int Ocopy(int eid)
}
/* Build Extract Command */
- if ( etab[eid].ps ) {
+ if ( etab[eid].ps > 1 ) {
--- End diff --
Hi Suresh, if ps > 1, odb will split select job to number of ps thread, for example, if we have [1, 12] range of data and ps == 3, the data will first be divided to 3 group: [1, 4), [4, 7), [7, 10), then change last group to [7, 12), the coding deal with last group in line 5830, but there make assumption ps > 1. So for this bug, data range is [6, 10], will be split to [6, 10), because of ps == 1, this group has no chance to be changed. so data == 10 lost. I found many expression "if (etab[no].ps) " make an assumption that ps > 1 and below line(5835) contains a potential bug. so I will make another commit to fix this. thank you for making me aware of these issues.
etab[no].sbmax = ( j + 1 ) == etab[l].ps ? (sbmax + 1 ) : etab[no].sbmin + d ;
---