You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Jordan Birdsell (Code Review)" <ge...@cloudera.org> on 2016/09/23 00:15:06 UTC
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Jordan Birdsell has uploaded a new change for review.
http://gerrit.cloudera.org:8080/4520
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
KUDU-1612 - [python] Enable setting of read mode for scanning
Currently the python client is unable to set the read mode for
scanning, so all scans are done as READ_LATEST. This patch enables
the ability to set the read mode so that the python client can read
at snapshots. This patch includes tests.
Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
---
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
M python/kudu/tests/util.py
6 files changed, 244 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/4520/1
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4520
to look at the new patch set (#10).
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
KUDU-1612 - [python] Enable setting of read mode for scanning
Currently the python client is unable to set the read mode for
scanning, so all scans are done as READ_LATEST. This patch enables
the ability to set the read mode so that the python client can read
at snapshots. This patch includes multiple tests.
Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
---
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
M python/kudu/tests/util.py
M python/kudu/util.py
7 files changed, 289 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/4520/10
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4520
to look at the new patch set (#7).
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
KUDU-1612 - [python] Enable setting of read mode for scanning
Currently the python client is unable to set the read mode for
scanning, so all scans are done as READ_LATEST. This patch enables
the ability to set the read mode so that the python client can read
at snapshots. This patch includes multiple tests.
Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
---
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
2 files changed, 2 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/4520/7
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 7
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Jordan Birdsell has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 10:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4520/4/python/kudu/tests/test_scanner.py
File python/kudu/tests/test_scanner.py:
Line 176: """
> Making sure I'm clear, are we talking about looping until we get the correc
done
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Jordan Birdsell has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 10:
Revised*... autocorrect
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 10: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Jordan Birdsell has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/4520/2/python/kudu/client.pyx
File python/kudu/client.pyx:
Line 1305: "Snapshot Timestamps be greater than the unix epoch.")
> is there a way that we can return the error status returned by SetSnapshotM
We wouldn't get that far here really, the error returned otherwise is an overflow error b/c of the unsigned int and I didnt think that would be preferred.
http://gerrit.cloudera.org:8080/#/c/4520/2/python/kudu/tests/util.py
File python/kudu/tests/util.py:
Line 122: self.snapshot_timestamp = datetime.datetime.utcnow()
> there is no guarantee that the snapshot timestamp will be in between the de
Since this op is being flushed why would it not be guaranteed? Since its blocking the call, would that not guarantee that the latest observed timestamp after the op is at least less than this "snapshot timestamp"? I get that using the latest observed timestamp is probably better form but I want to make sure I understand, also the TestScanAtSnapshot C++ test seems to also use system time, so we may need to fix that as well if its an issue.
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Jordan Birdsell has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 5:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/4520/4/python/kudu/client.pyx
File python/kudu/client.pyx:
Line 257: Note: The latest observed timestamp can also be used to start a
> there's ongoing discussion the this is more like a hack, and not why this a
Done
http://gerrit.cloudera.org:8080/#/c/4520/4/python/kudu/tests/test_scanner.py
File python/kudu/tests/test_scanner.py:
Line 176: scanner.set_read_mode(kudu.READ_LATEST)\
> I imagine you'd have to loop to make this non-flaky.
To account for a non-leader replica selection?
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Jordan Birdsell has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 3:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4520/2/python/kudu/tests/util.py
File python/kudu/tests/util.py:
Line 122: self.snapshot_timestamp = self.client.latest_observed_timestamp()
> because the server's clock might not have the same time as the system's clo
I follow, done. I took an approach that converts hybridtime to datetimes (via micros) to keep it a bit cleaner on the interface side if someone wanted to do anything with the latest observed timestamp value.
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 4:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/4520/4/python/kudu/client.pyx
File python/kudu/client.pyx:
Line 257: or previously read by this client.
there's ongoing discussion the this is more like a hack, and not why this api is there in the first place. My suggestion is that we focus the method comment on it's true purpose (getting external consistency across clients for CLIENT_PROPAGATED consistency) and then add a note that it can be used as a hack for RYW (https://gerrit.cloudera.org/#/c/4569/ might be relevant here).
Also noteworthy that it can be used for RYW only _if_ you add +1 to the value.
http://gerrit.cloudera.org:8080/#/c/4520/4/python/kudu/tests/test_scanner.py
File python/kudu/tests/test_scanner.py:
Line 176: scanner.set_read_mode(kudu.READ_LATEST)\
I imagine you'd have to loop to make this non-flaky.
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 4
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4520
to look at the new patch set (#3).
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
KUDU-1612 - [python] Enable setting of read mode for scanning
Currently the python client is unable to set the read mode for
scanning, so all scans are done as READ_LATEST. This patch enables
the ability to set the read mode so that the python client can read
at snapshots. This patch includes multiple tests.
Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
---
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
M python/kudu/tests/util.py
M python/kudu/util.py
7 files changed, 275 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/4520/3
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4520/4/python/kudu/tests/test_scanner.py
File python/kudu/tests/test_scanner.py:
Line 176: scanner.set_read_mode(kudu.READ_LATEST)\
> Making sure I'm clear, are we talking about looping until we get the correc
yeah, looping until you get the right result.
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 6
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Jordan Birdsell has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4520/4/python/kudu/tests/test_scanner.py
File python/kudu/tests/test_scanner.py:
Line 176: scanner.set_read_mode(kudu.READ_LATEST)\
> yeah, precisely.
Making sure I'm clear, are we talking about looping until we get the correct result or, rather, some sort of 50 ms pause or something (Not sure what the appropriate wait time here would be).
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Jordan Birdsell has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 10:
This is revised to master so you're good
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4520/2/python/kudu/tests/util.py
File python/kudu/tests/util.py:
Line 122: self.snapshot_timestamp = datetime.datetime.utcnow()
> Since this op is being flushed why would it not be guaranteed? Since its bl
because the server's clock might not have the same time as the system's clock (it's not a pure physical clock). i.e. the time returned by the system clock might behind or ahead of the time the server assigned to the write. it's unlikely but it can happen and thus it would make this test flaky.
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/4520/2/python/kudu/client.pyx
File python/kudu/client.pyx:
Line 1305: "Snapshot Timestamps be greater than the unix epoch.")
is there a way that we can return the error status returned by SetSnapshotMicros
http://gerrit.cloudera.org:8080/#/c/4520/2/python/kudu/tests/util.py
File python/kudu/tests/util.py:
Line 122: self.snapshot_timestamp = datetime.datetime.utcnow()
there is no guarantee that the snapshot timestamp will be in between the delete and the insert below, what you should do is to get the lastest observed timestamp from the KuduClient after the delete (which will be guaranteed to see the delete but not the insert).
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 10:
I guess you meant rebased :)
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4520
to look at the new patch set (#2).
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
KUDU-1612 - [python] Enable setting of read mode for scanning
Currently the python client is unable to set the read mode for
scanning, so all scans are done as READ_LATEST. This patch enables
the ability to set the read mode so that the python client can read
at snapshots. This patch includes multiple tests.
Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
---
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
M python/kudu/tests/util.py
6 files changed, 244 insertions(+), 6 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/4520/2
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4520/4/python/kudu/tests/test_scanner.py
File python/kudu/tests/test_scanner.py:
Line 176: scanner.set_read_mode(kudu.READ_LATEST)\
> To account for a non-leader replica selection?
yeah, precisely.
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4520
to look at the new patch set (#5).
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
KUDU-1612 - [python] Enable setting of read mode for scanning
Currently the python client is unable to set the read mode for
scanning, so all scans are done as READ_LATEST. This patch enables
the ability to set the read mode so that the python client can read
at snapshots. This patch includes multiple tests.
Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
---
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
M python/kudu/tests/util.py
M python/kudu/util.py
7 files changed, 280 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/4520/5
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4520
to look at the new patch set (#8).
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
KUDU-1612 - [python] Enable setting of read mode for scanning
Currently the python client is unable to set the read mode for
scanning, so all scans are done as READ_LATEST. This patch enables
the ability to set the read mode so that the python client can read
at snapshots. This patch includes multiple tests.
Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
---
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
M python/kudu/tests/util.py
M python/kudu/util.py
7 files changed, 281 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/4520/8
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 8
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has submitted this change and it was merged.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
KUDU-1612 - [python] Enable setting of read mode for scanning
Currently the python client is unable to set the read mode for
scanning, so all scans are done as READ_LATEST. This patch enables
the ability to set the read mode so that the python client can read
at snapshots. This patch includes multiple tests.
Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Reviewed-on: http://gerrit.cloudera.org:8080/4520
Tested-by: Kudu Jenkins
Reviewed-by: David Ribeiro Alves <dr...@apache.org>
---
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
M python/kudu/tests/util.py
M python/kudu/util.py
7 files changed, 289 insertions(+), 7 deletions(-)
Approvals:
David Ribeiro Alves: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 11
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "Jordan Birdsell (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4520
to look at the new patch set (#9).
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
KUDU-1612 - [python] Enable setting of read mode for scanning
Currently the python client is unable to set the read mode for
scanning, so all scans are done as READ_LATEST. This patch enables
the ability to set the read mode so that the python client can read
at snapshots. This patch includes multiple tests.
Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
---
M python/kudu/__init__.py
M python/kudu/client.pyx
M python/kudu/libkudu_client.pxd
M python/kudu/tests/test_scanner.py
M python/kudu/tests/test_scantoken.py
M python/kudu/tests/util.py
M python/kudu/util.py
7 files changed, 287 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/20/4520/9
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 9
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
[kudu-CR] KUDU-1612 - [python] Enable setting of read mode for scanning
Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.
Change subject: KUDU-1612 - [python] Enable setting of read mode for scanning
......................................................................
Patch Set 10:
Is is safe to merge this without the parent?
--
To view, visit http://gerrit.cloudera.org:8080/4520
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I2c61ef09f6e15bad2c44d9caf85b2cc2582b8a49
Gerrit-PatchSet: 10
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Jordan Birdsell <jo...@gmail.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Will Berkeley <wd...@gmail.com>
Gerrit-HasComments: No