You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by GitBox <gi...@apache.org> on 2020/01/13 18:59:12 UTC
[GitHub] [phoenix] swaroopak opened a new pull request #678: PHOENIX-5644
and PHOENIX-5651 addendum patch
swaroopak opened a new pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] priyankporwal commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
priyankporwal commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r365978792
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
Why change startWaitTime here?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] swaroopak closed pull request #678: PHOENIX-5644 and
PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
swaroopak closed pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] priyankporwal commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
priyankporwal commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r366039114
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
Got that. Not sure how that helps though. StartWaitTime is an input this function, and should normally never be updated .. besides, why keep shifting the starting time? What you really need is how much longer to wait until originalStartWaitTime+GLOBAL_INDEX_CHECKER_ENABLED_MAP_EXPIRATION_MIN+1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] swaroopak commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
swaroopak commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r366027171
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
in case the sleep is interrupted.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] swaroopak commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
swaroopak commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r366027556
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Thread.sleep(waitMore);
- waited = true;
+ isWaitComplete = true;
} catch (InterruptedException e) {
endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
I think this patch is good to overcome the problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] priyankporwal commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
priyankporwal commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r366039669
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Thread.sleep(waitMore);
- waited = true;
+ isWaitComplete = true;
} catch (InterruptedException e) {
endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
In fact, remove endWaitTime entirely and let getWaitMoreTime() use EnvironmentEdgeManager.currentTimeMillis() directly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] swaroopak commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
swaroopak commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r366026953
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Thread.sleep(waitMore);
- waited = true;
+ isWaitComplete = true;
} catch (InterruptedException e) {
endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
This is exactly what I was doing before considering @gjacoby126 's suggestion to not ignore sleep interruptions and avoid cache incoherency.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] swaroopak commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
swaroopak commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r366043710
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Thread.sleep(waitMore);
- waited = true;
+ isWaitComplete = true;
} catch (InterruptedException e) {
endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
I personally don't like while (true) :(
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] priyankporwal commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
priyankporwal commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r366047572
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Thread.sleep(waitMore);
- waited = true;
+ isWaitComplete = true;
} catch (InterruptedException e) {
endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
Alternative to appease your dislike:
do { ... } while(waitMore > 0) by pulling out waitMore out of the body
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] swaroopak commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
swaroopak commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r366027556
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Thread.sleep(waitMore);
- waited = true;
+ isWaitComplete = true;
} catch (InterruptedException e) {
endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
I think this patch is good to overcome the problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] swaroopak commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
swaroopak commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r366026953
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Thread.sleep(waitMore);
- waited = true;
+ isWaitComplete = true;
} catch (InterruptedException e) {
endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
This is exactly what I was doing before considering @gjacoby126 's suggestion to not ignore sleep interruptions and avoid cache incoherency.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [phoenix] priyankporwal commented on a change in pull request #678:
PHOENIX-5644 and PHOENIX-5651 addendum patch
Posted by GitBox <gi...@apache.org>.
priyankporwal commented on a change in pull request #678: PHOENIX-5644 and PHOENIX-5651 addendum patch
URL: https://github.com/apache/phoenix/pull/678#discussion_r365981964
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/IndexUpgradeTool.java
##########
@@ -424,20 +424,21 @@ private void enableImmutableTables(ConnectionQueryServices queryServices,
long startWaitTime) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
- while (waitMore>0) {
+ while (waitMore>0 && !isWaitComplete) {
// If the table is immutable, we need to wait for clients to purge
// their caches of table metadata
LOGGER.info("waiting for more " + waitMore + " ms for client cache "
+ "to expire for immutable tables");
try {
startWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Thread.sleep(waitMore);
- waited = true;
+ isWaitComplete = true;
} catch (InterruptedException e) {
endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
Review comment:
@swaroopak This logic of splitting endWaitTime and startWaitTime and two booleans is getting more bug prone now.
I would suggest the below reorg
while(true) {
long endWaitTime = EnvironmentEdgeManager.currentTimeMillis();
long waitMore = getWaitMoreTime(endWaitTime, startWaitTime);
if (waitMore <= 0)
{
isWaitComplete = true; break;
}
try {
Thread.sleep(waitMore);
isWaitComplete = true;
}
catch((InterruptedException e) {
LOGGER.warning("Sleep before starting index rebuild is interrupted. "
+ "Attempting to sleep again! " + e.getMessage());
}
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services