You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by maryannxue <gi...@git.apache.org> on 2014/02/28 21:05:35 UTC

[GitHub] incubator-phoenix pull request: PHOENIX-85 Refine error handling i...

GitHub user maryannxue opened a pull request:

    https://github.com/apache/incubator-phoenix/pull/10

    PHOENIX-85 Refine error handling in HashJoinPlan execution and add warnings

    Also include fix for:
    PHOENIX-90 MemoryChunk objects might be orphaned when creating new caches in TenantCacheImpl

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/maryannxue/incubator-phoenix master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-phoenix/pull/10.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #10
    
----
commit 05b584181fe36102feca765886fcda75b770b2b2
Author: maryannxue <ma...@apache.org>
Date:   2014-02-12T21:41:53Z

    Fix PHOENIX-44 Cannot create index on tables with case-sensitive schema names

commit 02af57c96476c9d205d07ba96974c49188a282fa
Author: maryannxue <ma...@apache.org>
Date:   2014-02-21T02:54:17Z

    Merge remote-tracking branch 'asf/master'

commit bd20346da5336d3d240493fdbfbcd5cc03280810
Author: maryannxue <ma...@apache.org>
Date:   2014-02-21T03:08:23Z

    PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries

commit 9a2ced8936d6cedbc44a4ed2de2c6f72f4f4307f
Author: maryannxue <ma...@apache.org>
Date:   2014-02-21T04:59:29Z

    PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries

commit 20a97904e48829f8b412bcfb0ac2084259133126
Author: maryannxue <ma...@apache.org>
Date:   2014-02-21T05:53:21Z

    PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries

commit 7568516fabfb416f5dcbfa89468b839963802a87
Author: maryannxue <ma...@apache.org>
Date:   2014-02-21T22:14:14Z

    Revert "PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries"
    
    This reverts commit 20a97904e48829f8b412bcfb0ac2084259133126.

commit 3fb95819959d22deb2c03e895d976e7d933007f2
Author: maryannxue <ma...@apache.org>
Date:   2014-02-21T22:14:56Z

    Revert "PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries"
    
    This reverts commit 9a2ced8936d6cedbc44a4ed2de2c6f72f4f4307f.

commit ca0b55d048a6c547a5d6b081dd84dea01011e5c1
Author: maryannxue <ma...@apache.org>
Date:   2014-02-21T22:16:13Z

    Revert "PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries"
    
    This reverts commit bd20346da5336d3d240493fdbfbcd5cc03280810.

commit c4df65791a163070362662aad7ce5916faa3c3d7
Author: maryannxue <ma...@apache.org>
Date:   2014-02-21T22:18:13Z

    Merge remote-tracking branch 'asf/master'

commit 9b5df132480b6bd25c0c33dd818278a5fc1b3f55
Author: maryannxue <ma...@apache.org>
Date:   2014-02-21T22:22:38Z

    PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries

commit 3e677489c0cb811c1bb3d266581330fe11fdd674
Author: maryannxue <ma...@apache.org>
Date:   2014-02-22T00:17:15Z

    PHOENIX-68 Add Option "FAVOR_STAR_JOIN" to indicate if star join optimization is preferred for multi inner join queries

commit 776c1a11b286ebd358f9111f3ff1c99c3f7c7883
Author: maryannxue <ma...@apache.org>
Date:   2014-02-27T18:41:58Z

    Merge remote-tracking branch 'asf/master'

commit 6d26e78096091624502d4382de23f655626bca58
Author: maryannxue <ma...@apache.org>
Date:   2014-02-28T19:36:38Z

    Merge remote-tracking branch 'asf/master'

commit df561c8d8ab59651d7e797dc0da501cceb4c6030
Author: maryannxue <ma...@apache.org>
Date:   2014-02-28T19:48:06Z

    PHOENIX-85 Refine error handling in HashJoinPlan execution and add warnings

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-phoenix pull request: PHOENIX-85 Refine error handling i...

Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue commented on a diff in the pull request:

    https://github.com/apache/incubator-phoenix/pull/10#discussion_r10185785
  
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/cache/TenantCacheImpl.java ---
    @@ -82,9 +82,17 @@ public Closeable getServerCache(ImmutableBytesPtr cacheId) {
         @Override
         public Closeable addServerCache(ImmutableBytesPtr cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException {
             MemoryChunk chunk = this.getMemoryManager().allocate(cachePtr.getLength());
    -        Closeable element = cacheFactory.newCache(cachePtr, chunk);
    -        getServerCaches().put(cacheId, element);
    -        return element;
    +        try {
    --- End diff --
    
    Agree. Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-phoenix pull request: PHOENIX-85 Refine error handling i...

Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:

    https://github.com/apache/incubator-phoenix/pull/10#discussion_r10185708
  
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/cache/TenantCacheImpl.java ---
    @@ -82,9 +82,17 @@ public Closeable getServerCache(ImmutableBytesPtr cacheId) {
         @Override
         public Closeable addServerCache(ImmutableBytesPtr cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException {
             MemoryChunk chunk = this.getMemoryManager().allocate(cachePtr.getLength());
    -        Closeable element = cacheFactory.newCache(cachePtr, chunk);
    -        getServerCaches().put(cacheId, element);
    -        return element;
    +        try {
    --- End diff --
    
    Good catch, but I'd code this like this instead so that you guarantee to close the chunk and ignore any exception that occurs when you close it, as you'd be in the middle of throwing another, different exception:
    
        boolean success = false;
        try {
            Closeable element = cacheFactory.newCache(cachePtr, chunk);
            getServerCaches().put(cacheId, element);
            success = true;
            return element;
        } finally {
            if (!success) Closeables.closeQuietly(chunk); // com.google.common.io.Closeables
        }


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-phoenix pull request: PHOENIX-85 Refine error handling i...

Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue commented on a diff in the pull request:

    https://github.com/apache/incubator-phoenix/pull/10#discussion_r10186139
  
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/execute/HashJoinPlan.java ---
    @@ -99,8 +109,14 @@ public ResultIterator iterator() throws SQLException {
                     @Override
                     public ServerCache call() throws Exception {
                         QueryPlan hashPlan = hashPlans[index];
    -                    return hashClient.addHashCache(ranges, hashPlan.iterator(), 
    +                    ServerCache cache = hashClient.addHashCache(ranges, hashPlan.iterator(), 
                                 hashPlan.getEstimatedSize(), hashExpressions[index], plan.getTableRef());
    +                    long endTime = System.currentTimeMillis();
    +                    boolean isSet = firstJobEndTime.compareAndSet(0, endTime);
    +                    if (!isSet && (endTime - firstJobEndTime.get()) > maxServerCacheTimeToLive) {
    +                        LOG.warn("Hash plan [" + index + "] execution seems too slow. Earlier hash cache(s) might have expired on servers.");
    --- End diff --
    
    Just when doing star-join, say, several inner tables are being scanned and added to server cache, and the outer table scan won't start until every inner table has been added to server cache. Then if one or more of the table scans run seriously slower than the others, when these scans finish, the other tables that finish much earlier might have expired in the cache.
    However, I checked the spec for "guava cache". Seems that cache expiration and cleanup does not always actively happen, so that the cache map won't be locked in conflict with user writes. So I suppose warning here can act as a good reminder for potential issue whether or not the problem actually occurs.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-phoenix pull request: PHOENIX-85 Refine error handling i...

Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on a diff in the pull request:

    https://github.com/apache/incubator-phoenix/pull/10#discussion_r10185769
  
    --- Diff: phoenix-core/src/main/java/org/apache/phoenix/execute/HashJoinPlan.java ---
    @@ -99,8 +109,14 @@ public ResultIterator iterator() throws SQLException {
                     @Override
                     public ServerCache call() throws Exception {
                         QueryPlan hashPlan = hashPlans[index];
    -                    return hashClient.addHashCache(ranges, hashPlan.iterator(), 
    +                    ServerCache cache = hashClient.addHashCache(ranges, hashPlan.iterator(), 
                                 hashPlan.getEstimatedSize(), hashExpressions[index], plan.getTableRef());
    +                    long endTime = System.currentTimeMillis();
    +                    boolean isSet = firstJobEndTime.compareAndSet(0, endTime);
    +                    if (!isSet && (endTime - firstJobEndTime.get()) > maxServerCacheTimeToLive) {
    +                        LOG.warn("Hash plan [" + index + "] execution seems too slow. Earlier hash cache(s) might have expired on servers.");
    --- End diff --
    
    Just curious - what would cause this? Would it make sense to throw here and let HBase do it's retry logic?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-phoenix pull request: PHOENIX-85 Refine error handling i...

Posted by JamesRTaylor <gi...@git.apache.org>.
Github user JamesRTaylor commented on the pull request:

    https://github.com/apache/incubator-phoenix/pull/10#issuecomment-36404869
  
    +1 other than that minor change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-phoenix pull request: PHOENIX-85 Refine error handling i...

Posted by maryannxue <gi...@git.apache.org>.
Github user maryannxue closed the pull request at:

    https://github.com/apache/incubator-phoenix/pull/10


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---