You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by jacques-n <gi...@git.apache.org> on 2016/01/14 05:04:38 UTC
[GitHub] drill pull request: DRILL-4246: Fix Allocator concurrency bug and ...
GitHub user jacques-n opened a pull request:
https://github.com/apache/drill/pull/323
DRILL-4246: Fix Allocator concurrency bug and improve error detection
- Rename the internal DrillBuf field to udle to better express its purpose.
- Rename AllocatorManager to AllocationManager to better express its purpose.
- Address situation where dangling ledger could be transferred into while it was being released released by protecting association and release inside the AllocationManager.
- Add allocator assertions to ensure allocator operations are done while the allocator is open.
- Simplify AllocationManager locking model.
- Exclude HDFS reference to netty-all
- Improve debugging messages for allocators (and fix debug message bugs)
Tests pending but evaluation shows previously frequently failing flatten tests to consistently pass.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jacques-n/drill DRILL-4246
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/323.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #323
----
commit 4b193d71a9f3e06aadf59c66832bcbcb4b060c92
Author: Jacques Nadeau <ja...@apache.org>
Date: 2016-01-11T04:08:03Z
DRILL-4246: Fix Allocator concurrency bug and improve error detection
- Rename the internal DrillBuf field to udle to better express its purpose.
- Rename AllocatorManager to AllocationManager to better express its purpose.
- Address situation where dangling ledger could be transferred into while it was being released released by protecting association and release inside the AllocationManager.
- Add allocator assertions to ensure allocator operations are done while the allocator is open.
- Simplify AllocationManager locking model.
- Exclude HDFS reference to netty-all
- Improve debugging messages for allocators (and fix debug message bugs)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] drill pull request: DRILL-4246: Fix Allocator concurrency bug and ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/drill/pull/323
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] drill pull request: DRILL-4246: Fix Allocator concurrency bug and ...
Posted by parthchandra <gi...@git.apache.org>.
Github user parthchandra commented on the pull request:
https://github.com/apache/drill/pull/323#issuecomment-172053123
LGTM. +1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] drill pull request: DRILL-4246: Fix Allocator concurrency bug and ...
Posted by parthchandra <gi...@git.apache.org>.
Github user parthchandra commented on a diff in the pull request:
https://github.com/apache/drill/pull/323#discussion_r49890071
--- Diff: exec/memory/base/src/main/java/org/apache/drill/exec/memory/AllocationManager.java ---
@@ -168,20 +213,26 @@ private BufferLedger(BaseAllocator allocator, ReleaseListener listener) {
/**
* Transfer any balance the current ledger has to the target ledger. In the case that the current ledger holds no
* memory, no transfer is made to the new ledger.
- *
* @param target
* The ledger to transfer ownership account to.
* @return Whether transfer fit within target ledgers limits.
*/
- public boolean transferBalance(BufferLedger target) {
+ public boolean transferBalance(final BufferLedger target) {
Preconditions.checkNotNull(target);
Preconditions.checkArgument(allocator.root == target.allocator.root,
"You can only transfer between two allocators that share the same root.");
+ allocator.assertOpen();
+
+ target.allocator.assertOpen();
+ // if we're transferring to ourself, just return.
+ if (target == this) {
+ return true;
+ }
// since two balance transfers out from the allocator manager could cause incorrect accounting, we need to ensure
// that this won't happen by synchronizing on the allocator manager instance.
--- End diff --
Comment needs to be updated
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] drill pull request: DRILL-4246: Fix Allocator concurrency bug and ...
Posted by jacques-n <gi...@git.apache.org>.
Github user jacques-n commented on the pull request:
https://github.com/apache/drill/pull/323#issuecomment-171673739
Passes unit and regression suites
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---