You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@deltacloud.apache.org by "Jan Provaznik (JIRA)" <ji...@apache.org> on 2012/08/01 11:33:03 UTC
[jira] [Created] (DTACLOUD-290) Memory leaking on Fedora 17
Jan Provaznik created DTACLOUD-290:
--------------------------------------
Summary: Memory leaking on Fedora 17
Key: DTACLOUD-290
URL: https://issues.apache.org/jira/browse/DTACLOUD-290
Project: DeltaCloud
Issue Type: Bug
Components: Server
Environment: deltacloud-core-1.0.0-4.fc17.noarch (though same issue with dc-core from git)
sinatra (1.3.2)
sinatra-rabbit (1.0.6)
Fedora 17
Reporter: Jan Provaznik
Priority: Critical
Memory used by deltacloudd process increases with time. After 12 hours, it's common it uses ~4G of virt memory and >1G of physical memory (only conductor's dbomatic script which periodically checks running instances every minute was running).
You can reproduce the memory leak by this simple script:
#!/usr/bin/ruby
require 'rubygems'
require 'deltacloud'
1000.times do |i| { DeltaCloud.new('mockuser', 'mockpassword', 'http://localhost:3002/api') }
Used memory will increase by >10M (both virt and physical). The problem is probably with lambda in check_capability usage in lib/deltacloud/collections/* files:
check_capability :for => lambda { |m| driver.respond_to? m }
if the lambda is replaced by true or other static value, the memory usage remains constant.
I see that in git the 'check_capability' call was replaced by 'set' call, this change has no influence on this problem.
Also it's weird that after starting deltacloudd, used memory is 233m/33m (virt/res), but after first request, used memory jumps to 776m/90m (virt/res) - this is far more than on Fedora 16, where used memory remains around 160m/50m.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DTACLOUD-290) Memory leaking on Fedora 17
Posted by "Jan Provaznik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DTACLOUD-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426573#comment-13426573 ]
Jan Provaznik commented on DTACLOUD-290:
----------------------------------------
The updated patch works fine - no memory leakage, also now it's significantly faster ;).
> Memory leaking on Fedora 17
> ---------------------------
>
> Key: DTACLOUD-290
> URL: https://issues.apache.org/jira/browse/DTACLOUD-290
> Project: DeltaCloud
> Issue Type: Bug
> Components: Server
> Environment: deltacloud-core-1.0.0-4.fc17.noarch (though same issue with dc-core from git)
> sinatra (1.3.2)
> sinatra-rabbit (1.0.6)
> Fedora 17
> Reporter: Jan Provaznik
> Assignee: Michal Fojtik
> Priority: Critical
>
> Memory used by deltacloudd process increases with time. After 12 hours, it's common it uses ~4G of virt memory and >1G of physical memory (only conductor's dbomatic script which periodically checks running instances every minute was running).
> You can reproduce the memory leak by this simple script:
> #!/usr/bin/ruby
> require 'rubygems'
> require 'deltacloud'
> 1000.times do |i| { DeltaCloud.new('mockuser', 'mockpassword', 'http://localhost:3002/api') }
> Used memory will increase by >10M (both virt and physical). The problem is probably with lambda in check_capability usage in lib/deltacloud/collections/* files:
> check_capability :for => lambda { |m| driver.respond_to? m }
> if the lambda is replaced by true or other static value, the memory usage remains constant.
> I see that in git the 'check_capability' call was replaced by 'set' call, this change has no influence on this problem.
> Also it's weird that after starting deltacloudd, used memory is 233m/33m (virt/res), but after first request, used memory jumps to 776m/90m (virt/res) - this is far more than on Fedora 16, where used memory remains around 160m/50m.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (DTACLOUD-290) Memory leaking on Fedora 17
Posted by "Michal Fojtik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DTACLOUD-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michal Fojtik reassigned DTACLOUD-290:
--------------------------------------
Assignee: Michal Fojtik
> Memory leaking on Fedora 17
> ---------------------------
>
> Key: DTACLOUD-290
> URL: https://issues.apache.org/jira/browse/DTACLOUD-290
> Project: DeltaCloud
> Issue Type: Bug
> Components: Server
> Environment: deltacloud-core-1.0.0-4.fc17.noarch (though same issue with dc-core from git)
> sinatra (1.3.2)
> sinatra-rabbit (1.0.6)
> Fedora 17
> Reporter: Jan Provaznik
> Assignee: Michal Fojtik
> Priority: Critical
>
> Memory used by deltacloudd process increases with time. After 12 hours, it's common it uses ~4G of virt memory and >1G of physical memory (only conductor's dbomatic script which periodically checks running instances every minute was running).
> You can reproduce the memory leak by this simple script:
> #!/usr/bin/ruby
> require 'rubygems'
> require 'deltacloud'
> 1000.times do |i| { DeltaCloud.new('mockuser', 'mockpassword', 'http://localhost:3002/api') }
> Used memory will increase by >10M (both virt and physical). The problem is probably with lambda in check_capability usage in lib/deltacloud/collections/* files:
> check_capability :for => lambda { |m| driver.respond_to? m }
> if the lambda is replaced by true or other static value, the memory usage remains constant.
> I see that in git the 'check_capability' call was replaced by 'set' call, this change has no influence on this problem.
> Also it's weird that after starting deltacloudd, used memory is 233m/33m (virt/res), but after first request, used memory jumps to 776m/90m (virt/res) - this is far more than on Fedora 16, where used memory remains around 160m/50m.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DTACLOUD-290) Memory leaking on Fedora 17
Posted by "Jan Provaznik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DTACLOUD-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426545#comment-13426545 ]
Jan Provaznik commented on DTACLOUD-290:
----------------------------------------
I can confirm that the original patch in first comment fixes memory leak. But I don't see the modified version of the patch or link to it (if you need confirm for it).
> Memory leaking on Fedora 17
> ---------------------------
>
> Key: DTACLOUD-290
> URL: https://issues.apache.org/jira/browse/DTACLOUD-290
> Project: DeltaCloud
> Issue Type: Bug
> Components: Server
> Environment: deltacloud-core-1.0.0-4.fc17.noarch (though same issue with dc-core from git)
> sinatra (1.3.2)
> sinatra-rabbit (1.0.6)
> Fedora 17
> Reporter: Jan Provaznik
> Assignee: Michal Fojtik
> Priority: Critical
>
> Memory used by deltacloudd process increases with time. After 12 hours, it's common it uses ~4G of virt memory and >1G of physical memory (only conductor's dbomatic script which periodically checks running instances every minute was running).
> You can reproduce the memory leak by this simple script:
> #!/usr/bin/ruby
> require 'rubygems'
> require 'deltacloud'
> 1000.times do |i| { DeltaCloud.new('mockuser', 'mockpassword', 'http://localhost:3002/api') }
> Used memory will increase by >10M (both virt and physical). The problem is probably with lambda in check_capability usage in lib/deltacloud/collections/* files:
> check_capability :for => lambda { |m| driver.respond_to? m }
> if the lambda is replaced by true or other static value, the memory usage remains constant.
> I see that in git the 'check_capability' call was replaced by 'set' call, this change has no influence on this problem.
> Also it's weird that after starting deltacloudd, used memory is 233m/33m (virt/res), but after first request, used memory jumps to 776m/90m (virt/res) - this is far more than on Fedora 16, where used memory remains around 160m/50m.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (DTACLOUD-290) Memory leaking on Fedora 17
Posted by "Ronelle Landy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DTACLOUD-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ronelle Landy closed DTACLOUD-290.
----------------------------------
Resolution: Fixed
Closing issue based on Jan's comments above ... changed was pushed to master in commit 5cefcb418e502b70baf016658471ed0d6a2a995d
> Memory leaking on Fedora 17
> ---------------------------
>
> Key: DTACLOUD-290
> URL: https://issues.apache.org/jira/browse/DTACLOUD-290
> Project: DeltaCloud
> Issue Type: Bug
> Components: Server
> Environment: deltacloud-core-1.0.0-4.fc17.noarch (though same issue with dc-core from git)
> sinatra (1.3.2)
> sinatra-rabbit (1.0.6)
> Fedora 17
> Reporter: Jan Provaznik
> Assignee: Michal Fojtik
> Priority: Critical
>
> Memory used by deltacloudd process increases with time. After 12 hours, it's common it uses ~4G of virt memory and >1G of physical memory (only conductor's dbomatic script which periodically checks running instances every minute was running).
> You can reproduce the memory leak by this simple script:
> #!/usr/bin/ruby
> require 'rubygems'
> require 'deltacloud'
> 1000.times do |i| { DeltaCloud.new('mockuser', 'mockpassword', 'http://localhost:3002/api') }
> Used memory will increase by >10M (both virt and physical). The problem is probably with lambda in check_capability usage in lib/deltacloud/collections/* files:
> check_capability :for => lambda { |m| driver.respond_to? m }
> if the lambda is replaced by true or other static value, the memory usage remains constant.
> I see that in git the 'check_capability' call was replaced by 'set' call, this change has no influence on this problem.
> Also it's weird that after starting deltacloudd, used memory is 233m/33m (virt/res), but after first request, used memory jumps to 776m/90m (virt/res) - this is far more than on Fedora 16, where used memory remains around 160m/50m.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DTACLOUD-290) Memory leaking on Fedora 17
Posted by "Michal Fojtik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DTACLOUD-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426512#comment-13426512 ]
Michal Fojtik commented on DTACLOUD-290:
----------------------------------------
Hi,
After some debugging we tried this patch that seems to solve the problem of memory growing. However, the memory usage after first request still jump to 1.5gb.
diff --git a/server/lib/deltacloud/helpers/driver_helper.rb b/server/lib/deltacloud/helpers/driver_helper.r
index 5c5e7c6..96fd1d7 100644
--- a/server/lib/deltacloud/helpers/driver_helper.rb
+++ b/server/lib/deltacloud/helpers/driver_helper.rb
@@ -49,8 +49,12 @@ module Deltacloud::Helpers
def driver
$:.unshift File.join(File.dirname(__FILE__), '..', '..')
begin
- require_relative driver_source_name
- driver_class
+ @@driver ||= nil
+ unless @@driver
+ puts "Loading driver..."
+ require_relative driver_source_name
+ end
+ @@driver ||= driver_class
rescue LoadError => e
raise "[ERROR] The driver '#{driver_name}' is unknown or not installed (#{driver_source_name})\n"
"\n#{e.message}\n"
> Memory leaking on Fedora 17
> ---------------------------
>
> Key: DTACLOUD-290
> URL: https://issues.apache.org/jira/browse/DTACLOUD-290
> Project: DeltaCloud
> Issue Type: Bug
> Components: Server
> Environment: deltacloud-core-1.0.0-4.fc17.noarch (though same issue with dc-core from git)
> sinatra (1.3.2)
> sinatra-rabbit (1.0.6)
> Fedora 17
> Reporter: Jan Provaznik
> Priority: Critical
>
> Memory used by deltacloudd process increases with time. After 12 hours, it's common it uses ~4G of virt memory and >1G of physical memory (only conductor's dbomatic script which periodically checks running instances every minute was running).
> You can reproduce the memory leak by this simple script:
> #!/usr/bin/ruby
> require 'rubygems'
> require 'deltacloud'
> 1000.times do |i| { DeltaCloud.new('mockuser', 'mockpassword', 'http://localhost:3002/api') }
> Used memory will increase by >10M (both virt and physical). The problem is probably with lambda in check_capability usage in lib/deltacloud/collections/* files:
> check_capability :for => lambda { |m| driver.respond_to? m }
> if the lambda is replaced by true or other static value, the memory usage remains constant.
> I see that in git the 'check_capability' call was replaced by 'set' call, this change has no influence on this problem.
> Also it's weird that after starting deltacloudd, used memory is 233m/33m (virt/res), but after first request, used memory jumps to 776m/90m (virt/res) - this is far more than on Fedora 16, where used memory remains around 160m/50m.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DTACLOUD-290) Memory leaking on Fedora 17
Posted by "Michal Fojtik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DTACLOUD-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426513#comment-13426513 ]
Michal Fojtik commented on DTACLOUD-290:
----------------------------------------
(remove the 'puts' ;-)
> Memory leaking on Fedora 17
> ---------------------------
>
> Key: DTACLOUD-290
> URL: https://issues.apache.org/jira/browse/DTACLOUD-290
> Project: DeltaCloud
> Issue Type: Bug
> Components: Server
> Environment: deltacloud-core-1.0.0-4.fc17.noarch (though same issue with dc-core from git)
> sinatra (1.3.2)
> sinatra-rabbit (1.0.6)
> Fedora 17
> Reporter: Jan Provaznik
> Priority: Critical
>
> Memory used by deltacloudd process increases with time. After 12 hours, it's common it uses ~4G of virt memory and >1G of physical memory (only conductor's dbomatic script which periodically checks running instances every minute was running).
> You can reproduce the memory leak by this simple script:
> #!/usr/bin/ruby
> require 'rubygems'
> require 'deltacloud'
> 1000.times do |i| { DeltaCloud.new('mockuser', 'mockpassword', 'http://localhost:3002/api') }
> Used memory will increase by >10M (both virt and physical). The problem is probably with lambda in check_capability usage in lib/deltacloud/collections/* files:
> check_capability :for => lambda { |m| driver.respond_to? m }
> if the lambda is replaced by true or other static value, the memory usage remains constant.
> I see that in git the 'check_capability' call was replaced by 'set' call, this change has no influence on this problem.
> Also it's weird that after starting deltacloudd, used memory is 233m/33m (virt/res), but after first request, used memory jumps to 776m/90m (virt/res) - this is far more than on Fedora 16, where used memory remains around 160m/50m.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (DTACLOUD-290) Memory leaking on Fedora 17
Posted by "Michal Fojtik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DTACLOUD-290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426536#comment-13426536 ]
Michal Fojtik commented on DTACLOUD-290:
----------------------------------------
I modified the patch to work nicely with our driver switching. Patch is on the list. Once you confirm it works for you I'll push it and apply
as hot-fix for F17 RPM.
> Memory leaking on Fedora 17
> ---------------------------
>
> Key: DTACLOUD-290
> URL: https://issues.apache.org/jira/browse/DTACLOUD-290
> Project: DeltaCloud
> Issue Type: Bug
> Components: Server
> Environment: deltacloud-core-1.0.0-4.fc17.noarch (though same issue with dc-core from git)
> sinatra (1.3.2)
> sinatra-rabbit (1.0.6)
> Fedora 17
> Reporter: Jan Provaznik
> Priority: Critical
>
> Memory used by deltacloudd process increases with time. After 12 hours, it's common it uses ~4G of virt memory and >1G of physical memory (only conductor's dbomatic script which periodically checks running instances every minute was running).
> You can reproduce the memory leak by this simple script:
> #!/usr/bin/ruby
> require 'rubygems'
> require 'deltacloud'
> 1000.times do |i| { DeltaCloud.new('mockuser', 'mockpassword', 'http://localhost:3002/api') }
> Used memory will increase by >10M (both virt and physical). The problem is probably with lambda in check_capability usage in lib/deltacloud/collections/* files:
> check_capability :for => lambda { |m| driver.respond_to? m }
> if the lambda is replaced by true or other static value, the memory usage remains constant.
> I see that in git the 'check_capability' call was replaced by 'set' call, this change has no influence on this problem.
> Also it's weird that after starting deltacloudd, used memory is 233m/33m (virt/res), but after first request, used memory jumps to 776m/90m (virt/res) - this is far more than on Fedora 16, where used memory remains around 160m/50m.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira