[jira] [Created] (JCLOUDS-310) jclouds Karaf caches results from BlobStore.list in an unbounded Multimap

["Andrew Gaul (JIRA)" <ji...@apache.org>]

Andrew Gaul created JCLOUDS-310:
-----------------------------------

             Summary: jclouds Karaf caches results from BlobStore.list in an unbounded Multimap
                 Key: JCLOUDS-310
                 URL: https://issues.apache.org/jira/browse/JCLOUDS-310
             Project: jclouds
          Issue Type: Bug
          Components: jclouds-karaf
    Affects Versions: 1.6.2, 1.7.0
            Reporter: Andrew Gaul
            Assignee: Andrew Gaul


This leads to OutOfMemoryError on large blobstores; I tested with 88 million blobs but can only enumerate 22 million.

[~iocanel] What is the intent of this cache?  Should I remove use of it in the blobstore list command, or do you want to change BasicCacheProvider to use a Cache instead of a Multimap?  I am also unclear where we use this.

{noformat}
$ time jclouds-cli-1.6.3-SNAPSHOT/bin/jclouds blobstore list mybigcontainer | wc -l
java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.regex.Pattern.matcher(Pattern.java:1088)
        at java.util.regex.Pattern.split(Pattern.java:1197)
        at java.util.regex.Pattern.split(Pattern.java:1259)
        at sun.security.util.DisabledAlgorithmConstraints.decomposes(DisabledAlgorithmConstraints.java:174)
        at sun.security.ssl.SSLAlgorithmConstraints$X509DisabledAlgConstraints.decomposes(SSLAlgorithmConstraints.java:465)
        at sun.security.util.DisabledAlgorithmConstraints.permits(DisabledAlgorithmConstraints.java:113)
        at sun.security.ssl.SSLAlgorithmConstraints.permits(SSLAlgorithmConstraints.java:135)
        at sun.security.ssl.SignatureAndHashAlgorithm.getSupportedAlgorithms(SignatureAndHashAlgorithm.java:162)
        at sun.security.ssl.Handshaker.getLocalSupportedSignAlgs(Handshaker.java:410)   
        at sun.security.ssl.Handshaker.activate(Handshaker.java:476)
        at sun.security.ssl.SSLSocketImpl.kickstartHandshake(SSLSocketImpl.java:1418)   
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1288)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:563)
        at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:767)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
        at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.invoke(JavaUrlHttpCommandExecutorService.java:108)
        at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.invoke(JavaUrlHttpCommandExecutorService.java:73)
        at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:141)
        at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.invoke(InvokeSyncToAsyncHttpMethod.java:131)
        at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:97)
        at org.jclouds.rest.internal.InvokeSyncToAsyncHttpMethod.apply(InvokeSyncToAsyncHttpMethod.java:58)
        at org.jclouds.rest.internal.DelegatesToInvocationFunction.handle(DelegatesToInvocationFunction.java:157)
        at org.jclouds.rest.internal.DelegatesToInvocationFunction.invoke(DelegatesToInvocationFunction.java:124)
        at com.sun.proxy.$Proxy51.listBucket(Unknown Source)
        at org.jclouds.s3.blobstore.S3BlobStore.list(S3BlobStore.java:146)
        at org.jclouds.karaf.commands.blobstore.BlobListCommand.doExecute(BlobListCommand.java:70)
        at org.jclouds.karaf.commands.blobstore.BlobStoreCommandWithOptions.execute(BlobStoreCommandWithOptions.java:70)
        at org.apache.felix.gogo.commands.basic.AbstractCommand.execute(AbstractCommand.java:35)
2241000

real    22m49.988s
user    35m29.157s
sys     0m42.995s
{noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)