[jira] [Created] (JCLOUDS-631) Slowdown of putBlob operation for the FileSystem provider

["Nikola Knezevic (JIRA)" <ji...@apache.org>]

Nikola Knezevic created JCLOUDS-631:
---------------------------------------

             Summary: Slowdown of putBlob operation for the FileSystem provider
                 Key: JCLOUDS-631
                 URL: https://issues.apache.org/jira/browse/JCLOUDS-631
             Project: jclouds
          Issue Type: Bug
          Components: jclouds-blobstore
    Affects Versions: 1.8.0
            Reporter: Nikola Knezevic


The following (incomplete) piece of code when run for the filesystem provider and swift, for uploading a 1GB file shows 2-3x slowdown for the filesystem provider.

Note that the code only _writes_ through jclouds, there is no reading involved.

{code}
 public void sendLargeFilesOnlyJclouds() throws Exception
    {
        LOG.info("Working with {}", p);
        Environment env = Environment.builder().setName("SomeName").addProperties(p).build();

        String providerName = env.getConnectionModules().get(0).getName();
        String provider = checkNotNull(env.getProperty(providerName + ".provider"), new ICStoreException("No provider specified"));

        Iterable<? extends Module> modules =
                ImmutableSet.of(
                        new SLF4JLoggingModule()
                );

        if(provider.equals("s3"))
        {
            provider = "aws-s3";
            modules =
                    ImmutableSet.<Module>builder()
                            .addAll(modules)
                            .add(new ICStoreMultipartUploadModule())
                            .build();
        }
        Properties properties = new Properties();
        // store provider credentials in properties
        properties.setProperty(provider + ".identity", env.getProperty( providerName + ".identity", ""));
        properties.setProperty(provider + ".credential", env.getProperty(providerName + ".credential", ""));
        properties.putAll(env.getPropertiesForModule(providerName));

        final BlobStoreContext blobStoreContext = ContextBuilder.newBuilder(provider)
                .overrides(properties)
                .modules(modules)
                .buildView(BlobStoreContext.class);

        final BlobStore bs = blobStoreContext.getBlobStore();

        bs.createContainerInLocation(null, "mycontainer");

        for (int i = 0; i < num_invocations; i++)
        {
            InputStream fis = new FakeInputStream(FAKE_IS_SEED, blobSize);
            Blob blob = bs.blobBuilder("myblob").payload(fis).build();

            LOG.info("Invocation number {}", i);
            long startTime = System.currentTimeMillis();
            bs.putBlob("mycontainer", blob);
            long stopTime = System.currentTimeMillis();

            LOG.info("Running time for one blob is {}", stopTime - startTime);
        }
    }
{code}

Where {{FakeInputStream}} is an input stream that produces a deterministic output (though method {{read()}}, of a given size, and {{Environment}} is just a smarter container for {{Properties}}.




--
This message was sent by Atlassian JIRA
(v6.2#6252)