You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jclouds.apache.org by felipe gutierrez <fe...@usto.re> on 2014/08/11 22:08:16 UTC
How does jclouds storage my files?
Hi,
I am new with jclouds. What I am doing is: Opening a disk with java and
formatting it with jclouds. So, instead of write Files.write() I use
jclouds.
When I format I can see about 63 blocks of 2MB. The blocks next to it, say
the 30 neighbors, are the indexes of my files copied to the disk. It is so
true that if I save 100 files for example, I can keep these blocks and move
the another blocks to my cloud and I can still open the disk and see all
files inside. Only see, of course. If I tried to open I need to move back
the exactly blocks of that file I want to open.
My question: When I copied 50GB of movies I still can see all movies on my
disk. When I open the movie I need to restore all blocks. It is what my app
does. But I got a claim saying the index of the movie are not integrate. I
still can see the size of the movie is 800MB, but when I open it says only
20 minutes of movie. So I would like to restore all the blocks of that
movie, not only what my app is asking to read.
Did I make myself clear?
Thanks in advanced, Felipe.
Re: How does jclouds storage my files?
Posted by Andrew Phillips <an...@apache.org>.
Hi Filipe
> I am new with jclouds. What I am doing is: Opening a disk with java and
> formatting it with jclouds. So, instead of write Files.write() I use
> jclouds.
Which provider or API are you using? And which underlying system (a
file system, I suspect?) are you talking to?
Which commands are you using to store the blobs, and which to read
them? What is the error?
Could you create a small test case that we could run to reproduce the
behaviour you're seeing?
Regards
ap
Re: How does jclouds storage my files?
Posted by felipe gutierrez <fe...@usto.re>.
ok, I understood, thanks Andrew. I believe I need to check it at the iSCSI
protocol and create meta-data for it.
On Tue, Aug 12, 2014 at 12:35 PM, Andrew Phillips <an...@apache.org>
wrote:
> yes. But What I am doing is editing jscsi code. So they don't know how to
>> answer because they don't move the blobs to other place.
>>
>
> Ah, OK. Still, the specific details of how a blob is stored are either
> jSCSI-specific, or (as Andrew Gaul pointed out) specific to the native
> filesystem provider in the JVM, which in turn depends on the underlying
> filesystem.
>
> The jclouds filesystem API does not do any of this work itself: it simply
> uses the standard Java file libraries to write and write files.
>
> So I suspect you will have to either check with jSCSI or examine the file
> handling implementation of the JVM on which you are running :-(
>
> ap
>
Re: How does jclouds storage my files?
Posted by Andrew Phillips <an...@apache.org>.
> yes. But What I am doing is editing jscsi code. So they don't know how to
> answer because they don't move the blobs to other place.
Ah, OK. Still, the specific details of how a blob is stored are either
jSCSI-specific, or (as Andrew Gaul pointed out) specific to the native
filesystem provider in the JVM, which in turn depends on the
underlying filesystem.
The jclouds filesystem API does not do any of this work itself: it
simply uses the standard Java file libraries to write and write files.
So I suspect you will have to either check with jSCSI or examine the
file handling implementation of the JVM on which you are running :-(
ap
Re: How does jclouds storage my files?
Posted by felipe gutierrez <fe...@usto.re>.
yes. But What I am doing is editing jscsi code. So they don't know how to
answer because they don't move the blobs to other place.
On Tue, Aug 12, 2014 at 10:27 AM, Andrew Phillips <an...@apache.org>
wrote:
> 2 - I use this implementation of iSCSI http://jscsi.org/. And it is this
>> class that saves my blobs
>> http://grepcode.com/file/repo1.maven.org/maven2/org.
>> jscsi/target/2.5.2/org/jscsi/target/storage/
>> JCloudsStorageModule.java?av=f
>>
>
> As far as I can see, the questions you have are related to jSCSI, which is
> not part of jclouds. Have you tried asking on the jSCSI mailing list [1]?
>
> ap
>
> [1] https://github.com/disy/jSCSI/blob/master/pom.xml#L65-L72
>
Re: How does jclouds storage my files?
Posted by Andrew Phillips <an...@apache.org>.
> 2 - I use this implementation of iSCSI http://jscsi.org/. And it is this
> class that saves my blobs
> http://grepcode.com/file/repo1.maven.org/maven2/org.jscsi/target/2.5.2/org/jscsi/target/storage/JCloudsStorageModule.java?av=f
As far as I can see, the questions you have are related to jSCSI,
which is not part of jclouds. Have you tried asking on the jSCSI
mailing list [1]?
ap
[1] https://github.com/disy/jSCSI/blob/master/pom.xml#L65-L72
Re: How does jclouds storage my files?
Posted by felipe gutierrez <fe...@usto.re>.
ok, thanks for reply.
1 - To store the Blobs I am using it:
private BlobStore mStore;
mStore = mContext.getBlobStore();
if (!mStore.containerExists(mContainerName)) {
Location locToSet = null;
for (Location loc : mStore.listAssignableLocations()) {
if (loc.getId().equals("eu-west-1")) {
locToSet = loc;
break;
}
}
mStore.createContainerInLocation(locToSet, mContainerName);
}
Blob blob = mStore.blobBuilder(Integer.toString(mBucketIndex)).build();
blob.setPayload(data);
mStore.putBlob(mContainerName, blob);
2 - I use this implementation of iSCSI http://jscsi.org/. And it is this
class that saves my blobs
http://grepcode.com/file/repo1.maven.org/maven2/org.jscsi/target/2.5.2/org/jscsi/target/storage/JCloudsStorageModule.java?av=f
3 - I do not know how to create a small test case. What it should have?
4 - Format means I have a disk that I can partition it and format.
5 - The indexes are the blobs that indicates there are some files on my
driver, but not really the file content.
Hope this answer.
On Mon, Aug 11, 2014 at 9:33 PM, Andrew Gaul <ga...@apache.org> wrote:
> On Mon, Aug 11, 2014 at 05:08:16PM -0300, felipe gutierrez wrote:
> > I am new with jclouds. What I am doing is: Opening a disk with java and
> > formatting it with jclouds. So, instead of write Files.write() I use
> > jclouds.
> >
> > When I format I can see about 63 blocks of 2MB. The blocks next to it,
> say
> > the 30 neighbors, are the indexes of my files copied to the disk. It is
> so
> > true that if I save 100 files for example, I can keep these blocks and
> move
> > the another blocks to my cloud and I can still open the disk and see all
> > files inside. Only see, of course. If I tried to open I need to move back
> > the exactly blocks of that file I want to open.
> >
> > My question: When I copied 50GB of movies I still can see all movies on
> my
> > disk. When I open the movie I need to restore all blocks. It is what my
> app
> > does. But I got a claim saying the index of the movie are not integrate.
> I
> > still can see the size of the movie is 800MB, but when I open it says
> only
> > 20 minutes of movie. So I would like to restore all the blocks of that
> > movie, not only what my app is asking to read.
>
> Can you explain how you open a disk and what it means to format it with
> jclouds? Also what does an index mean? The jclouds filesystem provider
> does not provide any indexing; the underlying operating system provides
> this.
>
> --
> Andrew Gaul
> http://gaul.org/
>
Re: How does jclouds storage my files?
Posted by Andrew Gaul <ga...@apache.org>.
On Mon, Aug 11, 2014 at 05:08:16PM -0300, felipe gutierrez wrote:
> I am new with jclouds. What I am doing is: Opening a disk with java and
> formatting it with jclouds. So, instead of write Files.write() I use
> jclouds.
>
> When I format I can see about 63 blocks of 2MB. The blocks next to it, say
> the 30 neighbors, are the indexes of my files copied to the disk. It is so
> true that if I save 100 files for example, I can keep these blocks and move
> the another blocks to my cloud and I can still open the disk and see all
> files inside. Only see, of course. If I tried to open I need to move back
> the exactly blocks of that file I want to open.
>
> My question: When I copied 50GB of movies I still can see all movies on my
> disk. When I open the movie I need to restore all blocks. It is what my app
> does. But I got a claim saying the index of the movie are not integrate. I
> still can see the size of the movie is 800MB, but when I open it says only
> 20 minutes of movie. So I would like to restore all the blocks of that
> movie, not only what my app is asking to read.
Can you explain how you open a disk and what it means to format it with
jclouds? Also what does an index mean? The jclouds filesystem provider
does not provide any indexing; the underlying operating system provides
this.
--
Andrew Gaul
http://gaul.org/