You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@jclouds.apache.org by Veit Guna <ve...@gmx.de> on 2016/04/23 15:24:02 UTC
2.0.0-SNAPSHOT Filesystemprovider permission problem
Hi.
I'm using jclouds 2.0.0-SNAPSHOT and I'm encountering strange permission
problems using the Filesystem provider under Windows 7 (with UAC)
using JDK8.
I'm trying to put a blob in a container directory starting in the root
of a harddisk (not C: - which is even more "special").
My path is (D:) /tmp/storage/mycontainer/file
When I try to put the blob using:
blobStore.createContainerInLocation(null, directory,
CreateContainerOptions.Builder.publicRead());
etag = blobStore.putBlob(directory, fileBlob);
I'll get:
Caused by: java.io.IOException: Could not rename file
\tmp\storage\mycontainer\file-cb84907b-a601-4cff-b21e-9ee0f268b567 to
\tmp\storage\mycontainer\file
at
org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.putBlob(FilesystemStorageStrategyImpl.java:500)
~[filesystem-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at
org.jclouds.blobstore.config.LocalBlobStore.putBlob(LocalBlobStore.java:757)
~[jclouds-blobstore-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at
org.jclouds.blobstore.config.LocalBlobStore.putBlob(LocalBlobStore.java:522)
~[jclouds-blobstore-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
...
Looking at the directory, /tmp/storage/mycontainer exists, but it is
empty. Trying to create files in this directory via Windows Explorer,
works. But renaming this
file afterwards doesn't. It complains about missing Administrator rights
- which I have.
Looking at the mycontainer directory permissions, it looks strange.
Every group/user is set to "special" permission - all other permissions
are removed.
Looking at this "special" permission, only read permissions are set.
That explains, why rename isn't allowed here.
Changing _any_ permission, e.g. "taking ownership" and saving it, allows
renaming again.
After this manual change, jclouds operates without any errors for
further puttBlobs in this container.
I already deleted the directory, made a filesystem check and rebootet
the machine. Behavior stays the same. I also disabled my Antivirus
solution - just in case.
Funny thing: changing the path from (D:) /tmp/... to the Windows temp
directory %TMP% (e.g.
C:\Users\<user>\AppData\Local\Temp\mycontainer\...) does not lead
to the exception above!
I know that jclouds is doing some Acl magic depending on the OS e.g.
getting the "Everyone" group from Windows and setting special
permissions to the container
and files. Maybe it's a Windows 7 specialty when not using the Windows
temp directory? Where are the ITs putting their stuff for testing? Are
there even Windows
related tests for the Filesystem storage provider ;)?
Any help is appreciated.
Thanks
Veit
Re: 2.0.0-SNAPSHOT Filesystemprovider permission problem
Posted by Veit Guna <ve...@gmx.de>.
I was curious and ran the tests for the filesystem storage provider
2.0.0-SNAPSHOT.
Indeed there are many tests skipped. If I look into the target dir, only
.\target\basedir\fun-blobstore-test\ss
exists after the tests have failed.
fun-blobstore-test and ss have the same strange permissions as I
mentioned in my earlier post.
I guess that's the reason why they're failing.
Tests run: 225, Failures: 3, Errors: 0, Skipped: 215, Time elapsed:
22.794 sec <<< FAILURE! - in TestSuite
testBlobExists(org.jclouds.filesystem.FilesystemBlobStoreTest) Time
elapsed: 0.145 sec <<< FAILURE!
java.io.IOException: Unable to create parent directories of
.\target\basedir\fun-blobstore-test\ss\asdas\0f50457a-21ec-4d0f-aeb9-0277e152d131.jpg
at com.google.common.io.Files.createParentDirs(Files.java:645)
at
org.jclouds.filesystem.utils.TestUtils.createBlobAsFile(TestUtils.java:194)
at
org.jclouds.filesystem.FilesystemBlobStoreTest.testBlobExists(FilesystemBlobStoreTest.java:602)
tearDown(org.jclouds.filesystem.FilesystemBlobStoreTest) Time elapsed:
10.081 sec <<< FAILURE!
java.io.IOException: Could not delete:
.\target\basedir\fun-blobstore-test\ss
at org.jclouds.filesystem.util.Utils.delete(Utils.java:101)
at
org.jclouds.filesystem.util.Utils.deleteRecursively(Utils.java:76)
at
org.jclouds.filesystem.util.Utils.deleteRecursively(Utils.java:71)
at
org.jclouds.filesystem.util.Utils.deleteRecursively(Utils.java:71)
at
org.jclouds.filesystem.FilesystemBlobStoreTest.tearDown(FilesystemBlobStoreTest.java:105)
setUp(org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImplTest)
Time elapsed: 9.742 sec <<< FAILURE!
java.io.IOException: Could not delete:
.\target\basedir\fun-blobstore-test\ss
at org.jclouds.filesystem.util.Utils.delete(Utils.java:101)
at
org.jclouds.filesystem.util.Utils.deleteRecursively(Utils.java:76)
at
org.jclouds.filesystem.util.Utils.deleteRecursively(Utils.java:71)
at
org.jclouds.filesystem.utils.TestUtils.cleanDirectoryContent(TestUtils.java:175)
at
org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImplTest.setUp(FilesystemStorageStrategyImplTest.java:97)
Results :
Failed tests:
FilesystemBlobStoreTest.testBlobExists:602 » IO Unable to create
parent direct...
FilesystemBlobStoreTest.tearDown:105 » IO Could not delete:
.\target\basedir\f...
FilesystemStorageStrategyImplTest.setUp:97 » IO Could not delete:
.\target\bas...
Tests run: 225, Failures: 3, Errors: 0, Skipped: 215
Am 30.04.2016 um 22:03 schrieb Veit Guna:
> Hi Andrew.
>
> Thanks for looking into this!
>
> I've tried the 1.9.2 version and now the error below pops up. Google
> pointed me to:
>
> https://issues.apache.org/jira/browse/JCLOUDS-1015
>
> Which has seemed to fix the problem. I'm not sure whether this has made
> it into 1.9.2 - apparently not :)?
> I have to admit, that I have a german Windows version. So that matches
> the error and the fix.
>
> I'll try to look into the testcases tomorrow. Thanks so far!
>
>
> java.lang.RuntimeException:
> java.nio.file.attribute.UserPrincipalNotFoundException
> at
> org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.setBlobAccess(FilesystemStorageStrategyImpl.java:547)
> at
> org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.putBlob(FilesystemStorageStrategyImpl.java:464)
> at
> org.jclouds.blobstore.config.LocalBlobStore.putBlob(LocalBlobStore.java:487)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37)
> .....
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at java.lang.reflect.Method.invoke(Method.java:497)
> Caused by: java.nio.file.attribute.UserPrincipalNotFoundException
> at org.jclouds.filesystem.util.Utils.setPrivate(Utils.java:83)
> at
> org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.setBlobAccess(FilesystemStorageStrategyImpl.java:542)
> ... 11 more
>
> Am 30.04.2016 um 21:46 schrieb Andrew Phillips:
>>> Based on the following, I suspect PUBLIC_READ may not work reliably on
>>> transient blobstores such as the filesystem provider, though:
>> Looks like I jumped the gun here a bit ;-) Do you see the same
>> behaviour with 1.9.2, i.e before the change you referred to with the
>> "Everyone lookup" was introduced?
>>
>> Regards
>>
>> ap
>
Re: 2.0.0-SNAPSHOT Filesystemprovider permission problem
Posted by Veit Guna <ve...@gmx.de>.
Hi Andrew.
Thanks for looking into this!
I've tried the 1.9.2 version and now the error below pops up. Google
pointed me to:
https://issues.apache.org/jira/browse/JCLOUDS-1015
Which has seemed to fix the problem. I'm not sure whether this has made
it into 1.9.2 - apparently not :)?
I have to admit, that I have a german Windows version. So that matches
the error and the fix.
I'll try to look into the testcases tomorrow. Thanks so far!
java.lang.RuntimeException:
java.nio.file.attribute.UserPrincipalNotFoundException
at
org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.setBlobAccess(FilesystemStorageStrategyImpl.java:547)
at
org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.putBlob(FilesystemStorageStrategyImpl.java:464)
at
org.jclouds.blobstore.config.LocalBlobStore.putBlob(LocalBlobStore.java:487)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:497)
at
com.google.inject.internal.DelegatingInvocationHandler.invoke(DelegatingInvocationHandler.java:37)
.....
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:497)
Caused by: java.nio.file.attribute.UserPrincipalNotFoundException
at org.jclouds.filesystem.util.Utils.setPrivate(Utils.java:83)
at
org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.setBlobAccess(FilesystemStorageStrategyImpl.java:542)
... 11 more
Am 30.04.2016 um 21:46 schrieb Andrew Phillips:
>> Based on the following, I suspect PUBLIC_READ may not work reliably on
>> transient blobstores such as the filesystem provider, though:
>
> Looks like I jumped the gun here a bit ;-) Do you see the same
> behaviour with 1.9.2, i.e before the change you referred to with the
> "Everyone lookup" was introduced?
>
> Regards
>
> ap
Re: 2.0.0-SNAPSHOT Filesystemprovider permission problem
Posted by Andrew Phillips <an...@apache.org>.
> Based on the following, I suspect PUBLIC_READ may not work reliably on
> transient blobstores such as the filesystem provider, though:
Looks like I jumped the gun here a bit ;-) Do you see the same behaviour
with 1.9.2, i.e before the change you referred to with the "Everyone
lookup" was introduced?
Regards
ap
Re: 2.0.0-SNAPSHOT Filesystemprovider permission problem
Posted by Andrew Phillips <an...@apache.org>.
>> and files. Maybe it's a Windows 7 specialty when not using the Windows
>> temp directory? Where are the ITs putting their stuff for testing? Are
>> there even Windows related tests for the Filesystem storage provider
>> ;)?
There are indeed a number of filesystem tests, although many of them are
skipped on Windows:
https://github.com/jclouds/jclouds/tree/master/apis/filesystem/src/test/java/org/jclouds/filesystem
If you could run the tests on your setup to see if any of them fail,
that would be helpful in terms of providing a reproducible case to work
on.
Based on the following, I suspect PUBLIC_READ may not work reliably on
transient blobstores such as the filesystem provider, though:
https://github.com/jclouds/jclouds/blob/master/apis/filesystem/src/test/java/org/jclouds/filesystem/integration/FilesystemContainerIntegrationTest.java#L192
To your question of testing blobstore work in general: do you have a
testing account with the target provider that you could use? The
semantics of local blobstores, and what is and is not supported, can be
sufficiently different that test results are not necessarily very
predictive of what will happen when you talk to your actual endpoint.
Regards
ap
Re: 2.0.0-SNAPSHOT Filesystemprovider permission problem
Posted by Veit Guna <ve...@gmx.de>.
Anyone?
Am 23.04.2016 um 15:24 schrieb Veit Guna:
> Hi.
>
> I'm using jclouds 2.0.0-SNAPSHOT and I'm encountering strange permission
> problems using the Filesystem provider under Windows 7 (with UAC)
> using JDK8.
>
> I'm trying to put a blob in a container directory starting in the root
> of a harddisk (not C: - which is even more "special").
> My path is (D:) /tmp/storage/mycontainer/file
>
> When I try to put the blob using:
>
> blobStore.createContainerInLocation(null, directory,
> CreateContainerOptions.Builder.publicRead());
> etag = blobStore.putBlob(directory, fileBlob);
>
> I'll get:
>
> Caused by: java.io.IOException: Could not rename file
> \tmp\storage\mycontainer\file-cb84907b-a601-4cff-b21e-9ee0f268b567 to
> \tmp\storage\mycontainer\file
> at
> org.jclouds.filesystem.strategy.internal.FilesystemStorageStrategyImpl.putBlob(FilesystemStorageStrategyImpl.java:500)
> ~[filesystem-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
> at
> org.jclouds.blobstore.config.LocalBlobStore.putBlob(LocalBlobStore.java:757)
> ~[jclouds-blobstore-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
> at
> org.jclouds.blobstore.config.LocalBlobStore.putBlob(LocalBlobStore.java:522)
> ~[jclouds-blobstore-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
> ...
>
> Looking at the directory, /tmp/storage/mycontainer exists, but it is
> empty. Trying to create files in this directory via Windows Explorer,
> works. But renaming this
> file afterwards doesn't. It complains about missing Administrator rights
> - which I have.
>
> Looking at the mycontainer directory permissions, it looks strange.
> Every group/user is set to "special" permission - all other permissions
> are removed.
> Looking at this "special" permission, only read permissions are set.
> That explains, why rename isn't allowed here.
> Changing _any_ permission, e.g. "taking ownership" and saving it, allows
> renaming again.
>
> After this manual change, jclouds operates without any errors for
> further puttBlobs in this container.
> I already deleted the directory, made a filesystem check and rebootet
> the machine. Behavior stays the same. I also disabled my Antivirus
> solution - just in case.
>
> Funny thing: changing the path from (D:) /tmp/... to the Windows temp
> directory %TMP% (e.g.
> C:\Users\<user>\AppData\Local\Temp\mycontainer\...) does not lead
> to the exception above!
>
> I know that jclouds is doing some Acl magic depending on the OS e.g.
> getting the "Everyone" group from Windows and setting special
> permissions to the container
> and files. Maybe it's a Windows 7 specialty when not using the Windows
> temp directory? Where are the ITs putting their stuff for testing? Are
> there even Windows
> related tests for the Filesystem storage provider ;)?
>
> Any help is appreciated.
>
> Thanks
> Veit
>
>
>
>