You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by syed <gi...@git.apache.org> on 2016/01/12 22:09:28 UTC
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
GitHub user syed opened a pull request:
https://github.com/apache/cloudstack/pull/1332
Add ability to download templates in Swift
This PR adds the ability to download templates when using Swift as a secondary storage. Uses the "temp_url" feature of Swift so that tempates can be downloaded without authenticaiton.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/syed/cloudstack swift-download
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/cloudstack/pull/1332.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1332
----
commit 0015d94f4ef185ecee94c00a4b8cf1e019b62276
Author: Syed <sy...@gmail.com>
Date: 2015-12-14T22:32:56Z
Add ability to download templates in Swift
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by wido <gi...@git.apache.org>.
Github user wido commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r51040901
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
+
+ Map<String, String> tempKeyMap = new HashMap<>();
+ tempKeyMap.put("Temp-URL-Key", tempKey);
+ return postMeta(cfg, "", "", tempKeyMap);
+
+ }
+
+ public static URL generateTempUrl(SwiftClientCfg cfg, String container, String object, String tempKey, int urlExpirationInterval) {
+
+ int currentTime = (int) (System.currentTimeMillis() / 1000L);
+ int expirationSeconds = currentTime + urlExpirationInterval;
+
+ try {
+
+ URL endpoint = new URL(cfg.getEndPoint());
+ String method = "GET";
+ String path = String.format("/v1/AUTH_%s/%s/%s", cfg.getAccount(), container, object);
+
+ //sign the request
+ String hmacBody = String.format("%s\n%d\n%s", method, expirationSeconds, path);
+ String signature = calculateRFC2104HMAC(hmacBody, tempKey);
+ path += String.format("?temp_url_sig=%s&temp_url_expires=%d", signature, expirationSeconds);
+
+ //generate the temp url
+ URL tempUrl = new URL(endpoint.getProtocol(), endpoint.getHost(), endpoint.getPort(), path);
+
+ return tempUrl;
+
+ } catch (Exception e) {
+ logger.error(e.getMessage());
+ throw new CloudRuntimeException(e.getMessage());
+ }
+
+ }
+
+
+ private static String calculateRFC2104HMAC(String data, String key)
+ throws SignatureException, NoSuchAlgorithmException, InvalidKeyException {
+
+ SecretKeySpec signingKey = new SecretKeySpec(key.getBytes(), HMAC_SHA1_ALGORITHM);
+ Mac mac = Mac.getInstance(HMAC_SHA1_ALGORITHM);
+ mac.init(signingKey);
+ return toHexString(mac.doFinal(data.getBytes()));
+
+ }
+
+ private static String toHexString(byte[] bytes) {
--- End diff --
Same here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r50452491
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
+
+ Map<String, String> tempKeyMap = new HashMap<>();
+ tempKeyMap.put("Temp-URL-Key", tempKey);
+ return postMeta(cfg, "", "", tempKeyMap);
+
+ }
+
+ public static URL generateTempUrl(SwiftClientCfg cfg, String container, String object, String tempKey, int urlExpirationInterval) {
+
+ int currentTime = (int) (System.currentTimeMillis() / 1000L);
+ int expirationSeconds = currentTime + urlExpirationInterval;
+
+ try {
+
+ URL endpoint = new URL(cfg.getEndPoint());
+ String method = "GET";
+ String path = String.format("/v1/AUTH_%s/%s/%s", cfg.getAccount(), container, object);
+
+ //sign the request
+ String hmacBody = String.format("%s\n%d\n%s", method, expirationSeconds, path);
+ String signature = calculateRFC2104HMAC(hmacBody, tempKey);
+ path += String.format("?temp_url_sig=%s&temp_url_expires=%d", signature, expirationSeconds);
+
+ //generate the temp url
+ URL tempUrl = new URL(endpoint.getProtocol(), endpoint.getHost(), endpoint.getPort(), path);
+
+ return tempUrl;
+
+ } catch (Exception e) {
+ logger.error("Unable to generate temperoary URL " + e.getMessage());
--- End diff --
rethrowing exception
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by wido <gi...@git.apache.org>.
Github user wido commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-175824353
Looks good at first, but I would like to see some UnitTests around methods. I'll comment on those inline.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r51153892
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
+
+ Map<String, String> tempKeyMap = new HashMap<>();
+ tempKeyMap.put("Temp-URL-Key", tempKey);
+ return postMeta(cfg, "", "", tempKeyMap);
+
+ }
+
+ public static URL generateTempUrl(SwiftClientCfg cfg, String container, String object, String tempKey, int urlExpirationInterval) {
+
+ int currentTime = (int) (System.currentTimeMillis() / 1000L);
+ int expirationSeconds = currentTime + urlExpirationInterval;
+
+ try {
+
+ URL endpoint = new URL(cfg.getEndPoint());
+ String method = "GET";
+ String path = String.format("/v1/AUTH_%s/%s/%s", cfg.getAccount(), container, object);
+
+ //sign the request
+ String hmacBody = String.format("%s\n%d\n%s", method, expirationSeconds, path);
+ String signature = calculateRFC2104HMAC(hmacBody, tempKey);
+ path += String.format("?temp_url_sig=%s&temp_url_expires=%d", signature, expirationSeconds);
+
+ //generate the temp url
+ URL tempUrl = new URL(endpoint.getProtocol(), endpoint.getHost(), endpoint.getPort(), path);
+
+ return tempUrl;
+
+ } catch (Exception e) {
+ logger.error(e.getMessage());
+ throw new CloudRuntimeException(e.getMessage());
+ }
+
+ }
+
+
+ private static String calculateRFC2104HMAC(String data, String key)
+ throws SignatureException, NoSuchAlgorithmException, InvalidKeyException {
+
+ SecretKeySpec signingKey = new SecretKeySpec(key.getBytes(), HMAC_SHA1_ALGORITHM);
+ Mac mac = Mac.getInstance(HMAC_SHA1_ALGORITHM);
+ mac.init(signingKey);
+ return toHexString(mac.doFinal(data.getBytes()));
+
+ }
+
+ private static String toHexString(byte[] bytes) {
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r50451763
--- Diff: plugins/storage/image/swift/src/org/apache/cloudstack/storage/datastore/driver/SwiftImageStoreDriverImpl.java ---
@@ -67,7 +73,28 @@ public DataStoreTO getStoreTO(DataStore store) {
@Override
public String createEntityExtractUrl(DataStore store, String installPath, ImageFormat format, DataObject dataObject) {
- throw new UnsupportedServiceException("Extract entity url is not yet supported for Swift image store provider");
+
+ SwiftTO swiftTO = (SwiftTO)store.getTO();
+ String tempKey = UUID.randomUUID().toString();
+ boolean result = SwiftUtil.setTempKey(swiftTO, tempKey);
+
+ if (!result) {
+ s_logger.error("Unable to set Temp-Key: " + tempKey);
+ return "";
--- End diff --
Fixed. Throwing exception instead
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-179502412
Can we get this merged?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by DaanHoogland <gi...@git.apache.org>.
Github user DaanHoogland commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-171277602
looks good @syed Can you add test data/description
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r51153907
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
+
+ Map<String, String> tempKeyMap = new HashMap<>();
+ tempKeyMap.put("Temp-URL-Key", tempKey);
+ return postMeta(cfg, "", "", tempKeyMap);
+
+ }
+
+ public static URL generateTempUrl(SwiftClientCfg cfg, String container, String object, String tempKey, int urlExpirationInterval) {
+
+ int currentTime = (int) (System.currentTimeMillis() / 1000L);
+ int expirationSeconds = currentTime + urlExpirationInterval;
+
+ try {
+
+ URL endpoint = new URL(cfg.getEndPoint());
+ String method = "GET";
+ String path = String.format("/v1/AUTH_%s/%s/%s", cfg.getAccount(), container, object);
+
+ //sign the request
+ String hmacBody = String.format("%s\n%d\n%s", method, expirationSeconds, path);
+ String signature = calculateRFC2104HMAC(hmacBody, tempKey);
+ path += String.format("?temp_url_sig=%s&temp_url_expires=%d", signature, expirationSeconds);
+
+ //generate the temp url
+ URL tempUrl = new URL(endpoint.getProtocol(), endpoint.getHost(), endpoint.getPort(), path);
+
+ return tempUrl;
+
+ } catch (Exception e) {
+ logger.error(e.getMessage());
+ throw new CloudRuntimeException(e.getMessage());
+ }
+
+ }
+
+
+ private static String calculateRFC2104HMAC(String data, String key)
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by wido <gi...@git.apache.org>.
Github user wido commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r51040885
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
+
+ Map<String, String> tempKeyMap = new HashMap<>();
+ tempKeyMap.put("Temp-URL-Key", tempKey);
+ return postMeta(cfg, "", "", tempKeyMap);
+
+ }
+
+ public static URL generateTempUrl(SwiftClientCfg cfg, String container, String object, String tempKey, int urlExpirationInterval) {
+
+ int currentTime = (int) (System.currentTimeMillis() / 1000L);
+ int expirationSeconds = currentTime + urlExpirationInterval;
+
+ try {
+
+ URL endpoint = new URL(cfg.getEndPoint());
+ String method = "GET";
+ String path = String.format("/v1/AUTH_%s/%s/%s", cfg.getAccount(), container, object);
+
+ //sign the request
+ String hmacBody = String.format("%s\n%d\n%s", method, expirationSeconds, path);
+ String signature = calculateRFC2104HMAC(hmacBody, tempKey);
+ path += String.format("?temp_url_sig=%s&temp_url_expires=%d", signature, expirationSeconds);
+
+ //generate the temp url
+ URL tempUrl = new URL(endpoint.getProtocol(), endpoint.getHost(), endpoint.getPort(), path);
+
+ return tempUrl;
+
+ } catch (Exception e) {
+ logger.error(e.getMessage());
+ throw new CloudRuntimeException(e.getMessage());
+ }
+
+ }
+
+
+ private static String calculateRFC2104HMAC(String data, String key)
--- End diff --
Same here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/cloudstack/pull/1332
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-205981012
Sure @swill . I've sqashed them
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by swill <gi...@git.apache.org>.
Github user swill commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-205926253
@syed can you squash the commits. This is best applied as a single commit. Thx.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by pdube <gi...@git.apache.org>.
Github user pdube commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r49606982
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
+
+ Map<String, String> tempKeyMap = new HashMap<>();
+ tempKeyMap.put("Temp-URL-Key", tempKey);
+ return postMeta(cfg, "", "", tempKeyMap);
+
+ }
+
+ public static URL generateTempUrl(SwiftClientCfg cfg, String container, String object, String tempKey, int urlExpirationInterval) {
+
+ int currentTime = (int) (System.currentTimeMillis() / 1000L);
+ int expirationSeconds = currentTime + urlExpirationInterval;
+
+ try {
+
+ URL endpoint = new URL(cfg.getEndPoint());
+ String method = "GET";
+ String path = String.format("/v1/AUTH_%s/%s/%s", cfg.getAccount(), container, object);
+
+ //sign the request
+ String hmacBody = String.format("%s\n%d\n%s", method, expirationSeconds, path);
+ String signature = calculateRFC2104HMAC(hmacBody, tempKey);
+ path += String.format("?temp_url_sig=%s&temp_url_expires=%d", signature, expirationSeconds);
+
+ //generate the temp url
+ URL tempUrl = new URL(endpoint.getProtocol(), endpoint.getHost(), endpoint.getPort(), path);
+
+ return tempUrl;
+
+ } catch (Exception e) {
+ logger.error("Unable to generate temperoary URL " + e.getMessage());
--- End diff --
Should log the exception, not just the message. And why catch all exceptions? What exceptions are you expecting?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r51153916
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
+
+ Map<String, String> tempKeyMap = new HashMap<>();
+ tempKeyMap.put("Temp-URL-Key", tempKey);
+ return postMeta(cfg, "", "", tempKeyMap);
+
+ }
+
+ public static URL generateTempUrl(SwiftClientCfg cfg, String container, String object, String tempKey, int urlExpirationInterval) {
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by wido <gi...@git.apache.org>.
Github user wido commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r51040823
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
--- End diff --
Could you write a UnitTest for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-173688390
JIRA: [CLOUDSTACK-9248](https://issues.apache.org/jira/browse/CLOUDSTACK-9248)
Tested this on our local setup. Created a template and generated the download link. Verified the download and it is the same as the template.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by pdube <gi...@git.apache.org>.
Github user pdube commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r49605888
--- Diff: plugins/storage/image/swift/src/org/apache/cloudstack/storage/datastore/driver/SwiftImageStoreDriverImpl.java ---
@@ -67,7 +73,28 @@ public DataStoreTO getStoreTO(DataStore store) {
@Override
public String createEntityExtractUrl(DataStore store, String installPath, ImageFormat format, DataObject dataObject) {
- throw new UnsupportedServiceException("Extract entity url is not yet supported for Swift image store provider");
+
+ SwiftTO swiftTO = (SwiftTO)store.getTO();
+ String tempKey = UUID.randomUUID().toString();
+ boolean result = SwiftUtil.setTempKey(swiftTO, tempKey);
+
+ if (!result) {
+ s_logger.error("Unable to set Temp-Key: " + tempKey);
+ return "";
--- End diff --
Why return empty here, and null later?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by pdube <gi...@git.apache.org>.
Github user pdube commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-176825081
LGTM Tested with a XenServer host and Swift as secondary storage. Got the link and downloaded the template
<img width="885" alt="screen shot 2016-01-29 at 10 02 13 am" src="https://cloud.githubusercontent.com/assets/2355044/12679915/ea6b1a40-c674-11e5-9a95-617ece9e1c33.png">
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-205922970
@bvbharatk it looks to be a problem with the test. Volume creation should work irrespective of my changes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-176289614
Added unit tests
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by wido <gi...@git.apache.org>.
Github user wido commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-176629568
With the tests this looks good to me. I have no way of verifying the Swift working itself, but based on the code and tests: LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by wido <gi...@git.apache.org>.
Github user wido commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r51040863
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
+
+ Map<String, String> tempKeyMap = new HashMap<>();
+ tempKeyMap.put("Temp-URL-Key", tempKey);
+ return postMeta(cfg, "", "", tempKeyMap);
+
+ }
+
+ public static URL generateTempUrl(SwiftClientCfg cfg, String container, String object, String tempKey, int urlExpirationInterval) {
--- End diff --
Same here. This method is perfectly suited for a Unit Test
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by bvbharatk <gi...@git.apache.org>.
Github user bvbharatk commented on the pull request:
https://github.com/apache/cloudstack/pull/1332#issuecomment-201084616
_Link to logs Folder (search by build_no):_ https://www.dropbox.com/sh/yj3wnzbceo9uef2/AAB6u-Iap-xztdm6jHX9SjPja?dl=0
### ACS CI BVT Run
**Sumarry:**
Build Number 125
Hypervisor xenserver
NetworkType Advanced
Passed=105
Failed=1
Skipped=4
**Failed tests:**
* integration.smoke.test_volumes.TestCreateVolume
* test_01_create_volume Failed
**Skipped tests:**
test_vm_nic_adapter_vmxnet3
test_deploy_vgpu_enabled_vm
test_06_copy_template
test_06_copy_iso
**Passed test suits:**
integration.smoke.test_deploy_vm_with_userdata.TestDeployVmWithUserData
integration.smoke.test_affinity_groups_projects.TestDeployVmWithAffinityGroup
integration.smoke.test_portable_publicip.TestPortablePublicIPAcquire
integration.smoke.test_over_provisioning.TestUpdateOverProvision
integration.smoke.test_global_settings.TestUpdateConfigWithScope
integration.smoke.test_guest_vlan_range.TestDedicateGuestVlanRange
integration.smoke.test_scale_vm.TestScaleVm
integration.smoke.test_service_offerings.TestCreateServiceOffering
integration.smoke.test_loadbalance.TestLoadBalance
integration.smoke.test_routers.TestRouterServices
integration.smoke.test_reset_vm_on_reboot.TestResetVmOnReboot
integration.smoke.test_snapshots.TestSnapshotRootDisk
integration.smoke.test_deploy_vms_with_varied_deploymentplanners.TestDeployVmWithVariedPlanners
integration.smoke.test_network.TestDeleteAccount
integration.smoke.test_non_contigiousvlan.TestUpdatePhysicalNetwork
integration.smoke.test_deploy_vm_iso.TestDeployVMFromISO
integration.smoke.test_public_ip_range.TestDedicatePublicIPRange
integration.smoke.test_multipleips_per_nic.TestDeployVM
integration.smoke.test_regions.TestRegions
integration.smoke.test_affinity_groups.TestDeployVmWithAffinityGroup
integration.smoke.test_network_acl.TestNetworkACL
integration.smoke.test_pvlan.TestPVLAN
integration.smoke.test_ssvm.TestSSVMs
integration.smoke.test_nic.TestNic
integration.smoke.test_deploy_vm_root_resize.TestDeployVM
integration.smoke.test_resource_detail.TestResourceDetail
integration.smoke.test_secondary_storage.TestSecStorageServices
integration.smoke.test_vm_life_cycle.TestDeployVM
integration.smoke.test_disk_offerings.TestCreateDiskOffering
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by pdube <gi...@git.apache.org>.
Github user pdube commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r49606572
--- Diff: plugins/storage/image/swift/src/org/apache/cloudstack/storage/datastore/driver/SwiftImageStoreDriverImpl.java ---
@@ -67,7 +73,28 @@ public DataStoreTO getStoreTO(DataStore store) {
@Override
public String createEntityExtractUrl(DataStore store, String installPath, ImageFormat format, DataObject dataObject) {
- throw new UnsupportedServiceException("Extract entity url is not yet supported for Swift image store provider");
+
+ SwiftTO swiftTO = (SwiftTO)store.getTO();
+ String tempKey = UUID.randomUUID().toString();
+ boolean result = SwiftUtil.setTempKey(swiftTO, tempKey);
+
+ if (!result) {
+ s_logger.error("Unable to set Temp-Key: " + tempKey);
+ return "";
+ }
+
+ String containerName = SwiftUtil.getContainerName(dataObject.getType().toString(), dataObject.getId());
+ String objectName = installPath.split("\\/")[1];
+ // Get extract url expiration interval set in global configuration (in seconds)
+ int urlExpirationInterval = Integer.parseInt(_configDao.getValue(Config.ExtractURLExpirationInterval.toString()));
+
+ URL swiftUrl = SwiftUtil.generateTempUrl(swiftTO, containerName, objectName, tempKey, urlExpirationInterval);
+ if (swiftUrl != null) {
+ s_logger.info("Swift temp-url: " + swiftUrl.toString());
+ return swiftUrl.toString();
+ }
+
--- End diff --
Could instead add a log statement here that indicates that the extraction url could not be created for the container and object
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r51154027
--- Diff: utils/src/main/java/com/cloud/utils/SwiftUtil.java ---
@@ -236,4 +247,60 @@ public static boolean deleteObject(SwiftClientCfg cfg, String path) {
command.execute(parser);
return true;
}
+
+ public static boolean setTempKey(SwiftClientCfg cfg, String tempKey){
--- End diff --
I don't think I can write a unit test for this as it relies on an external script to work to give the correct output. I could mock the output but that would defeat the purpose of the unit test.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by pdube <gi...@git.apache.org>.
Github user pdube commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r49606475
--- Diff: plugins/storage/image/swift/src/org/apache/cloudstack/storage/datastore/driver/SwiftImageStoreDriverImpl.java ---
@@ -67,7 +73,28 @@ public DataStoreTO getStoreTO(DataStore store) {
@Override
public String createEntityExtractUrl(DataStore store, String installPath, ImageFormat format, DataObject dataObject) {
- throw new UnsupportedServiceException("Extract entity url is not yet supported for Swift image store provider");
+
+ SwiftTO swiftTO = (SwiftTO)store.getTO();
+ String tempKey = UUID.randomUUID().toString();
+ boolean result = SwiftUtil.setTempKey(swiftTO, tempKey);
+
+ if (!result) {
+ s_logger.error("Unable to set Temp-Key: " + tempKey);
+ return "";
+ }
+
+ String containerName = SwiftUtil.getContainerName(dataObject.getType().toString(), dataObject.getId());
+ String objectName = installPath.split("\\/")[1];
+ // Get extract url expiration interval set in global configuration (in seconds)
+ int urlExpirationInterval = Integer.parseInt(_configDao.getValue(Config.ExtractURLExpirationInterval.toString()));
+
+ URL swiftUrl = SwiftUtil.generateTempUrl(swiftTO, containerName, objectName, tempKey, urlExpirationInterval);
+ if (swiftUrl != null) {
+ s_logger.info("Swift temp-url: " + swiftUrl.toString());
--- End diff --
Should probably be DEBUG instead of INFO, or remove this the log statement
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] cloudstack pull request: Add ability to download templates in Swif...
Posted by syed <gi...@git.apache.org>.
Github user syed commented on a diff in the pull request:
https://github.com/apache/cloudstack/pull/1332#discussion_r50451895
--- Diff: plugins/storage/image/swift/src/org/apache/cloudstack/storage/datastore/driver/SwiftImageStoreDriverImpl.java ---
@@ -67,7 +73,28 @@ public DataStoreTO getStoreTO(DataStore store) {
@Override
public String createEntityExtractUrl(DataStore store, String installPath, ImageFormat format, DataObject dataObject) {
- throw new UnsupportedServiceException("Extract entity url is not yet supported for Swift image store provider");
+
+ SwiftTO swiftTO = (SwiftTO)store.getTO();
+ String tempKey = UUID.randomUUID().toString();
+ boolean result = SwiftUtil.setTempKey(swiftTO, tempKey);
+
+ if (!result) {
+ s_logger.error("Unable to set Temp-Key: " + tempKey);
+ return "";
+ }
+
+ String containerName = SwiftUtil.getContainerName(dataObject.getType().toString(), dataObject.getId());
+ String objectName = installPath.split("\\/")[1];
+ // Get extract url expiration interval set in global configuration (in seconds)
+ int urlExpirationInterval = Integer.parseInt(_configDao.getValue(Config.ExtractURLExpirationInterval.toString()));
+
+ URL swiftUrl = SwiftUtil.generateTempUrl(swiftTO, containerName, objectName, tempKey, urlExpirationInterval);
+ if (swiftUrl != null) {
+ s_logger.info("Swift temp-url: " + swiftUrl.toString());
--- End diff --
changed to DEBUG
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---