You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by Everett Toews <no...@github.com> on 2013/12/12 05:01:43 UTC
[jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
https://issues.apache.org/jira/browse/JCLOUDS-340
To run the Claim API live tests:
cd rackspace-cloudqueues-us/
mvn -Plive -Dtest.rackspace-cloudqueues-us.identity=<username> -Dtest.rackspace-cloudqueues-us.credential=<apikey> -Dtest.rackspace-cloudqueues-us.zone=ORD -Dtest=CloudQueuesUSClaimApiLiveTest clean test
You can merge this Pull Request by running:
git pull https://github.com/rackerlabs/jclouds-labs-openstack marconi-claim-api
Or you can view, comment on it, or merge it online at:
https://github.com/jclouds/jclouds-labs-openstack/pull/60
-- Commit Summary --
* The OpenStack Marconi Claim API.
-- File Changes --
M openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/MarconiApi.java (23)
A openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/domain/Claim.java (164)
M openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/domain/Message.java (31)
A openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/features/ClaimApi.java (144)
M openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/features/MessageApi.java (30)
A openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/functions/ParseClaim.java (78)
M openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/functions/ParseMessagesCreated.java (4)
M openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/functions/ParseMessagesToStream.java (47)
M openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/functions/ParseQueueStats.java (6)
M openstack-marconi/src/main/java/org/jclouds/openstack/marconi/v1/options/StreamMessagesOptions.java (16)
A openstack-marconi/src/test/java/org/jclouds/openstack/marconi/v1/features/ClaimApiLiveTest.java (134)
A openstack-marconi/src/test/java/org/jclouds/openstack/marconi/v1/features/ClaimApiMockTest.java (126)
M openstack-marconi/src/test/java/org/jclouds/openstack/marconi/v1/features/MessageApiLiveTest.java (15)
M openstack-marconi/src/test/java/org/jclouds/openstack/marconi/v1/features/MessageApiMockTest.java (23)
A rackspace-cloudqueues-us/src/test/java/org/jclouds/rackspace/cloudqueues/us/CloudQueuesUSClaimApiLiveTest.java (30)
-- Patch Links --
https://github.com/jclouds/jclouds-labs-openstack/pull/60.patch
https://github.com/jclouds/jclouds-labs-openstack/pull/60.diff
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
> + * </p>
> + * When a claim expires, it is released. If the original worker failed to process the message, another client worker
> + * can then claim the message.
> + * </p>
> + * Note that claim creation is best-effort, meaning the worker may claim and return less than the requested number
> + * of messages.
> + * </p>
> + * To deal with workers that have stopped responding (for up to 1209600 seconds or 14 days, including claim
> + * lifetime), the server extends the lifetime of claimed messages to be at least as long as the lifetime of the
> + * claim itself, plus the specified grace period. If a claimed message would normally live longer than the grace
> + * period, its expiration is not adjusted.
> + *
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
> + * must be between 60 and 43200 seconds (12 hours). You must include a value for this attribute in
> + * your request.
> + * @param grace The grace attribute specifies the message grace period in seconds. The value of grace value must
Fixed.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8347034
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
Bad timing.
[FATAL] Non-resolvable parent POM: Could not transfer artifact org.apache.jclouds:jclouds-project:pom:1.7.0-SNAPSHOT from/to apache-snapshots (https://repository.apache.org/content/repositories/snapshots): Failed to transfer file: https://repository.apache.org/content/repositories/snapshots/org/apache/jclouds/jclouds-project/1.7.0-SNAPSHOT/jclouds-project-1.7.0-SNAPSHOT.pom.
Return code is: 503 , ReasonPhrase:Service Temporarily Unavailable. and 'parent.relativePath' points at wrong local POM @ line 23, column 11
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60#issuecomment-30388011
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
> + * When a claim expires, it is released. If the original worker failed to process the message, another client worker
> + * can then claim the message.
> + * </p>
> + * Note that claim creation is best-effort, meaning the worker may claim and return less than the requested number
> + * of messages.
> + * </p>
> + * To deal with workers that have stopped responding (for up to 1209600 seconds or 14 days, including claim
> + * lifetime), the server extends the lifetime of claimed messages to be at least as long as the lifetime of the
> + * claim itself, plus the specified grace period. If a claimed message would normally live longer than the grace
> + * period, its expiration is not adjusted.
> + *
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
> + * must be between 60 and 43200 seconds (12 hours). You must include a value for this attribute in
> + * your request.
> + * @param grace The grace attribute specifies the message grace period in seconds. The value of grace value must
> + * be between 60 and 43200 seconds (12 hours). You must include a value for thia attribute in your
Fixed.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8347046
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
> + * Strip off everything but the message id.
> + */
> + private static String getIdFromHref(String rawMessageHref) {
> + int indexOfQuestionMark = rawMessageHref.indexOf('?');
> + int lastIndexOfSlash = rawMessageHref.lastIndexOf('/') + 1;
> +
> + if (indexOfQuestionMark > 0) {
> + return rawMessageHref.substring(lastIndexOfSlash, indexOfQuestionMark);
> + }
> + else {
> + return rawMessageHref.substring(lastIndexOfSlash);
> + }
> + }
> +
> + private static String getClaimIdFromHref(String rawMessageHref) {
> + int indexOfQuestionMark = rawMessageHref.indexOf('?') + 1;
I like it. Fixed.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8347623
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Andrew Phillips <no...@github.com>.
> + * When a claim expires, it is released. If the original worker failed to process the message, another client worker
> + * can then claim the message.
> + * </p>
> + * Note that claim creation is best-effort, meaning the worker may claim and return less than the requested number
> + * of messages.
> + * </p>
> + * To deal with workers that have stopped responding (for up to 1209600 seconds or 14 days, including claim
> + * lifetime), the server extends the lifetime of claimed messages to be at least as long as the lifetime of the
> + * claim itself, plus the specified grace period. If a claimed message would normally live longer than the grace
> + * period, its expiration is not adjusted.
> + *
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
> + * must be between 60 and 43200 seconds (12 hours). You must include a value for this attribute in
> + * your request.
> + * @param grace The grace attribute specifies the message grace period in seconds. The value of grace value must
> + * be between 60 and 43200 seconds (12 hours). You must include a value for thia attribute in your
[typo] "this"
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8318579
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Andrew Phillips <no...@github.com>.
> + * Strip off everything but the message id.
> + */
> + private static String getIdFromHref(String rawMessageHref) {
> + int indexOfQuestionMark = rawMessageHref.indexOf('?');
> + int lastIndexOfSlash = rawMessageHref.lastIndexOf('/') + 1;
> +
> + if (indexOfQuestionMark > 0) {
> + return rawMessageHref.substring(lastIndexOfSlash, indexOfQuestionMark);
> + }
> + else {
> + return rawMessageHref.substring(lastIndexOfSlash);
> + }
> + }
> +
> + private static String getClaimIdFromHref(String rawMessageHref) {
> + int indexOfQuestionMark = rawMessageHref.indexOf('?') + 1;
```
String query = new URI(rawMessageHref).getQuery();
if (query == null) {
return null;
}
Multimap<String, String> queryParams = queryParser().apply(query);
return getOnlyElement(queryParams.get("claim_id"), null);
```
or something like that?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8318939
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by BuildHive <no...@github.com>.
[jclouds ยป jclouds-labs-openstack #722](https://buildhive.cloudbees.com/job/jclouds/job/jclouds-labs-openstack/722/) FAILURE
Looks like there's a problem with this pull request
[(what's this?)](https://www.cloudbees.com/what-is-buildhive)
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60#issuecomment-30387891
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
> + * from the original TTL). The server resets the age of the claim and applies the new TTL.
> + *
> + * @param claimId Specific claim ID of the message to get.
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
> + * must be between 60 and 43200 seconds (12 hours). You must include a value for this attribute in
> + * your request.
> + */
> + // TODO: revisit this when we figure out what's wrong with PATCH
> + // @Named("claim:update")
> + // @PATCH
> + // @Path("/claims/{claim_id}")
> + // @Fallback(FalseOnNotFoundOr404.class)
> + // @Payload("%7B\"ttl\":{ttl}%7D")
> + // @Produces(MediaType.APPLICATION_JSON)
> + // boolean update(@PathParam("claim_id") String claimId,
> + // @PayloadParam("ttl") int ttl);
Leaving this in for this commit. I'll deal with it in my next PR.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8347118
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
> + return claimWithHref.toBuilder()
> + .id(claimId)
> + .messages(messages)
> + .build();
> + }
> + };
> + private final ParseJson<ClaimWithHref> json;
> +
> + @Inject
> + ParseClaim(ParseJson<ClaimWithHref> json) {
> + this.json = checkNotNull(json, "json");
> + }
> +
> + @Override
> + public Claim apply(HttpResponse response) {
> + ClaimWithHref claimWithHref = json.apply(response);
Nope. If we've gotten this far, we know we have a valid response to parse.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8347375
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
>
> return new Messages(messages, messagesWithHref.getLinks());
> }
>
> - private static String getMessageId(String rawMessageHref) {
> - // strip off everything but the message id
> - return rawMessageHref.substring(rawMessageHref.lastIndexOf('/')+1);
> + /**
> + * Strip off everything but the message id.
> + */
> + private static String getIdFromHref(String rawMessageHref) {
> + int indexOfQuestionMark = rawMessageHref.indexOf('?');
> + int lastIndexOfSlash = rawMessageHref.lastIndexOf('/') + 1;
> +
> + if (indexOfQuestionMark > 0) {
Fixed.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8347457
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Andrew Phillips <no...@github.com>.
> +import static com.google.common.base.Preconditions.checkNotNull;
> +import static com.google.common.collect.Iterables.transform;
> +import static com.google.common.collect.Lists.newArrayList;
> +import static org.jclouds.openstack.marconi.v1.functions.ParseMessagesToStream.MessageWithHref;
> +import static org.jclouds.openstack.marconi.v1.functions.ParseMessagesToStream.TO_ID_FROM_HREF;
> +import static org.jclouds.openstack.marconi.v1.functions.ParseMessagesToStream.TO_MESSAGE;
> +
> +/**
> + * @author Everett Toews
> + */
> +public class ParseClaim implements Function<HttpResponse, Claim> {
> +
> + protected static final Function<ClaimWithHref, Claim> TO_CLAIM = new Function<ClaimWithHref, Claim>() {
> + @Override
> + public Claim apply(ClaimWithHref claimWithHref) {
> + List<Message> messages = newArrayList(transform(claimWithHref.messagesWithHref, TO_MESSAGE));
[minor] Use an ImmutableList?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8318716
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Andrew Phillips <no...@github.com>.
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
> + * must be between 60 and 43200 seconds (12 hours). You must include a value for this attribute in
> + * your request.
> + */
> + // TODO: revisit this when we figure out what's wrong with PATCH
> + // @Named("claim:update")
> + // @PATCH
> + // @Path("/claims/{claim_id}")
> + // @Fallback(FalseOnNotFoundOr404.class)
> + // @Payload("%7B\"ttl\":{ttl}%7D")
> + // @Produces(MediaType.APPLICATION_JSON)
> + // boolean update(@PathParam("claim_id") String claimId,
> + // @PayloadParam("ttl") int ttl);
> +
> + /**
> + * This operation immediately releases a claim, making any remaining, undeleted) messages that are associated with
Remove ")"
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8318631
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by CloudBees pull request builder plugin <no...@github.com>.
[jclouds-labs-openstack-pull-requests #111](https://jclouds.ci.cloudbees.com/job/jclouds-labs-openstack-pull-requests/111/) FAILURE
Looks like there's a problem with this pull request
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60#issuecomment-30387884
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
> + * must be between 60 and 43200 seconds (12 hours). You must include a value for this attribute in
> + * your request.
> + */
> + // TODO: revisit this when we figure out what's wrong with PATCH
> + // @Named("claim:update")
> + // @PATCH
> + // @Path("/claims/{claim_id}")
> + // @Fallback(FalseOnNotFoundOr404.class)
> + // @Payload("%7B\"ttl\":{ttl}%7D")
> + // @Produces(MediaType.APPLICATION_JSON)
> + // boolean update(@PathParam("claim_id") String claimId,
> + // @PayloadParam("ttl") int ttl);
> +
> + /**
> + * This operation immediately releases a claim, making any remaining, undeleted) messages that are associated with
Fixed.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8347130
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by CloudBees pull request builder plugin <no...@github.com>.
[jclouds-labs-openstack-pull-requests #112](https://jclouds.ci.cloudbees.com/job/jclouds-labs-openstack-pull-requests/112/) SUCCESS
This pull request looks good
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60#issuecomment-30389079
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Andrew Phillips <no...@github.com>.
> + return claimWithHref.toBuilder()
> + .id(claimId)
> + .messages(messages)
> + .build();
> + }
> + };
> + private final ParseJson<ClaimWithHref> json;
> +
> + @Inject
> + ParseClaim(ParseJson<ClaimWithHref> json) {
> + this.json = checkNotNull(json, "json");
> + }
> +
> + @Override
> + public Claim apply(HttpResponse response) {
> + ClaimWithHref claimWithHref = json.apply(response);
Do we need a `checkNotNull` here?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8318732
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Andrew Phillips <no...@github.com>.
> + * </p>
> + * The age given for a claim is relative to the server's clock. The claim's age is useful for determining how
> + * quickly messages are getting processed and whether a given message's claim is about to expire.
> + * </p>
> + * When a claim expires, it is released. If the original worker failed to process the message, another client worker
> + * can then claim the message.
> + * </p>
> + * Note that claim creation is best-effort, meaning the worker may claim and return less than the requested number
> + * of messages.
> + * </p>
> + * To deal with workers that have stopped responding (for up to 1209600 seconds or 14 days, including claim
> + * lifetime), the server extends the lifetime of claimed messages to be at least as long as the lifetime of the
> + * claim itself, plus the specified grace period. If a claimed message would normally live longer than the grace
> + * period, its expiration is not adjusted.
> + *
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
[minor] "TTL" (as below) or "ttl"?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8318604
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
> + * </p>
> + * The age given for a claim is relative to the server's clock. The claim's age is useful for determining how
> + * quickly messages are getting processed and whether a given message's claim is about to expire.
> + * </p>
> + * When a claim expires, it is released. If the original worker failed to process the message, another client worker
> + * can then claim the message.
> + * </p>
> + * Note that claim creation is best-effort, meaning the worker may claim and return less than the requested number
> + * of messages.
> + * </p>
> + * To deal with workers that have stopped responding (for up to 1209600 seconds or 14 days, including claim
> + * lifetime), the server extends the lifetime of claimed messages to be at least as long as the lifetime of the
> + * claim itself, plus the specified grace period. If a claimed message would normally live longer than the grace
> + * period, its expiration is not adjusted.
> + *
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
Fixed.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8347022
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Andrew Phillips <no...@github.com>.
>
> return new Messages(messages, messagesWithHref.getLinks());
> }
>
> - private static String getMessageId(String rawMessageHref) {
> - // strip off everything but the message id
> - return rawMessageHref.substring(rawMessageHref.lastIndexOf('/')+1);
> + /**
> + * Strip off everything but the message id.
> + */
> + private static String getIdFromHref(String rawMessageHref) {
> + int indexOfQuestionMark = rawMessageHref.indexOf('?');
> + int lastIndexOfSlash = rawMessageHref.lastIndexOf('/') + 1;
> +
> + if (indexOfQuestionMark > 0) {
Safer to do `if (indexOfQuestionMark > lastIndexOfSlash )`?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8318853
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Andrew Phillips <no...@github.com>.
> + * from the original TTL). The server resets the age of the claim and applies the new TTL.
> + *
> + * @param claimId Specific claim ID of the message to get.
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
> + * must be between 60 and 43200 seconds (12 hours). You must include a value for this attribute in
> + * your request.
> + */
> + // TODO: revisit this when we figure out what's wrong with PATCH
> + // @Named("claim:update")
> + // @PATCH
> + // @Path("/claims/{claim_id}")
> + // @Fallback(FalseOnNotFoundOr404.class)
> + // @Payload("%7B\"ttl\":{ttl}%7D")
> + // @Produces(MediaType.APPLICATION_JSON)
> + // boolean update(@PathParam("claim_id") String claimId,
> + // @PayloadParam("ttl") int ttl);
Remove commented-out code? And how does Javadoc deal with the "double Javadoc" now?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8318620
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Everett Toews <no...@github.com>.
> +import static com.google.common.base.Preconditions.checkNotNull;
> +import static com.google.common.collect.Iterables.transform;
> +import static com.google.common.collect.Lists.newArrayList;
> +import static org.jclouds.openstack.marconi.v1.functions.ParseMessagesToStream.MessageWithHref;
> +import static org.jclouds.openstack.marconi.v1.functions.ParseMessagesToStream.TO_ID_FROM_HREF;
> +import static org.jclouds.openstack.marconi.v1.functions.ParseMessagesToStream.TO_MESSAGE;
> +
> +/**
> + * @author Everett Toews
> + */
> +public class ParseClaim implements Function<HttpResponse, Claim> {
> +
> + protected static final Function<ClaimWithHref, Claim> TO_CLAIM = new Function<ClaimWithHref, Claim>() {
> + @Override
> + public Claim apply(ClaimWithHref claimWithHref) {
> + List<Message> messages = newArrayList(transform(claimWithHref.messagesWithHref, TO_MESSAGE));
Fixed.
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8347328
Re: [jclouds-labs-openstack] The OpenStack Marconi Claim API. (#60)
Posted by Andrew Phillips <no...@github.com>.
> + * </p>
> + * When a claim expires, it is released. If the original worker failed to process the message, another client worker
> + * can then claim the message.
> + * </p>
> + * Note that claim creation is best-effort, meaning the worker may claim and return less than the requested number
> + * of messages.
> + * </p>
> + * To deal with workers that have stopped responding (for up to 1209600 seconds or 14 days, including claim
> + * lifetime), the server extends the lifetime of claimed messages to be at least as long as the lifetime of the
> + * claim itself, plus the specified grace period. If a claimed message would normally live longer than the grace
> + * period, its expiration is not adjusted.
> + *
> + * @param ttl The ttl attribute specifies how long the server waits before releasing the claim. The ttl value
> + * must be between 60 and 43200 seconds (12 hours). You must include a value for this attribute in
> + * your request.
> + * @param grace The grace attribute specifies the message grace period in seconds. The value of grace value must
"The grace value" or "The value of grace"?
---
Reply to this email directly or view it on GitHub:
https://github.com/jclouds/jclouds-labs-openstack/pull/60/files#r8318563