You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jclouds.apache.org by Noorul Islam K M <no...@collab.net> on 2013/08/26 07:26:05 UTC
[jclouds-chef] Support for Chef server 11
We are planning to use jclouds-chef library to talk to community Chef
Server 11. In my initial testings, I get the following exception when I
try to get cookbook details using
api.getCookbook("openssl", "1.0.2")
| java.lang.IllegalArgumentException: duplicate key: openssl
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
at com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
at com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
at org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
at org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
at com.google.gson.Gson.fromJson(Gson.java:795)
at com.google.gson.Gson.fromJson(Gson.java:761)
at com.google.gson.Gson.fromJson(Gson.java:710)
at org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
README.md says that
It currently supports versions 0.9 and 0.10 of the standard Chef server
apis, and an initial and very basic (still in progress) implementation
of the user and organization api of the Hosted and Private Chef
flavours.
I think that the issue I am facing is because I am using Chef server 11.
Is there any plans to officially support Chef Server 11 in near future?
Thanks and Regards
Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Ignasi <ig...@gmail.com> writes:
> Hi Noorul!
>
> There is definitely the intention to support Chef 11 as soon as possible. I
> have a work in progress branch [1] where I fixed all Enterprise Chef live
> tests and also made all live tests pass against a Chef 11 except the
> "listCookbooks" one.
>
> I get the exact same error than you, and it happens when deserializing the
> json for the cookbook into the CookbookVersion object. It seems that the
> Chef Server is returning a duplicate key in the json structure (although it
> shouldn't because we are forcing by default a 0.10 version header to make
> it return the same responses than in previous versions), and the json
> parser fails to build the corresponding map.
>
> I'm currently working on this but it's taking longer than expected, so any
> help would be much appreciated. If you could build my branch and enable
> jclouds wire logging [2], you would be able to see the json returned for
> your cookbook. Could you share that json? Also, any help tracking/isolating
> the error (or patch :)) is appreciated too!
>
Here is what server is returning
https://gist.github.com/noorul/6339505
Thanks and Regards
Noorul
> Ignasi
>
> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
> [2]
> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>
>>
>> We are planning to use jclouds-chef library to talk to community Chef
>> Server 11. In my initial testings, I get the following exception when I
>> try to get cookbook details using
>>
>> api.getCookbook("openssl", "1.0.2")
>>
>>
>> | java.lang.IllegalArgumentException: duplicate key: openssl
>> at
>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>> at
>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>> at
>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>> at
>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>> at
>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>> at
>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>> at com.google.gson.Gson.fromJson(Gson.java:795)
>> at com.google.gson.Gson.fromJson(Gson.java:761)
>> at com.google.gson.Gson.fromJson(Gson.java:710)
>> at
>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>
>> README.md says that
>>
>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>> apis, and an initial and very basic (still in progress) implementation
>> of the user and organization api of the Hosted and Private Chef
>> flavours.
>>
>> I think that the issue I am facing is because I am using Chef server 11.
>> Is there any plans to officially support Chef Server 11 in near future?
>>
>> Thanks and Regards
>> Noorul
>>
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
BTW, if you want to get the list of all cookbooks you might prefer
doing it like:
ChefService chefService = context.getChefService();
Iterable< ? extends CookbookVersion> cookbooks =
chefService.listCookbookVersions();
On 26 August 2013 14:17, Ignasi <ig...@gmail.com> wrote:
> Thanks a lot for your feedback Noorul!
>
> You can run the 'core' live tests as follows:
>
> mvn clean install -pl core -Plive \
> -Dtest.chef.endpoint=<you chef server url> \
> -Dtest.chef.identity=<identity> \
> -Dtest.chef.credential=<path to the PEM private key>
>
> And if you have a hosted chef account, you can run them like this:
>
> mvn clean install -pl enterprise -Plive \
> -Dtest.enterprisechef.org=<you organization name> \
> -Dtest.enterprisechef.identity=<identity> \
> -Dtest.enterprisechef.credential=<path to the PEM private key>
>
>
>
> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>> Ignasi <ig...@gmail.com> writes:
>>
>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>> one, which seems to be the one with the right content. You can take a
>>> look at the commit [2] if you want to know the details.
>>>
>>> I've tried the api call that was failing to you, and now it works and
>>> the cookbook is properly deserialized. However, the live test to list
>>> all cookbooks is still failing due to a signature error (which is
>>> weird, because the signature is generated in a generic filter [3] that
>>> applies to all requests).
>>>
>>> I think it might be related to my Chef 11 installation. To make sure,
>>> could you update the branch and run the live tests, or try to list the
>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>
>>
>> I tested these methods after pulling latest from your branch.
>>
>> getCookbook()
>> listCookbooks()
>> listEnvironmentCookbooks()
>>
>> Everything seems to be working fine.
>>
>> How do I run live tests? Is this documented somewhere?
>>
>> Thanks and Regards
>> Noorul
>>
>>>
>>> Thanks!
>>>
>>>
>>> Ignasi
>>>
>>>
>>>
>>>
>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>
>>>
>>>
>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>> As soon as possible.
>>>>
>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>> backward compatible (just a massive package and artifactId renaming),
>>>> I sent a mail last week to let users express an opinion on the change
>>>> [2]. I plan to merge it this week, since there are no opinions
>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>
>>>> You could expect to have it merged by the end of the week, if I finish
>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>> fix this afternoon; I'll let you know when I have something usable so
>>>> you can test and validate it.
>>>>
>>>> Thanks for your help!
>>>>
>>>>
>>>> Ignasi
>>>>
>>>>
>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>
>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>
>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>> so you can have a look.
>>>>>>
>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>> to build a workaround in the meanwhile.
>>>>>>
>>>>>
>>>>> I would like to know, usually how long it takes something to get into
>>>>> upstream from the branch that you are working on?
>>>>>
>>>>> I can provide any help needed to make this working but need to be sure
>>>>> that I can use this in production as soon as possible.
>>>>>
>>>>> Thanks and Regards
>>>>> Noorul
>>>>>
>>>>>>
>>>>>>
>>>>>> [1] http://pastie.org/8270391
>>>>>>
>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>> Hi Noorul!
>>>>>>>
>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>> "listCookbooks" one.
>>>>>>>
>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>> fails to build the corresponding map.
>>>>>>>
>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>
>>>>>>> Ignasi
>>>>>>>
>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>> [2]
>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>
>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>
>>>>>>>>
>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>> try to get cookbook details using
>>>>>>>>
>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>
>>>>>>>>
>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>> at
>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>> at
>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>> at
>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>> at
>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>> at
>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>> at
>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>> at
>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>> at
>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>> at
>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>> at
>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>> at
>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>
>>>>>>>> README.md says that
>>>>>>>>
>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>> flavours.
>>>>>>>>
>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>
>>>>>>>> Thanks and Regards
>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Noorul Islam K M <no...@collab.net> writes:
> Ignasi <ig...@gmail.com> writes:
>
>> It is not intentional if there are some missing apis. We added the
>> environment API time ago [1], but not all endpoints were added then,
>> and are still missing.
>>
>> Mind opening an Improvement in JIRA to add the missing ones?
>>
>> Also, if you want to contribute them [2], that would be very welcome!
>> :) I plan to merge JCLOUDS-246 as it might block contributions (due to
>> the change of the project structure).
>>
>
> Created https://issues.apache.org/jira/browse/JCLOUDS-256
>
> But I am not able to assign it to myself. So I left a comment saying
> that I am already working on this.
>
Here is the initial commit on a branch.
https://github.com/noorul/jclouds-chef/commit/d97e14f0a445b0d2ae731ced212664b9ff3b2896
I will wait for you to merge live-test so that I can add a live test and
create a pull request.
Thanks and Regards
Noorul
> Thanks and Regards
> Noorul
>
>>
>> Ignasi
>>
>> [1] https://github.com/jclouds/legacy-jclouds-chef/pull/61
>> [2] https://wiki.apache.org/jclouds/How%20to%20Contribute
>>
>> On 28 August 2013 11:03, Noorul Islam K M <no...@collab.net> wrote:
>>> Ignasi <ig...@gmail.com> writes:
>>>
>>>> Hi Noorul,
>>>>
>>>> Finally, I've fixed the issue! :) Here is the commit [1] that fixes
>>>> it. Basically we don't have to encode the '+' and ' ' characters in
>>>> the Signature query parameter. With this fix applied, all live tests
>>>> are passing for Chef 0.10, Chef 11 and Enterprise Chef.
>>>>
>>>> Feel free to checkout my branch [2] and validate the live tests
>>>> (specially the testDownloadCookbooks).
>>>>
>>>
>>> Thanks a lot!
>>>
>>> I see that some of the APIs are not exposed, for example
>>>
>>> http://docs.opscode.com/api_chef_server_environments_recipe.html
>>>
>>> Is this something that consciously avoided?
>>>
>>> Thanks and Regards
>>> Noorul
>>>
>>>>
>>>> Ignasi
>>>>
>>>>
>>>> P.S. I'll submit the PR to have it merged in 1.7.0-SNAPSHOT as soon as
>>>> JCLOUDS-246 [2] gets merged.
>>>>
>>>>
>>>> [1] https://github.com/nacx/jclouds-chef/commit/67c1545aab3b742e9508f8bd3837b57de25c47e5
>>>> [2] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>> [3] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>
>>>> On 27 August 2013 13:18, Noorul Islam K M <no...@collab.net> wrote:
>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>
>>>>>> Hi Noorul,
>>>>>>
>>>>>> Just a follow up. I've been able to isolate (I think) the issue and
>>>>>> I've opened the corresponding JIRA [1]. I asked the chef-dev mailing
>>>>>> list [2] to see if someone can give us the details of how to properly
>>>>>> sign the URLs that are failing. It seems that some change may be
>>>>>> required in the signing filter [3].
>>>>>>
>>>>>> Let's see if we can get this fixed!
>>>>>>
>>>>>
>>>>> Thank you!
>>>>>
>>>>> Thanks and Regards
>>>>> Noorul
>>>>>
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-254
>>>>>> [2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html
>>>>>> [3] https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>>
>>>>>> On 26 August 2013 16:54, Ignasi <ig...@gmail.com> wrote:
>>>>>>> Ok, I just added a few cookbooks more to my Hosted Chef account and
>>>>>>> the download is failing the same way.
>>>>>>> I still haven't figured out what can be failing in the signature (it
>>>>>>> shouldn't have changed since Chef 0.10), so any help would be
>>>>>>> appreciated!
>>>>>>>
>>>>>>> On 26 August 2013 15:55, Ignasi <ig...@gmail.com> wrote:
>>>>>>>> Hi again,
>>>>>>>>
>>>>>>>> The problem appears in the "getResourceContents" api call. After
>>>>>>>> debugging for a while I haven't found an explanation. It fails
>>>>>>>> randomly with random resources, and there is no pattern for that :(
>>>>>>>> Sometimes it fails to download a root file resource, sometimes not.
>>>>>>>> Sometimes it fails downloading a recipe file, sometimes it just works.
>>>>>>>>
>>>>>>>> I executed the live test suite against a Chef 0.10.8, and against
>>>>>>>> Enterprise Chef and all tests are passing, so I assume it might be an
>>>>>>>> issue in Chef 11; otherwise I can not explain why the same resource
>>>>>>>> sometimes is downloaded and sometimes it is not. The authentication
>>>>>>>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise
>>>>>>>> Chef, it should work for Chef 11 too (or at least have some
>>>>>>>> deterministic behavior).
>>>>>>>>
>>>>>>>>
>>>>>>>> To properly reflect that in the live tests, I've separated the
>>>>>>>> "testListCookbooks" in two different tests: "testListCookbooks" and
>>>>>>>> "testDownloadCookbooks". If you update my branch, you should get those
>>>>>>>> changes.
>>>>>>>>
>>>>>>>> I'll search if there is any open issue and file a new one if appropriate.
>>>>>>>>
>>>>>>>> Anyway, let me know if you find something that could help getting rid of this!
>>>>>>>>
>>>>>>>>
>>>>>>>> Ignasi
>>>>>>>>
>>>>>>>>
>>>>>>>> On 26 August 2013 14:29, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>
>>>>>>>>>> Thanks a lot for your feedback Noorul!
>>>>>>>>>>
>>>>>>>>>> You can run the 'core' live tests as follows:
>>>>>>>>>>
>>>>>>>>>> mvn clean install -pl core -Plive \
>>>>>>>>>> -Dtest.chef.endpoint=<you chef server url> \
>>>>>>>>>> -Dtest.chef.identity=<identity> \
>>>>>>>>>> -Dtest.chef.credential=<path to the PEM private key>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I think, I get the same as you
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
>>>>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
>>>>>>>>>
>>>>>>>>> Results :
>>>>>>>>>
>>>>>>>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
>>>>>>>>> </Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
>>>>>>>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
>>>>>>>>> reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
>>>>>>>>>
>>>>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
>>>>>>>>>
>>>>>>>>> Thanks and Regards
>>>>>>>>> Noorul
>>>>>>>>>
>>>>>>>>>> And if you have a hosted chef account, you can run them like this:
>>>>>>>>>>
>>>>>>>>>> mvn clean install -pl enterprise -Plive \
>>>>>>>>>> -Dtest.enterprisechef.org=<you organization name> \
>>>>>>>>>> -Dtest.enterprisechef.identity=<identity> \
>>>>>>>>>> -Dtest.enterprisechef.credential=<path to the PEM private key>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>>>
>>>>>>>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>>>>>>>>>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>>>>>>>>>>> one, which seems to be the one with the right content. You can take a
>>>>>>>>>>>> look at the commit [2] if you want to know the details.
>>>>>>>>>>>>
>>>>>>>>>>>> I've tried the api call that was failing to you, and now it works and
>>>>>>>>>>>> the cookbook is properly deserialized. However, the live test to list
>>>>>>>>>>>> all cookbooks is still failing due to a signature error (which is
>>>>>>>>>>>> weird, because the signature is generated in a generic filter [3] that
>>>>>>>>>>>> applies to all requests).
>>>>>>>>>>>>
>>>>>>>>>>>> I think it might be related to my Chef 11 installation. To make sure,
>>>>>>>>>>>> could you update the branch and run the live tests, or try to list the
>>>>>>>>>>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I tested these methods after pulling latest from your branch.
>>>>>>>>>>>
>>>>>>>>>>> getCookbook()
>>>>>>>>>>> listCookbooks()
>>>>>>>>>>> listEnvironmentCookbooks()
>>>>>>>>>>>
>>>>>>>>>>> Everything seems to be working fine.
>>>>>>>>>>>
>>>>>>>>>>> How do I run live tests? Is this documented somewhere?
>>>>>>>>>>>
>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>> Noorul
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ignasi
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>>>>>>>>>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>>>>> As soon as possible.
>>>>>>>>>>>>>
>>>>>>>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>>>>>>>>>>> backward compatible (just a massive package and artifactId renaming),
>>>>>>>>>>>>> I sent a mail last week to let users express an opinion on the change
>>>>>>>>>>>>> [2]. I plan to merge it this week, since there are no opinions
>>>>>>>>>>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>>>>>>>>>>
>>>>>>>>>>>>> You could expect to have it merged by the end of the week, if I finish
>>>>>>>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>>>>>>>>>>> fix this afternoon; I'll let you know when I have something usable so
>>>>>>>>>>>>> you can test and validate it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for your help!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ignasi
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>>>>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>>>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>>>>>>>>>>> so you can have a look.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>>>>>>>>>>> to build a workaround in the meanwhile.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I would like to know, usually how long it takes something to get into
>>>>>>>>>>>>>> upstream from the branch that you are working on?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I can provide any help needed to make this working but need to be sure
>>>>>>>>>>>>>> that I can use this in production as soon as possible.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>>>> Noorul
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1] http://pastie.org/8270391
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>>>>>>>> Hi Noorul!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>>>>>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>>>>>>>>>>> "listCookbooks" one.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>>>>>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>>>>>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>>>>>>>>>>> fails to build the corresponding map.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>>>>>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>>>>>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>>>>>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>>>>>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ignasi
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>>>>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>>>>>>>>>>> try to get cookbook details using
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>>>>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> README.md says that
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>>>>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>>>>>>>>>>> flavours.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Ignasi <ig...@gmail.com> writes:
> It is not intentional if there are some missing apis. We added the
> environment API time ago [1], but not all endpoints were added then,
> and are still missing.
>
> Mind opening an Improvement in JIRA to add the missing ones?
>
> Also, if you want to contribute them [2], that would be very welcome!
> :) I plan to merge JCLOUDS-246 as it might block contributions (due to
> the change of the project structure).
>
Created https://issues.apache.org/jira/browse/JCLOUDS-256
But I am not able to assign it to myself. So I left a comment saying
that I am already working on this.
Thanks and Regards
Noorul
>
> Ignasi
>
> [1] https://github.com/jclouds/legacy-jclouds-chef/pull/61
> [2] https://wiki.apache.org/jclouds/How%20to%20Contribute
>
> On 28 August 2013 11:03, Noorul Islam K M <no...@collab.net> wrote:
>> Ignasi <ig...@gmail.com> writes:
>>
>>> Hi Noorul,
>>>
>>> Finally, I've fixed the issue! :) Here is the commit [1] that fixes
>>> it. Basically we don't have to encode the '+' and ' ' characters in
>>> the Signature query parameter. With this fix applied, all live tests
>>> are passing for Chef 0.10, Chef 11 and Enterprise Chef.
>>>
>>> Feel free to checkout my branch [2] and validate the live tests
>>> (specially the testDownloadCookbooks).
>>>
>>
>> Thanks a lot!
>>
>> I see that some of the APIs are not exposed, for example
>>
>> http://docs.opscode.com/api_chef_server_environments_recipe.html
>>
>> Is this something that consciously avoided?
>>
>> Thanks and Regards
>> Noorul
>>
>>>
>>> Ignasi
>>>
>>>
>>> P.S. I'll submit the PR to have it merged in 1.7.0-SNAPSHOT as soon as
>>> JCLOUDS-246 [2] gets merged.
>>>
>>>
>>> [1] https://github.com/nacx/jclouds-chef/commit/67c1545aab3b742e9508f8bd3837b57de25c47e5
>>> [2] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>> [3] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>
>>> On 27 August 2013 13:18, Noorul Islam K M <no...@collab.net> wrote:
>>>> Ignasi <ig...@gmail.com> writes:
>>>>
>>>>> Hi Noorul,
>>>>>
>>>>> Just a follow up. I've been able to isolate (I think) the issue and
>>>>> I've opened the corresponding JIRA [1]. I asked the chef-dev mailing
>>>>> list [2] to see if someone can give us the details of how to properly
>>>>> sign the URLs that are failing. It seems that some change may be
>>>>> required in the signing filter [3].
>>>>>
>>>>> Let's see if we can get this fixed!
>>>>>
>>>>
>>>> Thank you!
>>>>
>>>> Thanks and Regards
>>>> Noorul
>>>>
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-254
>>>>> [2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html
>>>>> [3] https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>
>>>>> On 26 August 2013 16:54, Ignasi <ig...@gmail.com> wrote:
>>>>>> Ok, I just added a few cookbooks more to my Hosted Chef account and
>>>>>> the download is failing the same way.
>>>>>> I still haven't figured out what can be failing in the signature (it
>>>>>> shouldn't have changed since Chef 0.10), so any help would be
>>>>>> appreciated!
>>>>>>
>>>>>> On 26 August 2013 15:55, Ignasi <ig...@gmail.com> wrote:
>>>>>>> Hi again,
>>>>>>>
>>>>>>> The problem appears in the "getResourceContents" api call. After
>>>>>>> debugging for a while I haven't found an explanation. It fails
>>>>>>> randomly with random resources, and there is no pattern for that :(
>>>>>>> Sometimes it fails to download a root file resource, sometimes not.
>>>>>>> Sometimes it fails downloading a recipe file, sometimes it just works.
>>>>>>>
>>>>>>> I executed the live test suite against a Chef 0.10.8, and against
>>>>>>> Enterprise Chef and all tests are passing, so I assume it might be an
>>>>>>> issue in Chef 11; otherwise I can not explain why the same resource
>>>>>>> sometimes is downloaded and sometimes it is not. The authentication
>>>>>>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise
>>>>>>> Chef, it should work for Chef 11 too (or at least have some
>>>>>>> deterministic behavior).
>>>>>>>
>>>>>>>
>>>>>>> To properly reflect that in the live tests, I've separated the
>>>>>>> "testListCookbooks" in two different tests: "testListCookbooks" and
>>>>>>> "testDownloadCookbooks". If you update my branch, you should get those
>>>>>>> changes.
>>>>>>>
>>>>>>> I'll search if there is any open issue and file a new one if appropriate.
>>>>>>>
>>>>>>> Anyway, let me know if you find something that could help getting rid of this!
>>>>>>>
>>>>>>>
>>>>>>> Ignasi
>>>>>>>
>>>>>>>
>>>>>>> On 26 August 2013 14:29, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>
>>>>>>>>> Thanks a lot for your feedback Noorul!
>>>>>>>>>
>>>>>>>>> You can run the 'core' live tests as follows:
>>>>>>>>>
>>>>>>>>> mvn clean install -pl core -Plive \
>>>>>>>>> -Dtest.chef.endpoint=<you chef server url> \
>>>>>>>>> -Dtest.chef.identity=<identity> \
>>>>>>>>> -Dtest.chef.credential=<path to the PEM private key>
>>>>>>>>>
>>>>>>>>
>>>>>>>> I think, I get the same as you
>>>>>>>>
>>>>>>>>
>>>>>>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
>>>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
>>>>>>>>
>>>>>>>> Results :
>>>>>>>>
>>>>>>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
>>>>>>>> </Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
>>>>>>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
>>>>>>>> reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
>>>>>>>>
>>>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
>>>>>>>>
>>>>>>>> Thanks and Regards
>>>>>>>> Noorul
>>>>>>>>
>>>>>>>>> And if you have a hosted chef account, you can run them like this:
>>>>>>>>>
>>>>>>>>> mvn clean install -pl enterprise -Plive \
>>>>>>>>> -Dtest.enterprisechef.org=<you organization name> \
>>>>>>>>> -Dtest.enterprisechef.identity=<identity> \
>>>>>>>>> -Dtest.enterprisechef.credential=<path to the PEM private key>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>>
>>>>>>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>>>>>>>>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>>>>>>>>>> one, which seems to be the one with the right content. You can take a
>>>>>>>>>>> look at the commit [2] if you want to know the details.
>>>>>>>>>>>
>>>>>>>>>>> I've tried the api call that was failing to you, and now it works and
>>>>>>>>>>> the cookbook is properly deserialized. However, the live test to list
>>>>>>>>>>> all cookbooks is still failing due to a signature error (which is
>>>>>>>>>>> weird, because the signature is generated in a generic filter [3] that
>>>>>>>>>>> applies to all requests).
>>>>>>>>>>>
>>>>>>>>>>> I think it might be related to my Chef 11 installation. To make sure,
>>>>>>>>>>> could you update the branch and run the live tests, or try to list the
>>>>>>>>>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I tested these methods after pulling latest from your branch.
>>>>>>>>>>
>>>>>>>>>> getCookbook()
>>>>>>>>>> listCookbooks()
>>>>>>>>>> listEnvironmentCookbooks()
>>>>>>>>>>
>>>>>>>>>> Everything seems to be working fine.
>>>>>>>>>>
>>>>>>>>>> How do I run live tests? Is this documented somewhere?
>>>>>>>>>>
>>>>>>>>>> Thanks and Regards
>>>>>>>>>> Noorul
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Ignasi
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>>>>>>>>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>>>> As soon as possible.
>>>>>>>>>>>>
>>>>>>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>>>>>>>>>> backward compatible (just a massive package and artifactId renaming),
>>>>>>>>>>>> I sent a mail last week to let users express an opinion on the change
>>>>>>>>>>>> [2]. I plan to merge it this week, since there are no opinions
>>>>>>>>>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>>>>>>>>>
>>>>>>>>>>>> You could expect to have it merged by the end of the week, if I finish
>>>>>>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>>>>>>>>>> fix this afternoon; I'll let you know when I have something usable so
>>>>>>>>>>>> you can test and validate it.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for your help!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ignasi
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>>>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>>>>>>>>>
>>>>>>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>>>>>>>>>> so you can have a look.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>>>>>>>>>> to build a workaround in the meanwhile.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would like to know, usually how long it takes something to get into
>>>>>>>>>>>>> upstream from the branch that you are working on?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I can provide any help needed to make this working but need to be sure
>>>>>>>>>>>>> that I can use this in production as soon as possible.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>>> Noorul
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1] http://pastie.org/8270391
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>>>>>>> Hi Noorul!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>>>>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>>>>>>>>>> "listCookbooks" one.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>>>>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>>>>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>>>>>>>>>> fails to build the corresponding map.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>>>>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>>>>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>>>>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>>>>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ignasi
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>>>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>>>>>>>>>> try to get cookbook details using
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>>>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> README.md says that
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>>>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>>>>>>>>>> flavours.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
It is not intentional if there are some missing apis. We added the
environment API time ago [1], but not all endpoints were added then,
and are still missing.
Mind opening an Improvement in JIRA to add the missing ones?
Also, if you want to contribute them [2], that would be very welcome!
:) I plan to merge JCLOUDS-246 as it might block contributions (due to
the change of the project structure).
Ignasi
[1] https://github.com/jclouds/legacy-jclouds-chef/pull/61
[2] https://wiki.apache.org/jclouds/How%20to%20Contribute
On 28 August 2013 11:03, Noorul Islam K M <no...@collab.net> wrote:
> Ignasi <ig...@gmail.com> writes:
>
>> Hi Noorul,
>>
>> Finally, I've fixed the issue! :) Here is the commit [1] that fixes
>> it. Basically we don't have to encode the '+' and ' ' characters in
>> the Signature query parameter. With this fix applied, all live tests
>> are passing for Chef 0.10, Chef 11 and Enterprise Chef.
>>
>> Feel free to checkout my branch [2] and validate the live tests
>> (specially the testDownloadCookbooks).
>>
>
> Thanks a lot!
>
> I see that some of the APIs are not exposed, for example
>
> http://docs.opscode.com/api_chef_server_environments_recipe.html
>
> Is this something that consciously avoided?
>
> Thanks and Regards
> Noorul
>
>>
>> Ignasi
>>
>>
>> P.S. I'll submit the PR to have it merged in 1.7.0-SNAPSHOT as soon as
>> JCLOUDS-246 [2] gets merged.
>>
>>
>> [1] https://github.com/nacx/jclouds-chef/commit/67c1545aab3b742e9508f8bd3837b57de25c47e5
>> [2] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>> [3] https://issues.apache.org/jira/browse/JCLOUDS-246
>>
>> On 27 August 2013 13:18, Noorul Islam K M <no...@collab.net> wrote:
>>> Ignasi <ig...@gmail.com> writes:
>>>
>>>> Hi Noorul,
>>>>
>>>> Just a follow up. I've been able to isolate (I think) the issue and
>>>> I've opened the corresponding JIRA [1]. I asked the chef-dev mailing
>>>> list [2] to see if someone can give us the details of how to properly
>>>> sign the URLs that are failing. It seems that some change may be
>>>> required in the signing filter [3].
>>>>
>>>> Let's see if we can get this fixed!
>>>>
>>>
>>> Thank you!
>>>
>>> Thanks and Regards
>>> Noorul
>>>
>>>>
>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-254
>>>> [2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html
>>>> [3] https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>
>>>> On 26 August 2013 16:54, Ignasi <ig...@gmail.com> wrote:
>>>>> Ok, I just added a few cookbooks more to my Hosted Chef account and
>>>>> the download is failing the same way.
>>>>> I still haven't figured out what can be failing in the signature (it
>>>>> shouldn't have changed since Chef 0.10), so any help would be
>>>>> appreciated!
>>>>>
>>>>> On 26 August 2013 15:55, Ignasi <ig...@gmail.com> wrote:
>>>>>> Hi again,
>>>>>>
>>>>>> The problem appears in the "getResourceContents" api call. After
>>>>>> debugging for a while I haven't found an explanation. It fails
>>>>>> randomly with random resources, and there is no pattern for that :(
>>>>>> Sometimes it fails to download a root file resource, sometimes not.
>>>>>> Sometimes it fails downloading a recipe file, sometimes it just works.
>>>>>>
>>>>>> I executed the live test suite against a Chef 0.10.8, and against
>>>>>> Enterprise Chef and all tests are passing, so I assume it might be an
>>>>>> issue in Chef 11; otherwise I can not explain why the same resource
>>>>>> sometimes is downloaded and sometimes it is not. The authentication
>>>>>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise
>>>>>> Chef, it should work for Chef 11 too (or at least have some
>>>>>> deterministic behavior).
>>>>>>
>>>>>>
>>>>>> To properly reflect that in the live tests, I've separated the
>>>>>> "testListCookbooks" in two different tests: "testListCookbooks" and
>>>>>> "testDownloadCookbooks". If you update my branch, you should get those
>>>>>> changes.
>>>>>>
>>>>>> I'll search if there is any open issue and file a new one if appropriate.
>>>>>>
>>>>>> Anyway, let me know if you find something that could help getting rid of this!
>>>>>>
>>>>>>
>>>>>> Ignasi
>>>>>>
>>>>>>
>>>>>> On 26 August 2013 14:29, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>
>>>>>>>> Thanks a lot for your feedback Noorul!
>>>>>>>>
>>>>>>>> You can run the 'core' live tests as follows:
>>>>>>>>
>>>>>>>> mvn clean install -pl core -Plive \
>>>>>>>> -Dtest.chef.endpoint=<you chef server url> \
>>>>>>>> -Dtest.chef.identity=<identity> \
>>>>>>>> -Dtest.chef.credential=<path to the PEM private key>
>>>>>>>>
>>>>>>>
>>>>>>> I think, I get the same as you
>>>>>>>
>>>>>>>
>>>>>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
>>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
>>>>>>>
>>>>>>> Results :
>>>>>>>
>>>>>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
>>>>>>> </Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
>>>>>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
>>>>>>> reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
>>>>>>>
>>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
>>>>>>>
>>>>>>> Thanks and Regards
>>>>>>> Noorul
>>>>>>>
>>>>>>>> And if you have a hosted chef account, you can run them like this:
>>>>>>>>
>>>>>>>> mvn clean install -pl enterprise -Plive \
>>>>>>>> -Dtest.enterprisechef.org=<you organization name> \
>>>>>>>> -Dtest.enterprisechef.identity=<identity> \
>>>>>>>> -Dtest.enterprisechef.credential=<path to the PEM private key>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>
>>>>>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>>>>>>>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>>>>>>>>> one, which seems to be the one with the right content. You can take a
>>>>>>>>>> look at the commit [2] if you want to know the details.
>>>>>>>>>>
>>>>>>>>>> I've tried the api call that was failing to you, and now it works and
>>>>>>>>>> the cookbook is properly deserialized. However, the live test to list
>>>>>>>>>> all cookbooks is still failing due to a signature error (which is
>>>>>>>>>> weird, because the signature is generated in a generic filter [3] that
>>>>>>>>>> applies to all requests).
>>>>>>>>>>
>>>>>>>>>> I think it might be related to my Chef 11 installation. To make sure,
>>>>>>>>>> could you update the branch and run the live tests, or try to list the
>>>>>>>>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I tested these methods after pulling latest from your branch.
>>>>>>>>>
>>>>>>>>> getCookbook()
>>>>>>>>> listCookbooks()
>>>>>>>>> listEnvironmentCookbooks()
>>>>>>>>>
>>>>>>>>> Everything seems to be working fine.
>>>>>>>>>
>>>>>>>>> How do I run live tests? Is this documented somewhere?
>>>>>>>>>
>>>>>>>>> Thanks and Regards
>>>>>>>>> Noorul
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ignasi
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>>>>>>>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>>> As soon as possible.
>>>>>>>>>>>
>>>>>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>>>>>>>>> backward compatible (just a massive package and artifactId renaming),
>>>>>>>>>>> I sent a mail last week to let users express an opinion on the change
>>>>>>>>>>> [2]. I plan to merge it this week, since there are no opinions
>>>>>>>>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>>>>>>>>
>>>>>>>>>>> You could expect to have it merged by the end of the week, if I finish
>>>>>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>>>>>>>>> fix this afternoon; I'll let you know when I have something usable so
>>>>>>>>>>> you can test and validate it.
>>>>>>>>>>>
>>>>>>>>>>> Thanks for your help!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Ignasi
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>>>>>>>>
>>>>>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>>>>
>>>>>>>>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>>>>>>>>> so you can have a look.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>>>>>>>>> to build a workaround in the meanwhile.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I would like to know, usually how long it takes something to get into
>>>>>>>>>>>> upstream from the branch that you are working on?
>>>>>>>>>>>>
>>>>>>>>>>>> I can provide any help needed to make this working but need to be sure
>>>>>>>>>>>> that I can use this in production as soon as possible.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>> Noorul
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] http://pastie.org/8270391
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>>>>>> Hi Noorul!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>>>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>>>>>>>>> "listCookbooks" one.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>>>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>>>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>>>>>>>>> fails to build the corresponding map.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>>>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>>>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>>>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>>>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ignasi
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>>>>>>>>> try to get cookbook details using
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> README.md says that
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>>>>>>>>> flavours.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Ignasi <ig...@gmail.com> writes:
> Hi Noorul,
>
> Finally, I've fixed the issue! :) Here is the commit [1] that fixes
> it. Basically we don't have to encode the '+' and ' ' characters in
> the Signature query parameter. With this fix applied, all live tests
> are passing for Chef 0.10, Chef 11 and Enterprise Chef.
>
> Feel free to checkout my branch [2] and validate the live tests
> (specially the testDownloadCookbooks).
>
Thanks a lot!
I see that some of the APIs are not exposed, for example
http://docs.opscode.com/api_chef_server_environments_recipe.html
Is this something that consciously avoided?
Thanks and Regards
Noorul
>
> Ignasi
>
>
> P.S. I'll submit the PR to have it merged in 1.7.0-SNAPSHOT as soon as
> JCLOUDS-246 [2] gets merged.
>
>
> [1] https://github.com/nacx/jclouds-chef/commit/67c1545aab3b742e9508f8bd3837b57de25c47e5
> [2] https://github.com/nacx/jclouds-chef/tree/245-live-tests
> [3] https://issues.apache.org/jira/browse/JCLOUDS-246
>
> On 27 August 2013 13:18, Noorul Islam K M <no...@collab.net> wrote:
>> Ignasi <ig...@gmail.com> writes:
>>
>>> Hi Noorul,
>>>
>>> Just a follow up. I've been able to isolate (I think) the issue and
>>> I've opened the corresponding JIRA [1]. I asked the chef-dev mailing
>>> list [2] to see if someone can give us the details of how to properly
>>> sign the URLs that are failing. It seems that some change may be
>>> required in the signing filter [3].
>>>
>>> Let's see if we can get this fixed!
>>>
>>
>> Thank you!
>>
>> Thanks and Regards
>> Noorul
>>
>>>
>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-254
>>> [2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html
>>> [3] https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>
>>> On 26 August 2013 16:54, Ignasi <ig...@gmail.com> wrote:
>>>> Ok, I just added a few cookbooks more to my Hosted Chef account and
>>>> the download is failing the same way.
>>>> I still haven't figured out what can be failing in the signature (it
>>>> shouldn't have changed since Chef 0.10), so any help would be
>>>> appreciated!
>>>>
>>>> On 26 August 2013 15:55, Ignasi <ig...@gmail.com> wrote:
>>>>> Hi again,
>>>>>
>>>>> The problem appears in the "getResourceContents" api call. After
>>>>> debugging for a while I haven't found an explanation. It fails
>>>>> randomly with random resources, and there is no pattern for that :(
>>>>> Sometimes it fails to download a root file resource, sometimes not.
>>>>> Sometimes it fails downloading a recipe file, sometimes it just works.
>>>>>
>>>>> I executed the live test suite against a Chef 0.10.8, and against
>>>>> Enterprise Chef and all tests are passing, so I assume it might be an
>>>>> issue in Chef 11; otherwise I can not explain why the same resource
>>>>> sometimes is downloaded and sometimes it is not. The authentication
>>>>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise
>>>>> Chef, it should work for Chef 11 too (or at least have some
>>>>> deterministic behavior).
>>>>>
>>>>>
>>>>> To properly reflect that in the live tests, I've separated the
>>>>> "testListCookbooks" in two different tests: "testListCookbooks" and
>>>>> "testDownloadCookbooks". If you update my branch, you should get those
>>>>> changes.
>>>>>
>>>>> I'll search if there is any open issue and file a new one if appropriate.
>>>>>
>>>>> Anyway, let me know if you find something that could help getting rid of this!
>>>>>
>>>>>
>>>>> Ignasi
>>>>>
>>>>>
>>>>> On 26 August 2013 14:29, Noorul Islam K M <no...@collab.net> wrote:
>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>
>>>>>>> Thanks a lot for your feedback Noorul!
>>>>>>>
>>>>>>> You can run the 'core' live tests as follows:
>>>>>>>
>>>>>>> mvn clean install -pl core -Plive \
>>>>>>> -Dtest.chef.endpoint=<you chef server url> \
>>>>>>> -Dtest.chef.identity=<identity> \
>>>>>>> -Dtest.chef.credential=<path to the PEM private key>
>>>>>>>
>>>>>>
>>>>>> I think, I get the same as you
>>>>>>
>>>>>>
>>>>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
>>>>>>
>>>>>> Results :
>>>>>>
>>>>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
>>>>>> </Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
>>>>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
>>>>>> reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
>>>>>>
>>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
>>>>>>
>>>>>> Thanks and Regards
>>>>>> Noorul
>>>>>>
>>>>>>> And if you have a hosted chef account, you can run them like this:
>>>>>>>
>>>>>>> mvn clean install -pl enterprise -Plive \
>>>>>>> -Dtest.enterprisechef.org=<you organization name> \
>>>>>>> -Dtest.enterprisechef.identity=<identity> \
>>>>>>> -Dtest.enterprisechef.credential=<path to the PEM private key>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>
>>>>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>>>>>>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>>>>>>>> one, which seems to be the one with the right content. You can take a
>>>>>>>>> look at the commit [2] if you want to know the details.
>>>>>>>>>
>>>>>>>>> I've tried the api call that was failing to you, and now it works and
>>>>>>>>> the cookbook is properly deserialized. However, the live test to list
>>>>>>>>> all cookbooks is still failing due to a signature error (which is
>>>>>>>>> weird, because the signature is generated in a generic filter [3] that
>>>>>>>>> applies to all requests).
>>>>>>>>>
>>>>>>>>> I think it might be related to my Chef 11 installation. To make sure,
>>>>>>>>> could you update the branch and run the live tests, or try to list the
>>>>>>>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>>>>>>>
>>>>>>>>
>>>>>>>> I tested these methods after pulling latest from your branch.
>>>>>>>>
>>>>>>>> getCookbook()
>>>>>>>> listCookbooks()
>>>>>>>> listEnvironmentCookbooks()
>>>>>>>>
>>>>>>>> Everything seems to be working fine.
>>>>>>>>
>>>>>>>> How do I run live tests? Is this documented somewhere?
>>>>>>>>
>>>>>>>> Thanks and Regards
>>>>>>>> Noorul
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ignasi
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>>>>>>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>> As soon as possible.
>>>>>>>>>>
>>>>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>>>>>>>> backward compatible (just a massive package and artifactId renaming),
>>>>>>>>>> I sent a mail last week to let users express an opinion on the change
>>>>>>>>>> [2]. I plan to merge it this week, since there are no opinions
>>>>>>>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>>>>>>>
>>>>>>>>>> You could expect to have it merged by the end of the week, if I finish
>>>>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>>>>>>>> fix this afternoon; I'll let you know when I have something usable so
>>>>>>>>>> you can test and validate it.
>>>>>>>>>>
>>>>>>>>>> Thanks for your help!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ignasi
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>>>>>>>
>>>>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>>>
>>>>>>>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>>>>>>>> so you can have a look.
>>>>>>>>>>>>
>>>>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>>>>>>>> to build a workaround in the meanwhile.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I would like to know, usually how long it takes something to get into
>>>>>>>>>>> upstream from the branch that you are working on?
>>>>>>>>>>>
>>>>>>>>>>> I can provide any help needed to make this working but need to be sure
>>>>>>>>>>> that I can use this in production as soon as possible.
>>>>>>>>>>>
>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>> Noorul
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [1] http://pastie.org/8270391
>>>>>>>>>>>>
>>>>>>>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>>>>> Hi Noorul!
>>>>>>>>>>>>>
>>>>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>>>>>>>> "listCookbooks" one.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>>>>>>>> fails to build the corresponding map.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ignasi
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>>>>> [2]
>>>>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>>>>>>>
>>>>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>>>>>>>> try to get cookbook details using
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>>>>>>>> at
>>>>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> README.md says that
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>>>>>>>> flavours.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
Hi Noorul,
Finally, I've fixed the issue! :) Here is the commit [1] that fixes
it. Basically we don't have to encode the '+' and ' ' characters in
the Signature query parameter. With this fix applied, all live tests
are passing for Chef 0.10, Chef 11 and Enterprise Chef.
Feel free to checkout my branch [2] and validate the live tests
(specially the testDownloadCookbooks).
Ignasi
P.S. I'll submit the PR to have it merged in 1.7.0-SNAPSHOT as soon as
JCLOUDS-246 [2] gets merged.
[1] https://github.com/nacx/jclouds-chef/commit/67c1545aab3b742e9508f8bd3837b57de25c47e5
[2] https://github.com/nacx/jclouds-chef/tree/245-live-tests
[3] https://issues.apache.org/jira/browse/JCLOUDS-246
On 27 August 2013 13:18, Noorul Islam K M <no...@collab.net> wrote:
> Ignasi <ig...@gmail.com> writes:
>
>> Hi Noorul,
>>
>> Just a follow up. I've been able to isolate (I think) the issue and
>> I've opened the corresponding JIRA [1]. I asked the chef-dev mailing
>> list [2] to see if someone can give us the details of how to properly
>> sign the URLs that are failing. It seems that some change may be
>> required in the signing filter [3].
>>
>> Let's see if we can get this fixed!
>>
>
> Thank you!
>
> Thanks and Regards
> Noorul
>
>>
>> [1] https://issues.apache.org/jira/browse/JCLOUDS-254
>> [2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html
>> [3] https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>
>> On 26 August 2013 16:54, Ignasi <ig...@gmail.com> wrote:
>>> Ok, I just added a few cookbooks more to my Hosted Chef account and
>>> the download is failing the same way.
>>> I still haven't figured out what can be failing in the signature (it
>>> shouldn't have changed since Chef 0.10), so any help would be
>>> appreciated!
>>>
>>> On 26 August 2013 15:55, Ignasi <ig...@gmail.com> wrote:
>>>> Hi again,
>>>>
>>>> The problem appears in the "getResourceContents" api call. After
>>>> debugging for a while I haven't found an explanation. It fails
>>>> randomly with random resources, and there is no pattern for that :(
>>>> Sometimes it fails to download a root file resource, sometimes not.
>>>> Sometimes it fails downloading a recipe file, sometimes it just works.
>>>>
>>>> I executed the live test suite against a Chef 0.10.8, and against
>>>> Enterprise Chef and all tests are passing, so I assume it might be an
>>>> issue in Chef 11; otherwise I can not explain why the same resource
>>>> sometimes is downloaded and sometimes it is not. The authentication
>>>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise
>>>> Chef, it should work for Chef 11 too (or at least have some
>>>> deterministic behavior).
>>>>
>>>>
>>>> To properly reflect that in the live tests, I've separated the
>>>> "testListCookbooks" in two different tests: "testListCookbooks" and
>>>> "testDownloadCookbooks". If you update my branch, you should get those
>>>> changes.
>>>>
>>>> I'll search if there is any open issue and file a new one if appropriate.
>>>>
>>>> Anyway, let me know if you find something that could help getting rid of this!
>>>>
>>>>
>>>> Ignasi
>>>>
>>>>
>>>> On 26 August 2013 14:29, Noorul Islam K M <no...@collab.net> wrote:
>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>
>>>>>> Thanks a lot for your feedback Noorul!
>>>>>>
>>>>>> You can run the 'core' live tests as follows:
>>>>>>
>>>>>> mvn clean install -pl core -Plive \
>>>>>> -Dtest.chef.endpoint=<you chef server url> \
>>>>>> -Dtest.chef.identity=<identity> \
>>>>>> -Dtest.chef.credential=<path to the PEM private key>
>>>>>>
>>>>>
>>>>> I think, I get the same as you
>>>>>
>>>>>
>>>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
>>>>>
>>>>> Results :
>>>>>
>>>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
>>>>> </Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
>>>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
>>>>> reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
>>>>>
>>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
>>>>>
>>>>> Thanks and Regards
>>>>> Noorul
>>>>>
>>>>>> And if you have a hosted chef account, you can run them like this:
>>>>>>
>>>>>> mvn clean install -pl enterprise -Plive \
>>>>>> -Dtest.enterprisechef.org=<you organization name> \
>>>>>> -Dtest.enterprisechef.identity=<identity> \
>>>>>> -Dtest.enterprisechef.credential=<path to the PEM private key>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>
>>>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>>>>>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>>>>>>> one, which seems to be the one with the right content. You can take a
>>>>>>>> look at the commit [2] if you want to know the details.
>>>>>>>>
>>>>>>>> I've tried the api call that was failing to you, and now it works and
>>>>>>>> the cookbook is properly deserialized. However, the live test to list
>>>>>>>> all cookbooks is still failing due to a signature error (which is
>>>>>>>> weird, because the signature is generated in a generic filter [3] that
>>>>>>>> applies to all requests).
>>>>>>>>
>>>>>>>> I think it might be related to my Chef 11 installation. To make sure,
>>>>>>>> could you update the branch and run the live tests, or try to list the
>>>>>>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>>>>>>
>>>>>>>
>>>>>>> I tested these methods after pulling latest from your branch.
>>>>>>>
>>>>>>> getCookbook()
>>>>>>> listCookbooks()
>>>>>>> listEnvironmentCookbooks()
>>>>>>>
>>>>>>> Everything seems to be working fine.
>>>>>>>
>>>>>>> How do I run live tests? Is this documented somewhere?
>>>>>>>
>>>>>>> Thanks and Regards
>>>>>>> Noorul
>>>>>>>
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>>
>>>>>>>> Ignasi
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>>>>>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>> As soon as possible.
>>>>>>>>>
>>>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>>>>>>> backward compatible (just a massive package and artifactId renaming),
>>>>>>>>> I sent a mail last week to let users express an opinion on the change
>>>>>>>>> [2]. I plan to merge it this week, since there are no opinions
>>>>>>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>>>>>>
>>>>>>>>> You could expect to have it merged by the end of the week, if I finish
>>>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>>>>>>> fix this afternoon; I'll let you know when I have something usable so
>>>>>>>>> you can test and validate it.
>>>>>>>>>
>>>>>>>>> Thanks for your help!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ignasi
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>>>>>>
>>>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>>
>>>>>>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>>>>>>> so you can have a look.
>>>>>>>>>>>
>>>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>>>>>>> to build a workaround in the meanwhile.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I would like to know, usually how long it takes something to get into
>>>>>>>>>> upstream from the branch that you are working on?
>>>>>>>>>>
>>>>>>>>>> I can provide any help needed to make this working but need to be sure
>>>>>>>>>> that I can use this in production as soon as possible.
>>>>>>>>>>
>>>>>>>>>> Thanks and Regards
>>>>>>>>>> Noorul
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [1] http://pastie.org/8270391
>>>>>>>>>>>
>>>>>>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>>>> Hi Noorul!
>>>>>>>>>>>>
>>>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>>>>>>> "listCookbooks" one.
>>>>>>>>>>>>
>>>>>>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>>>>>>> fails to build the corresponding map.
>>>>>>>>>>>>
>>>>>>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>>>>>>
>>>>>>>>>>>> Ignasi
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>>>> [2]
>>>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>>>>>>
>>>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>>>>>>> try to get cookbook details using
>>>>>>>>>>>>>
>>>>>>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>>>>>>
>>>>>>>>>>>>> README.md says that
>>>>>>>>>>>>>
>>>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>>>>>>> flavours.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Ignasi <ig...@gmail.com> writes:
> Hi Noorul,
>
> Just a follow up. I've been able to isolate (I think) the issue and
> I've opened the corresponding JIRA [1]. I asked the chef-dev mailing
> list [2] to see if someone can give us the details of how to properly
> sign the URLs that are failing. It seems that some change may be
> required in the signing filter [3].
>
> Let's see if we can get this fixed!
>
Thank you!
Thanks and Regards
Noorul
>
> [1] https://issues.apache.org/jira/browse/JCLOUDS-254
> [2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html
> [3] https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>
> On 26 August 2013 16:54, Ignasi <ig...@gmail.com> wrote:
>> Ok, I just added a few cookbooks more to my Hosted Chef account and
>> the download is failing the same way.
>> I still haven't figured out what can be failing in the signature (it
>> shouldn't have changed since Chef 0.10), so any help would be
>> appreciated!
>>
>> On 26 August 2013 15:55, Ignasi <ig...@gmail.com> wrote:
>>> Hi again,
>>>
>>> The problem appears in the "getResourceContents" api call. After
>>> debugging for a while I haven't found an explanation. It fails
>>> randomly with random resources, and there is no pattern for that :(
>>> Sometimes it fails to download a root file resource, sometimes not.
>>> Sometimes it fails downloading a recipe file, sometimes it just works.
>>>
>>> I executed the live test suite against a Chef 0.10.8, and against
>>> Enterprise Chef and all tests are passing, so I assume it might be an
>>> issue in Chef 11; otherwise I can not explain why the same resource
>>> sometimes is downloaded and sometimes it is not. The authentication
>>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise
>>> Chef, it should work for Chef 11 too (or at least have some
>>> deterministic behavior).
>>>
>>>
>>> To properly reflect that in the live tests, I've separated the
>>> "testListCookbooks" in two different tests: "testListCookbooks" and
>>> "testDownloadCookbooks". If you update my branch, you should get those
>>> changes.
>>>
>>> I'll search if there is any open issue and file a new one if appropriate.
>>>
>>> Anyway, let me know if you find something that could help getting rid of this!
>>>
>>>
>>> Ignasi
>>>
>>>
>>> On 26 August 2013 14:29, Noorul Islam K M <no...@collab.net> wrote:
>>>> Ignasi <ig...@gmail.com> writes:
>>>>
>>>>> Thanks a lot for your feedback Noorul!
>>>>>
>>>>> You can run the 'core' live tests as follows:
>>>>>
>>>>> mvn clean install -pl core -Plive \
>>>>> -Dtest.chef.endpoint=<you chef server url> \
>>>>> -Dtest.chef.identity=<identity> \
>>>>> -Dtest.chef.credential=<path to the PEM private key>
>>>>>
>>>>
>>>> I think, I get the same as you
>>>>
>>>>
>>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
>>>>
>>>> Results :
>>>>
>>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
>>>> </Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
>>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
>>>> reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
>>>>
>>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
>>>>
>>>> Thanks and Regards
>>>> Noorul
>>>>
>>>>> And if you have a hosted chef account, you can run them like this:
>>>>>
>>>>> mvn clean install -pl enterprise -Plive \
>>>>> -Dtest.enterprisechef.org=<you organization name> \
>>>>> -Dtest.enterprisechef.identity=<identity> \
>>>>> -Dtest.enterprisechef.credential=<path to the PEM private key>
>>>>>
>>>>>
>>>>>
>>>>> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>
>>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>>>>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>>>>>> one, which seems to be the one with the right content. You can take a
>>>>>>> look at the commit [2] if you want to know the details.
>>>>>>>
>>>>>>> I've tried the api call that was failing to you, and now it works and
>>>>>>> the cookbook is properly deserialized. However, the live test to list
>>>>>>> all cookbooks is still failing due to a signature error (which is
>>>>>>> weird, because the signature is generated in a generic filter [3] that
>>>>>>> applies to all requests).
>>>>>>>
>>>>>>> I think it might be related to my Chef 11 installation. To make sure,
>>>>>>> could you update the branch and run the live tests, or try to list the
>>>>>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>>>>>
>>>>>>
>>>>>> I tested these methods after pulling latest from your branch.
>>>>>>
>>>>>> getCookbook()
>>>>>> listCookbooks()
>>>>>> listEnvironmentCookbooks()
>>>>>>
>>>>>> Everything seems to be working fine.
>>>>>>
>>>>>> How do I run live tests? Is this documented somewhere?
>>>>>>
>>>>>> Thanks and Regards
>>>>>> Noorul
>>>>>>
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>>
>>>>>>> Ignasi
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>>>>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>>>>>> As soon as possible.
>>>>>>>>
>>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>>>>>> backward compatible (just a massive package and artifactId renaming),
>>>>>>>> I sent a mail last week to let users express an opinion on the change
>>>>>>>> [2]. I plan to merge it this week, since there are no opinions
>>>>>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>>>>>
>>>>>>>> You could expect to have it merged by the end of the week, if I finish
>>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>>>>>> fix this afternoon; I'll let you know when I have something usable so
>>>>>>>> you can test and validate it.
>>>>>>>>
>>>>>>>> Thanks for your help!
>>>>>>>>
>>>>>>>>
>>>>>>>> Ignasi
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>>>>>
>>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>>
>>>>>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>>>>>> so you can have a look.
>>>>>>>>>>
>>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>>>>>> to build a workaround in the meanwhile.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I would like to know, usually how long it takes something to get into
>>>>>>>>> upstream from the branch that you are working on?
>>>>>>>>>
>>>>>>>>> I can provide any help needed to make this working but need to be sure
>>>>>>>>> that I can use this in production as soon as possible.
>>>>>>>>>
>>>>>>>>> Thanks and Regards
>>>>>>>>> Noorul
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1] http://pastie.org/8270391
>>>>>>>>>>
>>>>>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>>> Hi Noorul!
>>>>>>>>>>>
>>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>>>>>> "listCookbooks" one.
>>>>>>>>>>>
>>>>>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>>>>>> fails to build the corresponding map.
>>>>>>>>>>>
>>>>>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>>>>>
>>>>>>>>>>> Ignasi
>>>>>>>>>>>
>>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>>> [2]
>>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>>>>>
>>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>>>>>> try to get cookbook details using
>>>>>>>>>>>>
>>>>>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>>>>>> at
>>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>>> at
>>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>>>>>> at
>>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>>>>>
>>>>>>>>>>>> README.md says that
>>>>>>>>>>>>
>>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>>>>>> flavours.
>>>>>>>>>>>>
>>>>>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
Hi Noorul,
Just a follow up. I've been able to isolate (I think) the issue and
I've opened the corresponding JIRA [1]. I asked the chef-dev mailing
list [2] to see if someone can give us the details of how to properly
sign the URLs that are failing. It seems that some change may be
required in the signing filter [3].
Let's see if we can get this fixed!
[1] https://issues.apache.org/jira/browse/JCLOUDS-254
[2] http://lists.opscode.com/sympa/arc/chef-dev/2013-08/msg00034.html
[3] https://github.com/jclouds/jclouds-chef/blob/master/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
On 26 August 2013 16:54, Ignasi <ig...@gmail.com> wrote:
> Ok, I just added a few cookbooks more to my Hosted Chef account and
> the download is failing the same way.
> I still haven't figured out what can be failing in the signature (it
> shouldn't have changed since Chef 0.10), so any help would be
> appreciated!
>
> On 26 August 2013 15:55, Ignasi <ig...@gmail.com> wrote:
>> Hi again,
>>
>> The problem appears in the "getResourceContents" api call. After
>> debugging for a while I haven't found an explanation. It fails
>> randomly with random resources, and there is no pattern for that :(
>> Sometimes it fails to download a root file resource, sometimes not.
>> Sometimes it fails downloading a recipe file, sometimes it just works.
>>
>> I executed the live test suite against a Chef 0.10.8, and against
>> Enterprise Chef and all tests are passing, so I assume it might be an
>> issue in Chef 11; otherwise I can not explain why the same resource
>> sometimes is downloaded and sometimes it is not. The authentication
>> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise
>> Chef, it should work for Chef 11 too (or at least have some
>> deterministic behavior).
>>
>>
>> To properly reflect that in the live tests, I've separated the
>> "testListCookbooks" in two different tests: "testListCookbooks" and
>> "testDownloadCookbooks". If you update my branch, you should get those
>> changes.
>>
>> I'll search if there is any open issue and file a new one if appropriate.
>>
>> Anyway, let me know if you find something that could help getting rid of this!
>>
>>
>> Ignasi
>>
>>
>> On 26 August 2013 14:29, Noorul Islam K M <no...@collab.net> wrote:
>>> Ignasi <ig...@gmail.com> writes:
>>>
>>>> Thanks a lot for your feedback Noorul!
>>>>
>>>> You can run the 'core' live tests as follows:
>>>>
>>>> mvn clean install -pl core -Plive \
>>>> -Dtest.chef.endpoint=<you chef server url> \
>>>> -Dtest.chef.identity=<identity> \
>>>> -Dtest.chef.credential=<path to the PEM private key>
>>>>
>>>
>>> I think, I get the same as you
>>>
>>>
>>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
>>>
>>> Results :
>>>
>>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
>>> </Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
>>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
>>> reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
>>>
>>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
>>>
>>> Thanks and Regards
>>> Noorul
>>>
>>>> And if you have a hosted chef account, you can run them like this:
>>>>
>>>> mvn clean install -pl enterprise -Plive \
>>>> -Dtest.enterprisechef.org=<you organization name> \
>>>> -Dtest.enterprisechef.identity=<identity> \
>>>> -Dtest.enterprisechef.credential=<path to the PEM private key>
>>>>
>>>>
>>>>
>>>> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>
>>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>>>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>>>>> one, which seems to be the one with the right content. You can take a
>>>>>> look at the commit [2] if you want to know the details.
>>>>>>
>>>>>> I've tried the api call that was failing to you, and now it works and
>>>>>> the cookbook is properly deserialized. However, the live test to list
>>>>>> all cookbooks is still failing due to a signature error (which is
>>>>>> weird, because the signature is generated in a generic filter [3] that
>>>>>> applies to all requests).
>>>>>>
>>>>>> I think it might be related to my Chef 11 installation. To make sure,
>>>>>> could you update the branch and run the live tests, or try to list the
>>>>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>>>>
>>>>>
>>>>> I tested these methods after pulling latest from your branch.
>>>>>
>>>>> getCookbook()
>>>>> listCookbooks()
>>>>> listEnvironmentCookbooks()
>>>>>
>>>>> Everything seems to be working fine.
>>>>>
>>>>> How do I run live tests? Is this documented somewhere?
>>>>>
>>>>> Thanks and Regards
>>>>> Noorul
>>>>>
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>>
>>>>>> Ignasi
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>>>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>>>>> As soon as possible.
>>>>>>>
>>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>>>>> backward compatible (just a massive package and artifactId renaming),
>>>>>>> I sent a mail last week to let users express an opinion on the change
>>>>>>> [2]. I plan to merge it this week, since there are no opinions
>>>>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>>>>
>>>>>>> You could expect to have it merged by the end of the week, if I finish
>>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>>>>> fix this afternoon; I'll let you know when I have something usable so
>>>>>>> you can test and validate it.
>>>>>>>
>>>>>>> Thanks for your help!
>>>>>>>
>>>>>>>
>>>>>>> Ignasi
>>>>>>>
>>>>>>>
>>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>>>>
>>>>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>>
>>>>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>>>>> so you can have a look.
>>>>>>>>>
>>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>>>>> to build a workaround in the meanwhile.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I would like to know, usually how long it takes something to get into
>>>>>>>> upstream from the branch that you are working on?
>>>>>>>>
>>>>>>>> I can provide any help needed to make this working but need to be sure
>>>>>>>> that I can use this in production as soon as possible.
>>>>>>>>
>>>>>>>> Thanks and Regards
>>>>>>>> Noorul
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] http://pastie.org/8270391
>>>>>>>>>
>>>>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>>> Hi Noorul!
>>>>>>>>>>
>>>>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>>>>> "listCookbooks" one.
>>>>>>>>>>
>>>>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>>>>> fails to build the corresponding map.
>>>>>>>>>>
>>>>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>>>>
>>>>>>>>>> Ignasi
>>>>>>>>>>
>>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>>> [2]
>>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>>>>
>>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>>>>> try to get cookbook details using
>>>>>>>>>>>
>>>>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>>>>> at
>>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>>>>> at
>>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>>>>> at
>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>>>>> at
>>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>>>>> at
>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>>>>> at
>>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>>>>> at
>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>> at
>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>> at
>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>>> at
>>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>>>>> at
>>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>>>>
>>>>>>>>>>> README.md says that
>>>>>>>>>>>
>>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>>>>> flavours.
>>>>>>>>>>>
>>>>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>>>>
>>>>>>>>>>> Thanks and Regards
>>>>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
Ok, I just added a few cookbooks more to my Hosted Chef account and
the download is failing the same way.
I still haven't figured out what can be failing in the signature (it
shouldn't have changed since Chef 0.10), so any help would be
appreciated!
On 26 August 2013 15:55, Ignasi <ig...@gmail.com> wrote:
> Hi again,
>
> The problem appears in the "getResourceContents" api call. After
> debugging for a while I haven't found an explanation. It fails
> randomly with random resources, and there is no pattern for that :(
> Sometimes it fails to download a root file resource, sometimes not.
> Sometimes it fails downloading a recipe file, sometimes it just works.
>
> I executed the live test suite against a Chef 0.10.8, and against
> Enterprise Chef and all tests are passing, so I assume it might be an
> issue in Chef 11; otherwise I can not explain why the same resource
> sometimes is downloaded and sometimes it is not. The authentication
> mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise
> Chef, it should work for Chef 11 too (or at least have some
> deterministic behavior).
>
>
> To properly reflect that in the live tests, I've separated the
> "testListCookbooks" in two different tests: "testListCookbooks" and
> "testDownloadCookbooks". If you update my branch, you should get those
> changes.
>
> I'll search if there is any open issue and file a new one if appropriate.
>
> Anyway, let me know if you find something that could help getting rid of this!
>
>
> Ignasi
>
>
> On 26 August 2013 14:29, Noorul Islam K M <no...@collab.net> wrote:
>> Ignasi <ig...@gmail.com> writes:
>>
>>> Thanks a lot for your feedback Noorul!
>>>
>>> You can run the 'core' live tests as follows:
>>>
>>> mvn clean install -pl core -Plive \
>>> -Dtest.chef.endpoint=<you chef server url> \
>>> -Dtest.chef.identity=<identity> \
>>> -Dtest.chef.credential=<path to the PEM private key>
>>>
>>
>> I think, I get the same as you
>>
>>
>> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
>>
>> Results :
>>
>> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
>> </Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
>> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
>> reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
>>
>> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
>>
>> Thanks and Regards
>> Noorul
>>
>>> And if you have a hosted chef account, you can run them like this:
>>>
>>> mvn clean install -pl enterprise -Plive \
>>> -Dtest.enterprisechef.org=<you organization name> \
>>> -Dtest.enterprisechef.identity=<identity> \
>>> -Dtest.enterprisechef.credential=<path to the PEM private key>
>>>
>>>
>>>
>>> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>>>> Ignasi <ig...@gmail.com> writes:
>>>>
>>>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>>>> one, which seems to be the one with the right content. You can take a
>>>>> look at the commit [2] if you want to know the details.
>>>>>
>>>>> I've tried the api call that was failing to you, and now it works and
>>>>> the cookbook is properly deserialized. However, the live test to list
>>>>> all cookbooks is still failing due to a signature error (which is
>>>>> weird, because the signature is generated in a generic filter [3] that
>>>>> applies to all requests).
>>>>>
>>>>> I think it might be related to my Chef 11 installation. To make sure,
>>>>> could you update the branch and run the live tests, or try to list the
>>>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>>>
>>>>
>>>> I tested these methods after pulling latest from your branch.
>>>>
>>>> getCookbook()
>>>> listCookbooks()
>>>> listEnvironmentCookbooks()
>>>>
>>>> Everything seems to be working fine.
>>>>
>>>> How do I run live tests? Is this documented somewhere?
>>>>
>>>> Thanks and Regards
>>>> Noorul
>>>>
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> Ignasi
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>>
>>>>>
>>>>>
>>>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>>>> As soon as possible.
>>>>>>
>>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>>>> backward compatible (just a massive package and artifactId renaming),
>>>>>> I sent a mail last week to let users express an opinion on the change
>>>>>> [2]. I plan to merge it this week, since there are no opinions
>>>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>>>
>>>>>> You could expect to have it merged by the end of the week, if I finish
>>>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>>>> fix this afternoon; I'll let you know when I have something usable so
>>>>>> you can test and validate it.
>>>>>>
>>>>>> Thanks for your help!
>>>>>>
>>>>>>
>>>>>> Ignasi
>>>>>>
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>>>
>>>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>>
>>>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>>>> so you can have a look.
>>>>>>>>
>>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>>>> to build a workaround in the meanwhile.
>>>>>>>>
>>>>>>>
>>>>>>> I would like to know, usually how long it takes something to get into
>>>>>>> upstream from the branch that you are working on?
>>>>>>>
>>>>>>> I can provide any help needed to make this working but need to be sure
>>>>>>> that I can use this in production as soon as possible.
>>>>>>>
>>>>>>> Thanks and Regards
>>>>>>> Noorul
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] http://pastie.org/8270391
>>>>>>>>
>>>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>>>> Hi Noorul!
>>>>>>>>>
>>>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>>>> "listCookbooks" one.
>>>>>>>>>
>>>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>>>> fails to build the corresponding map.
>>>>>>>>>
>>>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>>>
>>>>>>>>> Ignasi
>>>>>>>>>
>>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>>> [2]
>>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>>>
>>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>>>> try to get cookbook details using
>>>>>>>>>>
>>>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>>>> at
>>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>>>> at
>>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>>>> at
>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>>>> at
>>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>>>> at
>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>>>> at
>>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>>>> at
>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>> at
>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>> at
>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>>> at
>>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>>>> at
>>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>>>
>>>>>>>>>> README.md says that
>>>>>>>>>>
>>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>>>> flavours.
>>>>>>>>>>
>>>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>>>
>>>>>>>>>> Thanks and Regards
>>>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
Hi again,
The problem appears in the "getResourceContents" api call. After
debugging for a while I haven't found an explanation. It fails
randomly with random resources, and there is no pattern for that :(
Sometimes it fails to download a root file resource, sometimes not.
Sometimes it fails downloading a recipe file, sometimes it just works.
I executed the live test suite against a Chef 0.10.8, and against
Enterprise Chef and all tests are passing, so I assume it might be an
issue in Chef 11; otherwise I can not explain why the same resource
sometimes is downloaded and sometimes it is not. The authentication
mechanism hasn't changed, and if it works for Chef 0.10 and Enterprise
Chef, it should work for Chef 11 too (or at least have some
deterministic behavior).
To properly reflect that in the live tests, I've separated the
"testListCookbooks" in two different tests: "testListCookbooks" and
"testDownloadCookbooks". If you update my branch, you should get those
changes.
I'll search if there is any open issue and file a new one if appropriate.
Anyway, let me know if you find something that could help getting rid of this!
Ignasi
On 26 August 2013 14:29, Noorul Islam K M <no...@collab.net> wrote:
> Ignasi <ig...@gmail.com> writes:
>
>> Thanks a lot for your feedback Noorul!
>>
>> You can run the 'core' live tests as follows:
>>
>> mvn clean install -pl core -Plive \
>> -Dtest.chef.endpoint=<you chef server url> \
>> -Dtest.chef.identity=<identity> \
>> -Dtest.chef.credential=<path to the PEM private key>
>>
>
> I think, I get the same as you
>
>
> Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
>
> Results :
>
> Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
> </Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
> tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
> reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
>
> Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
>
> Thanks and Regards
> Noorul
>
>> And if you have a hosted chef account, you can run them like this:
>>
>> mvn clean install -pl enterprise -Plive \
>> -Dtest.enterprisechef.org=<you organization name> \
>> -Dtest.enterprisechef.identity=<identity> \
>> -Dtest.enterprisechef.credential=<path to the PEM private key>
>>
>>
>>
>> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>>> Ignasi <ig...@gmail.com> writes:
>>>
>>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>>> one, which seems to be the one with the right content. You can take a
>>>> look at the commit [2] if you want to know the details.
>>>>
>>>> I've tried the api call that was failing to you, and now it works and
>>>> the cookbook is properly deserialized. However, the live test to list
>>>> all cookbooks is still failing due to a signature error (which is
>>>> weird, because the signature is generated in a generic filter [3] that
>>>> applies to all requests).
>>>>
>>>> I think it might be related to my Chef 11 installation. To make sure,
>>>> could you update the branch and run the live tests, or try to list the
>>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>>
>>>
>>> I tested these methods after pulling latest from your branch.
>>>
>>> getCookbook()
>>> listCookbooks()
>>> listEnvironmentCookbooks()
>>>
>>> Everything seems to be working fine.
>>>
>>> How do I run live tests? Is this documented somewhere?
>>>
>>> Thanks and Regards
>>> Noorul
>>>
>>>>
>>>> Thanks!
>>>>
>>>>
>>>> Ignasi
>>>>
>>>>
>>>>
>>>>
>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>>
>>>>
>>>>
>>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>>> As soon as possible.
>>>>>
>>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>>> backward compatible (just a massive package and artifactId renaming),
>>>>> I sent a mail last week to let users express an opinion on the change
>>>>> [2]. I plan to merge it this week, since there are no opinions
>>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>>
>>>>> You could expect to have it merged by the end of the week, if I finish
>>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>>> fix this afternoon; I'll let you know when I have something usable so
>>>>> you can test and validate it.
>>>>>
>>>>> Thanks for your help!
>>>>>
>>>>>
>>>>> Ignasi
>>>>>
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>>
>>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>>
>>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>>> so you can have a look.
>>>>>>>
>>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>>> to build a workaround in the meanwhile.
>>>>>>>
>>>>>>
>>>>>> I would like to know, usually how long it takes something to get into
>>>>>> upstream from the branch that you are working on?
>>>>>>
>>>>>> I can provide any help needed to make this working but need to be sure
>>>>>> that I can use this in production as soon as possible.
>>>>>>
>>>>>> Thanks and Regards
>>>>>> Noorul
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [1] http://pastie.org/8270391
>>>>>>>
>>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>>> Hi Noorul!
>>>>>>>>
>>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>>> "listCookbooks" one.
>>>>>>>>
>>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>>> fails to build the corresponding map.
>>>>>>>>
>>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>>
>>>>>>>> Ignasi
>>>>>>>>
>>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>>> [2]
>>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>>
>>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>>> try to get cookbook details using
>>>>>>>>>
>>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>>> at
>>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>>> at
>>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>>> at
>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>>> at
>>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>>> at
>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>>> at
>>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>>> at
>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>> at
>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>> at
>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>>> at
>>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>>> at
>>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>>
>>>>>>>>> README.md says that
>>>>>>>>>
>>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>>> flavours.
>>>>>>>>>
>>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>>
>>>>>>>>> Thanks and Regards
>>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Ignasi <ig...@gmail.com> writes:
> Thanks a lot for your feedback Noorul!
>
> You can run the 'core' live tests as follows:
>
> mvn clean install -pl core -Plive \
> -Dtest.chef.endpoint=<you chef server url> \
> -Dtest.chef.identity=<identity> \
> -Dtest.chef.credential=<path to the PEM private key>
>
I think, I get the same as you
Test suite progress: tests succeeded: 41, failed: 1, skipped: 1.
Tests run: 43, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 187.134 sec <<< FAILURE!
Results :
Failed tests: testListCookbooks(org.jclouds.chef.ChefApiLiveTest): <?xml version="1.0" encoding="UTF-8"?><Error><Code>SignatureDoesNotMatch
</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><S
tringToSignBytes></StringToSignBytes><RequestId>g2gCZAATYm9va3NoZWxmQDEyNy4wLjAuMWgDYgAABWFiAAef4WIAC6/x</RequestId><HostId></HostId><Signatu
reProvided>hRkpN1RzN /pGeMmyXnDtRZUNnI=</SignatureProvided><StringToSign>GET(..)
Tests run: 43, Failures: 1, Errors: 0, Skipped: 1
Thanks and Regards
Noorul
> And if you have a hosted chef account, you can run them like this:
>
> mvn clean install -pl enterprise -Plive \
> -Dtest.enterprisechef.org=<you organization name> \
> -Dtest.enterprisechef.identity=<identity> \
> -Dtest.enterprisechef.credential=<path to the PEM private key>
>
>
>
> On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
>> Ignasi <ig...@gmail.com> writes:
>>
>>> Noorul, I've just uploaded to my branch [1] a change that registers a
>>> custom Map deserializer to allow repeated keys. It just keeps the last
>>> one, which seems to be the one with the right content. You can take a
>>> look at the commit [2] if you want to know the details.
>>>
>>> I've tried the api call that was failing to you, and now it works and
>>> the cookbook is properly deserialized. However, the live test to list
>>> all cookbooks is still failing due to a signature error (which is
>>> weird, because the signature is generated in a generic filter [3] that
>>> applies to all requests).
>>>
>>> I think it might be related to my Chef 11 installation. To make sure,
>>> could you update the branch and run the live tests, or try to list the
>>> cookbooks using the ChefApi and ChefServer, and share the results?
>>>
>>
>> I tested these methods after pulling latest from your branch.
>>
>> getCookbook()
>> listCookbooks()
>> listEnvironmentCookbooks()
>>
>> Everything seems to be working fine.
>>
>> How do I run live tests? Is this documented somewhere?
>>
>> Thanks and Regards
>> Noorul
>>
>>>
>>> Thanks!
>>>
>>>
>>> Ignasi
>>>
>>>
>>>
>>>
>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>>
>>>
>>>
>>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>>> As soon as possible.
>>>>
>>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>>> backward compatible (just a massive package and artifactId renaming),
>>>> I sent a mail last week to let users express an opinion on the change
>>>> [2]. I plan to merge it this week, since there are no opinions
>>>> against, and as soon as it is merged, I'll merge my current branch.
>>>>
>>>> You could expect to have it merged by the end of the week, if I finish
>>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>>> fix this afternoon; I'll let you know when I have something usable so
>>>> you can test and validate it.
>>>>
>>>> Thanks for your help!
>>>>
>>>>
>>>> Ignasi
>>>>
>>>>
>>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>>
>>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>>> Ignasi <ig...@gmail.com> writes:
>>>>>
>>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>>> so you can have a look.
>>>>>>
>>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>>> to build a workaround in the meanwhile.
>>>>>>
>>>>>
>>>>> I would like to know, usually how long it takes something to get into
>>>>> upstream from the branch that you are working on?
>>>>>
>>>>> I can provide any help needed to make this working but need to be sure
>>>>> that I can use this in production as soon as possible.
>>>>>
>>>>> Thanks and Regards
>>>>> Noorul
>>>>>
>>>>>>
>>>>>>
>>>>>> [1] http://pastie.org/8270391
>>>>>>
>>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>>> Hi Noorul!
>>>>>>>
>>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>>> "listCookbooks" one.
>>>>>>>
>>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>>> return the same responses than in previous versions), and the json parser
>>>>>>> fails to build the corresponding map.
>>>>>>>
>>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>>> the error (or patch :)) is appreciated too!
>>>>>>>
>>>>>>> Ignasi
>>>>>>>
>>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>>> [2]
>>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>>
>>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>>
>>>>>>>>
>>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>>> try to get cookbook details using
>>>>>>>>
>>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>>
>>>>>>>>
>>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>>> at
>>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>>> at
>>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>>> at
>>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>>> at
>>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>>> at
>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>>> at
>>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>>> at
>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>> at
>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>> at
>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>>> at
>>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>>> at
>>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>>
>>>>>>>> README.md says that
>>>>>>>>
>>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>>> flavours.
>>>>>>>>
>>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>>
>>>>>>>> Thanks and Regards
>>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
Thanks a lot for your feedback Noorul!
You can run the 'core' live tests as follows:
mvn clean install -pl core -Plive \
-Dtest.chef.endpoint=<you chef server url> \
-Dtest.chef.identity=<identity> \
-Dtest.chef.credential=<path to the PEM private key>
And if you have a hosted chef account, you can run them like this:
mvn clean install -pl enterprise -Plive \
-Dtest.enterprisechef.org=<you organization name> \
-Dtest.enterprisechef.identity=<identity> \
-Dtest.enterprisechef.credential=<path to the PEM private key>
On 26 August 2013 14:10, Noorul Islam K M <no...@collab.net> wrote:
> Ignasi <ig...@gmail.com> writes:
>
>> Noorul, I've just uploaded to my branch [1] a change that registers a
>> custom Map deserializer to allow repeated keys. It just keeps the last
>> one, which seems to be the one with the right content. You can take a
>> look at the commit [2] if you want to know the details.
>>
>> I've tried the api call that was failing to you, and now it works and
>> the cookbook is properly deserialized. However, the live test to list
>> all cookbooks is still failing due to a signature error (which is
>> weird, because the signature is generated in a generic filter [3] that
>> applies to all requests).
>>
>> I think it might be related to my Chef 11 installation. To make sure,
>> could you update the branch and run the live tests, or try to list the
>> cookbooks using the ChefApi and ChefServer, and share the results?
>>
>
> I tested these methods after pulling latest from your branch.
>
> getCookbook()
> listCookbooks()
> listEnvironmentCookbooks()
>
> Everything seems to be working fine.
>
> How do I run live tests? Is this documented somewhere?
>
> Thanks and Regards
> Noorul
>
>>
>> Thanks!
>>
>>
>> Ignasi
>>
>>
>>
>>
>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
>> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>>
>>
>>
>> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>>> As soon as possible.
>>>
>>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>>> backward compatible (just a massive package and artifactId renaming),
>>> I sent a mail last week to let users express an opinion on the change
>>> [2]. I plan to merge it this week, since there are no opinions
>>> against, and as soon as it is merged, I'll merge my current branch.
>>>
>>> You could expect to have it merged by the end of the week, if I finish
>>> the fix by then (which I hope :)). I plan to upload to my branch the
>>> fix this afternoon; I'll let you know when I have something usable so
>>> you can test and validate it.
>>>
>>> Thanks for your help!
>>>
>>>
>>> Ignasi
>>>
>>>
>>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>>
>>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>>> Ignasi <ig...@gmail.com> writes:
>>>>
>>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>>> so you can have a look.
>>>>>
>>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>>> to build a workaround in the meanwhile.
>>>>>
>>>>
>>>> I would like to know, usually how long it takes something to get into
>>>> upstream from the branch that you are working on?
>>>>
>>>> I can provide any help needed to make this working but need to be sure
>>>> that I can use this in production as soon as possible.
>>>>
>>>> Thanks and Regards
>>>> Noorul
>>>>
>>>>>
>>>>>
>>>>> [1] http://pastie.org/8270391
>>>>>
>>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>>> Hi Noorul!
>>>>>>
>>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>>> "listCookbooks" one.
>>>>>>
>>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>>> return the same responses than in previous versions), and the json parser
>>>>>> fails to build the corresponding map.
>>>>>>
>>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>>> help would be much appreciated. If you could build my branch and enable
>>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>>> the error (or patch :)) is appreciated too!
>>>>>>
>>>>>> Ignasi
>>>>>>
>>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>>> [2]
>>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>>
>>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>>
>>>>>>>
>>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>>> try to get cookbook details using
>>>>>>>
>>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>>
>>>>>>>
>>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>>> at
>>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>>> at
>>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>>> at
>>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>>> at
>>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>>> at
>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>>> at
>>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>>> at
>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>> at
>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>> at
>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>>> at
>>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>>> at
>>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>>
>>>>>>> README.md says that
>>>>>>>
>>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>>> flavours.
>>>>>>>
>>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>>
>>>>>>> Thanks and Regards
>>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Ignasi <ig...@gmail.com> writes:
> Noorul, I've just uploaded to my branch [1] a change that registers a
> custom Map deserializer to allow repeated keys. It just keeps the last
> one, which seems to be the one with the right content. You can take a
> look at the commit [2] if you want to know the details.
>
> I've tried the api call that was failing to you, and now it works and
> the cookbook is properly deserialized. However, the live test to list
> all cookbooks is still failing due to a signature error (which is
> weird, because the signature is generated in a generic filter [3] that
> applies to all requests).
>
> I think it might be related to my Chef 11 installation. To make sure,
> could you update the branch and run the live tests, or try to list the
> cookbooks using the ChefApi and ChefServer, and share the results?
>
I tested these methods after pulling latest from your branch.
getCookbook()
listCookbooks()
listEnvironmentCookbooks()
Everything seems to be working fine.
How do I run live tests? Is this documented somewhere?
Thanks and Regards
Noorul
>
> Thanks!
>
>
> Ignasi
>
>
>
>
> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
> [2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
> [3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
>
>
>
> On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
>> As soon as possible.
>>
>> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
>> backward compatible (just a massive package and artifactId renaming),
>> I sent a mail last week to let users express an opinion on the change
>> [2]. I plan to merge it this week, since there are no opinions
>> against, and as soon as it is merged, I'll merge my current branch.
>>
>> You could expect to have it merged by the end of the week, if I finish
>> the fix by then (which I hope :)). I plan to upload to my branch the
>> fix this afternoon; I'll let you know when I have something usable so
>> you can test and validate it.
>>
>> Thanks for your help!
>>
>>
>> Ignasi
>>
>>
>> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
>> [2] http://markmail.org/message/5jm2ivpypewnov3h
>>
>> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>>> Ignasi <ig...@gmail.com> writes:
>>>
>>>> Just tested the openssl cookbook and found that the returned Metadata
>>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>>> so you can have a look.
>>>>
>>>> I'll ping Chef people to see if it is an issue in their side, and try
>>>> to build a workaround in the meanwhile.
>>>>
>>>
>>> I would like to know, usually how long it takes something to get into
>>> upstream from the branch that you are working on?
>>>
>>> I can provide any help needed to make this working but need to be sure
>>> that I can use this in production as soon as possible.
>>>
>>> Thanks and Regards
>>> Noorul
>>>
>>>>
>>>>
>>>> [1] http://pastie.org/8270391
>>>>
>>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>>> Hi Noorul!
>>>>>
>>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>>> tests and also made all live tests pass against a Chef 11 except the
>>>>> "listCookbooks" one.
>>>>>
>>>>> I get the exact same error than you, and it happens when deserializing the
>>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>>> return the same responses than in previous versions), and the json parser
>>>>> fails to build the corresponding map.
>>>>>
>>>>> I'm currently working on this but it's taking longer than expected, so any
>>>>> help would be much appreciated. If you could build my branch and enable
>>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>>> the error (or patch :)) is appreciated too!
>>>>>
>>>>> Ignasi
>>>>>
>>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>>> [2]
>>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>>
>>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>>
>>>>>>
>>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>>> try to get cookbook details using
>>>>>>
>>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>>
>>>>>>
>>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>>> at
>>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>>> at
>>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>>> at
>>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>>> at
>>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>>> at
>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>>> at
>>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>>> at
>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>> at
>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>> at
>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>>> at
>>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>>> at
>>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>>
>>>>>> README.md says that
>>>>>>
>>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>>> of the user and organization api of the Hosted and Private Chef
>>>>>> flavours.
>>>>>>
>>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>>
>>>>>> Thanks and Regards
>>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
Noorul, I've just uploaded to my branch [1] a change that registers a
custom Map deserializer to allow repeated keys. It just keeps the last
one, which seems to be the one with the right content. You can take a
look at the commit [2] if you want to know the details.
I've tried the api call that was failing to you, and now it works and
the cookbook is properly deserialized. However, the live test to list
all cookbooks is still failing due to a signature error (which is
weird, because the signature is generated in a generic filter [3] that
applies to all requests).
I think it might be related to my Chef 11 installation. To make sure,
could you update the branch and run the live tests, or try to list the
cookbooks using the ChefApi and ChefServer, and share the results?
Thanks!
Ignasi
[1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
[2] https://github.com/nacx/jclouds-chef/commit/59115b5a08197547cf74f6831e265fe67a3b9ab8
[3] https://github.com/nacx/jclouds-chef/blob/245-live-tests/core/src/main/java/org/jclouds/chef/filters/SignedHeaderAuth.java
On 26 August 2013 12:24, Ignasi <ig...@gmail.com> wrote:
> As soon as possible.
>
> My branch depends on issue JCLOUDS-246 [1]. Since changes are not
> backward compatible (just a massive package and artifactId renaming),
> I sent a mail last week to let users express an opinion on the change
> [2]. I plan to merge it this week, since there are no opinions
> against, and as soon as it is merged, I'll merge my current branch.
>
> You could expect to have it merged by the end of the week, if I finish
> the fix by then (which I hope :)). I plan to upload to my branch the
> fix this afternoon; I'll let you know when I have something usable so
> you can test and validate it.
>
> Thanks for your help!
>
>
> Ignasi
>
>
> [1] https://issues.apache.org/jira/browse/JCLOUDS-246
> [2] http://markmail.org/message/5jm2ivpypewnov3h
>
> On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
>> Ignasi <ig...@gmail.com> writes:
>>
>>> Just tested the openssl cookbook and found that the returned Metadata
>>> has a duplicate key in the recipes list. I've pasted the json here [1]
>>> so you can have a look.
>>>
>>> I'll ping Chef people to see if it is an issue in their side, and try
>>> to build a workaround in the meanwhile.
>>>
>>
>> I would like to know, usually how long it takes something to get into
>> upstream from the branch that you are working on?
>>
>> I can provide any help needed to make this working but need to be sure
>> that I can use this in production as soon as possible.
>>
>> Thanks and Regards
>> Noorul
>>
>>>
>>>
>>> [1] http://pastie.org/8270391
>>>
>>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>>> Hi Noorul!
>>>>
>>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>>> tests and also made all live tests pass against a Chef 11 except the
>>>> "listCookbooks" one.
>>>>
>>>> I get the exact same error than you, and it happens when deserializing the
>>>> json for the cookbook into the CookbookVersion object. It seems that the
>>>> Chef Server is returning a duplicate key in the json structure (although it
>>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>>> return the same responses than in previous versions), and the json parser
>>>> fails to build the corresponding map.
>>>>
>>>> I'm currently working on this but it's taking longer than expected, so any
>>>> help would be much appreciated. If you could build my branch and enable
>>>> jclouds wire logging [2], you would be able to see the json returned for
>>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>>> the error (or patch :)) is appreciated too!
>>>>
>>>> Ignasi
>>>>
>>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>>> [2]
>>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>>
>>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>>
>>>>>
>>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>>> Server 11. In my initial testings, I get the following exception when I
>>>>> try to get cookbook details using
>>>>>
>>>>> api.getCookbook("openssl", "1.0.2")
>>>>>
>>>>>
>>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>>> at
>>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>>> at
>>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>>> at
>>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>>> at
>>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>>> at
>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>>> at
>>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>>> at
>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>> at
>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>> at
>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>>> at
>>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>>> at
>>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>>
>>>>> README.md says that
>>>>>
>>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>>> apis, and an initial and very basic (still in progress) implementation
>>>>> of the user and organization api of the Hosted and Private Chef
>>>>> flavours.
>>>>>
>>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>>
>>>>> Thanks and Regards
>>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
As soon as possible.
My branch depends on issue JCLOUDS-246 [1]. Since changes are not
backward compatible (just a massive package and artifactId renaming),
I sent a mail last week to let users express an opinion on the change
[2]. I plan to merge it this week, since there are no opinions
against, and as soon as it is merged, I'll merge my current branch.
You could expect to have it merged by the end of the week, if I finish
the fix by then (which I hope :)). I plan to upload to my branch the
fix this afternoon; I'll let you know when I have something usable so
you can test and validate it.
Thanks for your help!
Ignasi
[1] https://issues.apache.org/jira/browse/JCLOUDS-246
[2] http://markmail.org/message/5jm2ivpypewnov3h
On 26 August 2013 12:02, Noorul Islam K M <no...@collab.net> wrote:
> Ignasi <ig...@gmail.com> writes:
>
>> Just tested the openssl cookbook and found that the returned Metadata
>> has a duplicate key in the recipes list. I've pasted the json here [1]
>> so you can have a look.
>>
>> I'll ping Chef people to see if it is an issue in their side, and try
>> to build a workaround in the meanwhile.
>>
>
> I would like to know, usually how long it takes something to get into
> upstream from the branch that you are working on?
>
> I can provide any help needed to make this working but need to be sure
> that I can use this in production as soon as possible.
>
> Thanks and Regards
> Noorul
>
>>
>>
>> [1] http://pastie.org/8270391
>>
>> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>>> Hi Noorul!
>>>
>>> There is definitely the intention to support Chef 11 as soon as possible. I
>>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>>> tests and also made all live tests pass against a Chef 11 except the
>>> "listCookbooks" one.
>>>
>>> I get the exact same error than you, and it happens when deserializing the
>>> json for the cookbook into the CookbookVersion object. It seems that the
>>> Chef Server is returning a duplicate key in the json structure (although it
>>> shouldn't because we are forcing by default a 0.10 version header to make it
>>> return the same responses than in previous versions), and the json parser
>>> fails to build the corresponding map.
>>>
>>> I'm currently working on this but it's taking longer than expected, so any
>>> help would be much appreciated. If you could build my branch and enable
>>> jclouds wire logging [2], you would be able to see the json returned for
>>> your cookbook. Could you share that json? Also, any help tracking/isolating
>>> the error (or patch :)) is appreciated too!
>>>
>>> Ignasi
>>>
>>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>>> [2]
>>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>>
>>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>>
>>>>
>>>> We are planning to use jclouds-chef library to talk to community Chef
>>>> Server 11. In my initial testings, I get the following exception when I
>>>> try to get cookbook details using
>>>>
>>>> api.getCookbook("openssl", "1.0.2")
>>>>
>>>>
>>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>>> at
>>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>>> at
>>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>>> at
>>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>>> at
>>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>>> at
>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>>> at
>>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>>> at
>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>> at
>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>> at
>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>>> at
>>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>>> at
>>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>>
>>>> README.md says that
>>>>
>>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>>> apis, and an initial and very basic (still in progress) implementation
>>>> of the user and organization api of the Hosted and Private Chef
>>>> flavours.
>>>>
>>>> I think that the issue I am facing is because I am using Chef server 11.
>>>> Is there any plans to officially support Chef Server 11 in near future?
>>>>
>>>> Thanks and Regards
>>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Ignasi <ig...@gmail.com> writes:
> Just tested the openssl cookbook and found that the returned Metadata
> has a duplicate key in the recipes list. I've pasted the json here [1]
> so you can have a look.
>
> I'll ping Chef people to see if it is an issue in their side, and try
> to build a workaround in the meanwhile.
>
I would like to know, usually how long it takes something to get into
upstream from the branch that you are working on?
I can provide any help needed to make this working but need to be sure
that I can use this in production as soon as possible.
Thanks and Regards
Noorul
>
>
> [1] http://pastie.org/8270391
>
> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>> Hi Noorul!
>>
>> There is definitely the intention to support Chef 11 as soon as possible. I
>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>> tests and also made all live tests pass against a Chef 11 except the
>> "listCookbooks" one.
>>
>> I get the exact same error than you, and it happens when deserializing the
>> json for the cookbook into the CookbookVersion object. It seems that the
>> Chef Server is returning a duplicate key in the json structure (although it
>> shouldn't because we are forcing by default a 0.10 version header to make it
>> return the same responses than in previous versions), and the json parser
>> fails to build the corresponding map.
>>
>> I'm currently working on this but it's taking longer than expected, so any
>> help would be much appreciated. If you could build my branch and enable
>> jclouds wire logging [2], you would be able to see the json returned for
>> your cookbook. Could you share that json? Also, any help tracking/isolating
>> the error (or patch :)) is appreciated too!
>>
>> Ignasi
>>
>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>> [2]
>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>
>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>
>>>
>>> We are planning to use jclouds-chef library to talk to community Chef
>>> Server 11. In my initial testings, I get the following exception when I
>>> try to get cookbook details using
>>>
>>> api.getCookbook("openssl", "1.0.2")
>>>
>>>
>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>> at
>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>> at
>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>> at
>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>> at
>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>> at
>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>> at
>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>> at
>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>> at
>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>> at
>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>> at
>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>> at
>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>
>>> README.md says that
>>>
>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>> apis, and an initial and very basic (still in progress) implementation
>>> of the user and organization api of the Hosted and Private Chef
>>> flavours.
>>>
>>> I think that the issue I am facing is because I am using Chef server 11.
>>> Is there any plans to officially support Chef Server 11 in near future?
>>>
>>> Thanks and Regards
>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Ignasi <ig...@gmail.com> writes:
> Just tested the openssl cookbook and found that the returned Metadata
> has a duplicate key in the recipes list. I've pasted the json here [1]
> so you can have a look.
>
> I'll ping Chef people to see if it is an issue in their side, and try
> to build a workaround in the meanwhile.
>
>
I get the same duplicate key issue with other cookbooks also, like
apache2
Thanks and Regards
Noorul
>
> [1] http://pastie.org/8270391
>
> On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
>> Hi Noorul!
>>
>> There is definitely the intention to support Chef 11 as soon as possible. I
>> have a work in progress branch [1] where I fixed all Enterprise Chef live
>> tests and also made all live tests pass against a Chef 11 except the
>> "listCookbooks" one.
>>
>> I get the exact same error than you, and it happens when deserializing the
>> json for the cookbook into the CookbookVersion object. It seems that the
>> Chef Server is returning a duplicate key in the json structure (although it
>> shouldn't because we are forcing by default a 0.10 version header to make it
>> return the same responses than in previous versions), and the json parser
>> fails to build the corresponding map.
>>
>> I'm currently working on this but it's taking longer than expected, so any
>> help would be much appreciated. If you could build my branch and enable
>> jclouds wire logging [2], you would be able to see the json returned for
>> your cookbook. Could you share that json? Also, any help tracking/isolating
>> the error (or patch :)) is appreciated too!
>>
>> Ignasi
>>
>> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
>> [2]
>> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>>
>> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>>
>>>
>>> We are planning to use jclouds-chef library to talk to community Chef
>>> Server 11. In my initial testings, I get the following exception when I
>>> try to get cookbook details using
>>>
>>> api.getCookbook("openssl", "1.0.2")
>>>
>>>
>>> | java.lang.IllegalArgumentException: duplicate key: openssl
>>> at
>>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>>> at
>>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>>> at
>>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>>> at
>>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>>> at
>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>>> at
>>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>>> at
>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>> at
>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>> at
>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>>> at
>>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>>> at com.google.gson.Gson.fromJson(Gson.java:795)
>>> at com.google.gson.Gson.fromJson(Gson.java:761)
>>> at com.google.gson.Gson.fromJson(Gson.java:710)
>>> at
>>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>>
>>> README.md says that
>>>
>>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>>> apis, and an initial and very basic (still in progress) implementation
>>> of the user and organization api of the Hosted and Private Chef
>>> flavours.
>>>
>>> I think that the issue I am facing is because I am using Chef server 11.
>>> Is there any plans to officially support Chef Server 11 in near future?
>>>
>>> Thanks and Regards
>>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
Just tested the openssl cookbook and found that the returned Metadata
has a duplicate key in the recipes list. I've pasted the json here [1]
so you can have a look.
I'll ping Chef people to see if it is an issue in their side, and try
to build a workaround in the meanwhile.
[1] http://pastie.org/8270391
On 26 August 2013 08:28, Ignasi <ig...@gmail.com> wrote:
> Hi Noorul!
>
> There is definitely the intention to support Chef 11 as soon as possible. I
> have a work in progress branch [1] where I fixed all Enterprise Chef live
> tests and also made all live tests pass against a Chef 11 except the
> "listCookbooks" one.
>
> I get the exact same error than you, and it happens when deserializing the
> json for the cookbook into the CookbookVersion object. It seems that the
> Chef Server is returning a duplicate key in the json structure (although it
> shouldn't because we are forcing by default a 0.10 version header to make it
> return the same responses than in previous versions), and the json parser
> fails to build the corresponding map.
>
> I'm currently working on this but it's taking longer than expected, so any
> help would be much appreciated. If you could build my branch and enable
> jclouds wire logging [2], you would be able to see the json returned for
> your cookbook. Could you share that json? Also, any help tracking/isolating
> the error (or patch :)) is appreciated too!
>
> Ignasi
>
> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
> [2]
> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
>
> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>
>>
>> We are planning to use jclouds-chef library to talk to community Chef
>> Server 11. In my initial testings, I get the following exception when I
>> try to get cookbook details using
>>
>> api.getCookbook("openssl", "1.0.2")
>>
>>
>> | java.lang.IllegalArgumentException: duplicate key: openssl
>> at
>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>> at
>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>> at
>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>> at
>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>> at
>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>> at
>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>> at com.google.gson.Gson.fromJson(Gson.java:795)
>> at com.google.gson.Gson.fromJson(Gson.java:761)
>> at com.google.gson.Gson.fromJson(Gson.java:710)
>> at
>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>
>> README.md says that
>>
>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>> apis, and an initial and very basic (still in progress) implementation
>> of the user and organization api of the Hosted and Private Chef
>> flavours.
>>
>> I think that the issue I am facing is because I am using Chef server 11.
>> Is there any plans to officially support Chef Server 11 in near future?
>>
>> Thanks and Regards
>> Noorul
Re: [jclouds-chef] Support for Chef server 11
Posted by Noorul Islam K M <no...@collab.net>.
Ignasi <ig...@gmail.com> writes:
> Hi Noorul!
>
> There is definitely the intention to support Chef 11 as soon as possible. I
> have a work in progress branch [1] where I fixed all Enterprise Chef live
> tests and also made all live tests pass against a Chef 11 except the
> "listCookbooks" one.
>
> I get the exact same error than you, and it happens when deserializing the
> json for the cookbook into the CookbookVersion object. It seems that the
> Chef Server is returning a duplicate key in the json structure (although it
> shouldn't because we are forcing by default a 0.10 version header to make
> it return the same responses than in previous versions), and the json
> parser fails to build the corresponding map.
>
> I'm currently working on this but it's taking longer than expected, so any
> help would be much appreciated. If you could build my branch and
> enable
I built the branch that you mentioned but I could not get the logging to
work with that, I get
https://gist.github.com/noorul/6339541
But without logging I get the same exception that I get with 1.6.0 build.
Thanks and Regards
Noorul
> jclouds wire logging [2], you would be able to see the json returned for
> your cookbook. Could you share that json? Also, any help tracking/isolating
> the error (or patch :)) is appreciated too!
>
> Ignasi
>
> [1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
> [2]
> http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
> El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>
>>
>> We are planning to use jclouds-chef library to talk to community Chef
>> Server 11. In my initial testings, I get the following exception when I
>> try to get cookbook details using
>>
>> api.getCookbook("openssl", "1.0.2")
>>
>>
>> | java.lang.IllegalArgumentException: duplicate key: openssl
>> at
>> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
>> at
>> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
>> at
>> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
>> at
>> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
>> at
>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
>> at
>> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
>> at
>> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
>> at com.google.gson.Gson.fromJson(Gson.java:795)
>> at com.google.gson.Gson.fromJson(Gson.java:761)
>> at com.google.gson.Gson.fromJson(Gson.java:710)
>> at
>> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
>> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>>
>> README.md says that
>>
>> It currently supports versions 0.9 and 0.10 of the standard Chef server
>> apis, and an initial and very basic (still in progress) implementation
>> of the user and organization api of the Hosted and Private Chef
>> flavours.
>>
>> I think that the issue I am facing is because I am using Chef server 11.
>> Is there any plans to officially support Chef Server 11 in near future?
>>
>> Thanks and Regards
>> Noorul
>>
Re: [jclouds-chef] Support for Chef server 11
Posted by Ignasi <ig...@gmail.com>.
Hi Noorul!
There is definitely the intention to support Chef 11 as soon as possible. I
have a work in progress branch [1] where I fixed all Enterprise Chef live
tests and also made all live tests pass against a Chef 11 except the
"listCookbooks" one.
I get the exact same error than you, and it happens when deserializing the
json for the cookbook into the CookbookVersion object. It seems that the
Chef Server is returning a duplicate key in the json structure (although it
shouldn't because we are forcing by default a 0.10 version header to make
it return the same responses than in previous versions), and the json
parser fails to build the corresponding map.
I'm currently working on this but it's taking longer than expected, so any
help would be much appreciated. If you could build my branch and enable
jclouds wire logging [2], you would be able to see the json returned for
your cookbook. Could you share that json? Also, any help tracking/isolating
the error (or patch :)) is appreciated too!
Ignasi
[1] https://github.com/nacx/jclouds-chef/tree/245-live-tests
[2]
http://jclouds.incubator.apache.org/documentation/reference/jclouds-logging/
El 26/08/2013 07:27, "Noorul Islam K M" <no...@collab.net> escribió:
>
> We are planning to use jclouds-chef library to talk to community Chef
> Server 11. In my initial testings, I get the following exception when I
> try to get cookbook details using
>
> api.getCookbook("openssl", "1.0.2")
>
>
> | java.lang.IllegalArgumentException: duplicate key: openssl
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:119)
> at
> com.google.common.collect.RegularImmutableMap.<init>(RegularImmutableMap.java:67)
> at
> com.google.common.collect.ImmutableMap$Builder.fromEntryList(ImmutableMap.java:249)
> at
> com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:235)
> at
> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:313)
> at
> org.jclouds.json.internal.NullFilteringTypeAdapterFactories$MapTypeAdapter.read(NullFilteringTypeAdapterFactories.java:278)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
> at com.google.gson.Gson.fromJson(Gson.java:795)
> at com.google.gson.Gson.fromJson(Gson.java:761)
> at com.google.gson.Gson.fromJson(Gson.java:710)
> at
> org.jclouds.json.internal.GsonWrapper.fromJson(GsonWrapper.java:47)
> at org.jclouds.http.functions.ParseJson.apply(ParseJson.java:86)
>
> README.md says that
>
> It currently supports versions 0.9 and 0.10 of the standard Chef server
> apis, and an initial and very basic (still in progress) implementation
> of the user and organization api of the Hosted and Private Chef
> flavours.
>
> I think that the issue I am facing is because I am using Chef server 11.
> Is there any plans to officially support Chef Server 11 in near future?
>
> Thanks and Regards
> Noorul
>