You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Sergey Beryozkin <sb...@gmail.com> on 2017/02/03 15:22:07 UTC
Camel Azure component
Hi
I've started prototyping a Camel Azure component [1].
Azure has a number of services, Blob, Queue, Table and File, a lot of
commands, etc.
I'm starting with supporting a Blob Service. Queue service will follow,
and I guess the component will keep evolving to support other services too.
At the moment I have most of the Blob Service commands covered at the
Producer side. I'm not sure Consumer will need to be there given that
Azure Java API only offers an option to download the blobs to an output
stream (file most likely). May be only for getting the attributes if
really needed.
I guess Queue Service will though require a Consumer but that is a stage 2.
I'm using camel-aws component as a source of ideas.
The biggest issue it how to test it. I have several basic 'live' in-only
tests based on the copy from camel-aws/s3 which depend on a live
security key and the connection. Microsoft Azure offers an emulator
which can only be run on Windows. And I'm not keen using a mock client
which can become stale and won't really prove the blob has been uploaded.
I'd like to start with having these tests disabled on the master for
those who are interested to experiment with them using their own live
keys and in meantime I reckon we can either try to encourage the
Microsoft Azure team to release their service emulator for the Linux
platform asap or set up a Camel Windows build and run the emulator
integration test on Windows only.
Thanks, Sergey
[1] https://issues.apache.org/jira/browse/CAMEL-10786
Re: Camel Azure component
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi Claus
Thanks for the feedback,
On 14/02/17 08:08, Claus Ibsen wrote:
> Hi Sergey
>
> Good to see the commit and glad to hear you get time to work on Apache
> Camel as well.
>
> This a heads up that there has been a community user doing some Azure
> work on a PR
> https://github.com/apache/camel/pull/1213
>
> However there has not been any JIRA or information about what it was
> doing etc. But maybe you could take a look at the PR and see if it
> does something that our code dont do or something.
Right, I was not aware of it, I looked, at the moment there's no
conflict, as I've started with Azure Blob Service and this PR
concentrates on the Queue Service.
I don't know what to do in this case, I guess I will comment in that PR
and invite the PR authors to initiate a discussion on how to add their
code to the camel-azure, looks like it can go in without the conflict,
may be some coordination will be needed
Example, I started with a
asure-blob://camelazure/container1/blob?someOption
where the account name, the container name and the blob name form the
URI path as opposed to
asure-blob://blob?account=camelazure&container=container1&someOption
as I thought the 1st option was more compact and better conveys the
'must' requirement on the account & container names at least, but it is
a minor style issue...
So I will comment there and lets see what happens
>
> For the azure component does it work in Karaf? Eg is there a karaf
> feature for it, and can the JARs be installed as bundles. If not we
> may consider having the SMX team create OSGi bundles of those JARs.
>
Will investigate yes
> Also the component need to be in the apache-camel kit, which you do in
> the parent and apache-camel folder. Just do a git grep for another
> component name and see how its done for that guy.
>
Sure
Cheers. Sergey
>
>
> On Mon, Feb 13, 2017 at 10:52 PM, Sergey Beryozkin <sb...@gmail.com> wrote:
>> Yes, thanks, keep getting confused about the repos.
>>
>> So I merged now, and now I can feel less worried about boasting being a
>> Camel committer without having done a single commit :-).
>>
>> I've opened two more related issues: investigate what to do about the
>> integration tests and add the docs. My free account expires soon so I'll
>> play a bit more with Azure Blob Service or may be Queue before doing the
>> docs for the blob service and then once I'm done I suppose I will close
>> CAMEL-10786 and the Azure docs issue with more JIRAs to follow as needed
>> (for the queue, etc).
>>
>> Thanks, any concerns about my commit - let me know please
>>
>> Sergey
>>
>>
>> On 13/02/17 17:18, Colm O hEigeartaigh wrote:
>>>
>>> You could try cloning this repo instead:
>>> https://git-wip-us.apache.org/repos/asf/camel.git
>>>
>>> Colm.
>>>
>>> On Mon, Feb 13, 2017 at 4:43 PM, Sergey Beryozkin <sb...@gmail.com>
>>> wrote:
>>>
>>>> Hi Claus, All
>>>>
>>>> Am I actually allowed to commit yet ?
>>>>
>>>> I tried to commit but got an
>>>> "ERROR: Permission to apache/camel.git denied to sberyozkin."
>>>>
>>>> 'git remote show origin' shows I cloned from:
>>>>
>>>> git@github.com:apache/camel.git
>>>>
>>>> Can you please double check the permissions ?
>>>>
>>>> Thanks, Sergey
>>>>
>>>>
>>>> On 10/02/17 14:40, Sergey Beryozkin wrote:
>>>>
>>>>> Hi Claus
>>>>>
>>>>> Thanks, I've opened another issue for investigating on how to enable the
>>>>> Azure (integration) tests
>>>>> but I've also realized I do not have a simpler component/endpoint only
>>>>> test, so I'll work on it first and then I will commit
>>>>>
>>>>> Cheers, Sergey
>>>>> On 10/02/17 12:14, Claus Ibsen wrote:
>>>>>
>>>>>> Hi Sergey
>>>>>>
>>>>>> Yeah sure its good to get this out in the hands of the users. If
>>>>>> something is not yet implemented or something we can always state that
>>>>>> in the docs.
>>>>>>
>>>>>> And if there are things you think still must be done its a good idea
>>>>>> to log a JIRA with that.
>>>>>>
>>>>>> For tests we sometimes have a maven profile that runs real integration
>>>>>> tests against an online account or something, and then regular unit
>>>>>> tests for just simpler tests.
>>>>>>
>>>>>> As we have more and more of these it may be worthwhile to look into a
>>>>>> special jenkins job that only runs these integration tests with those
>>>>>> maven profiles enabled.
>>>>>>
>>>>>> But sure feel free to go ahead and get the code into the master branch.
>>>>>>
>>>>>> On Thu, Feb 9, 2017 at 6:42 PM, Sergey Beryozkin
>>>>>> <sb...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> I've finally attached a patch for the experts to have a glance, I
>>>>>>> know I can
>>>>>>> commit and look forward to doing my first Camel commit, but it is my
>>>>>>> first
>>>>>>> Camel component, so if something obvious is missing - let me know
>>>>>>> please.
>>>>>>>
>>>>>>> It is hard to create a complete functional component from a start but
>>>>>>> I hope
>>>>>>> it will be incrementally enhanced and improved.
>>>>>>> At the moment it covers an Azure Blob Service (with some tweaks likely
>>>>>>> needed later on once we experiment more with it), I'll have a look at
>>>>>>> adding
>>>>>>> a Queue service support later on.
>>>>>>>
>>>>>>> The most obvious thing which proved hard to solve is to have the
>>>>>>> tests not
>>>>>>> ignored - I run all the tests which are there against the live 30-day
>>>>>>> free
>>>>>>> account called 'camelazure' but it is a private account nonetheless.
>>>>>>>
>>>>>>> I spent a lot of time on trying to mock the clients but the Azure SDK
>>>>>>> creators made all of the clients final, so after messing with
>>>>>>> PowerMock I
>>>>>>> thought it was not worth it as it looks it can add some instability
>>>>>>> in the
>>>>>>> running tests. The idea of running the integration tests is also on
>>>>>>> the hold
>>>>>>> given that they do not have a Maven plugin for starting the emulator
>>>>>>> (and no
>>>>>>> emulator for Linux yet).
>>>>>>>
>>>>>>> So I'd like to go ahead with having these tests disabled and let
>>>>>>> those who
>>>>>>> are interested start playing with this component and provide the
>>>>>>> feedback
>>>>>>> until we figure out how to test it in Jenkins...
>>>>>>>
>>>>>>> Claus, what do you think, would you be OK for the commit to go to
>>>>>>> 2.19.0-SNAPSHOT in the current form ?
>>>>>>>
>>>>>>> Cheers, Sergey
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 05/02/17 17:06, Sergey Beryozkin wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Hi Guys,
>>>>>>>>
>>>>>>>> I've looked a bit more carefully at the Azure Emulator docs:
>>>>>>>>
>>>>>>>>
>>>>>>>> https://docs.microsoft.com/en-us/azure/storage/storage-use-e
>>>>>>>> mulator#start-and-initialize-the-storage-emulator
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> and I do not see now how it can be dynamically prepared to get the
>>>>>>>> integration tests running, my apologies I did not check it initially,
>>>>>>>> though if my earlier email would contribute to a Camel Windows build
>>>>>>>> be
>>>>>>>> set up then it would be good :-). I believe Francesco managed to do
>>>>>>>> it
>>>>>>>> quite easily for Syncope.
>>>>>>>>
>>>>>>>> I guess as far as this component is concerned I'd need to start with
>>>>>>>> the
>>>>>>>> basic mock client to get some coverage done.
>>>>>>>>
>>>>>>>> Cheers, Sergey
>>>>>>>>
>>>>>>>> On 05/02/17 12:39, Jean-Baptiste Onofr� wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> +1 for Windows build.
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> JB
>>>>>>>>>
>>>>>>>>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang
>>>>>>>>> <willem.jiang@gmail.com
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> +1 for setting up a Camel Windows build.
>>>>>>>>>> As some of our users are still using windows box.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Willem Jiang
>>>>>>>>>>
>>>>>>>>>> Blog: http://willemjiang.blogspot.com (English)
>>>>>>>>>> http://jnn.iteye.com (Chinese)
>>>>>>>>>> Twitter: willemjiang
>>>>>>>>>> Weibo: \u59dc\u5b81willem
>>>>>>>>>>
>>>>>>>>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>>>>>>>>> <sb...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I've started prototyping a Camel Azure component [1].
>>>>>>>>>>>
>>>>>>>>>>> Azure has a number of services, Blob, Queue, Table and File, a
>>>>>>>>>>> lot of
>>>>>>>>>>> commands, etc.
>>>>>>>>>>> I'm starting with supporting a Blob Service. Queue service will
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> follow,
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> and I guess the component will keep evolving to support other
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> services too.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> At the moment I have most of the Blob Service commands covered at
>>>>>>>>>>> the
>>>>>>>>>>> Producer side. I'm not sure Consumer will need to be there given
>>>>>>>>>>> that
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Azure
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Java API only offers an option to download the blobs to an output
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> stream
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> (file most likely). May be only for getting the attributes if
>>>>>>>>>>> really
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> needed.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I guess Queue Service will though require a Consumer but that is a
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> stage 2.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm using camel-aws component as a source of ideas.
>>>>>>>>>>>
>>>>>>>>>>> The biggest issue it how to test it. I have several basic 'live'
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> in-only
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> tests based on the copy from camel-aws/s3 which depend on a live
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> security
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> key and the connection. Microsoft Azure offers an emulator which
>>>>>>>>>>> can
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> only
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> be run on Windows. And I'm not keen using a mock client which can
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> become
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> stale and won't really prove the blob has been uploaded.
>>>>>>>>>>>
>>>>>>>>>>> I'd like to start with having these tests disabled on the master
>>>>>>>>>>> for
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> those
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> who are interested to experiment with them using their own live
>>>>>>>>>>> keys
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> and in
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> meantime I reckon we can either try to encourage the Microsoft
>>>>>>>>>>> Azure
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> team
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> to release their service emulator for the Linux platform asap or
>>>>>>>>>>> set
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> up a
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Camel Windows build and run the emulator integration test on
>>>>>>>>>>> Windows
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> only.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks, Sergey
>>>>>>>>>>>
>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
>
Re: Camel Azure component
Posted by Claus Ibsen <cl...@gmail.com>.
Hi Sergey
Good to see the commit and glad to hear you get time to work on Apache
Camel as well.
This a heads up that there has been a community user doing some Azure
work on a PR
https://github.com/apache/camel/pull/1213
However there has not been any JIRA or information about what it was
doing etc. But maybe you could take a look at the PR and see if it
does something that our code dont do or something.
For the azure component does it work in Karaf? Eg is there a karaf
feature for it, and can the JARs be installed as bundles. If not we
may consider having the SMX team create OSGi bundles of those JARs.
Also the component need to be in the apache-camel kit, which you do in
the parent and apache-camel folder. Just do a git grep for another
component name and see how its done for that guy.
On Mon, Feb 13, 2017 at 10:52 PM, Sergey Beryozkin <sb...@gmail.com> wrote:
> Yes, thanks, keep getting confused about the repos.
>
> So I merged now, and now I can feel less worried about boasting being a
> Camel committer without having done a single commit :-).
>
> I've opened two more related issues: investigate what to do about the
> integration tests and add the docs. My free account expires soon so I'll
> play a bit more with Azure Blob Service or may be Queue before doing the
> docs for the blob service and then once I'm done I suppose I will close
> CAMEL-10786 and the Azure docs issue with more JIRAs to follow as needed
> (for the queue, etc).
>
> Thanks, any concerns about my commit - let me know please
>
> Sergey
>
>
> On 13/02/17 17:18, Colm O hEigeartaigh wrote:
>>
>> You could try cloning this repo instead:
>> https://git-wip-us.apache.org/repos/asf/camel.git
>>
>> Colm.
>>
>> On Mon, Feb 13, 2017 at 4:43 PM, Sergey Beryozkin <sb...@gmail.com>
>> wrote:
>>
>>> Hi Claus, All
>>>
>>> Am I actually allowed to commit yet ?
>>>
>>> I tried to commit but got an
>>> "ERROR: Permission to apache/camel.git denied to sberyozkin."
>>>
>>> 'git remote show origin' shows I cloned from:
>>>
>>> git@github.com:apache/camel.git
>>>
>>> Can you please double check the permissions ?
>>>
>>> Thanks, Sergey
>>>
>>>
>>> On 10/02/17 14:40, Sergey Beryozkin wrote:
>>>
>>>> Hi Claus
>>>>
>>>> Thanks, I've opened another issue for investigating on how to enable the
>>>> Azure (integration) tests
>>>> but I've also realized I do not have a simpler component/endpoint only
>>>> test, so I'll work on it first and then I will commit
>>>>
>>>> Cheers, Sergey
>>>> On 10/02/17 12:14, Claus Ibsen wrote:
>>>>
>>>>> Hi Sergey
>>>>>
>>>>> Yeah sure its good to get this out in the hands of the users. If
>>>>> something is not yet implemented or something we can always state that
>>>>> in the docs.
>>>>>
>>>>> And if there are things you think still must be done its a good idea
>>>>> to log a JIRA with that.
>>>>>
>>>>> For tests we sometimes have a maven profile that runs real integration
>>>>> tests against an online account or something, and then regular unit
>>>>> tests for just simpler tests.
>>>>>
>>>>> As we have more and more of these it may be worthwhile to look into a
>>>>> special jenkins job that only runs these integration tests with those
>>>>> maven profiles enabled.
>>>>>
>>>>> But sure feel free to go ahead and get the code into the master branch.
>>>>>
>>>>> On Thu, Feb 9, 2017 at 6:42 PM, Sergey Beryozkin
>>>>> <sb...@gmail.com> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> I've finally attached a patch for the experts to have a glance, I
>>>>>> know I can
>>>>>> commit and look forward to doing my first Camel commit, but it is my
>>>>>> first
>>>>>> Camel component, so if something obvious is missing - let me know
>>>>>> please.
>>>>>>
>>>>>> It is hard to create a complete functional component from a start but
>>>>>> I hope
>>>>>> it will be incrementally enhanced and improved.
>>>>>> At the moment it covers an Azure Blob Service (with some tweaks likely
>>>>>> needed later on once we experiment more with it), I'll have a look at
>>>>>> adding
>>>>>> a Queue service support later on.
>>>>>>
>>>>>> The most obvious thing which proved hard to solve is to have the
>>>>>> tests not
>>>>>> ignored - I run all the tests which are there against the live 30-day
>>>>>> free
>>>>>> account called 'camelazure' but it is a private account nonetheless.
>>>>>>
>>>>>> I spent a lot of time on trying to mock the clients but the Azure SDK
>>>>>> creators made all of the clients final, so after messing with
>>>>>> PowerMock I
>>>>>> thought it was not worth it as it looks it can add some instability
>>>>>> in the
>>>>>> running tests. The idea of running the integration tests is also on
>>>>>> the hold
>>>>>> given that they do not have a Maven plugin for starting the emulator
>>>>>> (and no
>>>>>> emulator for Linux yet).
>>>>>>
>>>>>> So I'd like to go ahead with having these tests disabled and let
>>>>>> those who
>>>>>> are interested start playing with this component and provide the
>>>>>> feedback
>>>>>> until we figure out how to test it in Jenkins...
>>>>>>
>>>>>> Claus, what do you think, would you be OK for the commit to go to
>>>>>> 2.19.0-SNAPSHOT in the current form ?
>>>>>>
>>>>>> Cheers, Sergey
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 05/02/17 17:06, Sergey Beryozkin wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi Guys,
>>>>>>>
>>>>>>> I've looked a bit more carefully at the Azure Emulator docs:
>>>>>>>
>>>>>>>
>>>>>>> https://docs.microsoft.com/en-us/azure/storage/storage-use-e
>>>>>>> mulator#start-and-initialize-the-storage-emulator
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> and I do not see now how it can be dynamically prepared to get the
>>>>>>> integration tests running, my apologies I did not check it initially,
>>>>>>> though if my earlier email would contribute to a Camel Windows build
>>>>>>> be
>>>>>>> set up then it would be good :-). I believe Francesco managed to do
>>>>>>> it
>>>>>>> quite easily for Syncope.
>>>>>>>
>>>>>>> I guess as far as this component is concerned I'd need to start with
>>>>>>> the
>>>>>>> basic mock client to get some coverage done.
>>>>>>>
>>>>>>> Cheers, Sergey
>>>>>>>
>>>>>>> On 05/02/17 12:39, Jean-Baptiste Onofré wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> +1 for Windows build.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> JB
>>>>>>>>
>>>>>>>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang
>>>>>>>> <willem.jiang@gmail.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> +1 for setting up a Camel Windows build.
>>>>>>>>> As some of our users are still using windows box.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Willem Jiang
>>>>>>>>>
>>>>>>>>> Blog: http://willemjiang.blogspot.com (English)
>>>>>>>>> http://jnn.iteye.com (Chinese)
>>>>>>>>> Twitter: willemjiang
>>>>>>>>> Weibo: 姜宁willem
>>>>>>>>>
>>>>>>>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>>>>>>>> <sb...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I've started prototyping a Camel Azure component [1].
>>>>>>>>>>
>>>>>>>>>> Azure has a number of services, Blob, Queue, Table and File, a
>>>>>>>>>> lot of
>>>>>>>>>> commands, etc.
>>>>>>>>>> I'm starting with supporting a Blob Service. Queue service will
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> follow,
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> and I guess the component will keep evolving to support other
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> services too.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> At the moment I have most of the Blob Service commands covered at
>>>>>>>>>> the
>>>>>>>>>> Producer side. I'm not sure Consumer will need to be there given
>>>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Azure
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Java API only offers an option to download the blobs to an output
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> stream
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> (file most likely). May be only for getting the attributes if
>>>>>>>>>> really
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> needed.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I guess Queue Service will though require a Consumer but that is a
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> stage 2.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I'm using camel-aws component as a source of ideas.
>>>>>>>>>>
>>>>>>>>>> The biggest issue it how to test it. I have several basic 'live'
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> in-only
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> tests based on the copy from camel-aws/s3 which depend on a live
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> security
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> key and the connection. Microsoft Azure offers an emulator which
>>>>>>>>>> can
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> only
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> be run on Windows. And I'm not keen using a mock client which can
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> become
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> stale and won't really prove the blob has been uploaded.
>>>>>>>>>>
>>>>>>>>>> I'd like to start with having these tests disabled on the master
>>>>>>>>>> for
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> those
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> who are interested to experiment with them using their own live
>>>>>>>>>> keys
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> and in
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> meantime I reckon we can either try to encourage the Microsoft
>>>>>>>>>> Azure
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> team
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> to release their service emulator for the Linux platform asap or
>>>>>>>>>> set
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> up a
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Camel Windows build and run the emulator integration test on
>>>>>>>>>> Windows
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> only.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks, Sergey
>>>>>>>>>>
>>>>>>>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Re: Camel Azure component
Posted by Sergey Beryozkin <sb...@gmail.com>.
Yes, thanks, keep getting confused about the repos.
So I merged now, and now I can feel less worried about boasting being a
Camel committer without having done a single commit :-).
I've opened two more related issues: investigate what to do about the
integration tests and add the docs. My free account expires soon so I'll
play a bit more with Azure Blob Service or may be Queue before doing the
docs for the blob service and then once I'm done I suppose I will close
CAMEL-10786 and the Azure docs issue with more JIRAs to follow as needed
(for the queue, etc).
Thanks, any concerns about my commit - let me know please
Sergey
On 13/02/17 17:18, Colm O hEigeartaigh wrote:
> You could try cloning this repo instead:
> https://git-wip-us.apache.org/repos/asf/camel.git
>
> Colm.
>
> On Mon, Feb 13, 2017 at 4:43 PM, Sergey Beryozkin <sb...@gmail.com>
> wrote:
>
>> Hi Claus, All
>>
>> Am I actually allowed to commit yet ?
>>
>> I tried to commit but got an
>> "ERROR: Permission to apache/camel.git denied to sberyozkin."
>>
>> 'git remote show origin' shows I cloned from:
>>
>> git@github.com:apache/camel.git
>>
>> Can you please double check the permissions ?
>>
>> Thanks, Sergey
>>
>>
>> On 10/02/17 14:40, Sergey Beryozkin wrote:
>>
>>> Hi Claus
>>>
>>> Thanks, I've opened another issue for investigating on how to enable the
>>> Azure (integration) tests
>>> but I've also realized I do not have a simpler component/endpoint only
>>> test, so I'll work on it first and then I will commit
>>>
>>> Cheers, Sergey
>>> On 10/02/17 12:14, Claus Ibsen wrote:
>>>
>>>> Hi Sergey
>>>>
>>>> Yeah sure its good to get this out in the hands of the users. If
>>>> something is not yet implemented or something we can always state that
>>>> in the docs.
>>>>
>>>> And if there are things you think still must be done its a good idea
>>>> to log a JIRA with that.
>>>>
>>>> For tests we sometimes have a maven profile that runs real integration
>>>> tests against an online account or something, and then regular unit
>>>> tests for just simpler tests.
>>>>
>>>> As we have more and more of these it may be worthwhile to look into a
>>>> special jenkins job that only runs these integration tests with those
>>>> maven profiles enabled.
>>>>
>>>> But sure feel free to go ahead and get the code into the master branch.
>>>>
>>>> On Thu, Feb 9, 2017 at 6:42 PM, Sergey Beryozkin
>>>> <sb...@gmail.com> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> I've finally attached a patch for the experts to have a glance, I
>>>>> know I can
>>>>> commit and look forward to doing my first Camel commit, but it is my
>>>>> first
>>>>> Camel component, so if something obvious is missing - let me know
>>>>> please.
>>>>>
>>>>> It is hard to create a complete functional component from a start but
>>>>> I hope
>>>>> it will be incrementally enhanced and improved.
>>>>> At the moment it covers an Azure Blob Service (with some tweaks likely
>>>>> needed later on once we experiment more with it), I'll have a look at
>>>>> adding
>>>>> a Queue service support later on.
>>>>>
>>>>> The most obvious thing which proved hard to solve is to have the
>>>>> tests not
>>>>> ignored - I run all the tests which are there against the live 30-day
>>>>> free
>>>>> account called 'camelazure' but it is a private account nonetheless.
>>>>>
>>>>> I spent a lot of time on trying to mock the clients but the Azure SDK
>>>>> creators made all of the clients final, so after messing with
>>>>> PowerMock I
>>>>> thought it was not worth it as it looks it can add some instability
>>>>> in the
>>>>> running tests. The idea of running the integration tests is also on
>>>>> the hold
>>>>> given that they do not have a Maven plugin for starting the emulator
>>>>> (and no
>>>>> emulator for Linux yet).
>>>>>
>>>>> So I'd like to go ahead with having these tests disabled and let
>>>>> those who
>>>>> are interested start playing with this component and provide the
>>>>> feedback
>>>>> until we figure out how to test it in Jenkins...
>>>>>
>>>>> Claus, what do you think, would you be OK for the commit to go to
>>>>> 2.19.0-SNAPSHOT in the current form ?
>>>>>
>>>>> Cheers, Sergey
>>>>>
>>>>>
>>>>>
>>>>> On 05/02/17 17:06, Sergey Beryozkin wrote:
>>>>>
>>>>>>
>>>>>> Hi Guys,
>>>>>>
>>>>>> I've looked a bit more carefully at the Azure Emulator docs:
>>>>>>
>>>>>>
>>>>>> https://docs.microsoft.com/en-us/azure/storage/storage-use-e
>>>>>> mulator#start-and-initialize-the-storage-emulator
>>>>>>
>>>>>>
>>>>>>
>>>>>> and I do not see now how it can be dynamically prepared to get the
>>>>>> integration tests running, my apologies I did not check it initially,
>>>>>> though if my earlier email would contribute to a Camel Windows build be
>>>>>> set up then it would be good :-). I believe Francesco managed to do it
>>>>>> quite easily for Syncope.
>>>>>>
>>>>>> I guess as far as this component is concerned I'd need to start with
>>>>>> the
>>>>>> basic mock client to get some coverage done.
>>>>>>
>>>>>> Cheers, Sergey
>>>>>>
>>>>>> On 05/02/17 12:39, Jean-Baptiste Onofr� wrote:
>>>>>>
>>>>>>>
>>>>>>> +1 for Windows build.
>>>>>>>
>>>>>>> Regards
>>>>>>> JB
>>>>>>>
>>>>>>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang <willem.jiang@gmail.com
>>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> +1 for setting up a Camel Windows build.
>>>>>>>> As some of our users are still using windows box.
>>>>>>>>
>>>>>>>>
>>>>>>>> Willem Jiang
>>>>>>>>
>>>>>>>> Blog: http://willemjiang.blogspot.com (English)
>>>>>>>> http://jnn.iteye.com (Chinese)
>>>>>>>> Twitter: willemjiang
>>>>>>>> Weibo: \u59dc\u5b81willem
>>>>>>>>
>>>>>>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>>>>>>> <sb...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> I've started prototyping a Camel Azure component [1].
>>>>>>>>>
>>>>>>>>> Azure has a number of services, Blob, Queue, Table and File, a
>>>>>>>>> lot of
>>>>>>>>> commands, etc.
>>>>>>>>> I'm starting with supporting a Blob Service. Queue service will
>>>>>>>>>
>>>>>>>>
>>>>>>>> follow,
>>>>>>>>
>>>>>>>>>
>>>>>>>>> and I guess the component will keep evolving to support other
>>>>>>>>>
>>>>>>>>
>>>>>>>> services too.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> At the moment I have most of the Blob Service commands covered at
>>>>>>>>> the
>>>>>>>>> Producer side. I'm not sure Consumer will need to be there given
>>>>>>>>> that
>>>>>>>>>
>>>>>>>>
>>>>>>>> Azure
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Java API only offers an option to download the blobs to an output
>>>>>>>>>
>>>>>>>>
>>>>>>>> stream
>>>>>>>>
>>>>>>>>>
>>>>>>>>> (file most likely). May be only for getting the attributes if really
>>>>>>>>>
>>>>>>>>
>>>>>>>> needed.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I guess Queue Service will though require a Consumer but that is a
>>>>>>>>>
>>>>>>>>
>>>>>>>> stage 2.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm using camel-aws component as a source of ideas.
>>>>>>>>>
>>>>>>>>> The biggest issue it how to test it. I have several basic 'live'
>>>>>>>>>
>>>>>>>>
>>>>>>>> in-only
>>>>>>>>
>>>>>>>>>
>>>>>>>>> tests based on the copy from camel-aws/s3 which depend on a live
>>>>>>>>>
>>>>>>>>
>>>>>>>> security
>>>>>>>>
>>>>>>>>>
>>>>>>>>> key and the connection. Microsoft Azure offers an emulator which can
>>>>>>>>>
>>>>>>>>
>>>>>>>> only
>>>>>>>>
>>>>>>>>>
>>>>>>>>> be run on Windows. And I'm not keen using a mock client which can
>>>>>>>>>
>>>>>>>>
>>>>>>>> become
>>>>>>>>
>>>>>>>>>
>>>>>>>>> stale and won't really prove the blob has been uploaded.
>>>>>>>>>
>>>>>>>>> I'd like to start with having these tests disabled on the master for
>>>>>>>>>
>>>>>>>>
>>>>>>>> those
>>>>>>>>
>>>>>>>>>
>>>>>>>>> who are interested to experiment with them using their own live keys
>>>>>>>>>
>>>>>>>>
>>>>>>>> and in
>>>>>>>>
>>>>>>>>>
>>>>>>>>> meantime I reckon we can either try to encourage the Microsoft Azure
>>>>>>>>>
>>>>>>>>
>>>>>>>> team
>>>>>>>>
>>>>>>>>>
>>>>>>>>> to release their service emulator for the Linux platform asap or set
>>>>>>>>>
>>>>>>>>
>>>>>>>> up a
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Camel Windows build and run the emulator integration test on Windows
>>>>>>>>>
>>>>>>>>
>>>>>>>> only.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks, Sergey
>>>>>>>>>
>>>>>>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>
>
Re: Camel Azure component
Posted by Colm O hEigeartaigh <co...@apache.org>.
You could try cloning this repo instead:
https://git-wip-us.apache.org/repos/asf/camel.git
Colm.
On Mon, Feb 13, 2017 at 4:43 PM, Sergey Beryozkin <sb...@gmail.com>
wrote:
> Hi Claus, All
>
> Am I actually allowed to commit yet ?
>
> I tried to commit but got an
> "ERROR: Permission to apache/camel.git denied to sberyozkin."
>
> 'git remote show origin' shows I cloned from:
>
> git@github.com:apache/camel.git
>
> Can you please double check the permissions ?
>
> Thanks, Sergey
>
>
> On 10/02/17 14:40, Sergey Beryozkin wrote:
>
>> Hi Claus
>>
>> Thanks, I've opened another issue for investigating on how to enable the
>> Azure (integration) tests
>> but I've also realized I do not have a simpler component/endpoint only
>> test, so I'll work on it first and then I will commit
>>
>> Cheers, Sergey
>> On 10/02/17 12:14, Claus Ibsen wrote:
>>
>>> Hi Sergey
>>>
>>> Yeah sure its good to get this out in the hands of the users. If
>>> something is not yet implemented or something we can always state that
>>> in the docs.
>>>
>>> And if there are things you think still must be done its a good idea
>>> to log a JIRA with that.
>>>
>>> For tests we sometimes have a maven profile that runs real integration
>>> tests against an online account or something, and then regular unit
>>> tests for just simpler tests.
>>>
>>> As we have more and more of these it may be worthwhile to look into a
>>> special jenkins job that only runs these integration tests with those
>>> maven profiles enabled.
>>>
>>> But sure feel free to go ahead and get the code into the master branch.
>>>
>>> On Thu, Feb 9, 2017 at 6:42 PM, Sergey Beryozkin
>>> <sb...@gmail.com> wrote:
>>>
>>>> Hi
>>>>
>>>> I've finally attached a patch for the experts to have a glance, I
>>>> know I can
>>>> commit and look forward to doing my first Camel commit, but it is my
>>>> first
>>>> Camel component, so if something obvious is missing - let me know
>>>> please.
>>>>
>>>> It is hard to create a complete functional component from a start but
>>>> I hope
>>>> it will be incrementally enhanced and improved.
>>>> At the moment it covers an Azure Blob Service (with some tweaks likely
>>>> needed later on once we experiment more with it), I'll have a look at
>>>> adding
>>>> a Queue service support later on.
>>>>
>>>> The most obvious thing which proved hard to solve is to have the
>>>> tests not
>>>> ignored - I run all the tests which are there against the live 30-day
>>>> free
>>>> account called 'camelazure' but it is a private account nonetheless.
>>>>
>>>> I spent a lot of time on trying to mock the clients but the Azure SDK
>>>> creators made all of the clients final, so after messing with
>>>> PowerMock I
>>>> thought it was not worth it as it looks it can add some instability
>>>> in the
>>>> running tests. The idea of running the integration tests is also on
>>>> the hold
>>>> given that they do not have a Maven plugin for starting the emulator
>>>> (and no
>>>> emulator for Linux yet).
>>>>
>>>> So I'd like to go ahead with having these tests disabled and let
>>>> those who
>>>> are interested start playing with this component and provide the
>>>> feedback
>>>> until we figure out how to test it in Jenkins...
>>>>
>>>> Claus, what do you think, would you be OK for the commit to go to
>>>> 2.19.0-SNAPSHOT in the current form ?
>>>>
>>>> Cheers, Sergey
>>>>
>>>>
>>>>
>>>> On 05/02/17 17:06, Sergey Beryozkin wrote:
>>>>
>>>>>
>>>>> Hi Guys,
>>>>>
>>>>> I've looked a bit more carefully at the Azure Emulator docs:
>>>>>
>>>>>
>>>>> https://docs.microsoft.com/en-us/azure/storage/storage-use-e
>>>>> mulator#start-and-initialize-the-storage-emulator
>>>>>
>>>>>
>>>>>
>>>>> and I do not see now how it can be dynamically prepared to get the
>>>>> integration tests running, my apologies I did not check it initially,
>>>>> though if my earlier email would contribute to a Camel Windows build be
>>>>> set up then it would be good :-). I believe Francesco managed to do it
>>>>> quite easily for Syncope.
>>>>>
>>>>> I guess as far as this component is concerned I'd need to start with
>>>>> the
>>>>> basic mock client to get some coverage done.
>>>>>
>>>>> Cheers, Sergey
>>>>>
>>>>> On 05/02/17 12:39, Jean-Baptiste Onofré wrote:
>>>>>
>>>>>>
>>>>>> +1 for Windows build.
>>>>>>
>>>>>> Regards
>>>>>> JB
>>>>>>
>>>>>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang <willem.jiang@gmail.com
>>>>>> >
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> +1 for setting up a Camel Windows build.
>>>>>>> As some of our users are still using windows box.
>>>>>>>
>>>>>>>
>>>>>>> Willem Jiang
>>>>>>>
>>>>>>> Blog: http://willemjiang.blogspot.com (English)
>>>>>>> http://jnn.iteye.com (Chinese)
>>>>>>> Twitter: willemjiang
>>>>>>> Weibo: 姜宁willem
>>>>>>>
>>>>>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>>>>>> <sb...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi
>>>>>>>>
>>>>>>>> I've started prototyping a Camel Azure component [1].
>>>>>>>>
>>>>>>>> Azure has a number of services, Blob, Queue, Table and File, a
>>>>>>>> lot of
>>>>>>>> commands, etc.
>>>>>>>> I'm starting with supporting a Blob Service. Queue service will
>>>>>>>>
>>>>>>>
>>>>>>> follow,
>>>>>>>
>>>>>>>>
>>>>>>>> and I guess the component will keep evolving to support other
>>>>>>>>
>>>>>>>
>>>>>>> services too.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> At the moment I have most of the Blob Service commands covered at
>>>>>>>> the
>>>>>>>> Producer side. I'm not sure Consumer will need to be there given
>>>>>>>> that
>>>>>>>>
>>>>>>>
>>>>>>> Azure
>>>>>>>
>>>>>>>>
>>>>>>>> Java API only offers an option to download the blobs to an output
>>>>>>>>
>>>>>>>
>>>>>>> stream
>>>>>>>
>>>>>>>>
>>>>>>>> (file most likely). May be only for getting the attributes if really
>>>>>>>>
>>>>>>>
>>>>>>> needed.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I guess Queue Service will though require a Consumer but that is a
>>>>>>>>
>>>>>>>
>>>>>>> stage 2.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I'm using camel-aws component as a source of ideas.
>>>>>>>>
>>>>>>>> The biggest issue it how to test it. I have several basic 'live'
>>>>>>>>
>>>>>>>
>>>>>>> in-only
>>>>>>>
>>>>>>>>
>>>>>>>> tests based on the copy from camel-aws/s3 which depend on a live
>>>>>>>>
>>>>>>>
>>>>>>> security
>>>>>>>
>>>>>>>>
>>>>>>>> key and the connection. Microsoft Azure offers an emulator which can
>>>>>>>>
>>>>>>>
>>>>>>> only
>>>>>>>
>>>>>>>>
>>>>>>>> be run on Windows. And I'm not keen using a mock client which can
>>>>>>>>
>>>>>>>
>>>>>>> become
>>>>>>>
>>>>>>>>
>>>>>>>> stale and won't really prove the blob has been uploaded.
>>>>>>>>
>>>>>>>> I'd like to start with having these tests disabled on the master for
>>>>>>>>
>>>>>>>
>>>>>>> those
>>>>>>>
>>>>>>>>
>>>>>>>> who are interested to experiment with them using their own live keys
>>>>>>>>
>>>>>>>
>>>>>>> and in
>>>>>>>
>>>>>>>>
>>>>>>>> meantime I reckon we can either try to encourage the Microsoft Azure
>>>>>>>>
>>>>>>>
>>>>>>> team
>>>>>>>
>>>>>>>>
>>>>>>>> to release their service emulator for the Linux platform asap or set
>>>>>>>>
>>>>>>>
>>>>>>> up a
>>>>>>>
>>>>>>>>
>>>>>>>> Camel Windows build and run the emulator integration test on Windows
>>>>>>>>
>>>>>>>
>>>>>>> only.
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks, Sergey
>>>>>>>>
>>>>>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>
>
--
Colm O hEigeartaigh
Talend Community Coder
http://coders.talend.com
Re: Camel Azure component
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi Claus, All
Am I actually allowed to commit yet ?
I tried to commit but got an
"ERROR: Permission to apache/camel.git denied to sberyozkin."
'git remote show origin' shows I cloned from:
git@github.com:apache/camel.git
Can you please double check the permissions ?
Thanks, Sergey
On 10/02/17 14:40, Sergey Beryozkin wrote:
> Hi Claus
>
> Thanks, I've opened another issue for investigating on how to enable the
> Azure (integration) tests
> but I've also realized I do not have a simpler component/endpoint only
> test, so I'll work on it first and then I will commit
>
> Cheers, Sergey
> On 10/02/17 12:14, Claus Ibsen wrote:
>> Hi Sergey
>>
>> Yeah sure its good to get this out in the hands of the users. If
>> something is not yet implemented or something we can always state that
>> in the docs.
>>
>> And if there are things you think still must be done its a good idea
>> to log a JIRA with that.
>>
>> For tests we sometimes have a maven profile that runs real integration
>> tests against an online account or something, and then regular unit
>> tests for just simpler tests.
>>
>> As we have more and more of these it may be worthwhile to look into a
>> special jenkins job that only runs these integration tests with those
>> maven profiles enabled.
>>
>> But sure feel free to go ahead and get the code into the master branch.
>>
>> On Thu, Feb 9, 2017 at 6:42 PM, Sergey Beryozkin
>> <sb...@gmail.com> wrote:
>>> Hi
>>>
>>> I've finally attached a patch for the experts to have a glance, I
>>> know I can
>>> commit and look forward to doing my first Camel commit, but it is my
>>> first
>>> Camel component, so if something obvious is missing - let me know
>>> please.
>>>
>>> It is hard to create a complete functional component from a start but
>>> I hope
>>> it will be incrementally enhanced and improved.
>>> At the moment it covers an Azure Blob Service (with some tweaks likely
>>> needed later on once we experiment more with it), I'll have a look at
>>> adding
>>> a Queue service support later on.
>>>
>>> The most obvious thing which proved hard to solve is to have the
>>> tests not
>>> ignored - I run all the tests which are there against the live 30-day
>>> free
>>> account called 'camelazure' but it is a private account nonetheless.
>>>
>>> I spent a lot of time on trying to mock the clients but the Azure SDK
>>> creators made all of the clients final, so after messing with
>>> PowerMock I
>>> thought it was not worth it as it looks it can add some instability
>>> in the
>>> running tests. The idea of running the integration tests is also on
>>> the hold
>>> given that they do not have a Maven plugin for starting the emulator
>>> (and no
>>> emulator for Linux yet).
>>>
>>> So I'd like to go ahead with having these tests disabled and let
>>> those who
>>> are interested start playing with this component and provide the
>>> feedback
>>> until we figure out how to test it in Jenkins...
>>>
>>> Claus, what do you think, would you be OK for the commit to go to
>>> 2.19.0-SNAPSHOT in the current form ?
>>>
>>> Cheers, Sergey
>>>
>>>
>>>
>>> On 05/02/17 17:06, Sergey Beryozkin wrote:
>>>>
>>>> Hi Guys,
>>>>
>>>> I've looked a bit more carefully at the Azure Emulator docs:
>>>>
>>>>
>>>> https://docs.microsoft.com/en-us/azure/storage/storage-use-emulator#start-and-initialize-the-storage-emulator
>>>>
>>>>
>>>>
>>>> and I do not see now how it can be dynamically prepared to get the
>>>> integration tests running, my apologies I did not check it initially,
>>>> though if my earlier email would contribute to a Camel Windows build be
>>>> set up then it would be good :-). I believe Francesco managed to do it
>>>> quite easily for Syncope.
>>>>
>>>> I guess as far as this component is concerned I'd need to start with
>>>> the
>>>> basic mock client to get some coverage done.
>>>>
>>>> Cheers, Sergey
>>>>
>>>> On 05/02/17 12:39, Jean-Baptiste Onofr� wrote:
>>>>>
>>>>> +1 for Windows build.
>>>>>
>>>>> Regards
>>>>> JB
>>>>>
>>>>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang <wi...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> +1 for setting up a Camel Windows build.
>>>>>> As some of our users are still using windows box.
>>>>>>
>>>>>>
>>>>>> Willem Jiang
>>>>>>
>>>>>> Blog: http://willemjiang.blogspot.com (English)
>>>>>> http://jnn.iteye.com (Chinese)
>>>>>> Twitter: willemjiang
>>>>>> Weibo: \u59dc\u5b81willem
>>>>>>
>>>>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>>>>> <sb...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> I've started prototyping a Camel Azure component [1].
>>>>>>>
>>>>>>> Azure has a number of services, Blob, Queue, Table and File, a
>>>>>>> lot of
>>>>>>> commands, etc.
>>>>>>> I'm starting with supporting a Blob Service. Queue service will
>>>>>>
>>>>>> follow,
>>>>>>>
>>>>>>> and I guess the component will keep evolving to support other
>>>>>>
>>>>>> services too.
>>>>>>>
>>>>>>>
>>>>>>> At the moment I have most of the Blob Service commands covered at
>>>>>>> the
>>>>>>> Producer side. I'm not sure Consumer will need to be there given
>>>>>>> that
>>>>>>
>>>>>> Azure
>>>>>>>
>>>>>>> Java API only offers an option to download the blobs to an output
>>>>>>
>>>>>> stream
>>>>>>>
>>>>>>> (file most likely). May be only for getting the attributes if really
>>>>>>
>>>>>> needed.
>>>>>>>
>>>>>>>
>>>>>>> I guess Queue Service will though require a Consumer but that is a
>>>>>>
>>>>>> stage 2.
>>>>>>>
>>>>>>>
>>>>>>> I'm using camel-aws component as a source of ideas.
>>>>>>>
>>>>>>> The biggest issue it how to test it. I have several basic 'live'
>>>>>>
>>>>>> in-only
>>>>>>>
>>>>>>> tests based on the copy from camel-aws/s3 which depend on a live
>>>>>>
>>>>>> security
>>>>>>>
>>>>>>> key and the connection. Microsoft Azure offers an emulator which can
>>>>>>
>>>>>> only
>>>>>>>
>>>>>>> be run on Windows. And I'm not keen using a mock client which can
>>>>>>
>>>>>> become
>>>>>>>
>>>>>>> stale and won't really prove the blob has been uploaded.
>>>>>>>
>>>>>>> I'd like to start with having these tests disabled on the master for
>>>>>>
>>>>>> those
>>>>>>>
>>>>>>> who are interested to experiment with them using their own live keys
>>>>>>
>>>>>> and in
>>>>>>>
>>>>>>> meantime I reckon we can either try to encourage the Microsoft Azure
>>>>>>
>>>>>> team
>>>>>>>
>>>>>>> to release their service emulator for the Linux platform asap or set
>>>>>>
>>>>>> up a
>>>>>>>
>>>>>>> Camel Windows build and run the emulator integration test on Windows
>>>>>>
>>>>>> only.
>>>>>>>
>>>>>>>
>>>>>>> Thanks, Sergey
>>>>>>>
>>>>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>>>>
>>>>>
>>>>
>>>
>>
>>
>>
>
Re: Camel Azure component
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi Claus
Thanks, I've opened another issue for investigating on how to enable the
Azure (integration) tests
but I've also realized I do not have a simpler component/endpoint only
test, so I'll work on it first and then I will commit
Cheers, Sergey
On 10/02/17 12:14, Claus Ibsen wrote:
> Hi Sergey
>
> Yeah sure its good to get this out in the hands of the users. If
> something is not yet implemented or something we can always state that
> in the docs.
>
> And if there are things you think still must be done its a good idea
> to log a JIRA with that.
>
> For tests we sometimes have a maven profile that runs real integration
> tests against an online account or something, and then regular unit
> tests for just simpler tests.
>
> As we have more and more of these it may be worthwhile to look into a
> special jenkins job that only runs these integration tests with those
> maven profiles enabled.
>
> But sure feel free to go ahead and get the code into the master branch.
>
> On Thu, Feb 9, 2017 at 6:42 PM, Sergey Beryozkin <sb...@gmail.com> wrote:
>> Hi
>>
>> I've finally attached a patch for the experts to have a glance, I know I can
>> commit and look forward to doing my first Camel commit, but it is my first
>> Camel component, so if something obvious is missing - let me know please.
>>
>> It is hard to create a complete functional component from a start but I hope
>> it will be incrementally enhanced and improved.
>> At the moment it covers an Azure Blob Service (with some tweaks likely
>> needed later on once we experiment more with it), I'll have a look at adding
>> a Queue service support later on.
>>
>> The most obvious thing which proved hard to solve is to have the tests not
>> ignored - I run all the tests which are there against the live 30-day free
>> account called 'camelazure' but it is a private account nonetheless.
>>
>> I spent a lot of time on trying to mock the clients but the Azure SDK
>> creators made all of the clients final, so after messing with PowerMock I
>> thought it was not worth it as it looks it can add some instability in the
>> running tests. The idea of running the integration tests is also on the hold
>> given that they do not have a Maven plugin for starting the emulator (and no
>> emulator for Linux yet).
>>
>> So I'd like to go ahead with having these tests disabled and let those who
>> are interested start playing with this component and provide the feedback
>> until we figure out how to test it in Jenkins...
>>
>> Claus, what do you think, would you be OK for the commit to go to
>> 2.19.0-SNAPSHOT in the current form ?
>>
>> Cheers, Sergey
>>
>>
>>
>> On 05/02/17 17:06, Sergey Beryozkin wrote:
>>>
>>> Hi Guys,
>>>
>>> I've looked a bit more carefully at the Azure Emulator docs:
>>>
>>>
>>> https://docs.microsoft.com/en-us/azure/storage/storage-use-emulator#start-and-initialize-the-storage-emulator
>>>
>>>
>>> and I do not see now how it can be dynamically prepared to get the
>>> integration tests running, my apologies I did not check it initially,
>>> though if my earlier email would contribute to a Camel Windows build be
>>> set up then it would be good :-). I believe Francesco managed to do it
>>> quite easily for Syncope.
>>>
>>> I guess as far as this component is concerned I'd need to start with the
>>> basic mock client to get some coverage done.
>>>
>>> Cheers, Sergey
>>>
>>> On 05/02/17 12:39, Jean-Baptiste Onofr� wrote:
>>>>
>>>> +1 for Windows build.
>>>>
>>>> Regards
>>>> JB
>>>>
>>>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang <wi...@gmail.com>
>>>> wrote:
>>>>>
>>>>> +1 for setting up a Camel Windows build.
>>>>> As some of our users are still using windows box.
>>>>>
>>>>>
>>>>> Willem Jiang
>>>>>
>>>>> Blog: http://willemjiang.blogspot.com (English)
>>>>> http://jnn.iteye.com (Chinese)
>>>>> Twitter: willemjiang
>>>>> Weibo: \u59dc\u5b81willem
>>>>>
>>>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>>>> <sb...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> I've started prototyping a Camel Azure component [1].
>>>>>>
>>>>>> Azure has a number of services, Blob, Queue, Table and File, a lot of
>>>>>> commands, etc.
>>>>>> I'm starting with supporting a Blob Service. Queue service will
>>>>>
>>>>> follow,
>>>>>>
>>>>>> and I guess the component will keep evolving to support other
>>>>>
>>>>> services too.
>>>>>>
>>>>>>
>>>>>> At the moment I have most of the Blob Service commands covered at the
>>>>>> Producer side. I'm not sure Consumer will need to be there given that
>>>>>
>>>>> Azure
>>>>>>
>>>>>> Java API only offers an option to download the blobs to an output
>>>>>
>>>>> stream
>>>>>>
>>>>>> (file most likely). May be only for getting the attributes if really
>>>>>
>>>>> needed.
>>>>>>
>>>>>>
>>>>>> I guess Queue Service will though require a Consumer but that is a
>>>>>
>>>>> stage 2.
>>>>>>
>>>>>>
>>>>>> I'm using camel-aws component as a source of ideas.
>>>>>>
>>>>>> The biggest issue it how to test it. I have several basic 'live'
>>>>>
>>>>> in-only
>>>>>>
>>>>>> tests based on the copy from camel-aws/s3 which depend on a live
>>>>>
>>>>> security
>>>>>>
>>>>>> key and the connection. Microsoft Azure offers an emulator which can
>>>>>
>>>>> only
>>>>>>
>>>>>> be run on Windows. And I'm not keen using a mock client which can
>>>>>
>>>>> become
>>>>>>
>>>>>> stale and won't really prove the blob has been uploaded.
>>>>>>
>>>>>> I'd like to start with having these tests disabled on the master for
>>>>>
>>>>> those
>>>>>>
>>>>>> who are interested to experiment with them using their own live keys
>>>>>
>>>>> and in
>>>>>>
>>>>>> meantime I reckon we can either try to encourage the Microsoft Azure
>>>>>
>>>>> team
>>>>>>
>>>>>> to release their service emulator for the Linux platform asap or set
>>>>>
>>>>> up a
>>>>>>
>>>>>> Camel Windows build and run the emulator integration test on Windows
>>>>>
>>>>> only.
>>>>>>
>>>>>>
>>>>>> Thanks, Sergey
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>>>
>>>>
>>>
>>
>
>
>
Re: Camel Azure component
Posted by Claus Ibsen <cl...@gmail.com>.
Hi Sergey
Yeah sure its good to get this out in the hands of the users. If
something is not yet implemented or something we can always state that
in the docs.
And if there are things you think still must be done its a good idea
to log a JIRA with that.
For tests we sometimes have a maven profile that runs real integration
tests against an online account or something, and then regular unit
tests for just simpler tests.
As we have more and more of these it may be worthwhile to look into a
special jenkins job that only runs these integration tests with those
maven profiles enabled.
But sure feel free to go ahead and get the code into the master branch.
On Thu, Feb 9, 2017 at 6:42 PM, Sergey Beryozkin <sb...@gmail.com> wrote:
> Hi
>
> I've finally attached a patch for the experts to have a glance, I know I can
> commit and look forward to doing my first Camel commit, but it is my first
> Camel component, so if something obvious is missing - let me know please.
>
> It is hard to create a complete functional component from a start but I hope
> it will be incrementally enhanced and improved.
> At the moment it covers an Azure Blob Service (with some tweaks likely
> needed later on once we experiment more with it), I'll have a look at adding
> a Queue service support later on.
>
> The most obvious thing which proved hard to solve is to have the tests not
> ignored - I run all the tests which are there against the live 30-day free
> account called 'camelazure' but it is a private account nonetheless.
>
> I spent a lot of time on trying to mock the clients but the Azure SDK
> creators made all of the clients final, so after messing with PowerMock I
> thought it was not worth it as it looks it can add some instability in the
> running tests. The idea of running the integration tests is also on the hold
> given that they do not have a Maven plugin for starting the emulator (and no
> emulator for Linux yet).
>
> So I'd like to go ahead with having these tests disabled and let those who
> are interested start playing with this component and provide the feedback
> until we figure out how to test it in Jenkins...
>
> Claus, what do you think, would you be OK for the commit to go to
> 2.19.0-SNAPSHOT in the current form ?
>
> Cheers, Sergey
>
>
>
> On 05/02/17 17:06, Sergey Beryozkin wrote:
>>
>> Hi Guys,
>>
>> I've looked a bit more carefully at the Azure Emulator docs:
>>
>>
>> https://docs.microsoft.com/en-us/azure/storage/storage-use-emulator#start-and-initialize-the-storage-emulator
>>
>>
>> and I do not see now how it can be dynamically prepared to get the
>> integration tests running, my apologies I did not check it initially,
>> though if my earlier email would contribute to a Camel Windows build be
>> set up then it would be good :-). I believe Francesco managed to do it
>> quite easily for Syncope.
>>
>> I guess as far as this component is concerned I'd need to start with the
>> basic mock client to get some coverage done.
>>
>> Cheers, Sergey
>>
>> On 05/02/17 12:39, Jean-Baptiste Onofré wrote:
>>>
>>> +1 for Windows build.
>>>
>>> Regards
>>> JB
>>>
>>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang <wi...@gmail.com>
>>> wrote:
>>>>
>>>> +1 for setting up a Camel Windows build.
>>>> As some of our users are still using windows box.
>>>>
>>>>
>>>> Willem Jiang
>>>>
>>>> Blog: http://willemjiang.blogspot.com (English)
>>>> http://jnn.iteye.com (Chinese)
>>>> Twitter: willemjiang
>>>> Weibo: 姜宁willem
>>>>
>>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>>> <sb...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> I've started prototyping a Camel Azure component [1].
>>>>>
>>>>> Azure has a number of services, Blob, Queue, Table and File, a lot of
>>>>> commands, etc.
>>>>> I'm starting with supporting a Blob Service. Queue service will
>>>>
>>>> follow,
>>>>>
>>>>> and I guess the component will keep evolving to support other
>>>>
>>>> services too.
>>>>>
>>>>>
>>>>> At the moment I have most of the Blob Service commands covered at the
>>>>> Producer side. I'm not sure Consumer will need to be there given that
>>>>
>>>> Azure
>>>>>
>>>>> Java API only offers an option to download the blobs to an output
>>>>
>>>> stream
>>>>>
>>>>> (file most likely). May be only for getting the attributes if really
>>>>
>>>> needed.
>>>>>
>>>>>
>>>>> I guess Queue Service will though require a Consumer but that is a
>>>>
>>>> stage 2.
>>>>>
>>>>>
>>>>> I'm using camel-aws component as a source of ideas.
>>>>>
>>>>> The biggest issue it how to test it. I have several basic 'live'
>>>>
>>>> in-only
>>>>>
>>>>> tests based on the copy from camel-aws/s3 which depend on a live
>>>>
>>>> security
>>>>>
>>>>> key and the connection. Microsoft Azure offers an emulator which can
>>>>
>>>> only
>>>>>
>>>>> be run on Windows. And I'm not keen using a mock client which can
>>>>
>>>> become
>>>>>
>>>>> stale and won't really prove the blob has been uploaded.
>>>>>
>>>>> I'd like to start with having these tests disabled on the master for
>>>>
>>>> those
>>>>>
>>>>> who are interested to experiment with them using their own live keys
>>>>
>>>> and in
>>>>>
>>>>> meantime I reckon we can either try to encourage the Microsoft Azure
>>>>
>>>> team
>>>>>
>>>>> to release their service emulator for the Linux platform asap or set
>>>>
>>>> up a
>>>>>
>>>>> Camel Windows build and run the emulator integration test on Windows
>>>>
>>>> only.
>>>>>
>>>>>
>>>>> Thanks, Sergey
>>>>>
>>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>>
>>>
>>
>
--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
AW: Camel Azure component
Posted by "Walzer, Thomas" <th...@integratix.net>.
Sergey,
Thanks for the component.
I am really looking forward to the Azure Queue component.
Cheers, Thomas.
-----Ursprüngliche Nachricht-----
Von: Sergey Beryozkin [mailto:sberyozkin@gmail.com]
Gesendet: Donnerstag, 9. Februar 2017 18:42
An: dev@camel.apache.org
Betreff: Re: Camel Azure component
Hi
I've finally attached a patch for the experts to have a glance, I know I can commit and look forward to doing my first Camel commit, but it is my first Camel component, so if something obvious is missing - let me know please.
It is hard to create a complete functional component from a start but I hope it will be incrementally enhanced and improved.
At the moment it covers an Azure Blob Service (with some tweaks likely needed later on once we experiment more with it), I'll have a look at adding a Queue service support later on.
The most obvious thing which proved hard to solve is to have the tests not ignored - I run all the tests which are there against the live 30-day free account called 'camelazure' but it is a private account nonetheless.
I spent a lot of time on trying to mock the clients but the Azure SDK creators made all of the clients final, so after messing with PowerMock I thought it was not worth it as it looks it can add some instability in the running tests. The idea of running the integration tests is also on the hold given that they do not have a Maven plugin for starting the emulator (and no emulator for Linux yet).
So I'd like to go ahead with having these tests disabled and let those who are interested start playing with this component and provide the feedback until we figure out how to test it in Jenkins...
Claus, what do you think, would you be OK for the commit to go to 2.19.0-SNAPSHOT in the current form ?
Cheers, Sergey
On 05/02/17 17:06, Sergey Beryozkin wrote:
> Hi Guys,
>
> I've looked a bit more carefully at the Azure Emulator docs:
>
> https://docs.microsoft.com/en-us/azure/storage/storage-use-emulator#st
> art-and-initialize-the-storage-emulator
>
>
> and I do not see now how it can be dynamically prepared to get the
> integration tests running, my apologies I did not check it initially,
> though if my earlier email would contribute to a Camel Windows build
> be set up then it would be good :-). I believe Francesco managed to do
> it quite easily for Syncope.
>
> I guess as far as this component is concerned I'd need to start with
> the basic mock client to get some coverage done.
>
> Cheers, Sergey
>
> On 05/02/17 12:39, Jean-Baptiste Onofré wrote:
>> +1 for Windows build.
>>
>> Regards
>> JB
>>
>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang
>> <wi...@gmail.com>
>> wrote:
>>> +1 for setting up a Camel Windows build.
>>> As some of our users are still using windows box.
>>>
>>>
>>> Willem Jiang
>>>
>>> Blog: http://willemjiang.blogspot.com (English)
>>> http://jnn.iteye.com (Chinese)
>>> Twitter: willemjiang
>>> Weibo: 姜宁willem
>>>
>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>> <sb...@gmail.com>
>>> wrote:
>>>
>>>> Hi
>>>>
>>>> I've started prototyping a Camel Azure component [1].
>>>>
>>>> Azure has a number of services, Blob, Queue, Table and File, a lot
>>>> of commands, etc.
>>>> I'm starting with supporting a Blob Service. Queue service will
>>> follow,
>>>> and I guess the component will keep evolving to support other
>>> services too.
>>>>
>>>> At the moment I have most of the Blob Service commands covered at
>>>> the Producer side. I'm not sure Consumer will need to be there
>>>> given that
>>> Azure
>>>> Java API only offers an option to download the blobs to an output
>>> stream
>>>> (file most likely). May be only for getting the attributes if
>>>> really
>>> needed.
>>>>
>>>> I guess Queue Service will though require a Consumer but that is a
>>> stage 2.
>>>>
>>>> I'm using camel-aws component as a source of ideas.
>>>>
>>>> The biggest issue it how to test it. I have several basic 'live'
>>> in-only
>>>> tests based on the copy from camel-aws/s3 which depend on a live
>>> security
>>>> key and the connection. Microsoft Azure offers an emulator which
>>>> can
>>> only
>>>> be run on Windows. And I'm not keen using a mock client which can
>>> become
>>>> stale and won't really prove the blob has been uploaded.
>>>>
>>>> I'd like to start with having these tests disabled on the master
>>>> for
>>> those
>>>> who are interested to experiment with them using their own live
>>>> keys
>>> and in
>>>> meantime I reckon we can either try to encourage the Microsoft
>>>> Azure
>>> team
>>>> to release their service emulator for the Linux platform asap or
>>>> set
>>> up a
>>>> Camel Windows build and run the emulator integration test on
>>>> Windows
>>> only.
>>>>
>>>> Thanks, Sergey
>>>>
>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>
>>
>
Re: Camel Azure component
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi
I've finally attached a patch for the experts to have a glance, I know I
can commit and look forward to doing my first Camel commit, but it is my
first Camel component, so if something obvious is missing - let me know
please.
It is hard to create a complete functional component from a start but I
hope it will be incrementally enhanced and improved.
At the moment it covers an Azure Blob Service (with some tweaks likely
needed later on once we experiment more with it), I'll have a look at
adding a Queue service support later on.
The most obvious thing which proved hard to solve is to have the tests
not ignored - I run all the tests which are there against the live
30-day free account called 'camelazure' but it is a private account
nonetheless.
I spent a lot of time on trying to mock the clients but the Azure SDK
creators made all of the clients final, so after messing with PowerMock
I thought it was not worth it as it looks it can add some instability in
the running tests. The idea of running the integration tests is also on
the hold given that they do not have a Maven plugin for starting the
emulator (and no emulator for Linux yet).
So I'd like to go ahead with having these tests disabled and let those
who are interested start playing with this component and provide the
feedback until we figure out how to test it in Jenkins...
Claus, what do you think, would you be OK for the commit to go to
2.19.0-SNAPSHOT in the current form ?
Cheers, Sergey
On 05/02/17 17:06, Sergey Beryozkin wrote:
> Hi Guys,
>
> I've looked a bit more carefully at the Azure Emulator docs:
>
> https://docs.microsoft.com/en-us/azure/storage/storage-use-emulator#start-and-initialize-the-storage-emulator
>
>
> and I do not see now how it can be dynamically prepared to get the
> integration tests running, my apologies I did not check it initially,
> though if my earlier email would contribute to a Camel Windows build be
> set up then it would be good :-). I believe Francesco managed to do it
> quite easily for Syncope.
>
> I guess as far as this component is concerned I'd need to start with the
> basic mock client to get some coverage done.
>
> Cheers, Sergey
>
> On 05/02/17 12:39, Jean-Baptiste Onofr� wrote:
>> +1 for Windows build.
>>
>> Regards
>> JB
>>
>> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang <wi...@gmail.com>
>> wrote:
>>> +1 for setting up a Camel Windows build.
>>> As some of our users are still using windows box.
>>>
>>>
>>> Willem Jiang
>>>
>>> Blog: http://willemjiang.blogspot.com (English)
>>> http://jnn.iteye.com (Chinese)
>>> Twitter: willemjiang
>>> Weibo: \u59dc\u5b81willem
>>>
>>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>>> <sb...@gmail.com>
>>> wrote:
>>>
>>>> Hi
>>>>
>>>> I've started prototyping a Camel Azure component [1].
>>>>
>>>> Azure has a number of services, Blob, Queue, Table and File, a lot of
>>>> commands, etc.
>>>> I'm starting with supporting a Blob Service. Queue service will
>>> follow,
>>>> and I guess the component will keep evolving to support other
>>> services too.
>>>>
>>>> At the moment I have most of the Blob Service commands covered at the
>>>> Producer side. I'm not sure Consumer will need to be there given that
>>> Azure
>>>> Java API only offers an option to download the blobs to an output
>>> stream
>>>> (file most likely). May be only for getting the attributes if really
>>> needed.
>>>>
>>>> I guess Queue Service will though require a Consumer but that is a
>>> stage 2.
>>>>
>>>> I'm using camel-aws component as a source of ideas.
>>>>
>>>> The biggest issue it how to test it. I have several basic 'live'
>>> in-only
>>>> tests based on the copy from camel-aws/s3 which depend on a live
>>> security
>>>> key and the connection. Microsoft Azure offers an emulator which can
>>> only
>>>> be run on Windows. And I'm not keen using a mock client which can
>>> become
>>>> stale and won't really prove the blob has been uploaded.
>>>>
>>>> I'd like to start with having these tests disabled on the master for
>>> those
>>>> who are interested to experiment with them using their own live keys
>>> and in
>>>> meantime I reckon we can either try to encourage the Microsoft Azure
>>> team
>>>> to release their service emulator for the Linux platform asap or set
>>> up a
>>>> Camel Windows build and run the emulator integration test on Windows
>>> only.
>>>>
>>>> Thanks, Sergey
>>>>
>>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>>
>>
>
Re: Camel Azure component
Posted by Sergey Beryozkin <sb...@gmail.com>.
Hi Guys,
I've looked a bit more carefully at the Azure Emulator docs:
https://docs.microsoft.com/en-us/azure/storage/storage-use-emulator#start-and-initialize-the-storage-emulator
and I do not see now how it can be dynamically prepared to get the
integration tests running, my apologies I did not check it initially,
though if my earlier email would contribute to a Camel Windows build be
set up then it would be good :-). I believe Francesco managed to do it
quite easily for Syncope.
I guess as far as this component is concerned I'd need to start with the
basic mock client to get some coverage done.
Cheers, Sergey
On 05/02/17 12:39, Jean-Baptiste Onofr� wrote:
> +1 for Windows build.
>
> Regards
> JB
>
> On Feb 3, 2017, 21:27, at 21:27, Willem Jiang <wi...@gmail.com> wrote:
>> +1 for setting up a Camel Windows build.
>> As some of our users are still using windows box.
>>
>>
>> Willem Jiang
>>
>> Blog: http://willemjiang.blogspot.com (English)
>> http://jnn.iteye.com (Chinese)
>> Twitter: willemjiang
>> Weibo: \u59dc\u5b81willem
>>
>> On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
>> <sb...@gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> I've started prototyping a Camel Azure component [1].
>>>
>>> Azure has a number of services, Blob, Queue, Table and File, a lot of
>>> commands, etc.
>>> I'm starting with supporting a Blob Service. Queue service will
>> follow,
>>> and I guess the component will keep evolving to support other
>> services too.
>>>
>>> At the moment I have most of the Blob Service commands covered at the
>>> Producer side. I'm not sure Consumer will need to be there given that
>> Azure
>>> Java API only offers an option to download the blobs to an output
>> stream
>>> (file most likely). May be only for getting the attributes if really
>> needed.
>>>
>>> I guess Queue Service will though require a Consumer but that is a
>> stage 2.
>>>
>>> I'm using camel-aws component as a source of ideas.
>>>
>>> The biggest issue it how to test it. I have several basic 'live'
>> in-only
>>> tests based on the copy from camel-aws/s3 which depend on a live
>> security
>>> key and the connection. Microsoft Azure offers an emulator which can
>> only
>>> be run on Windows. And I'm not keen using a mock client which can
>> become
>>> stale and won't really prove the blob has been uploaded.
>>>
>>> I'd like to start with having these tests disabled on the master for
>> those
>>> who are interested to experiment with them using their own live keys
>> and in
>>> meantime I reckon we can either try to encourage the Microsoft Azure
>> team
>>> to release their service emulator for the Linux platform asap or set
>> up a
>>> Camel Windows build and run the emulator integration test on Windows
>> only.
>>>
>>> Thanks, Sergey
>>>
>>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>>
>
Re: Camel Azure component
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
+1 for Windows build.
Regards
JB
On Feb 3, 2017, 21:27, at 21:27, Willem Jiang <wi...@gmail.com> wrote:
>+1 for setting up a Camel Windows build.
>As some of our users are still using windows box.
>
>
>Willem Jiang
>
>Blog: http://willemjiang.blogspot.com (English)
> http://jnn.iteye.com (Chinese)
>Twitter: willemjiang
>Weibo: 姜宁willem
>
>On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin
><sb...@gmail.com>
>wrote:
>
>> Hi
>>
>> I've started prototyping a Camel Azure component [1].
>>
>> Azure has a number of services, Blob, Queue, Table and File, a lot of
>> commands, etc.
>> I'm starting with supporting a Blob Service. Queue service will
>follow,
>> and I guess the component will keep evolving to support other
>services too.
>>
>> At the moment I have most of the Blob Service commands covered at the
>> Producer side. I'm not sure Consumer will need to be there given that
>Azure
>> Java API only offers an option to download the blobs to an output
>stream
>> (file most likely). May be only for getting the attributes if really
>needed.
>>
>> I guess Queue Service will though require a Consumer but that is a
>stage 2.
>>
>> I'm using camel-aws component as a source of ideas.
>>
>> The biggest issue it how to test it. I have several basic 'live'
>in-only
>> tests based on the copy from camel-aws/s3 which depend on a live
>security
>> key and the connection. Microsoft Azure offers an emulator which can
>only
>> be run on Windows. And I'm not keen using a mock client which can
>become
>> stale and won't really prove the blob has been uploaded.
>>
>> I'd like to start with having these tests disabled on the master for
>those
>> who are interested to experiment with them using their own live keys
>and in
>> meantime I reckon we can either try to encourage the Microsoft Azure
>team
>> to release their service emulator for the Linux platform asap or set
>up a
>> Camel Windows build and run the emulator integration test on Windows
>only.
>>
>> Thanks, Sergey
>>
>> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>>
Re: Camel Azure component
Posted by Willem Jiang <wi...@gmail.com>.
+1 for setting up a Camel Windows build.
As some of our users are still using windows box.
Willem Jiang
Blog: http://willemjiang.blogspot.com (English)
http://jnn.iteye.com (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem
On Fri, Feb 3, 2017 at 11:22 PM, Sergey Beryozkin <sb...@gmail.com>
wrote:
> Hi
>
> I've started prototyping a Camel Azure component [1].
>
> Azure has a number of services, Blob, Queue, Table and File, a lot of
> commands, etc.
> I'm starting with supporting a Blob Service. Queue service will follow,
> and I guess the component will keep evolving to support other services too.
>
> At the moment I have most of the Blob Service commands covered at the
> Producer side. I'm not sure Consumer will need to be there given that Azure
> Java API only offers an option to download the blobs to an output stream
> (file most likely). May be only for getting the attributes if really needed.
>
> I guess Queue Service will though require a Consumer but that is a stage 2.
>
> I'm using camel-aws component as a source of ideas.
>
> The biggest issue it how to test it. I have several basic 'live' in-only
> tests based on the copy from camel-aws/s3 which depend on a live security
> key and the connection. Microsoft Azure offers an emulator which can only
> be run on Windows. And I'm not keen using a mock client which can become
> stale and won't really prove the blob has been uploaded.
>
> I'd like to start with having these tests disabled on the master for those
> who are interested to experiment with them using their own live keys and in
> meantime I reckon we can either try to encourage the Microsoft Azure team
> to release their service emulator for the Linux platform asap or set up a
> Camel Windows build and run the emulator integration test on Windows only.
>
> Thanks, Sergey
>
> [1] https://issues.apache.org/jira/browse/CAMEL-10786
>