You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Yoann Gini <yo...@gmail.com> on 2023/03/14 13:52:21 UTC
Learning steps / Artemis and bidirectional federated queue for IoT context.
Hello,
I'm new here and I'm working on an IoT issue where I'm wondering if Artemis and federated queue could help. I'm learning the software in the same time I'm trying to achieve something, so I would like to have your opinion on this: will it work? Is it a good idea? And do you have sample config / blog post to recommend to learn to achieve that?
The idea: IoT devices with some computing capability running local needed services as well as an Artemis local instance, with upstream set a Cloud located one.
Configuration in bidirectional to allow both Cloud to pass commands to IoT and IoT to report feedback to Cloud.
Goal of Artemis here in the IoT as federated layer (and not directly IoT service connected to the cloud one) would be to provide some sort of caching capabilities which would allow the local service to not have to handle disconnected state. Local service will still be able to publish to local Artemis and read what was lastly make available before a connectivity cut. And when the Internet recovers, the federation (if I understand it well) should resume bidirectional transfer of feedback and orders.
Does that make sense for you? Is it a good usage of the solution that has some chance to reliably works?
If so do you have any recommendations for the implementation or any blog post / conference I should look at to learn this?
Cheers
Yoann
Re: Learning steps / Artemis and bidirectional federated queue for IoT context.
Posted by Yoann Gini <yo...@gmail.com>.
Thanks a lot for the time you take Justin here.
Bellow information you requested
> Le 15 mars 2023 à 20:10, Justin Bertram <jb...@apache.org> a écrit :
>
> - How many clients will be using the edge broker, and how will they be
> using it (e.g. consumer, producer, protocol, multicast, anycast, etc.)?
On the central one, one consumer for all feedbacks, and one producer of orders
One each edge one, one consumer for all orders, one producer for feedbacks, and 2 to 3 services for local sync (each produce and consume local signals)
> - How many addresses & queues will be used on the edge broker? Is the
> configuration static in broker.xml or dynamic via auto-creation?
One for upstream, one for local, all statics
> - Which queues will be read to send messages to the cloud broker?
Only one
> - Which addresses will receive messages from the cloud broker?
Not yet at that level of knowledge on my side (still learning)
> Other considerations...
>
> - What happens if the edge broker crashes?
Nothing breaks and no one get hurts. It won't be used for active piloting but for feedback and signaling. No actions can't be started without full sync of requests on each clients, and loss of signals will mean loss of observability but not loss of safety systems.
> - What happens if connectivity to the cloud broker is absent for an
> extended period of time?
Current work is finished without issue, no new work can be sent to the cell, and we are blind to the status of the last one from the central point.
If the messages are kept until the network recovery, and sent once connectivity is back, we will just resume where we were.
> - Will you use any kind of flow-control for the producers to ensure the
> broker isn't overwhelmed with messages?
Not yet planned but can be
That's clearly the kind of help and wondering I was hopping here: questions to challenge our idea from people that know the technology and can lead us to appropriate config / training source / eventually consultant or support (once we pass the PoC).
Thanks a lot
Re: Learning steps / Artemis and bidirectional federated queue for IoT context.
Posted by Justin Bertram <jb...@apache.org>.
The business use-case isn't my primary concern. In order to make a
recommendation I need to know exactly how you want to use the broker. Some
questions to answer:
- How many clients will be using the edge broker, and how will they be
using it (e.g. consumer, producer, protocol, multicast, anycast, etc.)?
- How many addresses & queues will be used on the edge broker? Is the
configuration static in broker.xml or dynamic via auto-creation?
- Which queues will be read to send messages to the cloud broker?
- Which addresses will receive messages from the cloud broker?
Other considerations...
- What happens if the edge broker crashes?
- What happens if connectivity to the cloud broker is absent for an
extended period of time?
- Will you use any kind of flow-control for the producers to ensure the
broker isn't overwhelmed with messages?
Justin
On Wed, Mar 15, 2023 at 1:49 PM Yoann Gini <yo...@gmail.com> wrote:
> The message that will pass between instances are robots work orders in one
> way and robots feedback in the other way
>
> All of them are fixed robots (not automotive) and with on prem safety, not
> dependent of any networks
>
>
> Order are basic JSON document with list of instructions to pass once, and
> then during the execution and at the end the feedbacks allow us to report
> where we are in the process
>
>
> > Le 15 mars 2023 à 19:38, Justin Bertram <jb...@apache.org> a écrit :
> >
> > I'm not sure what exactly you're doing on your edge node so it's hard to
> > make a recommendation.
> >
> >
> > Justin
> >
> > On Wed, Mar 15, 2023 at 1:07 PM Yoann Gini <yo...@gmail.com> wrote:
> >
> >> Thanks a lot for that feedback!
> >>
> >> Between the three option (bridges, federation, and broker connection)
> what
> >> would be the most suited for a configuration only at the edge side?
> >>
> >> All our edges have a certificate available published by a private CA,
> >> ideally I would like my central instance to be with the minimum
> >> configuration possible, accepting anything from anyone with a valid
> >> certificate, letting the Edge instance in charge of all the replication
> >> (bi-directional) settings.
> >>
> >>
> >>> Le 15 mars 2023 à 17:38, Justin Bertram <jb...@apache.org> a écrit
> :
> >>>
> >>> The edge use-case with substantial hardware is quite different from the
> >>> classic IoT use-case. Also, the fact that you want to use messaging
> >> between
> >>> clients on the edge device is also a significant difference. Therefore,
> >> my
> >>> answer certainly would change given that the underlying assumptions
> from
> >> my
> >>> previous answer are invalid.
> >>>
> >>> I have no doubt that some of our users are running a broker on their
> edge
> >>> devices. The broker can be tuned to run effectively even on very modest
> >>> hardware when necessary. I can't rightly say whether that's a "good
> >> usage"
> >>> in your use-case when it's clear that important details are easy to
> omit.
> >>> Also, it's hard to say whether core bridges [1], federation, or even
> >> broker
> >>> connections [2] would best fit your needs.
> >>>
> >>>
> >>> Justin
> >>>
> >>> [1]
> >>>
> >>
> https://activemq.apache.org/components/artemis/documentation/latest/core-bridges.html
> >>> [1]
> >>>
> >>
> https://activemq.apache.org/components/artemis/documentation/latest/amqp-broker-connections.html
> >>>
> >>> On Tue, Mar 14, 2023 at 3:12 PM Yoann Gini <yo...@gmail.com>
> wrote:
> >>>
> >>>> Thanks for your input Justin!
> >>>>
> >>>> Maybe IoT was missleading here, if I speak about EdgeComputing does it
> >>>> change your answer?
> >>>>
> >>>> One of the reasons I was looking for that architecture and that I
> forgot
> >>>> to mention was the ability to use also the MQ between multiple
> services
> >>>> locally to the edge.
> >>>>
> >>>> So each edge would be autonomous for their own internal messaging and
> >> then
> >>>> have federations for upstream communications.
> >>>>
> >>>> I was in my context forgetting that IoT usually mean low end device.
> >>>>
> >>>> In my context it’s IoT but with 64 GB of RAM, i7 CPU and Cuda
> >> capabilities
> >>>> on the GPU.
> >>>>
> >>>> So maybe Edge Computer is more appropriate wording.
> >>>>
> >>>> Yoann
> >>>>
> >>>>> Le 14 mars 2023 à 20:02, Justin Bertram <jb...@apache.org> a
> écrit
> >> :
> >>>>>
> >>>>> Generally speaking, the software that runs on IoT devices is
> designed
> >>>> with
> >>>>> the unique constraints of IoT in mind (e.g. poor or intermittent
> >> network
> >>>>> connectivity, limited processing capacity, small amount of memory,
> low
> >>>>> power, etc.).
> >>>>>
> >>>>> MQTT, for example, is often used for messaging in IoT contexts and it
> >> is
> >>>>> designed to be light-weight and has features like will messages,
> >> retained
> >>>>> messages, link stealing, session expiration, etc. to facilitate
> unique
> >>>> IoT
> >>>>> use-cases.
> >>>>>
> >>>>> Furthermore, MQTT client implementations offer additional features
> not
> >>>>> outlined in the MQTT specification. For example, most Paho MQTT
> clients
> >>>> [1]
> >>>>> offer the ability to automatically reconnect, buffer messages when
> >>>> offline
> >>>>> (which will be sent automatically when the connection is
> >> re-established),
> >>>>> and persist messages (in case the application crashes before it has a
> >>>>> chance to send them).
> >>>>>
> >>>>> In my opinion, adding a broker to an IoT device is not a good idea.
> >> It's
> >>>>> adding complexity where it isn't really required. It will make
> >>>>> configuration, deployment, and debugging more difficult, and it will
> >>>>> require more hardware (which is notoriously limited in IoT). A good
> >> MQTT
> >>>>> client implementation should be able to provide you with all the
> >> features
> >>>>> you need to operate effectively in an IoT context.
> >>>>>
> >>>>>
> >>>>> Justin
> >>>>>
> >>>>> [1] https://www.eclipse.org/paho/index.php?page=downloads.php
> >>>>>
> >>>>>> On Tue, Mar 14, 2023 at 8:52 AM Yoann Gini <yo...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>> Hello,
> >>>>>>
> >>>>>> I'm new here and I'm working on an IoT issue where I'm wondering if
> >>>>>> Artemis and federated queue could help. I'm learning the software in
> >> the
> >>>>>> same time I'm trying to achieve something, so I would like to have
> >> your
> >>>>>> opinion on this: will it work? Is it a good idea? And do you have
> >> sample
> >>>>>> config / blog post to recommend to learn to achieve that?
> >>>>>>
> >>>>>> The idea: IoT devices with some computing capability running local
> >>>> needed
> >>>>>> services as well as an Artemis local instance, with upstream set a
> >> Cloud
> >>>>>> located one.
> >>>>>>
> >>>>>> Configuration in bidirectional to allow both Cloud to pass commands
> to
> >>>> IoT
> >>>>>> and IoT to report feedback to Cloud.
> >>>>>>
> >>>>>> Goal of Artemis here in the IoT as federated layer (and not directly
> >> IoT
> >>>>>> service connected to the cloud one) would be to provide some sort of
> >>>>>> caching capabilities which would allow the local service to not have
> >> to
> >>>>>> handle disconnected state. Local service will still be able to
> publish
> >>>> to
> >>>>>> local Artemis and read what was lastly make available before a
> >>>> connectivity
> >>>>>> cut. And when the Internet recovers, the federation (if I understand
> >> it
> >>>>>> well) should resume bidirectional transfer of feedback and orders.
> >>>>>>
> >>>>>> Does that make sense for you? Is it a good usage of the solution
> that
> >>>> has
> >>>>>> some chance to reliably works?
> >>>>>>
> >>>>>> If so do you have any recommendations for the implementation or any
> >> blog
> >>>>>> post / conference I should look at to learn this?
> >>>>>>
> >>>>>> Cheers
> >>>>>> Yoann
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
>
>
Re: Learning steps / Artemis and bidirectional federated queue for IoT context.
Posted by Yoann Gini <yo...@gmail.com>.
The message that will pass between instances are robots work orders in one way and robots feedback in the other way
All of them are fixed robots (not automotive) and with on prem safety, not dependent of any networks
Order are basic JSON document with list of instructions to pass once, and then during the execution and at the end the feedbacks allow us to report where we are in the process
> Le 15 mars 2023 à 19:38, Justin Bertram <jb...@apache.org> a écrit :
>
> I'm not sure what exactly you're doing on your edge node so it's hard to
> make a recommendation.
>
>
> Justin
>
> On Wed, Mar 15, 2023 at 1:07 PM Yoann Gini <yo...@gmail.com> wrote:
>
>> Thanks a lot for that feedback!
>>
>> Between the three option (bridges, federation, and broker connection) what
>> would be the most suited for a configuration only at the edge side?
>>
>> All our edges have a certificate available published by a private CA,
>> ideally I would like my central instance to be with the minimum
>> configuration possible, accepting anything from anyone with a valid
>> certificate, letting the Edge instance in charge of all the replication
>> (bi-directional) settings.
>>
>>
>>> Le 15 mars 2023 à 17:38, Justin Bertram <jb...@apache.org> a écrit :
>>>
>>> The edge use-case with substantial hardware is quite different from the
>>> classic IoT use-case. Also, the fact that you want to use messaging
>> between
>>> clients on the edge device is also a significant difference. Therefore,
>> my
>>> answer certainly would change given that the underlying assumptions from
>> my
>>> previous answer are invalid.
>>>
>>> I have no doubt that some of our users are running a broker on their edge
>>> devices. The broker can be tuned to run effectively even on very modest
>>> hardware when necessary. I can't rightly say whether that's a "good
>> usage"
>>> in your use-case when it's clear that important details are easy to omit.
>>> Also, it's hard to say whether core bridges [1], federation, or even
>> broker
>>> connections [2] would best fit your needs.
>>>
>>>
>>> Justin
>>>
>>> [1]
>>>
>> https://activemq.apache.org/components/artemis/documentation/latest/core-bridges.html
>>> [1]
>>>
>> https://activemq.apache.org/components/artemis/documentation/latest/amqp-broker-connections.html
>>>
>>> On Tue, Mar 14, 2023 at 3:12 PM Yoann Gini <yo...@gmail.com> wrote:
>>>
>>>> Thanks for your input Justin!
>>>>
>>>> Maybe IoT was missleading here, if I speak about EdgeComputing does it
>>>> change your answer?
>>>>
>>>> One of the reasons I was looking for that architecture and that I forgot
>>>> to mention was the ability to use also the MQ between multiple services
>>>> locally to the edge.
>>>>
>>>> So each edge would be autonomous for their own internal messaging and
>> then
>>>> have federations for upstream communications.
>>>>
>>>> I was in my context forgetting that IoT usually mean low end device.
>>>>
>>>> In my context it’s IoT but with 64 GB of RAM, i7 CPU and Cuda
>> capabilities
>>>> on the GPU.
>>>>
>>>> So maybe Edge Computer is more appropriate wording.
>>>>
>>>> Yoann
>>>>
>>>>> Le 14 mars 2023 à 20:02, Justin Bertram <jb...@apache.org> a écrit
>> :
>>>>>
>>>>> Generally speaking, the software that runs on IoT devices is designed
>>>> with
>>>>> the unique constraints of IoT in mind (e.g. poor or intermittent
>> network
>>>>> connectivity, limited processing capacity, small amount of memory, low
>>>>> power, etc.).
>>>>>
>>>>> MQTT, for example, is often used for messaging in IoT contexts and it
>> is
>>>>> designed to be light-weight and has features like will messages,
>> retained
>>>>> messages, link stealing, session expiration, etc. to facilitate unique
>>>> IoT
>>>>> use-cases.
>>>>>
>>>>> Furthermore, MQTT client implementations offer additional features not
>>>>> outlined in the MQTT specification. For example, most Paho MQTT clients
>>>> [1]
>>>>> offer the ability to automatically reconnect, buffer messages when
>>>> offline
>>>>> (which will be sent automatically when the connection is
>> re-established),
>>>>> and persist messages (in case the application crashes before it has a
>>>>> chance to send them).
>>>>>
>>>>> In my opinion, adding a broker to an IoT device is not a good idea.
>> It's
>>>>> adding complexity where it isn't really required. It will make
>>>>> configuration, deployment, and debugging more difficult, and it will
>>>>> require more hardware (which is notoriously limited in IoT). A good
>> MQTT
>>>>> client implementation should be able to provide you with all the
>> features
>>>>> you need to operate effectively in an IoT context.
>>>>>
>>>>>
>>>>> Justin
>>>>>
>>>>> [1] https://www.eclipse.org/paho/index.php?page=downloads.php
>>>>>
>>>>>> On Tue, Mar 14, 2023 at 8:52 AM Yoann Gini <yo...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I'm new here and I'm working on an IoT issue where I'm wondering if
>>>>>> Artemis and federated queue could help. I'm learning the software in
>> the
>>>>>> same time I'm trying to achieve something, so I would like to have
>> your
>>>>>> opinion on this: will it work? Is it a good idea? And do you have
>> sample
>>>>>> config / blog post to recommend to learn to achieve that?
>>>>>>
>>>>>> The idea: IoT devices with some computing capability running local
>>>> needed
>>>>>> services as well as an Artemis local instance, with upstream set a
>> Cloud
>>>>>> located one.
>>>>>>
>>>>>> Configuration in bidirectional to allow both Cloud to pass commands to
>>>> IoT
>>>>>> and IoT to report feedback to Cloud.
>>>>>>
>>>>>> Goal of Artemis here in the IoT as federated layer (and not directly
>> IoT
>>>>>> service connected to the cloud one) would be to provide some sort of
>>>>>> caching capabilities which would allow the local service to not have
>> to
>>>>>> handle disconnected state. Local service will still be able to publish
>>>> to
>>>>>> local Artemis and read what was lastly make available before a
>>>> connectivity
>>>>>> cut. And when the Internet recovers, the federation (if I understand
>> it
>>>>>> well) should resume bidirectional transfer of feedback and orders.
>>>>>>
>>>>>> Does that make sense for you? Is it a good usage of the solution that
>>>> has
>>>>>> some chance to reliably works?
>>>>>>
>>>>>> If so do you have any recommendations for the implementation or any
>> blog
>>>>>> post / conference I should look at to learn this?
>>>>>>
>>>>>> Cheers
>>>>>> Yoann
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>
Re: Learning steps / Artemis and bidirectional federated queue for IoT context.
Posted by Justin Bertram <jb...@apache.org>.
I'm not sure what exactly you're doing on your edge node so it's hard to
make a recommendation.
Justin
On Wed, Mar 15, 2023 at 1:07 PM Yoann Gini <yo...@gmail.com> wrote:
> Thanks a lot for that feedback!
>
> Between the three option (bridges, federation, and broker connection) what
> would be the most suited for a configuration only at the edge side?
>
> All our edges have a certificate available published by a private CA,
> ideally I would like my central instance to be with the minimum
> configuration possible, accepting anything from anyone with a valid
> certificate, letting the Edge instance in charge of all the replication
> (bi-directional) settings.
>
>
> > Le 15 mars 2023 à 17:38, Justin Bertram <jb...@apache.org> a écrit :
> >
> > The edge use-case with substantial hardware is quite different from the
> > classic IoT use-case. Also, the fact that you want to use messaging
> between
> > clients on the edge device is also a significant difference. Therefore,
> my
> > answer certainly would change given that the underlying assumptions from
> my
> > previous answer are invalid.
> >
> > I have no doubt that some of our users are running a broker on their edge
> > devices. The broker can be tuned to run effectively even on very modest
> > hardware when necessary. I can't rightly say whether that's a "good
> usage"
> > in your use-case when it's clear that important details are easy to omit.
> > Also, it's hard to say whether core bridges [1], federation, or even
> broker
> > connections [2] would best fit your needs.
> >
> >
> > Justin
> >
> > [1]
> >
> https://activemq.apache.org/components/artemis/documentation/latest/core-bridges.html
> > [1]
> >
> https://activemq.apache.org/components/artemis/documentation/latest/amqp-broker-connections.html
> >
> > On Tue, Mar 14, 2023 at 3:12 PM Yoann Gini <yo...@gmail.com> wrote:
> >
> >> Thanks for your input Justin!
> >>
> >> Maybe IoT was missleading here, if I speak about EdgeComputing does it
> >> change your answer?
> >>
> >> One of the reasons I was looking for that architecture and that I forgot
> >> to mention was the ability to use also the MQ between multiple services
> >> locally to the edge.
> >>
> >> So each edge would be autonomous for their own internal messaging and
> then
> >> have federations for upstream communications.
> >>
> >> I was in my context forgetting that IoT usually mean low end device.
> >>
> >> In my context it’s IoT but with 64 GB of RAM, i7 CPU and Cuda
> capabilities
> >> on the GPU.
> >>
> >> So maybe Edge Computer is more appropriate wording.
> >>
> >> Yoann
> >>
> >>> Le 14 mars 2023 à 20:02, Justin Bertram <jb...@apache.org> a écrit
> :
> >>>
> >>> Generally speaking, the software that runs on IoT devices is designed
> >> with
> >>> the unique constraints of IoT in mind (e.g. poor or intermittent
> network
> >>> connectivity, limited processing capacity, small amount of memory, low
> >>> power, etc.).
> >>>
> >>> MQTT, for example, is often used for messaging in IoT contexts and it
> is
> >>> designed to be light-weight and has features like will messages,
> retained
> >>> messages, link stealing, session expiration, etc. to facilitate unique
> >> IoT
> >>> use-cases.
> >>>
> >>> Furthermore, MQTT client implementations offer additional features not
> >>> outlined in the MQTT specification. For example, most Paho MQTT clients
> >> [1]
> >>> offer the ability to automatically reconnect, buffer messages when
> >> offline
> >>> (which will be sent automatically when the connection is
> re-established),
> >>> and persist messages (in case the application crashes before it has a
> >>> chance to send them).
> >>>
> >>> In my opinion, adding a broker to an IoT device is not a good idea.
> It's
> >>> adding complexity where it isn't really required. It will make
> >>> configuration, deployment, and debugging more difficult, and it will
> >>> require more hardware (which is notoriously limited in IoT). A good
> MQTT
> >>> client implementation should be able to provide you with all the
> features
> >>> you need to operate effectively in an IoT context.
> >>>
> >>>
> >>> Justin
> >>>
> >>> [1] https://www.eclipse.org/paho/index.php?page=downloads.php
> >>>
> >>>> On Tue, Mar 14, 2023 at 8:52 AM Yoann Gini <yo...@gmail.com>
> >> wrote:
> >>>>
> >>>> Hello,
> >>>>
> >>>> I'm new here and I'm working on an IoT issue where I'm wondering if
> >>>> Artemis and federated queue could help. I'm learning the software in
> the
> >>>> same time I'm trying to achieve something, so I would like to have
> your
> >>>> opinion on this: will it work? Is it a good idea? And do you have
> sample
> >>>> config / blog post to recommend to learn to achieve that?
> >>>>
> >>>> The idea: IoT devices with some computing capability running local
> >> needed
> >>>> services as well as an Artemis local instance, with upstream set a
> Cloud
> >>>> located one.
> >>>>
> >>>> Configuration in bidirectional to allow both Cloud to pass commands to
> >> IoT
> >>>> and IoT to report feedback to Cloud.
> >>>>
> >>>> Goal of Artemis here in the IoT as federated layer (and not directly
> IoT
> >>>> service connected to the cloud one) would be to provide some sort of
> >>>> caching capabilities which would allow the local service to not have
> to
> >>>> handle disconnected state. Local service will still be able to publish
> >> to
> >>>> local Artemis and read what was lastly make available before a
> >> connectivity
> >>>> cut. And when the Internet recovers, the federation (if I understand
> it
> >>>> well) should resume bidirectional transfer of feedback and orders.
> >>>>
> >>>> Does that make sense for you? Is it a good usage of the solution that
> >> has
> >>>> some chance to reliably works?
> >>>>
> >>>> If so do you have any recommendations for the implementation or any
> blog
> >>>> post / conference I should look at to learn this?
> >>>>
> >>>> Cheers
> >>>> Yoann
> >>>>
> >>>>
> >>
> >>
>
>
Re: Learning steps / Artemis and bidirectional federated queue for IoT context.
Posted by Yoann Gini <yo...@gmail.com>.
Thanks a lot for that feedback!
Between the three option (bridges, federation, and broker connection) what would be the most suited for a configuration only at the edge side?
All our edges have a certificate available published by a private CA, ideally I would like my central instance to be with the minimum configuration possible, accepting anything from anyone with a valid certificate, letting the Edge instance in charge of all the replication (bi-directional) settings.
> Le 15 mars 2023 à 17:38, Justin Bertram <jb...@apache.org> a écrit :
>
> The edge use-case with substantial hardware is quite different from the
> classic IoT use-case. Also, the fact that you want to use messaging between
> clients on the edge device is also a significant difference. Therefore, my
> answer certainly would change given that the underlying assumptions from my
> previous answer are invalid.
>
> I have no doubt that some of our users are running a broker on their edge
> devices. The broker can be tuned to run effectively even on very modest
> hardware when necessary. I can't rightly say whether that's a "good usage"
> in your use-case when it's clear that important details are easy to omit.
> Also, it's hard to say whether core bridges [1], federation, or even broker
> connections [2] would best fit your needs.
>
>
> Justin
>
> [1]
> https://activemq.apache.org/components/artemis/documentation/latest/core-bridges.html
> [1]
> https://activemq.apache.org/components/artemis/documentation/latest/amqp-broker-connections.html
>
> On Tue, Mar 14, 2023 at 3:12 PM Yoann Gini <yo...@gmail.com> wrote:
>
>> Thanks for your input Justin!
>>
>> Maybe IoT was missleading here, if I speak about EdgeComputing does it
>> change your answer?
>>
>> One of the reasons I was looking for that architecture and that I forgot
>> to mention was the ability to use also the MQ between multiple services
>> locally to the edge.
>>
>> So each edge would be autonomous for their own internal messaging and then
>> have federations for upstream communications.
>>
>> I was in my context forgetting that IoT usually mean low end device.
>>
>> In my context it’s IoT but with 64 GB of RAM, i7 CPU and Cuda capabilities
>> on the GPU.
>>
>> So maybe Edge Computer is more appropriate wording.
>>
>> Yoann
>>
>>> Le 14 mars 2023 à 20:02, Justin Bertram <jb...@apache.org> a écrit :
>>>
>>> Generally speaking, the software that runs on IoT devices is designed
>> with
>>> the unique constraints of IoT in mind (e.g. poor or intermittent network
>>> connectivity, limited processing capacity, small amount of memory, low
>>> power, etc.).
>>>
>>> MQTT, for example, is often used for messaging in IoT contexts and it is
>>> designed to be light-weight and has features like will messages, retained
>>> messages, link stealing, session expiration, etc. to facilitate unique
>> IoT
>>> use-cases.
>>>
>>> Furthermore, MQTT client implementations offer additional features not
>>> outlined in the MQTT specification. For example, most Paho MQTT clients
>> [1]
>>> offer the ability to automatically reconnect, buffer messages when
>> offline
>>> (which will be sent automatically when the connection is re-established),
>>> and persist messages (in case the application crashes before it has a
>>> chance to send them).
>>>
>>> In my opinion, adding a broker to an IoT device is not a good idea. It's
>>> adding complexity where it isn't really required. It will make
>>> configuration, deployment, and debugging more difficult, and it will
>>> require more hardware (which is notoriously limited in IoT). A good MQTT
>>> client implementation should be able to provide you with all the features
>>> you need to operate effectively in an IoT context.
>>>
>>>
>>> Justin
>>>
>>> [1] https://www.eclipse.org/paho/index.php?page=downloads.php
>>>
>>>> On Tue, Mar 14, 2023 at 8:52 AM Yoann Gini <yo...@gmail.com>
>> wrote:
>>>>
>>>> Hello,
>>>>
>>>> I'm new here and I'm working on an IoT issue where I'm wondering if
>>>> Artemis and federated queue could help. I'm learning the software in the
>>>> same time I'm trying to achieve something, so I would like to have your
>>>> opinion on this: will it work? Is it a good idea? And do you have sample
>>>> config / blog post to recommend to learn to achieve that?
>>>>
>>>> The idea: IoT devices with some computing capability running local
>> needed
>>>> services as well as an Artemis local instance, with upstream set a Cloud
>>>> located one.
>>>>
>>>> Configuration in bidirectional to allow both Cloud to pass commands to
>> IoT
>>>> and IoT to report feedback to Cloud.
>>>>
>>>> Goal of Artemis here in the IoT as federated layer (and not directly IoT
>>>> service connected to the cloud one) would be to provide some sort of
>>>> caching capabilities which would allow the local service to not have to
>>>> handle disconnected state. Local service will still be able to publish
>> to
>>>> local Artemis and read what was lastly make available before a
>> connectivity
>>>> cut. And when the Internet recovers, the federation (if I understand it
>>>> well) should resume bidirectional transfer of feedback and orders.
>>>>
>>>> Does that make sense for you? Is it a good usage of the solution that
>> has
>>>> some chance to reliably works?
>>>>
>>>> If so do you have any recommendations for the implementation or any blog
>>>> post / conference I should look at to learn this?
>>>>
>>>> Cheers
>>>> Yoann
>>>>
>>>>
>>
>>
Re: Learning steps / Artemis and bidirectional federated queue for IoT context.
Posted by Justin Bertram <jb...@apache.org>.
The edge use-case with substantial hardware is quite different from the
classic IoT use-case. Also, the fact that you want to use messaging between
clients on the edge device is also a significant difference. Therefore, my
answer certainly would change given that the underlying assumptions from my
previous answer are invalid.
I have no doubt that some of our users are running a broker on their edge
devices. The broker can be tuned to run effectively even on very modest
hardware when necessary. I can't rightly say whether that's a "good usage"
in your use-case when it's clear that important details are easy to omit.
Also, it's hard to say whether core bridges [1], federation, or even broker
connections [2] would best fit your needs.
Justin
[1]
https://activemq.apache.org/components/artemis/documentation/latest/core-bridges.html
[1]
https://activemq.apache.org/components/artemis/documentation/latest/amqp-broker-connections.html
On Tue, Mar 14, 2023 at 3:12 PM Yoann Gini <yo...@gmail.com> wrote:
> Thanks for your input Justin!
>
> Maybe IoT was missleading here, if I speak about EdgeComputing does it
> change your answer?
>
> One of the reasons I was looking for that architecture and that I forgot
> to mention was the ability to use also the MQ between multiple services
> locally to the edge.
>
> So each edge would be autonomous for their own internal messaging and then
> have federations for upstream communications.
>
> I was in my context forgetting that IoT usually mean low end device.
>
> In my context it’s IoT but with 64 GB of RAM, i7 CPU and Cuda capabilities
> on the GPU.
>
> So maybe Edge Computer is more appropriate wording.
>
> Yoann
>
> > Le 14 mars 2023 à 20:02, Justin Bertram <jb...@apache.org> a écrit :
> >
> > Generally speaking, the software that runs on IoT devices is designed
> with
> > the unique constraints of IoT in mind (e.g. poor or intermittent network
> > connectivity, limited processing capacity, small amount of memory, low
> > power, etc.).
> >
> > MQTT, for example, is often used for messaging in IoT contexts and it is
> > designed to be light-weight and has features like will messages, retained
> > messages, link stealing, session expiration, etc. to facilitate unique
> IoT
> > use-cases.
> >
> > Furthermore, MQTT client implementations offer additional features not
> > outlined in the MQTT specification. For example, most Paho MQTT clients
> [1]
> > offer the ability to automatically reconnect, buffer messages when
> offline
> > (which will be sent automatically when the connection is re-established),
> > and persist messages (in case the application crashes before it has a
> > chance to send them).
> >
> > In my opinion, adding a broker to an IoT device is not a good idea. It's
> > adding complexity where it isn't really required. It will make
> > configuration, deployment, and debugging more difficult, and it will
> > require more hardware (which is notoriously limited in IoT). A good MQTT
> > client implementation should be able to provide you with all the features
> > you need to operate effectively in an IoT context.
> >
> >
> > Justin
> >
> > [1] https://www.eclipse.org/paho/index.php?page=downloads.php
> >
> >> On Tue, Mar 14, 2023 at 8:52 AM Yoann Gini <yo...@gmail.com>
> wrote:
> >>
> >> Hello,
> >>
> >> I'm new here and I'm working on an IoT issue where I'm wondering if
> >> Artemis and federated queue could help. I'm learning the software in the
> >> same time I'm trying to achieve something, so I would like to have your
> >> opinion on this: will it work? Is it a good idea? And do you have sample
> >> config / blog post to recommend to learn to achieve that?
> >>
> >> The idea: IoT devices with some computing capability running local
> needed
> >> services as well as an Artemis local instance, with upstream set a Cloud
> >> located one.
> >>
> >> Configuration in bidirectional to allow both Cloud to pass commands to
> IoT
> >> and IoT to report feedback to Cloud.
> >>
> >> Goal of Artemis here in the IoT as federated layer (and not directly IoT
> >> service connected to the cloud one) would be to provide some sort of
> >> caching capabilities which would allow the local service to not have to
> >> handle disconnected state. Local service will still be able to publish
> to
> >> local Artemis and read what was lastly make available before a
> connectivity
> >> cut. And when the Internet recovers, the federation (if I understand it
> >> well) should resume bidirectional transfer of feedback and orders.
> >>
> >> Does that make sense for you? Is it a good usage of the solution that
> has
> >> some chance to reliably works?
> >>
> >> If so do you have any recommendations for the implementation or any blog
> >> post / conference I should look at to learn this?
> >>
> >> Cheers
> >> Yoann
> >>
> >>
>
>
Re: Learning steps / Artemis and bidirectional federated queue for IoT context.
Posted by Yoann Gini <yo...@gmail.com>.
Thanks for your input Justin!
Maybe IoT was missleading here, if I speak about EdgeComputing does it change your answer?
One of the reasons I was looking for that architecture and that I forgot to mention was the ability to use also the MQ between multiple services locally to the edge.
So each edge would be autonomous for their own internal messaging and then have federations for upstream communications.
I was in my context forgetting that IoT usually mean low end device.
In my context it’s IoT but with 64 GB of RAM, i7 CPU and Cuda capabilities on the GPU.
So maybe Edge Computer is more appropriate wording.
Yoann
> Le 14 mars 2023 à 20:02, Justin Bertram <jb...@apache.org> a écrit :
>
> Generally speaking, the software that runs on IoT devices is designed with
> the unique constraints of IoT in mind (e.g. poor or intermittent network
> connectivity, limited processing capacity, small amount of memory, low
> power, etc.).
>
> MQTT, for example, is often used for messaging in IoT contexts and it is
> designed to be light-weight and has features like will messages, retained
> messages, link stealing, session expiration, etc. to facilitate unique IoT
> use-cases.
>
> Furthermore, MQTT client implementations offer additional features not
> outlined in the MQTT specification. For example, most Paho MQTT clients [1]
> offer the ability to automatically reconnect, buffer messages when offline
> (which will be sent automatically when the connection is re-established),
> and persist messages (in case the application crashes before it has a
> chance to send them).
>
> In my opinion, adding a broker to an IoT device is not a good idea. It's
> adding complexity where it isn't really required. It will make
> configuration, deployment, and debugging more difficult, and it will
> require more hardware (which is notoriously limited in IoT). A good MQTT
> client implementation should be able to provide you with all the features
> you need to operate effectively in an IoT context.
>
>
> Justin
>
> [1] https://www.eclipse.org/paho/index.php?page=downloads.php
>
>> On Tue, Mar 14, 2023 at 8:52 AM Yoann Gini <yo...@gmail.com> wrote:
>>
>> Hello,
>>
>> I'm new here and I'm working on an IoT issue where I'm wondering if
>> Artemis and federated queue could help. I'm learning the software in the
>> same time I'm trying to achieve something, so I would like to have your
>> opinion on this: will it work? Is it a good idea? And do you have sample
>> config / blog post to recommend to learn to achieve that?
>>
>> The idea: IoT devices with some computing capability running local needed
>> services as well as an Artemis local instance, with upstream set a Cloud
>> located one.
>>
>> Configuration in bidirectional to allow both Cloud to pass commands to IoT
>> and IoT to report feedback to Cloud.
>>
>> Goal of Artemis here in the IoT as federated layer (and not directly IoT
>> service connected to the cloud one) would be to provide some sort of
>> caching capabilities which would allow the local service to not have to
>> handle disconnected state. Local service will still be able to publish to
>> local Artemis and read what was lastly make available before a connectivity
>> cut. And when the Internet recovers, the federation (if I understand it
>> well) should resume bidirectional transfer of feedback and orders.
>>
>> Does that make sense for you? Is it a good usage of the solution that has
>> some chance to reliably works?
>>
>> If so do you have any recommendations for the implementation or any blog
>> post / conference I should look at to learn this?
>>
>> Cheers
>> Yoann
>>
>>
Re: Learning steps / Artemis and bidirectional federated queue for IoT context.
Posted by Justin Bertram <jb...@apache.org>.
Generally speaking, the software that runs on IoT devices is designed with
the unique constraints of IoT in mind (e.g. poor or intermittent network
connectivity, limited processing capacity, small amount of memory, low
power, etc.).
MQTT, for example, is often used for messaging in IoT contexts and it is
designed to be light-weight and has features like will messages, retained
messages, link stealing, session expiration, etc. to facilitate unique IoT
use-cases.
Furthermore, MQTT client implementations offer additional features not
outlined in the MQTT specification. For example, most Paho MQTT clients [1]
offer the ability to automatically reconnect, buffer messages when offline
(which will be sent automatically when the connection is re-established),
and persist messages (in case the application crashes before it has a
chance to send them).
In my opinion, adding a broker to an IoT device is not a good idea. It's
adding complexity where it isn't really required. It will make
configuration, deployment, and debugging more difficult, and it will
require more hardware (which is notoriously limited in IoT). A good MQTT
client implementation should be able to provide you with all the features
you need to operate effectively in an IoT context.
Justin
[1] https://www.eclipse.org/paho/index.php?page=downloads.php
On Tue, Mar 14, 2023 at 8:52 AM Yoann Gini <yo...@gmail.com> wrote:
> Hello,
>
> I'm new here and I'm working on an IoT issue where I'm wondering if
> Artemis and federated queue could help. I'm learning the software in the
> same time I'm trying to achieve something, so I would like to have your
> opinion on this: will it work? Is it a good idea? And do you have sample
> config / blog post to recommend to learn to achieve that?
>
> The idea: IoT devices with some computing capability running local needed
> services as well as an Artemis local instance, with upstream set a Cloud
> located one.
>
> Configuration in bidirectional to allow both Cloud to pass commands to IoT
> and IoT to report feedback to Cloud.
>
> Goal of Artemis here in the IoT as federated layer (and not directly IoT
> service connected to the cloud one) would be to provide some sort of
> caching capabilities which would allow the local service to not have to
> handle disconnected state. Local service will still be able to publish to
> local Artemis and read what was lastly make available before a connectivity
> cut. And when the Internet recovers, the federation (if I understand it
> well) should resume bidirectional transfer of feedback and orders.
>
> Does that make sense for you? Is it a good usage of the solution that has
> some chance to reliably works?
>
> If so do you have any recommendations for the implementation or any blog
> post / conference I should look at to learn this?
>
> Cheers
> Yoann
>
>