You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@edgent.apache.org by Gayashan Amarasinghe <ga...@gmail.com> on 2017/05/31 03:11:46 UTC
Looking for samples and some help on Loosely coupled Edgent applications
Hi Edgent community,
I am a PhD student from University of Melbourne and I am interested in the
Apache Edgent project. I need some help to implement a scenario to submit a
topology externally to a provider. Unfortunately apart from some test cases
in the source code and [1], I couldn't find any documentation on this. So
here's what I have done so far,
- I have implemented the TopologyBuilder interface and created a
separate jar. This is the code in the class.
public class MyTopology implements TopologyBuilder {
@Override
public String getName() {
return "MyCPUTempApp";
}
@Override
public BiConsumer<Topology, JsonObject> getBuilder() {
CPUTempSensor cpuTempSensor = new CPUTempSensor();
return (t, c) -> t.poll(cpuTempSensor, 2, TimeUnit.SECONDS)
.filter(readings -> readings >
35).sink(System.out::println);
}
}
- In a separate project, I have created another class to instantiate a
DirectProvider and to load the previous jar using an AppService similar to
the IotAppServiceTest. Here's the code,
public class DynamicAppLoader {
public static void main(String[] args) {
DirectProvider directProvider = new DirectProvider();
JsonControlService jsonControlService = new JsonControlService();
directProvider.getServices().addService(ControlService.class,
jsonControlService);
ApplicationService service =
AppService.createAndRegister(directProvider, directProvider);
File cpuTempAppJar =
new
File("/home/gayashan/projects/research/cputemptopology/target/cputemptopology-1.0.0.jar");
try {
URL cpuTempAppJarUrl = cpuTempAppJar.toURI().toURL();
JsonObject registerJar =
newRegisterJarRequest(cpuTempAppJarUrl.toExternalForm());
jsonControlService.controlRequest(registerJar);
JsonObject submitCpuTempApp = newSubmitRequest("MyCPUTempApp");
jsonControlService.controlRequest(submitCpuTempApp);
System.out.println(service.getApplicationNames());
} catch (Exception e) {
System.err.println("Error: " + e.getLocalizedMessage());
}
}
...
// newRegisterJarRequest and newSubmitRequest methods are as same as in
the IotAppServiceTest class.
}
But unfortunately I don't understand how to get this to work. Can someone
help me with this and point where I am making a mistake? Apologies in
advance if this seems like a trivial question.
Thank you.
[1]
https://edgent.apache.org/recipes/recipe_dynamic_analytic_control.html#loosely-coupled-edgent-applications
Best regards,
Gayashan
Re: Looking for samples and some help on Loosely coupled Edgent
applications
Posted by Dale LaBossiere <dm...@gmail.com>.
Glad to help!
As you’ve seen, the JsonControlService impl has support for mapping a JSON object to a method invocation on a registered control bean. IotProvider then adds a builtin topology/job that maps an external “device command” to use that JsonControlService feature. The net is when you use IotProvider you don’t have to write any Edgent app machinery to support method invocation on a registered control bean. That includes invoking registerJar().
Edgent applications can define and register their own control beans too.
IotProvider also allows you to say “please build and submit this topology when the provider is started” (in addition to supporting submit via a remote submit device cmd).
There’s a sample code fragment for creating/initializing an IotProvider in its javadoc (at least the latest version, not sure about 1.1.0). Be sure to look at its package javadoc too for sample device commands.
One sample use of IotProvider (and IotDevice) is the IotpFullScenario sample[1]
The EdgeVideoAnalytics demo system uses it and also includes IoT hub clients that “send commands to" (publish device commands for) the Edgent app)[2]
That said, neither of those utilize the registerJar capability… though both inherently support it.
A third, preferred?, option for locally invoking registerJar and submit is to just lookup and use the registered ApplicationServiceMXBean created by AppService.createAndRegister(). e.g.
ApplicationServiceMXBean appControl = controlService.getControl(
ApplicationServiceMXBean.TYPE,
ApplicationService.ALIAS,
ApplicationServiceMXBean.class);
appControl.registerJar(…)
appControl.submit(…)
I say preferred because I think of the JSON / controlRequest() mechanism as existing mostly to support external/remote method invocation spec use case. (Which IotProvider does for you)
By “sample” I just mean some runnable code (and run script) that’s part of the Edgent distribution that demonstrates a feature. Just something under “samples” like IotFullScenario as opposed to a recipe on the website.
— Dale
[1] https://github.com/apache/incubator-edgent/blob/master/samples/scenarios/src/main/java/org/apache/edgent/samples/scenarios/iotp/IotpFullScenario.java <https://github.com/apache/incubator-edgent/blob/master/samples/scenarios/src/main/java/org/apache/edgent/samples/scenarios/iotp/IotpFullScenario.java>
[2] https://github.com/IBMStreams/streamsx.edgevideoanalytics.starterkit <https://github.com/IBMStreams/streamsx.edgevideoanalytics.starterkit>
Re: Looking for samples and some help on Loosely coupled Edgent applications
Posted by Gayashan Amarasinghe <ga...@gmail.com>.
Hi Dale,
Thank you for the details. Yes, my use case is to dynamically load and
control topologies run on edge devices by an external application. So as a
starter I am running a simple scenario. When the DynamicAppLoader class is
run, it'll simply load the jar containing the topology and execute it
straightaway. I tried doing this in two ways directly calling registerJar
and then calling it via reflection.
1. using AppService#registerJar method which is followed by a "submit"
control request.
service.registerJar(cpuTempAppJarUrl.toExternalForm(), null);
JsonObject submitCpuTempAppRequest = newOperationRequest("MyCPUTempApp",
"submit");
jsonControlService.controlRequest(submitCpuTempAppRequest);
2. using a "registerJar" control request which is followed by a "submit"
control request
JsonObject registerJarRequest =
newOperationRequest(cpuTempAppJarUrl.toExternalForm(), "registerJar");
jsonControlService.controlRequest(registerJarRequest);
JsonObject submitCpuTempAppRequest = newOperationRequest("MyCPUTempApp",
"submit");
jsonControlService.controlRequest(submitCpuTempAppRequest);
Are there any other (*recommended*) ways to do this?
I would be glad to contribute to the doc and code, and perhaps with a
sample as well. What do you mean by a sample exactly? It is not clear to me.
I'll check out the IotProvider as well. It seems interesting, but I don't
yet understand how to instantiate an IotProvider. I also need to understand
how the ControlService works as well. Need to dig in to the documents and
code more.
Thank you again for the support.
/Gayashan
On Thu, Jun 1, 2017 at 2:01 AM, Dale LaBossiere <dm...@gmail.com>
wrote:
> Yeah, this is a ~confusing and an under doc’d area.
> Glad you’re back on track.
>
> There has been some improvement in recent time - e.g., [1][2][3] all post
> 1.1.0.
> Any concrete suggestions regarding that would be appreciated.
> Please consider contributing doc improvements, a new sample or recipe,
> etc! :-)
>
> If you were not aware of IotProvider[4], you may want to check it out.
> IotProvider includes those services you’re manually instantiating/adding to
> your DirectProvider.
>
> Just to be sure… registerJar() is mostly about being able to dynamically
> add topology implementations to a running Edgent app/provider. Is that
> your use case / can you elaborate on your use case?
>
> — Dale
>
> [1] https://issues.apache.org/jira/browse/EDGENT-406
> [2] https://issues.apache.org/jira/browse/EDGENT-405
> [3] https://issues.apache.org/jira/browse/EDGENT-401
> [4] https://github.com/apache/incubator-edgent/blob/master/
> api/topology/src/main/java/org/apache/edgent/topology/
> services/ApplicationService.java <https://github.com/apache/
> incubator-edgent/blob/master/api/topology/src/main/java/
> org/apache/edgent/topology/services/ApplicationService.java>
>
> > On May 31, 2017, at 4:49 AM, Gayashan Amarasinghe <
> gayashan.amarasinghe@gmail.com> wrote:
> >
> > Hi all,
> >
> > After more digging I was able to figure out the issue. In my first jar (
> > cputemptopology-1.0.0.jar) which contains the TopologyBuilder
> > implementation, I haven't registered the service provider and hence the
> > ServiceLoader was unable to pick up the service. After fixing this I was
> > able to get it running.
> >
> > So my current cputemptopology-1.0.0.jar has the following structure.
> >
> > ├── META-INF
> > │ ├── MANIFEST.MF
> > │ ├── maven
> > │ │ └── org.amrsnghe.test.topology
> > │ │ └── cputemptopology
> > │ │ ├── pom.properties
> > │ │ └── pom.xml
> > │ └── services
> > │ └── org.apache.edgent.topology.services.TopologyBuilder
> > └── org
> > └── amrsnghe
> > └── test
> > └── topology
> > ├── CPUTempSensor.class
> > └── MyTopology.class
> >
> > And
> > the META-INF/services/org.apache.edgent.topology.services.
> TopologyBuilder
> > has the following line:
> > org.amrsnghe.test.topology.MyTopology
> >
> > Let me know if I am doing something wrong here. But this seems to work,
> and
> > the topology loaded from the jar is submitted to the direct provider and
> > executed without an issue.
> >
> > Thank you.
> >
> > /Gayashan
> >
> > On Wed, May 31, 2017 at 1:11 PM, Gayashan Amarasinghe <
> > gayashan.amarasinghe@gmail.com> wrote:
> >
> >> Hi Edgent community,
> >>
> >> I am a PhD student from University of Melbourne and I am interested in
> the
> >> Apache Edgent project. I need some help to implement a scenario to
> submit a
> >> topology externally to a provider. Unfortunately apart from some test
> cases
> >> in the source code and [1], I couldn't find any documentation on this.
> So
> >> here's what I have done so far,
> >>
> >> - I have implemented the TopologyBuilder interface and created a
> >> separate jar. This is the code in the class.
> >>
> >> public class MyTopology implements TopologyBuilder {
> >>
> >> @Override
> >>
> >> public String getName() {
> >>
> >> return "MyCPUTempApp";
> >>
> >> }
> >>
> >>
> >> @Override
> >>
> >> public BiConsumer<Topology, JsonObject> getBuilder() {
> >>
> >> CPUTempSensor cpuTempSensor = new CPUTempSensor();
> >>
> >> return (t, c) -> t.poll(cpuTempSensor, 2, TimeUnit.SECONDS)
> >>
> >> .filter(readings -> readings >
> >> 35).sink(System.out::println);
> >>
> >> }
> >>
> >> }
> >>
> >>
> >> - In a separate project, I have created another class to instantiate a
> >> DirectProvider and to load the previous jar using an AppService
> similar to
> >> the IotAppServiceTest. Here's the code,
> >>
> >> public class DynamicAppLoader {
> >> public static void main(String[] args) {
> >> DirectProvider directProvider = new DirectProvider();
> >> JsonControlService jsonControlService = new JsonControlService();
> >> directProvider.getServices().addService(ControlService.class,
> >> jsonControlService);
> >> ApplicationService service = AppService.createAndRegister(
> directProvider,
> >> directProvider);
> >> File cpuTempAppJar =
> >> new File("/home/gayashan/projects/
> >> research/cputemptopology/target/cputemptopology-1.0.0.jar");
> >> try {
> >> URL cpuTempAppJarUrl = cpuTempAppJar.toURI().toURL();
> >> JsonObject registerJar = newRegisterJarRequest(
> >> cpuTempAppJarUrl.toExternalForm());
> >> jsonControlService.controlRequest(registerJar);
> >> JsonObject submitCpuTempApp = newSubmitRequest("
> >> MyCPUTempApp");
> >> jsonControlService.controlRequest(submitCpuTempApp);
> >> System.out.println(service.getApplicationNames());
> >> } catch (Exception e) {
> >> System.err.println("Error: " + e.getLocalizedMessage());
> >> }
> >> }
> >>
> >> ...
> >> // newRegisterJarRequest and newSubmitRequest methods are as same as in
> >> the IotAppServiceTest class.
> >> }
> >>
> >> But unfortunately I don't understand how to get this to work. Can
> someone
> >> help me with this and point where I am making a mistake? Apologies in
> >> advance if this seems like a trivial question.
> >>
> >> Thank you.
> >>
> >> [1] https://edgent.apache.org/recipes/recipe_dynamic_
> >> analytic_control.html#loosely-coupled-edgent-applications
> >>
> >> Best regards,
> >> Gayashan
> >>
>
>
Re: Looking for samples and some help on Loosely coupled Edgent
applications
Posted by Dale LaBossiere <dm...@gmail.com>.
Yeah, this is a ~confusing and an under doc’d area.
Glad you’re back on track.
There has been some improvement in recent time - e.g., [1][2][3] all post 1.1.0.
Any concrete suggestions regarding that would be appreciated.
Please consider contributing doc improvements, a new sample or recipe, etc! :-)
If you were not aware of IotProvider[4], you may want to check it out. IotProvider includes those services you’re manually instantiating/adding to your DirectProvider.
Just to be sure… registerJar() is mostly about being able to dynamically add topology implementations to a running Edgent app/provider. Is that your use case / can you elaborate on your use case?
— Dale
[1] https://issues.apache.org/jira/browse/EDGENT-406
[2] https://issues.apache.org/jira/browse/EDGENT-405
[3] https://issues.apache.org/jira/browse/EDGENT-401
[4] https://github.com/apache/incubator-edgent/blob/master/api/topology/src/main/java/org/apache/edgent/topology/services/ApplicationService.java <https://github.com/apache/incubator-edgent/blob/master/api/topology/src/main/java/org/apache/edgent/topology/services/ApplicationService.java>
> On May 31, 2017, at 4:49 AM, Gayashan Amarasinghe <ga...@gmail.com> wrote:
>
> Hi all,
>
> After more digging I was able to figure out the issue. In my first jar (
> cputemptopology-1.0.0.jar) which contains the TopologyBuilder
> implementation, I haven't registered the service provider and hence the
> ServiceLoader was unable to pick up the service. After fixing this I was
> able to get it running.
>
> So my current cputemptopology-1.0.0.jar has the following structure.
>
> ├── META-INF
> │ ├── MANIFEST.MF
> │ ├── maven
> │ │ └── org.amrsnghe.test.topology
> │ │ └── cputemptopology
> │ │ ├── pom.properties
> │ │ └── pom.xml
> │ └── services
> │ └── org.apache.edgent.topology.services.TopologyBuilder
> └── org
> └── amrsnghe
> └── test
> └── topology
> ├── CPUTempSensor.class
> └── MyTopology.class
>
> And
> the META-INF/services/org.apache.edgent.topology.services.TopologyBuilder
> has the following line:
> org.amrsnghe.test.topology.MyTopology
>
> Let me know if I am doing something wrong here. But this seems to work, and
> the topology loaded from the jar is submitted to the direct provider and
> executed without an issue.
>
> Thank you.
>
> /Gayashan
>
> On Wed, May 31, 2017 at 1:11 PM, Gayashan Amarasinghe <
> gayashan.amarasinghe@gmail.com> wrote:
>
>> Hi Edgent community,
>>
>> I am a PhD student from University of Melbourne and I am interested in the
>> Apache Edgent project. I need some help to implement a scenario to submit a
>> topology externally to a provider. Unfortunately apart from some test cases
>> in the source code and [1], I couldn't find any documentation on this. So
>> here's what I have done so far,
>>
>> - I have implemented the TopologyBuilder interface and created a
>> separate jar. This is the code in the class.
>>
>> public class MyTopology implements TopologyBuilder {
>>
>> @Override
>>
>> public String getName() {
>>
>> return "MyCPUTempApp";
>>
>> }
>>
>>
>> @Override
>>
>> public BiConsumer<Topology, JsonObject> getBuilder() {
>>
>> CPUTempSensor cpuTempSensor = new CPUTempSensor();
>>
>> return (t, c) -> t.poll(cpuTempSensor, 2, TimeUnit.SECONDS)
>>
>> .filter(readings -> readings >
>> 35).sink(System.out::println);
>>
>> }
>>
>> }
>>
>>
>> - In a separate project, I have created another class to instantiate a
>> DirectProvider and to load the previous jar using an AppService similar to
>> the IotAppServiceTest. Here's the code,
>>
>> public class DynamicAppLoader {
>> public static void main(String[] args) {
>> DirectProvider directProvider = new DirectProvider();
>> JsonControlService jsonControlService = new JsonControlService();
>> directProvider.getServices().addService(ControlService.class,
>> jsonControlService);
>> ApplicationService service = AppService.createAndRegister(directProvider,
>> directProvider);
>> File cpuTempAppJar =
>> new File("/home/gayashan/projects/
>> research/cputemptopology/target/cputemptopology-1.0.0.jar");
>> try {
>> URL cpuTempAppJarUrl = cpuTempAppJar.toURI().toURL();
>> JsonObject registerJar = newRegisterJarRequest(
>> cpuTempAppJarUrl.toExternalForm());
>> jsonControlService.controlRequest(registerJar);
>> JsonObject submitCpuTempApp = newSubmitRequest("
>> MyCPUTempApp");
>> jsonControlService.controlRequest(submitCpuTempApp);
>> System.out.println(service.getApplicationNames());
>> } catch (Exception e) {
>> System.err.println("Error: " + e.getLocalizedMessage());
>> }
>> }
>>
>> ...
>> // newRegisterJarRequest and newSubmitRequest methods are as same as in
>> the IotAppServiceTest class.
>> }
>>
>> But unfortunately I don't understand how to get this to work. Can someone
>> help me with this and point where I am making a mistake? Apologies in
>> advance if this seems like a trivial question.
>>
>> Thank you.
>>
>> [1] https://edgent.apache.org/recipes/recipe_dynamic_
>> analytic_control.html#loosely-coupled-edgent-applications
>>
>> Best regards,
>> Gayashan
>>
Re: Looking for samples and some help on Loosely coupled Edgent applications
Posted by Gayashan Amarasinghe <ga...@gmail.com>.
Hi all,
After more digging I was able to figure out the issue. In my first jar (
cputemptopology-1.0.0.jar) which contains the TopologyBuilder
implementation, I haven't registered the service provider and hence the
ServiceLoader was unable to pick up the service. After fixing this I was
able to get it running.
So my current cputemptopology-1.0.0.jar has the following structure.
├── META-INF
│ ├── MANIFEST.MF
│ ├── maven
│ │ └── org.amrsnghe.test.topology
│ │ └── cputemptopology
│ │ ├── pom.properties
│ │ └── pom.xml
│ └── services
│ └── org.apache.edgent.topology.services.TopologyBuilder
└── org
└── amrsnghe
└── test
└── topology
├── CPUTempSensor.class
└── MyTopology.class
And
the META-INF/services/org.apache.edgent.topology.services.TopologyBuilder
has the following line:
org.amrsnghe.test.topology.MyTopology
Let me know if I am doing something wrong here. But this seems to work, and
the topology loaded from the jar is submitted to the direct provider and
executed without an issue.
Thank you.
/Gayashan
On Wed, May 31, 2017 at 1:11 PM, Gayashan Amarasinghe <
gayashan.amarasinghe@gmail.com> wrote:
> Hi Edgent community,
>
> I am a PhD student from University of Melbourne and I am interested in the
> Apache Edgent project. I need some help to implement a scenario to submit a
> topology externally to a provider. Unfortunately apart from some test cases
> in the source code and [1], I couldn't find any documentation on this. So
> here's what I have done so far,
>
> - I have implemented the TopologyBuilder interface and created a
> separate jar. This is the code in the class.
>
> public class MyTopology implements TopologyBuilder {
>
> @Override
>
> public String getName() {
>
> return "MyCPUTempApp";
>
> }
>
>
> @Override
>
> public BiConsumer<Topology, JsonObject> getBuilder() {
>
> CPUTempSensor cpuTempSensor = new CPUTempSensor();
>
> return (t, c) -> t.poll(cpuTempSensor, 2, TimeUnit.SECONDS)
>
> .filter(readings -> readings >
> 35).sink(System.out::println);
>
> }
>
> }
>
>
> - In a separate project, I have created another class to instantiate a
> DirectProvider and to load the previous jar using an AppService similar to
> the IotAppServiceTest. Here's the code,
>
> public class DynamicAppLoader {
> public static void main(String[] args) {
> DirectProvider directProvider = new DirectProvider();
> JsonControlService jsonControlService = new JsonControlService();
> directProvider.getServices().addService(ControlService.class,
> jsonControlService);
> ApplicationService service = AppService.createAndRegister(directProvider,
> directProvider);
> File cpuTempAppJar =
> new File("/home/gayashan/projects/
> research/cputemptopology/target/cputemptopology-1.0.0.jar");
> try {
> URL cpuTempAppJarUrl = cpuTempAppJar.toURI().toURL();
> JsonObject registerJar = newRegisterJarRequest(
> cpuTempAppJarUrl.toExternalForm());
> jsonControlService.controlRequest(registerJar);
> JsonObject submitCpuTempApp = newSubmitRequest("
> MyCPUTempApp");
> jsonControlService.controlRequest(submitCpuTempApp);
> System.out.println(service.getApplicationNames());
> } catch (Exception e) {
> System.err.println("Error: " + e.getLocalizedMessage());
> }
> }
>
> ...
> // newRegisterJarRequest and newSubmitRequest methods are as same as in
> the IotAppServiceTest class.
> }
>
> But unfortunately I don't understand how to get this to work. Can someone
> help me with this and point where I am making a mistake? Apologies in
> advance if this seems like a trivial question.
>
> Thank you.
>
> [1] https://edgent.apache.org/recipes/recipe_dynamic_
> analytic_control.html#loosely-coupled-edgent-applications
>
> Best regards,
> Gayashan
>