You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iota.apache.org by Gunnar Tapper <ta...@gmail.com> on 2017/02/05 06:30:11 UTC
Do you folks need help?
Hi,
I've been interested in IoT since I first read about Dr. Peter Hartwell's
IoT work in 2010. In previous products, I've used IoT-inspired design to
capture and analyze metrics from hardware, OS, databases, etc. Now, I think
it's time to take the next step and work on the real thing. :)
I'm a committer for Apache Trafodion where I created the website, set up
the documentation tree, wrote the contributor's guide, and designed the
next-generation installer. I want to do more hands on work for IoT but
could obviously document as I learn if that'd be useful for y'all. For
Trafodion, I used asciidoc but that's not a must.
So, do you need help and if so, where should I start? I'm studying the code
tree and the Jiras but I find that it's hard to get going based on just
"pick up one of the open Jiras", especially when you don't fully grasp the
terminology yet.
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Do you folks need help?
Posted by Gunnar Tapper <ta...@gmail.com>.
Hi Tony,
This is great. Let's add it to an introduction on the iota website. I
assume that the website is kept in SVN? I think you have to be a committer
to get to that part. If correct in both, it might make sense to add the
website to the git repository.
This is how we handled that part in Trafodion:
https://cwiki.apache.org/confluence/display/TRAFODION/Modify+Web+Site
Thanks,
Gunnar
On Sun, Feb 5, 2017 at 1:31 AM, Tony Faustini <to...@litbit.com> wrote:
> Hi Gunnar, absolutely we could do with your help. Thanks for offering. At
> Litbit iota is an integral part of the IoT platform we are building. It is
> an integral part of the SMACK stack (Spark, Mesos, Akka, Cassandra and
> Kafka) on which the platform is based. I assume you have taken a look at
> the iota website (http://iota.incubator.apache.org/) if not it would be a
> good place to start.
>
> Let’s begin with an overview of iota. iota consists of
>
> 1) The Fey engine - An Akka based framework that facilitates the
> definition of Fey actors each actor implementing a Fey component. Fey
> actors extend a generic Fey Actor FeyGenericActor and override generic
> functions, like onStart and execute , to define an autonomous computation.
> Each Fey actor should be provided through a .jar which the Fey engine loads
> to access the actors functionality. Note this comes from the README.md file
> under the fey-core directory. There is a lot of technical information in
> the README.md file. Take a look and let us know what you think? Perhaps we
> need to have a new place for this documentation.
>
> 2) Fey performers - Think of them as standalone continously operating
> agents/actors/functions. The can process messages that are sent to them,
> they can send messages down stream to other performers. Performers can also
> directly interact with the technologies that they interact with. For
> example a Kafka performer might directly interact with a Kafka Broker
> reading messages of a particular type and sending them on as messages in
> Fey to other performers that they are connected to. The same Kafta
> performer might process messages that it is being sent from other
> Performers and send them as messages to the Kafka Broker.
>
> Here is the “Hello World” performer that will propagate the message
> “Hello World” to any connected performers downstream. It will also
> propagate “Hello World” when it receives a message from performers upstream
> from the performer. Here is a snippet of code for such a performer: The
> performer below would generate a fey-HelloWorld.jar file that is what get’s
> used to deploy the Performer
>
> package org.apache.iota.fey.performer
>
> import akka.actor.ActorRef
> import org.apache.iota.fey.FeyGenericActor
>
> import scala.collection.immutable.Map
> import scala.concurrent.duration._
>
> class HelloWorld(override val params: Map[String, String] = Map.empty,
> override val backoff: FiniteDuration = 1.minutes,
> override val connectTo: Map[String, ActorRef] = Map.empty,
> override val schedulerTimeInterval: FiniteDuration
> = 30.seconds,
> override val orchestrationName: String = "",
> override val orchestrationID: String = "",
> override val autoScale: Boolean = false) extends FeyGenericActor
> {
>
> override def onStart = {
> }
>
> override def onStop = {
> }
>
> override def onRestart(reason: Throwable) = {
> }
>
> override def processMessage[T](message: T, sender: ActorRef): Unit = {
> propagateMessage(“Hello World”)
> }
>
> override def execute() = {
> propagateMessage(“Hello World")
> }
>
> Note each performer in iota should supply a README.md file that explains
> what it does and gives a sample of how it is connected to other performers.
> A network of integrating performers is known as an orchestration when each
> performer is operating autonomously. Here is an example of an orchestration
> for the HelloWorld performer. Note you could run this orchestration
> without knowing the inner workings (coding) of any of the performers used.
> They are just software components with a well know behavior that you can
> put together in a network. The network below is degenerate in that it is
> simple a single performer that is running continuously calling the
> execute method of the performer every 1000 milliseconds (1 second). Since
> it’s not connected to anything nothing can send it a message so the
> processMessage method of the performer is never called here.
>
> {
> "guid": “Hello World Orchestration",
> "command": "CREATE",
> "timestamp": "591997890",
> "name": “HelloWorld",
> "ensembles": [
> {
> "guid": “MyHelloWorldEnsemble",
> "command": "NONE",
> "performers": [
> {
> "guid": "Timestamp",
> "schedule": 1000,
> "backoff": 0,
> "source": {
> "name": “fey-helloWorld.jar",
> "classPath": "org.apache.iota.fey.performer.HelloWorld",
> "parameters": {
> }
> }
> }
> ],
> "connections": [
> ]
> }
> ]
> }
>
> Note the Fey engine uses a repository of performers (.jar file) and one or
> more orchestrations to define a computation and hence compute. The
> repository of performers and the orchestrations are stored in well defined
> locations see the README.md documents for details.
>
> Let’s create a a more complete orchestration. We first define a performer
> which bundles HelloWorld, Alive and LogMessage functionality into a single
> performer (.jar file). We will call this the tutorial.jar
>
> package org.apache.iota.fey.performer
>
> import akka.actor.ActorRef
> import org.apache.iota.fey.FeyGenericActor
>
> import scala.collection.immutable.Map
> import scala.concurrent.duration._
>
> class HelloWorld(override val params: Map[String, String] = Map.empty,
> override val backoff: FiniteDuration = 1.minutes,
> override val connectTo: Map[String, ActorRef] = Map.empty,
> override val schedulerTimeInterval: FiniteDuration
> = 30.seconds,
> override val orchestrationName: String = "",
> override val orchestrationID: String = "",
> override val autoScale: Boolean = false) extends FeyGenericActor
> {
>
> override def processMessage[T](message: T, sender: ActorRef): Unit = {
> propagateMessage(“Hello World”)
> }
>
> override def execute() = {
> propagateMessage(“Hello World")
> }
>
> package org.apache.iota.fey.performer
>
> import akka.actor.ActorRef
> import org.apache.iota.fey.FeyGenericActor
>
> import scala.collection.immutable.Map
> import scala.concurrent.duration._
>
> class Alive(override val params: Map[String, String] = Map.empty,
> override val backoff: FiniteDuration = 1.minutes,
> override val connectTo: Map[String, ActorRef] = Map.empty,
> override val schedulerTimeInterval: FiniteDuration
> = 30.seconds,
> override val orchestrationName: String = "",
> override val orchestrationID: String = "",
> override val autoScale: Boolean = false) extends FeyGenericActor
> {
>
> override def execute() = {
> propagateMessage(“Alive")
> }
>
> package org.apache.iota.fey.performer
>
> import akka.actor.ActorRef
> import org.apache.iota.fey.FeyGenericActor
>
> import scala.collection.immutable.Map
> import scala.concurrent.duration._
> class LogMessage(override val params: Map[String, String] = Map.empty,
> override val backoff: FiniteDuration = 1.minutes,
> override val connectTo: Map[String, ActorRef] = Map.empty,
> override val schedulerTimeInterval: FiniteDuration
> = 30.seconds,
> override val orchestrationName: String = "",
> override val orchestrationID: String = "",
> override val autoScale: Boolean = false) extends FeyGenericActor
> {
>
> override def processMessage[T](message: T, sender: ActorRef): Unit = {
> log.info(message)
> }
>
> The following Orchestration shows how performers can interact with each
> other. Note that in the ConnectedWorldEnsemble instantiates three “actors"
> an instance of the Alive performer is connected an instance of the
> HelloWorld performer. The HelloWorld Performer is connected to the
> LogMessage Performer. This orchestration is driven by messages being
> generated every second by the Alive Performer. The HelloWorld performer
> will itself generated the “Hello World” message on receiving the “Alive”
> message. In turn the logMessage performer will print the message “Hello
> World” to the log file.
>
> {
> "guid": “Connected Orchestration",
> "command": "CREATE",
> "timestamp": "591997890",
> "name": “ConnectedWorldEnsembles",
> "ensembles": [
> {
> "guid": “ConnectedWorldEnsemble",
> "command": "NONE",
> "performers": [
> {
> "guid": “Alive",
> "schedule": 1000,
> "backoff": 0,
> "source": {
> "name": “fey-tutorial.jar",
> "classPath": "org.apache.iota.fey.performer.Alive",
> "parameters": {
> }
> }
> },
> {
> "guid": “HellowWorld",
> "schedule": 0,
> "backoff": 0,
> "source": {
> "name": “fey-tutorial.jar",
> "classPath": "org.apache.iota.fey.performer.HelloWorld",
> "parameters": {
> }
> }
> },
> {
> "guid": “LogMessage",
> "schedule": 0,
> "backoff": 0,
> "source": {
> "name": “fey-tutorial.jar",
> "classPath": "org.apache.iota.fey.performer.LogMessage",
> "parameters": {
> }
> }
> }
> ],
> "connections": [ { “Alive": [
> "HelloWorld"
> ]
> } ,
> { “HelloWorld": [
> “LogMessage"
> ]
> }
> ]
> }
> ]
> }
>
> Gunnar is this helpful? Note for brevity I may have skipped some the code
> details in the snippets above.
>
> Thanks
>
> -Tony
>
>
>
>
--
Thanks,
Gunnar
*If you think you can you can, if you think you can't you're right.*
Re: Do you folks need help?
Posted by Tony Faustini <to...@litbit.com>.
Hi Gunnar, absolutely we could do with your help. Thanks for offering. At Litbit iota is an integral part of the IoT platform we are building. It is an integral part of the SMACK stack (Spark, Mesos, Akka, Cassandra and Kafka) on which the platform is based. I assume you have taken a look at the iota website (http://iota.incubator.apache.org/ <http://iota.incubator.apache.org/>) if not it would be a good place to start.
Let’s begin with an overview of iota. iota consists of
1) The Fey engine - An Akka based framework that facilitates the definition of Fey actors each actor implementing a Fey component. Fey actors extend a generic Fey Actor FeyGenericActor and override generic functions, like onStart and execute , to define an autonomous computation. Each Fey actor should be provided through a .jar which the Fey engine loads to access the actors functionality. Note this comes from the README.md file under the fey-core directory. There is a lot of technical information in the README.md file. Take a look and let us know what you think? Perhaps we need to have a new place for this documentation.
2) Fey performers - Think of them as standalone continously operating agents/actors/functions. The can process messages that are sent to them, they can send messages down stream to other performers. Performers can also directly interact with the technologies that they interact with. For example a Kafka performer might directly interact with a Kafka Broker reading messages of a particular type and sending them on as messages in Fey to other performers that they are connected to. The same Kafta performer might process messages that it is being sent from other Performers and send them as messages to the Kafka Broker.
Here is the “Hello World” performer that will propagate the message “Hello World” to any connected performers downstream. It will also propagate “Hello World” when it receives a message from performers upstream from the performer. Here is a snippet of code for such a performer: The performer below would generate a fey-HelloWorld.jar file that is what get’s used to deploy the Performer
package org.apache.iota.fey.performer
import akka.actor.ActorRef
import org.apache.iota.fey.FeyGenericActor
import scala.collection.immutable.Map
import scala.concurrent.duration._
class HelloWorld(override val params: Map[String, String] = Map.empty,
override val backoff: FiniteDuration = 1.minutes,
override val connectTo: Map[String, ActorRef] = Map.empty,
override val schedulerTimeInterval: FiniteDuration = 30.seconds,
override val orchestrationName: String = "",
override val orchestrationID: String = "",
override val autoScale: Boolean = false) extends FeyGenericActor {
override def onStart = {
}
override def onStop = {
}
override def onRestart(reason: Throwable) = {
}
override def processMessage[T](message: T, sender: ActorRef): Unit = {
propagateMessage(“Hello World”)
}
override def execute() = {
propagateMessage(“Hello World")
}
Note each performer in iota should supply a README.md file that explains what it does and gives a sample of how it is connected to other performers. A network of integrating performers is known as an orchestration when each performer is operating autonomously. Here is an example of an orchestration for the HelloWorld performer. Note you could run this orchestration without knowing the inner workings (coding) of any of the performers used. They are just software components with a well know behavior that you can put together in a network. The network below is degenerate in that it is simple a single performer that is running continuously calling the execute method of the performer every 1000 milliseconds (1 second). Since it’s not connected to anything nothing can send it a message so the processMessage method of the performer is never called here.
{
"guid": “Hello World Orchestration",
"command": "CREATE",
"timestamp": "591997890",
"name": “HelloWorld",
"ensembles": [
{
"guid": “MyHelloWorldEnsemble",
"command": "NONE",
"performers": [
{
"guid": "Timestamp",
"schedule": 1000,
"backoff": 0,
"source": {
"name": “fey-helloWorld.jar",
"classPath": "org.apache.iota.fey.performer.HelloWorld",
"parameters": {
}
}
}
],
"connections": [
]
}
]
}
Note the Fey engine uses a repository of performers (.jar file) and one or more orchestrations to define a computation and hence compute. The repository of performers and the orchestrations are stored in well defined locations see the README.md documents for details.
Let’s create a a more complete orchestration. We first define a performer which bundles HelloWorld, Alive and LogMessage functionality into a single performer (.jar file). We will call this the tutorial.jar
package org.apache.iota.fey.performer
import akka.actor.ActorRef
import org.apache.iota.fey.FeyGenericActor
import scala.collection.immutable.Map
import scala.concurrent.duration._
class HelloWorld(override val params: Map[String, String] = Map.empty,
override val backoff: FiniteDuration = 1.minutes,
override val connectTo: Map[String, ActorRef] = Map.empty,
override val schedulerTimeInterval: FiniteDuration = 30.seconds,
override val orchestrationName: String = "",
override val orchestrationID: String = "",
override val autoScale: Boolean = false) extends FeyGenericActor {
override def processMessage[T](message: T, sender: ActorRef): Unit = {
propagateMessage(“Hello World”)
}
override def execute() = {
propagateMessage(“Hello World")
}
package org.apache.iota.fey.performer
import akka.actor.ActorRef
import org.apache.iota.fey.FeyGenericActor
import scala.collection.immutable.Map
import scala.concurrent.duration._
class Alive(override val params: Map[String, String] = Map.empty,
override val backoff: FiniteDuration = 1.minutes,
override val connectTo: Map[String, ActorRef] = Map.empty,
override val schedulerTimeInterval: FiniteDuration = 30.seconds,
override val orchestrationName: String = "",
override val orchestrationID: String = "",
override val autoScale: Boolean = false) extends FeyGenericActor {
override def execute() = {
propagateMessage(“Alive")
}
package org.apache.iota.fey.performer
import akka.actor.ActorRef
import org.apache.iota.fey.FeyGenericActor
import scala.collection.immutable.Map
import scala.concurrent.duration._
class LogMessage(override val params: Map[String, String] = Map.empty,
override val backoff: FiniteDuration = 1.minutes,
override val connectTo: Map[String, ActorRef] = Map.empty,
override val schedulerTimeInterval: FiniteDuration = 30.seconds,
override val orchestrationName: String = "",
override val orchestrationID: String = "",
override val autoScale: Boolean = false) extends FeyGenericActor {
override def processMessage[T](message: T, sender: ActorRef): Unit = {
log.info(message)
}
The following Orchestration shows how performers can interact with each other. Note that in the ConnectedWorldEnsemble instantiates three “actors" an instance of the Alive performer is connected an instance of the HelloWorld performer. The HelloWorld Performer is connected to the LogMessage Performer. This orchestration is driven by messages being generated every second by the Alive Performer. The HelloWorld performer will itself generated the “Hello World” message on receiving the “Alive” message. In turn the logMessage performer will print the message “Hello World” to the log file.
{
"guid": “Connected Orchestration",
"command": "CREATE",
"timestamp": "591997890",
"name": “ConnectedWorldEnsembles",
"ensembles": [
{
"guid": “ConnectedWorldEnsemble",
"command": "NONE",
"performers": [
{
"guid": “Alive",
"schedule": 1000,
"backoff": 0,
"source": {
"name": “fey-tutorial.jar",
"classPath": "org.apache.iota.fey.performer.Alive",
"parameters": {
}
}
},
{
"guid": “HellowWorld",
"schedule": 0,
"backoff": 0,
"source": {
"name": “fey-tutorial.jar",
"classPath": "org.apache.iota.fey.performer.HelloWorld",
"parameters": {
}
}
},
{
"guid": “LogMessage",
"schedule": 0,
"backoff": 0,
"source": {
"name": “fey-tutorial.jar",
"classPath": "org.apache.iota.fey.performer.LogMessage",
"parameters": {
}
}
}
],
"connections": [ { “Alive": [
"HelloWorld"
]
} ,
{ “HelloWorld": [
“LogMessage"
]
}
]
}
]
}
Gunnar is this helpful? Note for brevity I may have skipped some the code details in the snippets above.
Thanks
-Tony