You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by AndreaKinn <ki...@hotmail.it> on 2017/08/24 15:19:59 UTC
Flink-HTM integration
Hi,
Is there here someone who used Flink-HTM library
https://github.com/htm-community/flink-htm
<https://github.com/htm-community/flink-htm> ?
I'm trying to implement it in my project but I have some fundamental
question to complete my thesis work.
Regards,
Andrea
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-HTM-integration-tp15113.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Flink-HTM integration
Posted by AndreaKinn <ki...@hotmail.it>.
Sure.
Firstly I followed the steps showed here to build the project: flink-htm
github <https://github.com/htm-community/flink-htm/wiki/Building>
In my project I want to perform anomaly detection of values in a stream. I
have a Kafka broker as source:
/DataStream<Tuple6<String, String, Date, String, String, Double>> stream
= env
.addSource(new FlinkKafkaConsumer010<>(TOPIC, new CustomDeserializer(),
properties))
.assignTimestampsAndWatermarks(new CustomTimestampExtractor())
.keyBy(0);/
The double field of Tuple6 is my target value. I tried to follow the example
find in the code download by GitHub, so I transformed my streams of Tuple6
in a stream of KafkaRecord (an object who has just a field value)
/DataStream<Harness.KafkaRecord> kafkaStream = stream.map(new
MapFunction<Tuple6<String, String, Date, String, String, Double>,
Harness.KafkaRecord>(){
@Override
public Harness.KafkaRecord map(Tuple6<String, String, Date, String,
String, Double> value) throws Exception {
return new Harness.KafkaRecord(value.f5);
}
}); /
Note that if I print stream or kafkaStream everything works fine.
So I used HTM functions as in the example:
/DataStream<Tuple2<Double,Double>> result = HTM.learn(kafkaStream, new
Harness.AnomalyNetwork())
.select(new InferenceSelectFunction<Harness.KafkaRecord,
Tuple2<Double, Double>>() {
@Override
public Tuple2<Double,Double>
select(Tuple2<Harness.KafkaRecord, NetworkInference> inference) throws
Exception {
return new Tuple2<Double, Double>(
3.333333 //fake value
inference.f1.getAnomalyScore());
}
}); /
I set my network in a way very similar to that described in example. I just
indicated my value is a "double" instead of a "number" and changed the
encoder from "ScalarEncoder" to "RandomDistributedScalarEncoder".
I have no error during the execution but if I try to print "result" nothing
is printed. Just for attempt, I tried to print datastream on a file, nothing
anyway.
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-HTM-integration-tp15113p15176.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Flink-HTM integration
Posted by Ted Yu <yu...@gmail.com>.
Can you give us some detail on how flink-HTM is integrated in your project ?
Cheers
On Sun, Aug 27, 2017 at 6:28 AM, AndreaKinn <ki...@hotmail.it> wrote:
> I think this is not a good idea, I don't know if it's a bug or a my fault.
> I believe I integrated correctly flink-HTM in my project (but I'm not sure,
> Flink-HTM is still an embryonal phase I think) and simply I can't see any
> output after the HTM elaboration, even there are no errors on console.
>
> I watched just a video presentation of Eron Wright on youtube about
> Flink-HTM example in Scala and an output is obviously generate. Instead, I
> use Java version of Flink-HTM
>
>
>
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Flink-HTM-
> integration-tp15113p15172.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>
Re: Flink-HTM integration
Posted by AndreaKinn <ki...@hotmail.it>.
I think this is not a good idea, I don't know if it's a bug or a my fault.
I believe I integrated correctly flink-HTM in my project (but I'm not sure,
Flink-HTM is still an embryonal phase I think) and simply I can't see any
output after the HTM elaboration, even there are no errors on console.
I watched just a video presentation of Eron Wright on youtube about
Flink-HTM example in Scala and an output is obviously generate. Instead, I
use Java version of Flink-HTM
--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-HTM-integration-tp15113p15172.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.
Re: Flink-HTM integration
Posted by Robert Metzger <rm...@apache.org>.
Hi, maybe creating an issue in the GitHub project will notify the project
members?
On Thu, Aug 24, 2017 at 5:19 PM, AndreaKinn <ki...@hotmail.it> wrote:
> Hi,
> Is there here someone who used Flink-HTM library
> https://github.com/htm-community/flink-htm
> <https://github.com/htm-community/flink-htm> ?
> I'm trying to implement it in my project but I have some fundamental
> question to complete my thesis work.
>
> Regards,
> Andrea
>
>
>
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Flink-HTM-
> integration-tp15113.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>