You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Eranga Heshan <er...@gmail.com> on 2016/11/17 06:26:55 UTC
Enforce an operation to run in an exact host in flink
Hi all,
I have 4 VMs. There is an input stream and an output stream in my job. To
get the exact time interval for an event to be processed, I need to run
both input and output bolts inside the same host/node.
Is there a way to run a bolt inside a host we define?
Thank you,
Regards,
Eranga Heshan
*Undergraduate*
Computer Science & Engineering
University of Moratuwa
Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
Email: eranga@wso2.com <os...@gmail.com>
<https://www.facebook.com/erangaheshan> <https://twitter.com/erangaheshan>
<https://www.linkedin.com/in/erangaheshan>
Re: Enforce an operation to run in an exact host in flink
Posted by Eranga Heshan <er...@gmail.com>.
Thank you Aijosha. 😃 I understood the concept.
Eranga Heshan
*Undergraduate*
Computer Science & Engineering
University of Moratuwa
Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
Email: eranga@wso2.com <os...@gmail.com>
<https://www.facebook.com/erangaheshan> <https://twitter.com/erangaheshan>
<https://www.linkedin.com/in/erangaheshan>
On Mon, Nov 21, 2016 at 10:28 PM, Aljoscha Krettek <al...@apache.org>
wrote:
> If I'm not mistaken Flink already has metrics that report latencies. These
> are not 100% correct because they machine one which an element originates
> is not necessarily the machine on which we measure the latency.
>
> I'm afraid that that's the best we can do right now, however. Other than
> running your pipeline on only one machine.
>
> On Fri, 18 Nov 2016 at 01:32 Eranga Heshan <er...@gmail.com> wrote:
>
>> Yeah, I need to process latency. Any suggestions?
>>
>> On Nov 18, 2016 12:47 AM, "Aljoscha Krettek" <al...@apache.org> wrote:
>>
>> Hi,
>> I'm afraid that it is currently not possible to specify the machine on
>> which a Flink operator should run. What do you need this for? Are you
>> trying to measure processing latency?
>>
>> Cheers,
>> Aljoscha
>>
>> On Thu, 17 Nov 2016 at 09:15 Eranga Heshan <er...@gmail.com> wrote:
>>
>> Any help please?
>>
>>
>> Eranga Heshan
>> *Undergraduate*
>> Computer Science & Engineering
>> University of Moratuwa
>> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
>> Email: eranga@wso2.com <os...@gmail.com>
>> <https://www.facebook.com/erangaheshan>
>> <https://twitter.com/erangaheshan>
>> <https://www.linkedin.com/in/erangaheshan>
>>
>> On Thu, Nov 17, 2016 at 11:56 AM, Eranga Heshan <er...@gmail.com>
>> wrote:
>>
>> Hi all,
>>
>> I have 4 VMs. There is an input stream and an output stream in my job. To
>> get the exact time interval for an event to be processed, I need to run
>> both input and output bolts inside the same host/node.
>>
>> Is there a way to run a bolt inside a host we define?
>>
>> Thank you,
>> Regards,
>>
>> Eranga Heshan
>> *Undergraduate*
>> Computer Science & Engineering
>> University of Moratuwa
>> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
>> Email: eranga@wso2.com <os...@gmail.com>
>> <https://www.facebook.com/erangaheshan>
>> <https://twitter.com/erangaheshan>
>> <https://www.linkedin.com/in/erangaheshan>
>>
>>
>>
Re: Enforce an operation to run in an exact host in flink
Posted by Aljoscha Krettek <al...@apache.org>.
If I'm not mistaken Flink already has metrics that report latencies. These
are not 100% correct because they machine one which an element originates
is not necessarily the machine on which we measure the latency.
I'm afraid that that's the best we can do right now, however. Other than
running your pipeline on only one machine.
On Fri, 18 Nov 2016 at 01:32 Eranga Heshan <er...@gmail.com> wrote:
> Yeah, I need to process latency. Any suggestions?
>
> On Nov 18, 2016 12:47 AM, "Aljoscha Krettek" <al...@apache.org> wrote:
>
> Hi,
> I'm afraid that it is currently not possible to specify the machine on
> which a Flink operator should run. What do you need this for? Are you
> trying to measure processing latency?
>
> Cheers,
> Aljoscha
>
> On Thu, 17 Nov 2016 at 09:15 Eranga Heshan <er...@gmail.com> wrote:
>
> Any help please?
>
>
> Eranga Heshan
> *Undergraduate*
> Computer Science & Engineering
> University of Moratuwa
> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
> Email: eranga@wso2.com <os...@gmail.com>
> <https://www.facebook.com/erangaheshan>
> <https://twitter.com/erangaheshan>
> <https://www.linkedin.com/in/erangaheshan>
>
> On Thu, Nov 17, 2016 at 11:56 AM, Eranga Heshan <er...@gmail.com>
> wrote:
>
> Hi all,
>
> I have 4 VMs. There is an input stream and an output stream in my job. To
> get the exact time interval for an event to be processed, I need to run
> both input and output bolts inside the same host/node.
>
> Is there a way to run a bolt inside a host we define?
>
> Thank you,
> Regards,
>
> Eranga Heshan
> *Undergraduate*
> Computer Science & Engineering
> University of Moratuwa
> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
> Email: eranga@wso2.com <os...@gmail.com>
> <https://www.facebook.com/erangaheshan>
> <https://twitter.com/erangaheshan>
> <https://www.linkedin.com/in/erangaheshan>
>
>
>
Re: Enforce an operation to run in an exact host in flink
Posted by Eranga Heshan <er...@gmail.com>.
Yeah, I need to process latency. Any suggestions?
On Nov 18, 2016 12:47 AM, "Aljoscha Krettek" <al...@apache.org> wrote:
> Hi,
> I'm afraid that it is currently not possible to specify the machine on
> which a Flink operator should run. What do you need this for? Are you
> trying to measure processing latency?
>
> Cheers,
> Aljoscha
>
> On Thu, 17 Nov 2016 at 09:15 Eranga Heshan <er...@gmail.com> wrote:
>
>> Any help please?
>>
>>
>> Eranga Heshan
>> *Undergraduate*
>> Computer Science & Engineering
>> University of Moratuwa
>> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
>> Email: eranga@wso2.com <os...@gmail.com>
>> <https://www.facebook.com/erangaheshan>
>> <https://twitter.com/erangaheshan>
>> <https://www.linkedin.com/in/erangaheshan>
>>
>> On Thu, Nov 17, 2016 at 11:56 AM, Eranga Heshan <er...@gmail.com>
>> wrote:
>>
>> Hi all,
>>
>> I have 4 VMs. There is an input stream and an output stream in my job. To
>> get the exact time interval for an event to be processed, I need to run
>> both input and output bolts inside the same host/node.
>>
>> Is there a way to run a bolt inside a host we define?
>>
>> Thank you,
>> Regards,
>>
>> Eranga Heshan
>> *Undergraduate*
>> Computer Science & Engineering
>> University of Moratuwa
>> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
>> Email: eranga@wso2.com <os...@gmail.com>
>> <https://www.facebook.com/erangaheshan>
>> <https://twitter.com/erangaheshan>
>> <https://www.linkedin.com/in/erangaheshan>
>>
>>
>>
Re: Enforce an operation to run in an exact host in flink
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
I'm afraid that it is currently not possible to specify the machine on
which a Flink operator should run. What do you need this for? Are you
trying to measure processing latency?
Cheers,
Aljoscha
On Thu, 17 Nov 2016 at 09:15 Eranga Heshan <er...@gmail.com> wrote:
> Any help please?
>
>
> Eranga Heshan
> *Undergraduate*
> Computer Science & Engineering
> University of Moratuwa
> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
> Email: eranga@wso2.com <os...@gmail.com>
> <https://www.facebook.com/erangaheshan>
> <https://twitter.com/erangaheshan>
> <https://www.linkedin.com/in/erangaheshan>
>
> On Thu, Nov 17, 2016 at 11:56 AM, Eranga Heshan <er...@gmail.com>
> wrote:
>
> Hi all,
>
> I have 4 VMs. There is an input stream and an output stream in my job. To
> get the exact time interval for an event to be processed, I need to run
> both input and output bolts inside the same host/node.
>
> Is there a way to run a bolt inside a host we define?
>
> Thank you,
> Regards,
>
> Eranga Heshan
> *Undergraduate*
> Computer Science & Engineering
> University of Moratuwa
> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
> Email: eranga@wso2.com <os...@gmail.com>
> <https://www.facebook.com/erangaheshan>
> <https://twitter.com/erangaheshan>
> <https://www.linkedin.com/in/erangaheshan>
>
>
>
Re: Enforce an operation to run in an exact host in flink
Posted by Eranga Heshan <er...@gmail.com>.
Any help please?
Eranga Heshan
*Undergraduate*
Computer Science & Engineering
University of Moratuwa
Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
Email: eranga@wso2.com <os...@gmail.com>
<https://www.facebook.com/erangaheshan> <https://twitter.com/erangaheshan>
<https://www.linkedin.com/in/erangaheshan>
On Thu, Nov 17, 2016 at 11:56 AM, Eranga Heshan <er...@gmail.com>
wrote:
> Hi all,
>
> I have 4 VMs. There is an input stream and an output stream in my job. To
> get the exact time interval for an event to be processed, I need to run
> both input and output bolts inside the same host/node.
>
> Is there a way to run a bolt inside a host we define?
>
> Thank you,
> Regards,
>
> Eranga Heshan
> *Undergraduate*
> Computer Science & Engineering
> University of Moratuwa
> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087>
> Email: eranga@wso2.com <os...@gmail.com>
> <https://www.facebook.com/erangaheshan>
> <https://twitter.com/erangaheshan>
> <https://www.linkedin.com/in/erangaheshan>
>