You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Alexey Romanenko <ar...@gmail.com> on 2021/10/12 14:23:42 UTC
Grafana/InfluxDb/Jenkins manual or doc
Hello,
Does anyone know if there is any manual or a doc about how to create a new metrics dashboard at http://metrics.beam.apache.org/ for Beam Grafana/InfluxDb combo and Jenkins job? I was trying here and there but seems it requires some additional admin privileges, iiic.
I’m asking this because of the need to run TPC-DS benchmark in the similar way as we do this with Nexmark for Beam SQL testing.
—
Alexey
Re: Grafana/InfluxDb/Jenkins manual or doc
Posted by Fernando Morales Martinez <fe...@wizeline.com>.
Will do!
It took some tweaking here and there to figure it out, but it is a
straightforward process.
I'll update the wiki with the instructions!
Thanks!
On Tue, Oct 12, 2021 at 7:49 PM Ahmet Altay <al...@google.com> wrote:
> Do you mind updating the wiki with any relevant information? This is an
> area where we do not have many experts and it would be good to make it as
> easy as possible to maintain.
>
> On Tue, Oct 12, 2021 at 9:55 AM Fernando Morales Martinez <
> fernando.morales@wizeline.com> wrote:
>
>> Yeah, that is correct!
>> Once the code is merged and the container restarted then the dashboard
>> will be available/displayed.
>> It usually takes a couple of hours for the container to be restarted
>> automatically.
>>
>> - Fer
>>
>> On Tue, Oct 12, 2021 at 10:50 AM Alexey Romanenko <
>> aromanenko.dev@gmail.com> wrote:
>>
>>> Hi Fernando,
>>>
>>> Many thanks for answer! Makes sense for me, I’ll try.
>>>
>>> I assume that all my changes will be applied only once they are merged.
>>> Is it correct?
>>>
>>> Alexey
>>>
>>> On 12 Oct 2021, at 18:31, Fernando Morales Martinez <
>>> fernando.morales@wizeline.com> wrote:
>>>
>>> Hi Alexey!
>>> I think I can help with this.
>>> Are you trying to create the dashboard directly on the website
>>> http://metrics.beam.apache.org/? Even with the admin credentials,
>>> creating the dashboard this way would result in it being deleted the next
>>> time the Grafana container gets redeployed.
>>>
>>> What I did was create the new dashboard using the Docker image here
>>> *.test-infra/metrics/grafana/Dockerfile*, then export the resulting
>>> json file.
>>> Afterwards, the exported json file has to be saved inside the*
>>> .test-infra/metrics/grafana/dashboards* folder and push the changes to
>>> github.
>>>
>>> I'm putting together a more thorough guide, but those are the
>>> general steps.
>>>
>>> Thanks!
>>>
>>> On Tue, Oct 12, 2021 at 8:23 AM Alexey Romanenko <
>>> aromanenko.dev@gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> Does anyone know if there is any manual or a doc about how to create a
>>>> new metrics dashboard at http://metrics.beam.apache.org/ for Beam
>>>> Grafana/InfluxDb combo and Jenkins job? I was trying here and there but
>>>> seems it requires some additional admin privileges, iiic.
>>>>
>>>> I’m asking this because of the need to run TPC-DS benchmark in the
>>>> similar way as we do this with Nexmark for Beam SQL testing.
>>>>
>>>>
>>>> —
>>>> Alexey
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *This email and its contents (including any attachments) are being sent
>>> toyou on the condition of confidentiality and may be protected by
>>> legalprivilege. Access to this email by anyone other than the intended
>>> recipientis unauthorized. If you are not the intended recipient, please
>>> immediatelynotify the sender by replying to this message and delete the
>>> materialimmediately from your system. Any further use, dissemination,
>>> distributionor reproduction of this email is strictly prohibited. Further,
>>> norepresentation is made with respect to any content contained in this
>>> email.*
>>>
>>>
>>>
>>
>>
>>
>>
>>
>>
>>
>> *This email and its contents (including any attachments) are being sent
>> toyou on the condition of confidentiality and may be protected by
>> legalprivilege. Access to this email by anyone other than the intended
>> recipientis unauthorized. If you are not the intended recipient, please
>> immediatelynotify the sender by replying to this message and delete the
>> materialimmediately from your system. Any further use, dissemination,
>> distributionor reproduction of this email is strictly prohibited. Further,
>> norepresentation is made with respect to any content contained in this
>> email.*
>
>
--
*This email and its contents (including any attachments) are being sent to
you on the condition of confidentiality and may be protected by legal
privilege. Access to this email by anyone other than the intended recipient
is unauthorized. If you are not the intended recipient, please immediately
notify the sender by replying to this message and delete the material
immediately from your system. Any further use, dissemination, distribution
or reproduction of this email is strictly prohibited. Further, no
representation is made with respect to any content contained in this email.*
Re: Grafana/InfluxDb/Jenkins manual or doc
Posted by Ahmet Altay <al...@google.com>.
Do you mind updating the wiki with any relevant information? This is an
area where we do not have many experts and it would be good to make it as
easy as possible to maintain.
On Tue, Oct 12, 2021 at 9:55 AM Fernando Morales Martinez <
fernando.morales@wizeline.com> wrote:
> Yeah, that is correct!
> Once the code is merged and the container restarted then the dashboard
> will be available/displayed.
> It usually takes a couple of hours for the container to be restarted
> automatically.
>
> - Fer
>
> On Tue, Oct 12, 2021 at 10:50 AM Alexey Romanenko <
> aromanenko.dev@gmail.com> wrote:
>
>> Hi Fernando,
>>
>> Many thanks for answer! Makes sense for me, I’ll try.
>>
>> I assume that all my changes will be applied only once they are merged.
>> Is it correct?
>>
>> Alexey
>>
>> On 12 Oct 2021, at 18:31, Fernando Morales Martinez <
>> fernando.morales@wizeline.com> wrote:
>>
>> Hi Alexey!
>> I think I can help with this.
>> Are you trying to create the dashboard directly on the website
>> http://metrics.beam.apache.org/? Even with the admin credentials,
>> creating the dashboard this way would result in it being deleted the next
>> time the Grafana container gets redeployed.
>>
>> What I did was create the new dashboard using the Docker image here
>> *.test-infra/metrics/grafana/Dockerfile*, then export the resulting json
>> file.
>> Afterwards, the exported json file has to be saved inside the*
>> .test-infra/metrics/grafana/dashboards* folder and push the changes to
>> github.
>>
>> I'm putting together a more thorough guide, but those are the
>> general steps.
>>
>> Thanks!
>>
>> On Tue, Oct 12, 2021 at 8:23 AM Alexey Romanenko <
>> aromanenko.dev@gmail.com> wrote:
>>
>>> Hello,
>>>
>>> Does anyone know if there is any manual or a doc about how to create a
>>> new metrics dashboard at http://metrics.beam.apache.org/ for Beam
>>> Grafana/InfluxDb combo and Jenkins job? I was trying here and there but
>>> seems it requires some additional admin privileges, iiic.
>>>
>>> I’m asking this because of the need to run TPC-DS benchmark in the
>>> similar way as we do this with Nexmark for Beam SQL testing.
>>>
>>>
>>> —
>>> Alexey
>>>
>>>
>>
>>
>>
>>
>>
>>
>>
>> *This email and its contents (including any attachments) are being sent
>> toyou on the condition of confidentiality and may be protected by
>> legalprivilege. Access to this email by anyone other than the intended
>> recipientis unauthorized. If you are not the intended recipient, please
>> immediatelynotify the sender by replying to this message and delete the
>> materialimmediately from your system. Any further use, dissemination,
>> distributionor reproduction of this email is strictly prohibited. Further,
>> norepresentation is made with respect to any content contained in this
>> email.*
>>
>>
>>
>
>
>
>
>
>
>
> *This email and its contents (including any attachments) are being sent
> toyou on the condition of confidentiality and may be protected by
> legalprivilege. Access to this email by anyone other than the intended
> recipientis unauthorized. If you are not the intended recipient, please
> immediatelynotify the sender by replying to this message and delete the
> materialimmediately from your system. Any further use, dissemination,
> distributionor reproduction of this email is strictly prohibited. Further,
> norepresentation is made with respect to any content contained in this
> email.*
Re: Grafana/InfluxDb/Jenkins manual or doc
Posted by Fernando Morales Martinez <fe...@wizeline.com>.
Yeah, that is correct!
Once the code is merged and the container restarted then the dashboard will
be available/displayed.
It usually takes a couple of hours for the container to be restarted
automatically.
- Fer
On Tue, Oct 12, 2021 at 10:50 AM Alexey Romanenko <ar...@gmail.com>
wrote:
> Hi Fernando,
>
> Many thanks for answer! Makes sense for me, I’ll try.
>
> I assume that all my changes will be applied only once they are merged. Is
> it correct?
>
> Alexey
>
> On 12 Oct 2021, at 18:31, Fernando Morales Martinez <
> fernando.morales@wizeline.com> wrote:
>
> Hi Alexey!
> I think I can help with this.
> Are you trying to create the dashboard directly on the website
> http://metrics.beam.apache.org/? Even with the admin credentials,
> creating the dashboard this way would result in it being deleted the next
> time the Grafana container gets redeployed.
>
> What I did was create the new dashboard using the Docker image here
> *.test-infra/metrics/grafana/Dockerfile*, then export the resulting json
> file.
> Afterwards, the exported json file has to be saved inside the*
> .test-infra/metrics/grafana/dashboards* folder and push the changes to
> github.
>
> I'm putting together a more thorough guide, but those are the
> general steps.
>
> Thanks!
>
> On Tue, Oct 12, 2021 at 8:23 AM Alexey Romanenko <ar...@gmail.com>
> wrote:
>
>> Hello,
>>
>> Does anyone know if there is any manual or a doc about how to create a
>> new metrics dashboard at http://metrics.beam.apache.org/ for Beam
>> Grafana/InfluxDb combo and Jenkins job? I was trying here and there but
>> seems it requires some additional admin privileges, iiic.
>>
>> I’m asking this because of the need to run TPC-DS benchmark in the
>> similar way as we do this with Nexmark for Beam SQL testing.
>>
>>
>> —
>> Alexey
>>
>>
>
>
>
>
>
>
>
> *This email and its contents (including any attachments) are being sent
> toyou on the condition of confidentiality and may be protected by
> legalprivilege. Access to this email by anyone other than the intended
> recipientis unauthorized. If you are not the intended recipient, please
> immediatelynotify the sender by replying to this message and delete the
> materialimmediately from your system. Any further use, dissemination,
> distributionor reproduction of this email is strictly prohibited. Further,
> norepresentation is made with respect to any content contained in this
> email.*
>
>
>
--
*This email and its contents (including any attachments) are being sent to
you on the condition of confidentiality and may be protected by legal
privilege. Access to this email by anyone other than the intended recipient
is unauthorized. If you are not the intended recipient, please immediately
notify the sender by replying to this message and delete the material
immediately from your system. Any further use, dissemination, distribution
or reproduction of this email is strictly prohibited. Further, no
representation is made with respect to any content contained in this email.*
Re: Grafana/InfluxDb/Jenkins manual or doc
Posted by Alexey Romanenko <ar...@gmail.com>.
Hi Fernando,
Many thanks for answer! Makes sense for me, I’ll try.
I assume that all my changes will be applied only once they are merged. Is it correct?
Alexey
> On 12 Oct 2021, at 18:31, Fernando Morales Martinez <fe...@wizeline.com> wrote:
>
> Hi Alexey!
> I think I can help with this.
> Are you trying to create the dashboard directly on the website http://metrics.beam.apache.org/ <http://metrics.beam.apache.org/>? Even with the admin credentials, creating the dashboard this way would result in it being deleted the next time the Grafana container gets redeployed.
>
> What I did was create the new dashboard using the Docker image here .test-infra/metrics/grafana/Dockerfile, then export the resulting json file.
> Afterwards, the exported json file has to be saved inside the .test-infra/metrics/grafana/dashboards folder and push the changes to github.
>
> I'm putting together a more thorough guide, but those are the general steps.
>
> Thanks!
>
> On Tue, Oct 12, 2021 at 8:23 AM Alexey Romanenko <aromanenko.dev@gmail.com <ma...@gmail.com>> wrote:
> Hello,
>
> Does anyone know if there is any manual or a doc about how to create a new metrics dashboard at http://metrics.beam.apache.org/ <http://metrics.beam.apache.org/> for Beam Grafana/InfluxDb combo and Jenkins job? I was trying here and there but seems it requires some additional admin privileges, iiic.
>
> I’m asking this because of the need to run TPC-DS benchmark in the similar way as we do this with Nexmark for Beam SQL testing.
>
>
> —
> Alexey
>
>
> This email and its contents (including any attachments) are being sent to
> you on the condition of confidentiality and may be protected by legal
> privilege. Access to this email by anyone other than the intended recipient
> is unauthorized. If you are not the intended recipient, please immediately
> notify the sender by replying to this message and delete the material
> immediately from your system. Any further use, dissemination, distribution
> or reproduction of this email is strictly prohibited. Further, no
> representation is made with respect to any content contained in this email.
Re: Grafana/InfluxDb/Jenkins manual or doc
Posted by Fernando Morales Martinez <fe...@wizeline.com>.
Hi Alexey!
I think I can help with this.
Are you trying to create the dashboard directly on the website
http://metrics.beam.apache.org/? Even with the admin credentials, creating
the dashboard this way would result in it being deleted the next time the
Grafana container gets redeployed.
What I did was create the new dashboard using the Docker image here
*.test-infra/metrics/grafana/Dockerfile*, then export the resulting json
file.
Afterwards, the exported json file has to be saved inside the*
.test-infra/metrics/grafana/dashboards* folder and push the changes to
github.
I'm putting together a more thorough guide, but those are the general steps.
Thanks!
On Tue, Oct 12, 2021 at 8:23 AM Alexey Romanenko <ar...@gmail.com>
wrote:
> Hello,
>
> Does anyone know if there is any manual or a doc about how to create a new
> metrics dashboard at http://metrics.beam.apache.org/ for Beam
> Grafana/InfluxDb combo and Jenkins job? I was trying here and there but
> seems it requires some additional admin privileges, iiic.
>
> I’m asking this because of the need to run TPC-DS benchmark in the similar
> way as we do this with Nexmark for Beam SQL testing.
>
>
> —
> Alexey
>
>
--
*This email and its contents (including any attachments) are being sent to
you on the condition of confidentiality and may be protected by legal
privilege. Access to this email by anyone other than the intended recipient
is unauthorized. If you are not the intended recipient, please immediately
notify the sender by replying to this message and delete the material
immediately from your system. Any further use, dissemination, distribution
or reproduction of this email is strictly prohibited. Further, no
representation is made with respect to any content contained in this email.*