[jira] [Commented] (STORM-1759) Viewing logs from the Storm UI doesn't work in dockerized environment

["Jorge Moraleda (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-1759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16562565#comment-16562565 ] 

Jorge Moraleda commented on STORM-1759:
---------------------------------------

I think routing all log viewing through a single host/port running storm ui as requested in [#2708] is the most desirable approach for log viewiing in a container based deployment.

The obvious alternative is  leaving storm logging as is. This forces one to either give up the convenience of accessing all logs from the browsers from the ui and instead requiring to ssh into each container, or an impractically complex setup that dynamically maps and open each supervisor's logviewer port to a port accessible from outside the cluster and handles container restarts and dynamic scaling of supervisors.

This issue has in some form been going on for almost 3 years in some form [#1342], but I believe it is now much more urgent than it used to be since deployment in containers is a lot more common now. Moreover I believe the solution should be orchestration agnostic. In particular it should not rely on runnning under mesos. It also should also not require running additional services: Since both a zookeeper and a nimbus are already required for any deployment, that should be sufficient for coordination for storm ui to display the logs of individual supervisors at its own port.

> Viewing logs from the Storm UI doesn't work in dockerized environment
> ---------------------------------------------------------------------
>
>                 Key: STORM-1759
>                 URL: https://issues.apache.org/jira/browse/STORM-1759
>             Project: Apache Storm
>          Issue Type: Bug
>          Components: storm-ui
>    Affects Versions: 0.10.0, 1.0.0, 0.9.6
>            Reporter: Elisey Zanko
>            Priority: Major
>
> I run the Storm using the following docker-compose.yml
> {code}
> version: '2'
> services:
>     zookeeper:
>         image: jplock/zookeeper:3.4.8
>         restart: always
>     nimbus:
>         image: 31z4/storm:1.0.0
>         command: nimbus -c storm.log.dir="/logs" -c storm.zookeeper.servers="[\"zookeeper\"]" -c nimbus.host="nimbus"
>         depends_on:
>             - zookeeper
>         restart: always
>         ports:
>             - 6627:6627
>         volumes:
>             - logs:/logs
>     supervisor:
>         image: 31z4/storm:1.0.0
>         command: supervisor -c storm.log.dir="/logs" -c storm.zookeeper.servers="[\"zookeeper\"]" -c nimbus.host="nimbus"
>         depends_on:
>             - nimbus
>         restart: always
>         volumes:
>             - logs:/logs
>     logviewer:
>         image: 31z4/storm:1.0.0
>         command: logviewer -c storm.log.dir="/logs"
>         restart: always
>         ports:
>             - 8000:8000
>         volumes:
>             - logs:/logs
>     ui:
>         image: 31z4/storm:1.0.0
>         command: ui -c storm.log.dir="/logs" -c nimbus.host="nimbus"
>         depends_on:
>             - nimbus
>             - logviewer
>         restart: always
>         ports:
>             - 8080:8080
>         volumes:
>             - logs:/log
> volumes:
>     logs: {}
> {code}
> And opening the logs from the Storm UI doesn't work because all links are pointing to different container ids as hosts.
> I guess adding an ability to explicitly specify the logviewer host in the storm.yaml would solve the issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)