[jira] [Commented] (STORM-885) Heartbeat Server (Pacemaker)

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988054#comment-14988054 ] 

ASF GitHub Bot commented on STORM-885:
--------------------------------------

Github user knusbaum commented on a diff in the pull request:

    https://github.com/apache/storm/pull/838#discussion_r43802385
  
    --- Diff: storm-core/test/clj/org/apache/storm/pacemaker_test.clj ---
    @@ -0,0 +1,227 @@
    +(ns org.apache.storm.pacemaker-test
    +  (:require [clojure.test :refer :all]
    +            [org.apache.storm.pacemaker [pacemaker :as pacemaker]]
    +            [conjure.core :as conjure])
    +  (:import [backtype.storm.generated
    +            HBExecutionException HBNodes HBRecords
    +            HBServerMessageType HBMessage HBMessageData HBPulse]))
    +
    +(defn- message-with-rand-id [type data]
    +  (let [mid (rand-int 1000)
    +        message (HBMessage. type data)]
    +    (.set_message_id message mid)
    +    [message mid]))
    +
    +(defn- string-to-bytes [string]
    +  (byte-array (map int string)))
    +
    +(defn- bytes-to-string [bytez]
    +  (apply str (map char bytez)))
    +
    +(defn- makenode [handler path]
    +  (.handleMessage handler
    +                  (HBMessage.
    +                   HBServerMessageType/SEND_PULSE
    +                   (HBMessageData/pulse
    +                    (doto (HBPulse.)
    +                      (.set_id path)
    +                      (.set_details (string-to-bytes "nothing")))))
    +                  true))
    +
    +(deftest pacemaker-server-create-path
    +  (conjure/stubbing
    +   [pacemaker/register nil]
    +   (let [handler (pacemaker/mk-handler {})]
    +     (testing "CREATE_PATH"
    +       (let [[message mid] (message-with-rand-id
    +                            HBServerMessageType/CREATE_PATH
    +                            (HBMessageData/path "/testpath"))
    +             response (.handleMessage handler message true)]
    +         (is (= (.get_message_id response) mid))
    +         (is (= (.get_type response) HBServerMessageType/CREATE_PATH_RESPONSE))
    +         (is (= (.get_data response) nil)))))))
    +
    +(deftest pacemaker-server-exists
    +  (conjure/stubbing
    +   [pacemaker/register nil]
    +   (let [handler (pacemaker/mk-handler {})]
    +     (testing "EXISTS - false"
    +       (let [[message mid] (message-with-rand-id HBServerMessageType/EXISTS
    +                                                 (HBMessageData/path "/testpath"))
    +             bad-response (.handleMessage handler message false)
    +             good-response (.handleMessage handler message true)]
    +         (is (= (.get_message_id bad-response) mid))
    +         (is (= (.get_type bad-response) HBServerMessageType/NOT_AUTHORIZED))
    +
    +         (is (= (.get_message_id good-response) mid))
    +         (is (= (.get_type good-response) HBServerMessageType/EXISTS_RESPONSE))
    +         (is (= (.get_boolval (.get_data good-response)) false))))
    +
    +     (testing "EXISTS - true"
    +       (let [path "/exists_path"
    +             data-string "pulse data"]
    +         (let [[send _] (message-with-rand-id
    +                         HBServerMessageType/SEND_PULSE
    +                         (HBMessageData/pulse
    +                          (doto (HBPulse.)
    +                            (.set_id path)
    +                            (.set_details (string-to-bytes data-string)))))
    +               _ (.handleMessage handler send true)
    +               [message mid] (message-with-rand-id HBServerMessageType/EXISTS
    +                                                   (HBMessageData/path path))
    +               bad-response (.handleMessage handler message false)
    +               good-response (.handleMessage handler message true)]
    +           (is (= (.get_message_id bad-response) mid))
    +          (is (= (.get_type bad-response) HBServerMessageType/NOT_AUTHORIZED))
    +
    +          (is (= (.get_message_id good-response) mid))
    +          (is (= (.get_type good-response) HBServerMessageType/EXISTS_RESPONSE))
    +          (is (= (.get_boolval (.get_data good-response)) true))))))))
    --- End diff --
    
    Spacing for a bunch of these `is` forms is off.


> Heartbeat Server (Pacemaker)
> ----------------------------
>
>                 Key: STORM-885
>                 URL: https://issues.apache.org/jira/browse/STORM-885
>             Project: Apache Storm
>          Issue Type: Improvement
>          Components: storm-core
>            Reporter: Robert Joseph Evans
>            Assignee: Kyle Nusbaum
>
> Large highly connected topologies and large clusters write a lot of data into ZooKeeper.  The heartbeats, that make up the majority of this data, do not need to be persisted to disk.  Pacemaker is intended to be a secure replacement for storing the heartbeats without changing anything within the heartbeats.  In the future as more metrics are added in, we may want to look into switching it over to look more like Heron, where a metrics server is running for each node/topology.  And can be used to aggregate/per-aggregate them in a more scalable manor.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)