You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2018/08/17 09:11:05 UTC
Slack digest for #general - 2018-08-17

2018-08-16 13:07:03 UTC - ben gordon: @ben gordon has joined the channel
----
2018-08-16 19:17:55 UTC - Ravi: @Ravi has joined the channel
----
2018-08-16 19:18:37 UTC - Ravi: hi
----
2018-08-16 19:20:02 UTC - Ravi: had an issue with testing docker standalone sandbox, Sending sentence: an apple a day keeps the doctor away
Traceback (most recent call last):
  File "./producer.py", line 42, in &lt;module&gt;
    main(sys.argv)
  File "./producer.py", line 36, in main
    producer.send(sentence) # Publish randomly selected sentence to Pulsar
----
2018-08-16 19:21:32 UTC - Matteo Merli: Hi @Ravi, can you paste your producer.py into <http://gist.github.com|gist.github.com> ?
----
2018-08-16 19:22:02 UTC - Ravi: there is no change, i did wget from your link
----
2018-08-16 19:22:32 UTC - Ravi: followed the same rules as mentioned in sequence
----
2018-08-16 19:23:21 UTC - Matteo Merli: Oh, ok
----
2018-08-16 19:23:40 UTC - Matteo Merli: can you show the logs / other errors in console?
----
2018-08-16 19:24:04 UTC - Ravi: 4:57:29.164 [0x7fffb1c93380] INFO  /Users/vagrant/incubator-pulsar/pulsar-client-cpp/lib/ConnectionPool.cc(63) - Created connection for <pulsar://localhost:6650>
14:57:29.165 [0x700009d90000] INFO  /Users/vagrant/incubator-pulsar/pulsar-client-cpp/lib/ClientConnection.cc(279) - [[::1]:60580 -&gt; [::1]:6650] Connected to broker
14:57:29.172 [0x700009d90000] INFO  /Users/vagrant/incubator-pulsar/pulsar-client-cpp/lib/HandlerBase.cc(53) - [<persistent://public/default/sentences>, ] Getting connection from pool
14:57:29.173 [0x700009d90000] INFO  /Users/vagrant/incubator-pulsar/pulsar-client-cpp/lib/ConnectionPool.cc(63) - Created connection for <pulsar://127.0.0.1:6650>
14:57:29.174 [0x700009d90000] INFO  /Users/vagrant/incubator-pulsar/pulsar-client-cpp/lib/ClientConnection.cc(281) - [[::1]:60581 -&gt; [::1]:6650] Connected to broker through proxy. Logical broker: <pulsar://127.0.0.1:6650>
14:57:29.179 [0x700009d90000] INFO  /Users/vagrant/incubator-pulsar/pulsar-client-cpp/lib/ProducerImpl.cc(154) - [<persistent://public/default/sentences>, ] Created producer on broker [[::1]:60581 -&gt; [::1]:6650]
Sending sentence: an apple a day keeps the doctor away
Traceback (most recent call last):
  File "./producer.py", line 42, in &lt;module&gt;
    main(sys.argv)
  File "./producer.py", line 36, in main
    producer.send(sentence) # Publish randomly selected sentence to Pulsar
  File "/Users/ravihanzo/anaconda3/lib/python3.6/site-packages/pulsar/__init__.py", line 577, in send
    replication_clusters, disable_replication)
  File "/Users/ravihanzo/anaconda3/lib/python3.6/site-packages/pulsar/__init__.py", line 638, in _build_msg
    _check_type(bytes, content, 'content')
  File "/Users/ravihanzo/anaconda3/lib/python3.6/site-packages/pulsar/__init__.py", line 812, in _check_type
    raise ValueError("Argument %s is expected to be of type '%s'" % (name, var_type.__name__))
ValueError: Argument content is expected to be of type 'bytes'
----
2018-08-16 19:24:33 UTC - Matteo Merli: oh, ok
----
2018-08-16 19:24:57 UTC - Matteo Merli: the problem is that the example only works with Py2.7 out of the box.
----
2018-08-16 19:25:03 UTC - Matteo Merli: there’s a simple fix:
----
2018-08-16 19:25:27 UTC - Matteo Merli: basically in py3 a string is != from bytes
----
2018-08-16 19:26:19 UTC - Matteo Merli: so, at line 36: `producer.send(sentence.encode('utf-8'))`
----
2018-08-16 19:28:25 UTC - Matteo Merli: (we’ll fix the example code!)
----
2018-08-16 19:31:29 UTC - Ravi: I kinda thought so, did not want to edit. i will try to fix it. Please update the blog.
+1 : Matteo Merli
----
2018-08-16 19:35:10 UTC - Ravi: resolved thanks
----
2018-08-16 21:21:18 UTC - Abhishek Sharma: Can a pulsar-dashboard polls data from multiple SERVICE_URL ?? Say one cluster running in my local as standalone mode &amp; other running on remote server ?
----
2018-08-16 21:22:07 UTC - Matteo Merli: Good point. It’s not currently possible but it should be easy to add/customize
----
2018-08-16 21:23:46 UTC - Abhishek Sharma: How is it so ? Do I have to make change in docker script?
----
2018-08-16 21:23:51 UTC - Matteo Merli: well, the current behavior is that it fetches stats from all clusters within a single logical “instance”
----
2018-08-16 21:24:55 UTC - Matteo Merli: so, if you setup replication across these clusters or just if you specify the clusters in the metadata, it will work already
----
2018-08-16 21:25:24 UTC - Matteo Merli: is that the case, or you mean completely separated clusters?
----
2018-08-16 21:25:47 UTC - Abhishek Sharma: completely separated clusters.
----
2018-08-16 21:26:12 UTC - Matteo Merli: ok, then you’d need to tweak a bit the config :
----
2018-08-16 21:27:35 UTC - Matteo Merli: The dashboard itself is composed of multiple parts runnning inside a docker container for convenience: 
 * collector — Python scripts that fetches info from brokers- It discover the list of clusters and brokers for each cluster and fetches all metrics 
 * Postgres — Where the metrics are stored 
 * Django app to visualize
----
2018-08-16 21:27:57 UTC - Matteo Merli: You can easily run multiple collector processes inside the docker image
----
2018-08-16 21:29:07 UTC - Matteo Merli: We use supervisor to manage the processes inside the container:
----
2018-08-16 21:29:10 UTC - Abhishek Sharma: okay .. I am looking in to the dashboard code. I think multiple `collectors.py` need to be triggered.
----
2018-08-16 21:29:15 UTC - Matteo Merli: <https://github.com/apache/incubator-pulsar/blob/master/dashboard/conf/supervisor-app.conf#L30>
----
2018-08-16 21:30:49 UTC - Matteo Merli: There is only one issue. Each time the collector runs, it attaches a timestamp (as epoch) to identify the latest version of stats. So that the webapp only does the query on that.
----
2018-08-16 21:31:04 UTC - Matteo Merli: If there are 2 collectors, they will step into each others :confused:
----
2018-08-16 21:33:16 UTC - Abhishek Sharma: okay let me have a complete look of the code. If you think it's a good feature to have, I can create a enhancement-ticket for that.
----
2018-08-16 21:35:42 UTC - Matteo Merli: :+1:
----