You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@pulsar.apache.org by Apache Pulsar Slack <ap...@gmail.com> on 2020/10/01 09:11:05 UTC
Slack digest for #general - 2020-10-01
2020-09-30 10:01:12 UTC - Vladimir Shchur: Hi! I'm trying to use state storage with pulsar functions, but it seems to be impossible with 2.6.1, I've tried specifying both --state-storage-service-url and --state-storage-url on function creation, but neither of them works `Unknown option: --state-storage-service-url` , I'm trying to run it in k8s cluster generated by helm chart
----
2020-09-30 10:25:56 UTC - Emanuel: Hi guys, looking for a way expose pulsar topics via http over json API's without deploying a container out of the pulsar cluster. Has anyone tried this before ?
----
2020-09-30 11:18:14 UTC - Zia Uddin: @Zia Uddin has joined the channel
----
2020-09-30 11:19:13 UTC - Jerry Wang: @Jerry Wang has joined the channel
----
2020-09-30 11:34:56 UTC - Lobo Xu: @Lobo Xu has joined the channel
----
2020-09-30 14:51:55 UTC - Shivji Kumar Jha: I am starting to work with the c++ client. Running <http://schemaTest.cc|schemaTest.cc> in the IDE (not the whole test suite) gives me the below error (Failed to establish connection: Connection refused).
```$ pulsar-client-cpp/cmake-build-debug/tests/main gtest_filter=SchemaTest.testSchema:SchemaTest/*.testSchema:*/SchemaTest.testSchema/*:*/SchemaTest/*.testSchema --gtest_color=no
Running 1 test from 1 test suite.2020-09-30 18:30:38.846 INFO ConnectionPool:85 | Created connection for <pulsar://localhost:6650>
2020-09-30 18:30:38.869 ERROR ClientConnection:385 | [<none> -> <pulsar://localhost:6650>] Failed to establish connection: Connection refused
2020-09-30 18:30:38.871 INFO ClientConnection:1354 | [<none> -> <pulsar://localhost:6650>] Connection closed
2020-09-30 18:30:38.872 ERROR ClientImpl:182 | Error Checking/Getting Partition Metadata while creating producer on <persistent://public/default/topic-avro> -- 5
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV) ```
What is the right way to run a single gtest?
----
2020-09-30 15:05:52 UTC - Shivji Kumar Jha: Running from the command line
```pulsar-client-cpp/cmake-build-debug/tests$ ./main --gtest_filter=SchemaTest.testSchema:SchemaTest/*.testSchema:*/SchemaTest.testSchema/*:*/SchemaTest/*.testSchema --gtest_color=no
Note: Google Test filter = SchemaTest.testSchema:SchemaTest/*.testSchema:*/SchemaTest.testSchema/*:*/SchemaTest/*.testSchema
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from SchemaTest
[ RUN ] SchemaTest.testSchema
2020-09-30 20:33:33.913 INFO ConnectionPool:85 | Created connection for <pulsar://localhost:6650>
2020-09-30 20:33:33.917 ERROR ClientConnection:385 | [<none> -> <pulsar://localhost:6650>] Failed to establish connection: Connection refused
2020-09-30 20:33:33.917 INFO ClientConnection:1354 | [<none> -> <pulsar://localhost:6650>] Connection closed
2020-09-30 20:33:33.917 ERROR ClientImpl:182 | Error Checking/Getting Partition Metadata while creating producer on <persistent://public/default/topic-avro> -- 5
Segmentation fault: 11```
----
2020-09-30 15:14:29 UTC - Shawn: @Sijie Guo When we had NACKs enabled the ackTimeout was `120s` . Without it, ackTimeout is set to `60s`.
This is the Go client (with the c++ bindings).
The installed c++ library is 2.4.0
<https://github.com/apache/pulsar/tree/7cdd354b0b3ff31a287fa0c44ad013de4ac83faa/pulsar-client-go>
----
2020-09-30 16:16:24 UTC - Addison Higham: I assume what that feature means is producer deduplication. I am not sure if there is a technical reason why that is difficult to achieve over websockets, but a quick glance at the code tells me that it is not currently possible. It would be a good discussion to start on github if you want to make an issue
----
2020-09-30 16:21:30 UTC - Yunze Xu: Did you run pulsar-test-service-start.sh first? The cpp unit tests need a local pulsar running
----
2020-09-30 16:24:17 UTC - Sijie Guo: There are 5 sessions in the #Pulsar track at ApacheCon. If you are interested in them, you can watch these session live at <https://hopin.to/events/apachecon-home>. The fulltrack schedule is at <https://www.apachecon.com/acah2020/tracks/pulsar.html>.
ok_hand : Shivji Kumar Jha, Josh Perryman
----
2020-09-30 16:25:18 UTC - Addison Higham: state storage is quite an "alpha" feature and fairly rough. There is some work in progress to improve it in one of the next releases. In other words, it isn't recommended for production.
If you want to try and work with it though there are a few prerequisites. Have you already enabled the options in bookkeeper that enable table storage? There may also be some settings to enable pulsar functions. I will find the doc I am looking for and share it, but I wanted to mention those caveats first
----
2020-09-30 16:27:03 UTC - Addison Higham: do you mean like an http API for sanding and receiving messages? There is an in progress HTTP protocol handler but I am not sure of the current state of it
----
2020-09-30 16:28:36 UTC - Emanuel: Thanks, got the answer actual. There Pulsar beam from the guys in Kafkaesque. Thanks for the replay !
----
2020-09-30 16:40:27 UTC - Vladimir Shchur: Thx, I've already discovered, that it should be alpha feature, but to be alpha there should be a way to try it. It looks like the only way is to use kafkaesque helm chart.
----
2020-09-30 16:56:19 UTC - Shivji Kumar Jha: ```-- Wait for Pulsar service to be ready
-- Pulsar service is ready -- Configure permissions```
```-- Ready to start tests
/pulsar/pulsar-client-cpp/tests /pulsar/pulsar-client-cpp
---- Run unit tests in parallel
Traceback (most recent call last):
File "/gtest-parallel/gtest-parallel", line 18, in <module>
sys.exit(gtest_parallel.main())
File "/gtest-parallel/gtest_parallel.py", line 801, in main
tasks = find_tests(binaries, additional_args, options, times)
File "/gtest-parallel/gtest_parallel.py", line 582, in find_tests
stderr=subprocess.STDOUT)
File "/usr/lib/python2.7/subprocess.py", line 567, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
OSError: [Errno 8] Exec format error```
@Yunze Xu the server log ^
----
2020-09-30 17:32:23 UTC - Ramesh Chandran: @Ramesh Chandran has joined the channel
----
2020-09-30 19:13:13 UTC - Nicoló Boschi: @Nicoló Boschi has joined the channel
----
2020-09-30 20:02:44 UTC - Jeff Schneller: Any updates on a windows C++ client?
----
2020-09-30 20:03:34 UTC - Matteo Merli: The C++ client should already be working on Windows
----
2020-09-30 20:04:33 UTC - Jeff Schneller: hmm... never been able to compile it as of a couple months ago. Are there updates docs for compiling on windows?
----
2020-09-30 20:17:17 UTC - Christopher Faini: @Christopher Faini has joined the channel
----
2020-09-30 20:22:02 UTC - Christopher Faini: Hello! I have a question about Apache Pulsar functions. I noticed that even though the Pulsar client supports C++, it is not listed as a supported language for Pulsar Functions. Are there any plans for C++ to be added as a supported language, or is there some sort of workaround that can be used?
I've only just started working with Pulsar, and this seemed like the best place to ask this question :slightly_smiling_face:
----
2020-09-30 20:22:59 UTC - Matteo Merli: I see. Definitely there were users compiling on Windows. The main issue is not having a windows CI in place to validate new changes.
----
2020-09-30 20:24:59 UTC - Christos Anagnostakis: Hello, I am running Pulsar using docker and the image pulsar-all. When I follow the instruction to use Presto as described in “Query Data with Pulsar SQL” and issue the command “presto> show tables in pulsar.“public/default”;” after generating the mock data, I get an error:
----
2020-09-30 20:25:14 UTC - Christos Anagnostakis: Query 20200930_201656_00003_hweyd failed: Unexpected response from <http://172.17.0.2:8081/v1/task/20200930_201656_00003_hweyd.2.0?summarize>
----
2020-09-30 20:25:33 UTC - Christos Anagnostakis: Any ideas? Many thanks in advance!
----
2020-09-30 22:42:03 UTC - Addison Higham: I don't know of any current plans, but more languages in functions is always something we are interested in and because of C++ client already existing, it is a relatively reasonable amount of effort.
A few questions for you though, for functions, we usually see them as not being very large or complex apps or having need for most optimal per message performance. Is there a primary reason for C++ beyond just language preference? (like library integrations, etc)
----
2020-10-01 04:38:39 UTC - Yunze Xu: Could you run the `main` binary without `gtest_parallel` ? If it still failed, there may be something wrong with the `main` binary.
----
2020-10-01 05:39:18 UTC - Jim M.: Pulsar 2.5.x and Java client. I have partition topics, where keying matter. In k8s. As my app scales, the consumer group never rebalances, is there something I have to do to force it to start handing off records to the new consumers. Im using Keyshared since the consumer must get the same keys.
----
2020-10-01 07:21:08 UTC - charles: @Addison Higham
Thank you Addison. I created a feature request in Github at: <https://github.com/apache/pulsar/issues/8175>
----