You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/05/26 07:34:35 UTC
[GitHub] [pulsar] BewareMyPower opened a new pull request #10712: [C++] Add SampleFileLogger C++ example
BewareMyPower opened a new pull request #10712:
URL: https://github.com/apache/pulsar/pull/10712
### Motivation
Some users are confused about how to customize the logger in C++ client, while they don't want to build their own cpp library with log4cxx support and change the log4cxx.conf file.
### Modifications
- Provide an example program that writes logs to a single file.
- Change other examples' topic name to v2 format instead of the old v1 topic format.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change is a trivial rework / code cleanup without any test coverage. If you ran the `SampleFileLogger` program in your local env without any pulsar server, a `pulsar-client-cpp.log` would be created and it contains the content like following.
```
Wed May 26 15:34:03 2021 INFO [0x10fde8e00] ConnectionPool:84 Created connection for pulsar://localhost:6650
Wed May 26 15:34:03 2021 ERROR [0x700002a2c000] ClientConnection:422 [<none> -> pulsar://localhost:6650] Failed to establish connection: Connection refused
Wed May 26 15:34:03 2021 INFO [0x700002a2c000] ClientConnection:1446 [<none> -> pulsar://localhost:6650] Connection closed
Wed May 26 15:34:03 2021 ERROR [0x700002a2c000] ClientImpl:181 Error Checking/Getting Partition Metadata while creating producer on persistent://public/default/my-topic -- ConnectError
Wed May 26 15:34:03 2021 INFO [0x700002a2c000] ClientConnection:261 [<none> -> pulsar://localhost:6650] Destroyed connection
Wed May 26 15:34:03 2021 INFO [0x10fde8e00] ClientImpl:483 Closing Pulsar client
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat merged pull request #10712: [C++] Add C++ single file logger factory
Posted by GitBox <gi...@apache.org>.
merlimat merged pull request #10712:
URL: https://github.com/apache/pulsar/pull/10712
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on a change in pull request #10712: [C++] Add C++ example for customizing the logger factory
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on a change in pull request #10712:
URL: https://github.com/apache/pulsar/pull/10712#discussion_r639923788
##########
File path: pulsar-client-cpp/examples/SampleFileLogger.cc
##########
@@ -0,0 +1,111 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <string.h>
+
+#include <chrono>
+#include <ctime>
+#include <fstream>
+#include <sstream>
+#include <thread>
+
+#include <pulsar/Client.h>
+
+class FileLogger : public pulsar::Logger {
+ public:
+ FileLogger(std::ofstream& os, Level level, const std::string& filename)
+ : os_(os), level_(level), filename_(filename) {}
+
+ bool isEnabled(Level level) override { return level >= level_; }
+
+ void log(Level level, int line, const std::string& message) override {
+ std::ostringstream oss;
+ auto now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
+ oss << currentTime() << " " << convertLevelToString(level) << " [" << std::this_thread::get_id()
+ << "] " << filename_ << ":" << line << " " << message << "\n";
+ os_ << oss.str();
+ os_.flush();
+ }
+
+ private:
+ std::ostream& os_;
+ const Level level_;
+ const std::string filename_;
+
+ static const char* currentTime() {
+ auto now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
+ char* s = ctime(&now); // ctime() returns a string with a newline at the end
+ auto newLinePos = strlen(s) - 1;
+ s[newLinePos] = '\0';
+ return s;
+ }
+
+ static const char* convertLevelToString(Level level) {
+ switch (level) {
+ case Level::LEVEL_DEBUG:
+ return "DEBUG";
+ case Level::LEVEL_INFO:
+ return "INFO";
+ case Level::LEVEL_WARN:
+ return "WARN";
+ case Level::LEVEL_ERROR:
+ return "ERROR";
+ default:
+ return "???";
+ }
+ }
+};
+
+/**
+ * A logger factory that is appending logs to a single file.
+ */
+class SingleFileLoggerFactory : public pulsar::LoggerFactory {
Review comment:
Sounds good. I'll expose it as the public interface soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on a change in pull request #10712: [C++] Add C++ single file logger factory
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on a change in pull request #10712:
URL: https://github.com/apache/pulsar/pull/10712#discussion_r641336087
##########
File path: pulsar-client-cpp/include/pulsar/SingleFileLoggerFactory.h
##########
@@ -0,0 +1,68 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#pragma once
+
+#include <fstream>
+#include <ios>
+#include <string>
+#include <pulsar/Logger.h>
+
+namespace pulsar {
+
+/**
+ * A logger factory that is appending logs to a single file.
+ *
+ * The log format is "yyyy-mm-dd hh:MM:ss.xxx <level> <thread-id> <file>:<line> | <msg>", like
+ *
+ * ```
+ * 2021-03-24 17:35:46.571 INFO [0x10a951e00] ConnectionPool:85 | Created connection for ...
+ * ```
+ *
+ * Example:
+ *
+ * ```c++
+ * #include <pulsar/SingleFileLoggerFactory.h>
+ *
+ * ClientConfiguration conf;
+ * conf.setLogger(new SingleFileLoggerFactory(Logger::LEVEL_DEBUG, "pulsar-client-cpp.log"));
+ * Client client("pulsar://localhost:6650", conf);
+ * ```
+ */
+class PULSAR_PUBLIC SingleFileLoggerFactory : public pulsar::LoggerFactory {
Review comment:
There's already a `SimpleLoggerFactory` that writes logs to console. I'll change its name in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on a change in pull request #10712: [C++] Add C++ example for customizing the logger factory
Posted by GitBox <gi...@apache.org>.
merlimat commented on a change in pull request #10712:
URL: https://github.com/apache/pulsar/pull/10712#discussion_r639917361
##########
File path: pulsar-client-cpp/examples/SampleFileLogger.cc
##########
@@ -0,0 +1,111 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <string.h>
+
+#include <chrono>
+#include <ctime>
+#include <fstream>
+#include <sstream>
+#include <thread>
+
+#include <pulsar/Client.h>
+
+class FileLogger : public pulsar::Logger {
+ public:
+ FileLogger(std::ofstream& os, Level level, const std::string& filename)
+ : os_(os), level_(level), filename_(filename) {}
+
+ bool isEnabled(Level level) override { return level >= level_; }
+
+ void log(Level level, int line, const std::string& message) override {
+ std::ostringstream oss;
+ auto now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
+ oss << currentTime() << " " << convertLevelToString(level) << " [" << std::this_thread::get_id()
+ << "] " << filename_ << ":" << line << " " << message << "\n";
+ os_ << oss.str();
+ os_.flush();
+ }
+
+ private:
+ std::ostream& os_;
+ const Level level_;
+ const std::string filename_;
+
+ static const char* currentTime() {
+ auto now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
+ char* s = ctime(&now); // ctime() returns a string with a newline at the end
+ auto newLinePos = strlen(s) - 1;
+ s[newLinePos] = '\0';
+ return s;
+ }
+
+ static const char* convertLevelToString(Level level) {
+ switch (level) {
+ case Level::LEVEL_DEBUG:
+ return "DEBUG";
+ case Level::LEVEL_INFO:
+ return "INFO";
+ case Level::LEVEL_WARN:
+ return "WARN";
+ case Level::LEVEL_ERROR:
+ return "ERROR";
+ default:
+ return "???";
+ }
+ }
+};
+
+/**
+ * A logger factory that is appending logs to a single file.
+ */
+class SingleFileLoggerFactory : public pulsar::LoggerFactory {
Review comment:
I think it would make sense to add this as part of the API, so there is a ready to use simple logger to file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10712: [C++] Add C++ single file logger factory
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10712:
URL: https://github.com/apache/pulsar/pull/10712#issuecomment-849460956
It looks like there's something wrong with the build on CentOS 7, I'll take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] merlimat commented on a change in pull request #10712: [C++] Add C++ single file logger factory
Posted by GitBox <gi...@apache.org>.
merlimat commented on a change in pull request #10712:
URL: https://github.com/apache/pulsar/pull/10712#discussion_r640788843
##########
File path: pulsar-client-cpp/include/pulsar/SingleFileLoggerFactory.h
##########
@@ -0,0 +1,68 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#pragma once
+
+#include <fstream>
+#include <ios>
+#include <string>
+#include <pulsar/Logger.h>
+
+namespace pulsar {
+
+/**
+ * A logger factory that is appending logs to a single file.
+ *
+ * The log format is "yyyy-mm-dd hh:MM:ss.xxx <level> <thread-id> <file>:<line> | <msg>", like
+ *
+ * ```
+ * 2021-03-24 17:35:46.571 INFO [0x10a951e00] ConnectionPool:85 | Created connection for ...
+ * ```
+ *
+ * Example:
+ *
+ * ```c++
+ * #include <pulsar/SingleFileLoggerFactory.h>
+ *
+ * ClientConfiguration conf;
+ * conf.setLogger(new SingleFileLoggerFactory(Logger::LEVEL_DEBUG, "pulsar-client-cpp.log"));
+ * Client client("pulsar://localhost:6650", conf);
+ * ```
+ */
+class PULSAR_PUBLIC SingleFileLoggerFactory : public pulsar::LoggerFactory {
Review comment:
Since in future we could extend this, why not call it `FileLoggerFactory` instead?
##########
File path: pulsar-client-cpp/include/pulsar/SingleFileLoggerFactory.h
##########
@@ -0,0 +1,68 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#pragma once
+
+#include <fstream>
+#include <ios>
+#include <string>
+#include <pulsar/Logger.h>
+
+namespace pulsar {
+
+/**
+ * A logger factory that is appending logs to a single file.
+ *
+ * The log format is "yyyy-mm-dd hh:MM:ss.xxx <level> <thread-id> <file>:<line> | <msg>", like
+ *
+ * ```
+ * 2021-03-24 17:35:46.571 INFO [0x10a951e00] ConnectionPool:85 | Created connection for ...
+ * ```
+ *
+ * Example:
+ *
+ * ```c++
+ * #include <pulsar/SingleFileLoggerFactory.h>
+ *
+ * ClientConfiguration conf;
+ * conf.setLogger(new SingleFileLoggerFactory(Logger::LEVEL_DEBUG, "pulsar-client-cpp.log"));
+ * Client client("pulsar://localhost:6650", conf);
+ * ```
+ */
+class PULSAR_PUBLIC SingleFileLoggerFactory : public pulsar::LoggerFactory {
+ public:
+ /**
+ * Create a SingleFileLoggerFactory instance.
+ *
+ * @param level the log level
+ * @param logFilePath the log file's path
+ */
+ SingleFileLoggerFactory(Logger::Level level, const std::string& logFilePath)
+ : level_(level), os_(logFilePath, std::ios_base::out | std::ios_base::app) {}
+
+ ~SingleFileLoggerFactory() { os_.close(); }
+
+ pulsar::Logger* getLogger(const std::string& filename) override;
+
+ private:
+ const pulsar::Logger::Level level_;
+ std::ofstream os_;
Review comment:
We should use PrivateImpl approach for all classes exposed in the API, so that we can change the implementation and retain binary compatibility.
##########
File path: pulsar-client-cpp/include/pulsar/SingleFileLoggerFactory.h
##########
@@ -0,0 +1,68 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#pragma once
+
+#include <fstream>
+#include <ios>
+#include <string>
+#include <pulsar/Logger.h>
+
+namespace pulsar {
+
+/**
+ * A logger factory that is appending logs to a single file.
+ *
+ * The log format is "yyyy-mm-dd hh:MM:ss.xxx <level> <thread-id> <file>:<line> | <msg>", like
+ *
+ * ```
+ * 2021-03-24 17:35:46.571 INFO [0x10a951e00] ConnectionPool:85 | Created connection for ...
+ * ```
+ *
+ * Example:
+ *
+ * ```c++
+ * #include <pulsar/SingleFileLoggerFactory.h>
+ *
+ * ClientConfiguration conf;
+ * conf.setLogger(new SingleFileLoggerFactory(Logger::LEVEL_DEBUG, "pulsar-client-cpp.log"));
+ * Client client("pulsar://localhost:6650", conf);
+ * ```
+ */
+class PULSAR_PUBLIC SingleFileLoggerFactory : public pulsar::LoggerFactory {
Review comment:
Also, in a different PR we could also expose a configurable `ConsoleLoggerFactory` so that users can set the log level and maybe format there as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on pull request #10712: [C++] Add C++ single file logger factory
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on pull request #10712:
URL: https://github.com/apache/pulsar/pull/10712#issuecomment-849526870
@merlimat Done. PTAL again
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] BewareMyPower commented on a change in pull request #10712: [C++] Add C++ example for customizing the logger factory
Posted by GitBox <gi...@apache.org>.
BewareMyPower commented on a change in pull request #10712:
URL: https://github.com/apache/pulsar/pull/10712#discussion_r639923788
##########
File path: pulsar-client-cpp/examples/SampleFileLogger.cc
##########
@@ -0,0 +1,111 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <string.h>
+
+#include <chrono>
+#include <ctime>
+#include <fstream>
+#include <sstream>
+#include <thread>
+
+#include <pulsar/Client.h>
+
+class FileLogger : public pulsar::Logger {
+ public:
+ FileLogger(std::ofstream& os, Level level, const std::string& filename)
+ : os_(os), level_(level), filename_(filename) {}
+
+ bool isEnabled(Level level) override { return level >= level_; }
+
+ void log(Level level, int line, const std::string& message) override {
+ std::ostringstream oss;
+ auto now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
+ oss << currentTime() << " " << convertLevelToString(level) << " [" << std::this_thread::get_id()
+ << "] " << filename_ << ":" << line << " " << message << "\n";
+ os_ << oss.str();
+ os_.flush();
+ }
+
+ private:
+ std::ostream& os_;
+ const Level level_;
+ const std::string filename_;
+
+ static const char* currentTime() {
+ auto now = std::chrono::system_clock::to_time_t(std::chrono::system_clock::now());
+ char* s = ctime(&now); // ctime() returns a string with a newline at the end
+ auto newLinePos = strlen(s) - 1;
+ s[newLinePos] = '\0';
+ return s;
+ }
+
+ static const char* convertLevelToString(Level level) {
+ switch (level) {
+ case Level::LEVEL_DEBUG:
+ return "DEBUG";
+ case Level::LEVEL_INFO:
+ return "INFO";
+ case Level::LEVEL_WARN:
+ return "WARN";
+ case Level::LEVEL_ERROR:
+ return "ERROR";
+ default:
+ return "???";
+ }
+ }
+};
+
+/**
+ * A logger factory that is appending logs to a single file.
+ */
+class SingleFileLoggerFactory : public pulsar::LoggerFactory {
Review comment:
Sounds good. I'll fix it soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org