You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "eldenmoon (via GitHub)" <gi...@apache.org> on 2023/06/26 09:29:22 UTC
[GitHub] [doris] eldenmoon commented on a diff in pull request #21109: [Improve](serde)update serialize and deserialize text for data type
eldenmoon commented on code in PR #21109:
URL: https://github.com/apache/doris/pull/21109#discussion_r1241877756
##########
be/src/vec/data_types/serde/data_type_serde.h:
##########
@@ -57,9 +59,20 @@ class IDataType;
// the developer does not know how many datatypes has to deal.
class DataTypeSerDe {
+public:
+ struct FormatOptions {
+ bool use_lib_format = false;
Review Comment:
`use_lib_format` could you add some example to explain more details of this flag?
##########
be/src/vec/data_types/serde/data_type_number_serde.cpp:
##########
@@ -92,6 +95,63 @@ void DataTypeNumberSerDe<T>::write_column_to_arrow(const IColumn& column, const
}
}
+template <typename T>
+Status DataTypeNumberSerDe<T>::deserialize_one_cell_from_text(IColumn& column, ReadBuffer& rb,
+ const FormatOptions& options) const {
+ auto& column_data = reinterpret_cast<ColumnType&>(column);
+ if constexpr (std::is_same<T, UInt128>::value) {
+ // TODO: support for Uint128
+ return Status::InvalidArgument("uint128 is not support");
+ } else if constexpr (std::is_same_v<T, float> || std::is_same_v<T, double>) {
+ T val = 0;
+ if (!read_float_text_fast_impl(val, rb)) {
+ return Status::InvalidArgument("parse number fail, string: '{}'",
+ std::string(rb.position(), rb.count()).c_str());
+ }
+ column_data.insert_value(val);
+ } else if constexpr (std::is_same_v<T, uint8_t>) {
+ // Note: here we should handle the bool type
+ T val = 0;
+ if (!try_read_bool_text(val, rb)) {
Review Comment:
what if doris provide uint_8 type? The input rb could be '123' and result Error?
##########
be/src/vec/data_types/serde/data_type_number_serde.cpp:
##########
@@ -92,6 +95,63 @@ void DataTypeNumberSerDe<T>::write_column_to_arrow(const IColumn& column, const
}
}
+template <typename T>
+Status DataTypeNumberSerDe<T>::deserialize_one_cell_from_text(IColumn& column, ReadBuffer& rb,
+ const FormatOptions& options) const {
+ auto& column_data = reinterpret_cast<ColumnType&>(column);
+ if constexpr (std::is_same<T, UInt128>::value) {
Review Comment:
why not handle float and double?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org