You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by "zhishengyk (via GitHub)" <gi...@apache.org> on 2025/12/13 09:44:29 UTC
[PR] [feat] add hamming_distance function [doris]
zhishengyk opened a new pull request, #59014:
URL: https://github.com/apache/doris/pull/59014
## Description
Add `hamming_distance` function to calculate Hamming distance between two strings.
## Changes
- **BE**: Add implementation in `function_string.cpp` with vectorized execution
- **FE**: Add `HammingDistance.java` with `AlwaysNullable` (returns NULL if strings have different lengths)
- **Test**: Add BE-UT test with `check_function_all_arg_comb` covering all argument combinations
- **Test**: Add distributed regression test
## Behavior
- Returns BIGINT: the number of positions where corresponding characters differ
- Returns NULL if:
- Either input is NULL
- The two strings have different lengths
## Documentation
Documentation PR: [link to your doc PR]
## Testing
- BE-UT: `check_function_all_arg_comb` covers all argument combinations (vector_vector, scalar_vector, vector_scalar, scalar_scalar)
- Regression test: `test_hamming_distance.groovy` covers various scenarios including NULL handling and different-length strings
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat] add hamming_distance function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on code in PR #59014:
URL: https://github.com/apache/doris/pull/59014#discussion_r2616348900
##########
be/src/vec/functions/function_string.cpp:
##########
@@ -1343,6 +1432,9 @@ using FunctionFromBase64 = FunctionStringOperateToNullType<FromBase64Impl>;
using FunctionStringAppendTrailingCharIfAbsent =
FunctionBinaryStringOperateToNullType<StringAppendTrailingCharIfAbsent>;
+using FunctionHammingDistance =
+ FunctionBinaryStringOperateToNullType<HammingDistanceImpl>;
Review Comment:
shouldn't use this template. for inputs with not same lengths, just raise an error. not null. so `FunctionBinaryToType` and `StringFunctionImpl` may be better?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat] add hamming_distance function [doris]
Posted by "Thearas (via GitHub)" <gi...@apache.org>.
Thearas commented on PR #59014:
URL: https://github.com/apache/doris/pull/59014#issuecomment-3649162999
Thank you for your contribution to Apache Doris.
Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR).
Please clearly describe your PR:
1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat] add hamming_distance function [doris]
Posted by "zhishengyk (via GitHub)" <gi...@apache.org>.
zhishengyk closed pull request #59014: [feat] add hamming_distance function
URL: https://github.com/apache/doris/pull/59014
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat] add hamming_distance function [doris]
Posted by "cai-jinlin (via GitHub)" <gi...@apache.org>.
cai-jinlin commented on PR #59014:
URL: https://github.com/apache/doris/pull/59014#issuecomment-3663351113
#ifndef CORE_HPP
#define CORE_HPP
#include <string>
// 核心计算函数
inline int64_t hamming_calc(const std::string& s1, const std::string& s2) {
if (s1.length() != s2.length()) {
throw std::runtime_error("length mismatch");
}
int64_t diff = 0;
for (size_t i = 0; i < s1.length(); i++) {
if (s1[i] != s2[i]) diff++;
}
return diff;
}
#endif
//选择能抛出异常的模版
#ifndef TEMPLATE_HPP
#define TEMPLATE_HPP
class FunctionBinaryToType {
public:
static int64_t call(const std::string& s1, const std::string& s2) {
// 这里起作用：它会调用核心函数
// 如果 hamming_calc 抛出异常，这里会让异常继续传播
return hamming_calc(s1, s2);
}
};
// 选择模板：这里 FunctionBinaryToType 被选中
using HammingDistance = FunctionBinaryToType;
#endif
//测试代码
#include <iostream>
#include "template.hpp"
// 用户调用的函数
int64_t hamming_distance(const std::string& s1, const std::string& s2) {
return HammingDistance::call(s1, s2);
}
int main() {
// 正常情况
std::cout << "1. 正常调用: ";
try {
int64_t r = hamming_distance("abc", "abc");
std::cout << "结果 = " << r << "\n";
} catch (...) {
std::cout << "抛出异常\n";
}
// 错误情况
std::cout << "\n2. 长度不同: ";
try {
int64_t r = hamming_distance("abc", "abcd");
std::cout << "结果 = " << r << "\n";
} catch (const std::exception& e) {
std::cout << "抛出异常: " << e.what() << "\n";
}
std::cout << "\nFunctionBinaryToType 的作用：传递异常\n";
return 0;
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat] add hamming_distance function [doris]
Posted by "cai-jinlin (via GitHub)" <gi...@apache.org>.
cai-jinlin commented on PR #59014:
URL: https://github.com/apache/doris/pull/59014#issuecomment-3663346499
#ifndef CORE_HPP
#define CORE_HPP
#include <string>
// 核心计算函数
inline int64_t hamming_calc(const std::string& s1, const std::string& s2) {
if (s1.length() != s2.length()) {
throw std::runtime_error("length mismatch");
}
int64_t diff = 0;
for (size_t i = 0; i < s1.length(); i++) {
if (s1[i] != s2[i]) diff++;
}
return diff;
}
#endif
//选择能抛出异常的模版
#ifndef TEMPLATE_HPP
#define TEMPLATE_HPP
class FunctionBinaryToType {
public:
static int64_t call(const std::string& s1, const std::string& s2) {
// 这里起作用：它会调用核心函数
// 如果 hamming_calc 抛出异常，这里会让异常继续传播
return hamming_calc(s1, s2);
}
};
// 选择模板：这里 FunctionBinaryToType 被选中
using HammingDistance = FunctionBinaryToType;
#endif
//测试代码
#include <iostream>
#include "template.hpp"
// 用户调用的函数
int64_t hamming_distance(const std::string& s1, const std::string& s2) {
return HammingDistance::call(s1, s2);
}
int main() {
// 正常情况
std::cout << "1. 正常调用: ";
try {
int64_t r = hamming_distance("abc", "abc");
std::cout << "结果 = " << r << "\n";
} catch (...) {
std::cout << "抛出异常\n";
}
// 错误情况
std::cout << "\n2. 长度不同: ";
try {
int64_t r = hamming_distance("abc", "abcd");
std::cout << "结果 = " << r << "\n";
} catch (const std::exception& e) {
std::cout << "抛出异常: " << e.what() << "\n";
}
std::cout << "\nFunctionBinaryToType 的作用：传递异常\n";
return 0;
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat] add hamming_distance function [doris]
Posted by "zclllyybb (via GitHub)" <gi...@apache.org>.
zclllyybb commented on PR #59014:
URL: https://github.com/apache/doris/pull/59014#issuecomment-3649505376
remember to change format the BE code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
Re: [PR] [feat] add hamming_distance function [doris]
Posted by "cai-jinlin (via GitHub)" <gi...@apache.org>.
cai-jinlin commented on PR #59014:
URL: https://github.com/apache/doris/pull/59014#issuecomment-3649182995
`// 核心计算逻辑：比较两个等长字符串的不同字符数
static int64_t hamming_distance_impl(const char* s1, const char* s2, size_t len) {
int64_t count = 0;
for (size_t i = 0; i < len; ++i) {
count += (s1[i] != s2[i]); // 字符不同时加1
}
return count;
}
// 向量化处理函数（Doris BE核心）
Status execute_impl(FunctionContext* context, Block& block, const ColumnNumbers& args,
size_t result_idx, size_t rows) override {
// 获取输入列
const ColumnPtr& col_left = block.get_by_position(args[0]).column;
const ColumnPtr& col_right = block.get_by_position(args[1]).column;
// 准备结果列
auto result_col = ColumnInt64::create();
auto null_map = ColumnUInt8::create();
auto& results = result_col->get_data();
auto& nulls = null_map->get_data();
results.resize(rows);
nulls.resize_fill(rows, 0);
// 批量处理每一行
for (size_t i = 0; i < rows; ++i) {
// 检查NULL值（Doris会自动处理，这里显式检查）
if (col_left->is_null_at(i) || col_right->is_null_at(i)) {
nulls[i] = 1;
continue;
}
// 获取字符串
auto s1 = col_left->get_data_at(i);
auto s2 = col_right->get_data_at(i);
// 长度不同返回NULL
if (s1.size != s2.size) {
nulls[i] = 1;
continue;
}
// 计算汉明距离
results[i] = hamming_distance_impl(s1.data, s2.data, s1.size);
}
// 包装为可空列
block.get_by_position(result_idx).column =
ColumnNullable::create(std::move(result_col), std::move(null_map));
return Status::OK();
}`
（汉明距离定义就是等长字符串对应位置不同字符的数量）
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org