You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/04/27 16:47:29 UTC
[GitHub] [arrow] Johnnathanalmeida opened a new pull request, #13015: [Gandiva][C++] Add regexp_extract function
Johnnathanalmeida opened a new pull request, #13015:
URL: https://github.com/apache/arrow/pull/13015
Implements the REGEXP_EXTRACT function based on [the Hive implementation](https://www.revisitclass.com/hadoop/regexp_extract-function-in-hive-with-examples/).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Johnnathanalmeida commented on a diff in pull request #13015: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
Johnnathanalmeida commented on code in PR #13015:
URL: https://github.com/apache/arrow/pull/13015#discussion_r861174411
##########
cpp/src/gandiva/tests/projector_test.cc:
##########
@@ -2624,4 +2624,54 @@ TEST_F(TestProjector, TestNextDay) {
// Validate results
EXPECT_ARROW_ARRAY_EQUALS(exp, outputs.at(0));
}
+
+TEST_F(TestProjector, TestRegexpExtract) {
+ // schema for input fields
+ auto field0 = field("f0", arrow::utf8());
+ auto field1 = field("f1", arrow::int32());
+ auto schema = arrow::schema({field0, field1});
+
+ // output fields
+ auto field_extract = field("extract", arrow::utf8());
+
+ // The pattern to match this sequence: string string - number
+ std::string pattern(R"((\w+) (\w+) - (\d+))");
+ auto literal = TreeExprBuilder::MakeStringLiteral(pattern);
+ auto node0 = TreeExprBuilder::MakeField(field0);
+ auto node1 = TreeExprBuilder::MakeField(field1);
+
+ // Build expression
+ auto regexp_extract_func = TreeExprBuilder::MakeFunction(
+ "regexp_extract", {node0, literal, node1}, arrow::utf8());
+ auto extract_expr = TreeExprBuilder::MakeExpression(regexp_extract_func, field_extract);
+
+ std::shared_ptr<Projector> projector;
+ auto status = Projector::Make(schema, {extract_expr}, TestConfiguration(), &projector);
+ EXPECT_TRUE(status.ok()) << status.message();
+
+ // Create a row-batch with some sample data
+ int num_records = 7;
+ auto array0 = MakeArrowArrayUtf8(
+ {"John Doe - 124", "John Doe - 124", "John Doe - 124", "John Doe - 124",
Review Comment:
> can you add test with UTF8 entry? e.g. : b路%$大 路%$大a 路%c$大
New tests cases with japanese characters were add
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Johnnathanalmeida commented on pull request #13015: ARROW-13052: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
Johnnathanalmeida commented on PR #13015:
URL: https://github.com/apache/arrow/pull/13015#issuecomment-1117235489
@github-actions crossbow submit java-jars
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13015: ARROW-13052: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13015:
URL: https://github.com/apache/arrow/pull/13015#issuecomment-1113172802
:warning: Ticket **has not been started in JIRA**, please click 'Start Progress'.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] Johnnathanalmeida commented on pull request #13015: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
Johnnathanalmeida commented on PR #13015:
URL: https://github.com/apache/arrow/pull/13015#issuecomment-1112069883
@github-actions crossbow submit java-jars
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ursabot commented on pull request #13015: ARROW-13052: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
ursabot commented on PR #13015:
URL: https://github.com/apache/arrow/pull/13015#issuecomment-1125605325
Benchmark runs are scheduled for baseline = 3b3ad2b3bb1ae052a848b2a4e680135e2a95c1f4 and contender = a8479e9c252482438b6fc2bc0383ac5cf6a09d59. a8479e9c252482438b6fc2bc0383ac5cf6a09d59 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] [ec2-t3-xlarge-us-east-2](https://conbench.ursa.dev/compare/runs/aa9ce19cc491401083dd0d1950463482...12b9b01f6829478ea1a8b604dc44e24d/)
[Finished :arrow_down:0.43% :arrow_up:0.04%] [test-mac-arm](https://conbench.ursa.dev/compare/runs/a3247d05f555449ea226e6c3c95a777f...680e316e8d324fefa371f9564808b551/)
[Finished :arrow_down:0.0% :arrow_up:0.36%] [ursa-i9-9960x](https://conbench.ursa.dev/compare/runs/fc04daf8de514165905f382ec4488e37...2945509275b04e5ebaa61ef12a77fd49/)
[Finished :arrow_down:0.32% :arrow_up:0.0%] [ursa-thinkcentre-m75q](https://conbench.ursa.dev/compare/runs/b7877061cfa2402d999eb381e4f95f49...896bd0fb06654959a22633cf129ed076/)
Buildkite builds:
[Finished] [`a8479e9c` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/738)
[Finished] [`a8479e9c` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/735)
[Finished] [`a8479e9c` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/725)
[Finished] [`a8479e9c` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/740)
[Finished] [`3b3ad2b3` ec2-t3-xlarge-us-east-2](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/737)
[Finished] [`3b3ad2b3` test-mac-arm](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/734)
[Finished] [`3b3ad2b3` ursa-i9-9960x](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/724)
[Finished] [`3b3ad2b3` ursa-thinkcentre-m75q](https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/739)
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] ViniciusSouzaRoque commented on a diff in pull request #13015: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
ViniciusSouzaRoque commented on code in PR #13015:
URL: https://github.com/apache/arrow/pull/13015#discussion_r860938249
##########
cpp/src/gandiva/tests/projector_test.cc:
##########
@@ -2624,4 +2624,54 @@ TEST_F(TestProjector, TestNextDay) {
// Validate results
EXPECT_ARROW_ARRAY_EQUALS(exp, outputs.at(0));
}
+
+TEST_F(TestProjector, TestRegexpExtract) {
+ // schema for input fields
+ auto field0 = field("f0", arrow::utf8());
+ auto field1 = field("f1", arrow::int32());
+ auto schema = arrow::schema({field0, field1});
+
+ // output fields
+ auto field_extract = field("extract", arrow::utf8());
+
+ // The pattern to match this sequence: string string - number
+ std::string pattern(R"((\w+) (\w+) - (\d+))");
+ auto literal = TreeExprBuilder::MakeStringLiteral(pattern);
+ auto node0 = TreeExprBuilder::MakeField(field0);
+ auto node1 = TreeExprBuilder::MakeField(field1);
+
+ // Build expression
+ auto regexp_extract_func = TreeExprBuilder::MakeFunction(
+ "regexp_extract", {node0, literal, node1}, arrow::utf8());
+ auto extract_expr = TreeExprBuilder::MakeExpression(regexp_extract_func, field_extract);
+
+ std::shared_ptr<Projector> projector;
+ auto status = Projector::Make(schema, {extract_expr}, TestConfiguration(), &projector);
+ EXPECT_TRUE(status.ok()) << status.message();
+
+ // Create a row-batch with some sample data
+ int num_records = 7;
+ auto array0 = MakeArrowArrayUtf8(
+ {"John Doe - 124", "John Doe - 124", "John Doe - 124", "John Doe - 124",
Review Comment:
can you add test with UTF8 entry?
e.g. :
b路%$大
路%$大a
路%c$大
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13015: ARROW-13052: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13015:
URL: https://github.com/apache/arrow/pull/13015#issuecomment-1113172785
https://issues.apache.org/jira/browse/ARROW-13052
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13015: ARROW-13052: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13015:
URL: https://github.com/apache/arrow/pull/13015#issuecomment-1117236606
Revision: 9481ab3421497d429696b732967ea1a3766a27f9
Submitted crossbow builds: [ursacomputing/crossbow @ actions-2013](https://github.com/ursacomputing/crossbow/branches/all?query=actions-2013)
|Task|Status|
|----|------|
|java-jars|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-2013-github-java-jars)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] pravindra closed pull request #13015: ARROW-13052: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
pravindra closed pull request #13015: ARROW-13052: [Gandiva][C++] Add regexp_extract function
URL: https://github.com/apache/arrow/pull/13015
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13015: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13015:
URL: https://github.com/apache/arrow/pull/13015#issuecomment-1111264192
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/master/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW
Opening JIRAs ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename pull request title in the following format?
ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - How to contribute patches](https://arrow.apache.org/docs/developers/contributing.html#how-to-contribute-patches)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #13015: [Gandiva][C++] Add regexp_extract function
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #13015:
URL: https://github.com/apache/arrow/pull/13015#issuecomment-1112070887
Revision: cdd06a726ec9763f7e94696ed9583908d5710a32
Submitted crossbow builds: [ursacomputing/crossbow @ actions-1985](https://github.com/ursacomputing/crossbow/branches/all?query=actions-1985)
|Task|Status|
|----|------|
|java-jars|[](https://github.com/ursacomputing/crossbow/actions?query=branch:actions-1985-github-java-jars)|
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org