You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "zuyu (via GitHub)" <gi...@apache.org> on 2024/09/23 12:31:48 UTC
[PR] [C++][CSV] ParseLine Refactor [arrow]
zuyu opened a new pull request, #44191:
URL: https://github.com/apache/arrow/pull/44191
<!--
Thanks for opening a pull request!
If this is your first pull request you can find detailed information on how
to contribute here:
* [New Contributor's Guide](https://arrow.apache.org/docs/dev/developers/guide/step_by_step/pr_lifecycle.html#reviews-and-merge-of-the-pull-request)
* [Contributing Overview](https://arrow.apache.org/docs/dev/developers/overview.html)
If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
In the case of PARQUET issues on JIRA the title also supports:
PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
-->
### Rationale for this change
<!--
Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.
-->
`ParseLine` uses lots of `goto`s, and it is not that hard to refactor it using loops.
### What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
`ParseLine` refactored.
### Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
Covered by existing tests.
### Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "zuyu (via GitHub)" <gi...@apache.org>.
zuyu commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368085564
@ursabot please benchmark lang=C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368708650
Thanks for your patience. Conbench analyzed the 2 benchmarking runs that have been run so far on PR commit 7e1af5017b72f25975272c706c8d71c01e1ca3e3.
There were 3 benchmark results indicating a performance regression:
- Pull Request Run on `amd64-m5-4xlarge-linux` at [2024-09-23 14:26:54Z](https://conbench.ursa.dev/compare/runs/c926aac687d843d595995fd5b618e68b...ba2c512156dc42d7828b8c779c8c852a/)
- [`BinaryMathOpAggregate` (C++) with source=cpp-micro, suite=arrow-decimal-benchmark](https://conbench.ursa.dev/compare/benchmarks/066ecad2199877638000a1e50b5b6280...066f17b1690a7e9f8000244738011540)
- [`FloatColumnMajorTensorConversionFixture` (C++) with params=<Int32Type>/ConvertToSparseCOOTensorInt32, source=cpp-micro, suite=arrow-tensor-conversion-benchmark](https://conbench.ursa.dev/compare/benchmarks/066ecad23a1c7ccc800084d12852ad26...066f17b1d94679118000af811cba466c)
- and 1 more (see the report linked below)
The [full Conbench report](https://github.com/apache/arrow/runs/30534617868) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "zuyu (via GitHub)" <gi...@apache.org>.
zuyu commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368096642
@ursabot please benchmark lang=C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368079898
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
Thanks for opening a pull request!
If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
Then could you also rename the pull request title in the following format?
GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
In the case of PARQUET issues on JIRA the title also supports:
PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
See also:
* [Other pull requests](https://github.com/apache/arrow/pulls/)
* [Contribution Guidelines - Contributing Overview](https://arrow.apache.org/docs/developers/overview.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "zuyu (via GitHub)" <gi...@apache.org>.
zuyu commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368218884
@ursabot please benchmark lang=C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368219126
Benchmark runs are scheduled for commit 7e1af5017b72f25975272c706c8d71c01e1ca3e3. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368744550
@zuyu Thanks for suggesting this. However, I don't think this is an improvement. The reason the code uses labels and `goto`s is that it's essentially a state machine, and the current style makes the states and state transitions explicit, making it easy to reason about them.
@felipecrv What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "conbench-apache-arrow[bot] (via GitHub)" <gi...@apache.org>.
conbench-apache-arrow[bot] commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368706111
Thanks for your patience. Conbench analyzed the 3 benchmarking runs that have been run so far on PR commit 66ea6de3bbccfb89fe95e88365cc0e9ea6779f49.
There were 2 benchmark results indicating a performance regression:
- Pull Request Run on `amd64-c6a-4xlarge-linux` at [2024-09-23 13:29:27Z](https://conbench.ursa.dev/compare/runs/65527422a04c4551b29d5ad2b7b6d3b6...341b72a9f81d40f589700f0e80d1c076/)
- [`BM_DeltaBitPackingEncode_Int32_Fixed` (C++) with params=1024, source=cpp-micro, suite=parquet-encoding-benchmark](https://conbench.ursa.dev/compare/benchmarks/066f134b083e770280009910306dc417...066f16d9c03c714580005e8cd66a0be0)
- Pull Request Run on `amd64-m5-4xlarge-linux` at [2024-09-23 13:41:38Z](https://conbench.ursa.dev/compare/runs/5da719ada1604c4ba111337d9cedfc39...fd011d7de3f94307ac8c8bfa7f43b6cf/)
- [`BitmapEqualsWithOffset` (C++) with params=8192, source=cpp-micro, suite=arrow-bit-util-benchmark](https://conbench.ursa.dev/compare/benchmarks/066f137699ff79d8800047cc95b6fc69...066f17097e94796c800092c469fbf076)
The [full Conbench report](https://github.com/apache/arrow/runs/30534555764) has more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "zuyu (via GitHub)" <gi...@apache.org>.
zuyu commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2370007312
Thank you for the comments. This *draft* PR was just to let the CI verify the correctness of the refactor. It does not need to get merge, and it seems no performance regressions. 😊
I do agree that sometimes `goto` is useful, and even necessary. Like @felipecrv said, I do observe some CSV parser uses an enum to represent the parsing states, and thus a huge `switch` statement for the state machine, but seems too complicate.
Finally, I really appreciate the code quality of the CSV parser by @pitrou. It is really easier to follow than the above parser.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "zuyu (via GitHub)" <gi...@apache.org>.
zuyu closed pull request #44191: [C++][CSV] ParseLine Refactor
URL: https://github.com/apache/arrow/pull/44191
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368085757
Benchmark runs are scheduled for commit 66ea6de3bbccfb89fe95e88365cc0e9ea6779f49. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "felipecrv (via GitHub)" <gi...@apache.org>.
felipecrv commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2369769432
I agree with @pitrou.
`goto` was deemed harmful by Dijkstra because it made proving the correctness of a program (specially termination checking) harder than with structured constructs, but in the case of a parser, the high-level languages we use don't give us great tools. As such you will find `goto` in many parsers in the wild even when most of the non-parser code in the same codebase doesn't use gotos.
A parser without goto is more branchy. All the parser state must be represented in data instead of being represented in the instruction pointer (which is what goto mutates). These extra branches might affect performance (if the compiler can't make flatten the structured code as we can).
If a label has multiple gotos pointing to it, the version without goto may need functions. This may lead to code repetition due to inlining and worse instruction cache usage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [C++][CSV] ParseLine Refactor [arrow]
Posted by "ursabot (via GitHub)" <gi...@apache.org>.
ursabot commented on PR #44191:
URL: https://github.com/apache/arrow/pull/44191#issuecomment-2368096759
Commit 66ea6de3bbccfb89fe95e88365cc0e9ea6779f49 already has scheduled benchmark runs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org