You are viewing a plain text version of this content. The canonical link for it is here.

Posted to github@arrow.apache.org by "JOBIN-SABU (via GitHub)" <gi...@apache.org> on 2025/02/06 06:44:20 UTC

[PR] Fix issue #45394: Handle single-line JSON without line ending [arrow]

JOBIN-SABU opened a new pull request, #45443:
URL: https://github.com/apache/arrow/pull/45443

   Fixes issue #45394: Handle single-line JSON without line ending
   
   - Previously, if a file contained only one line of JSON and lacked a newline character at the end, 
     `pyarrow.dataset.dataset` would correctly infer the schema but fail to load the values from that row.
   - This fix ensures that such cases are handled properly.
   - Additionally, I have fixed some other minor issues in the code.
   - Tests are still not passing, but they were already failing before these changes.
   - I believe the issue might be on the upstream side, but please review.
   
   Looking forward to feedback!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on PR #45443:
URL: https://github.com/apache/arrow/pull/45443#issuecomment-2638976522

   :warning: GitHub issue #45394 **has been automatically assigned in GitHub** to PR creator.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on PR #45443:
URL: https://github.com/apache/arrow/pull/45443#issuecomment-4030662208

   Thank you for your contribution. Unfortunately, this pull request has been marked as stale because it has had no activity in the past 365 days. Please remove the stale label or comment below, or this PR will be closed in 14 days.  Feel free to re-open this if it has been closed in error. If you do not have repository permissions to reopen the PR, please tag a maintainer.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Fix issue #45394: Handle single-line JSON without line ending [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] commented on PR #45443:
URL: https://github.com/apache/arrow/pull/45443#issuecomment-2638970042

   <!--
     Licensed to the Apache Software Foundation (ASF) under one
     or more contributor license agreements.  See the NOTICE file
     distributed with this work for additional information
     regarding copyright ownership.  The ASF licenses this file
     to you under the Apache License, Version 2.0 (the
     "License"); you may not use this file except in compliance
     with the License.  You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
   
     Unless required by applicable law or agreed to in writing,
     software distributed under the License is distributed on an
     "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
     KIND, either express or implied.  See the License for the
     specific language governing permissions and limitations
     under the License.
   -->
   
   Thanks for opening a pull request!
   
   If this is not a [minor PR](https://github.com/apache/arrow/blob/main/CONTRIBUTING.md#Minor-Fixes). Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose
   
   Opening GitHub issues ahead of time contributes to the [Openness](http://theapacheway.com/open/#:~:text=Openness%20allows%20new%20users%20the,must%20happen%20in%20the%20open.) of the Apache Arrow project.
   
   Then could you also rename the pull request title in the following format?
   
       GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}
   
   or
   
       MINOR: [${COMPONENT}] ${SUMMARY}
   
   See also:
   
     * [Other pull requests](https://github.com/apache/arrow/pulls/)
     * [Contribution Guidelines - Contributing Overview](https://arrow.apache.org/docs/developers/overview.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.

github-actions[bot] closed pull request #45443: GH-45394: [C++] Handle Single-Line JSON Without Line Ending
URL: https://github.com/apache/arrow/pull/45443


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.

raulcd commented on code in PR #45443:
URL: https://github.com/apache/arrow/pull/45443#discussion_r1944794062


##########
cpp/tests/test_file_json.cc:
##########
@@ -0,0 +1,81 @@
+#include <gtest/gtest.h>

Review Comment:
   is there any reason you're not using the existing `files_json_test.cc` file and infrastructure and you're adding all this new CMake?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "JOBIN-SABU (via GitHub)" <gi...@apache.org>.

JOBIN-SABU commented on PR #45443:
URL: https://github.com/apache/arrow/pull/45443#issuecomment-2692015391

Hi @mapleFU,

I haven’t provided specific inputs in the test cases yet, but I can describe the scenario and the expected behavior:

Test Case: Single Line JSON Without Newline

Description: This test case is meant to handle a JSON file containing a single line without a newline character.

Expected Behavior: The JSON object should be correctly parsed, and the values should be loaded.

Actual Behavior: The test fails to parse the JSON object and load the values correctly.

Additionally, I’ve encountered challenges related to the mixing of Release and Debug builds, which is causing build errors. Specifically, the mismatches in _ITERATOR_DEBUG_LEVEL and RuntimeLibrary settings are leading to issues during the build process.

I’ve made changes to address the core issue, but I’m encountering challenges with running the tests due to hardware and internet limitations. The tests are still not passing, and I suspect the issue might be upstream. I’d appreciate any guidance or assistance you can provide to help finalize these changes and get the tests passing.

Thank you for your help!

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.

raulcd commented on code in PR #45443:
URL: https://github.com/apache/arrow/pull/45443#discussion_r1944796509


##########
cpp/tests/test_file_json.cc:
##########
@@ -0,0 +1,81 @@
+#include <gtest/gtest.h>

Review Comment:
   https://github.com/apache/arrow/blob/16c7f1a0bbcfad20eba7c63bc86d3da784d1db34/cpp/src/arrow/dataset/file_json_test.cc



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "JOBIN-SABU (via GitHub)" <gi...@apache.org>.

JOBIN-SABU commented on code in PR #45443:
URL: https://github.com/apache/arrow/pull/45443#discussion_r1951983136


##########
cpp/tests/test_file_json.cc:
##########
@@ -0,0 +1,81 @@
+#include <gtest/gtest.h>

Review Comment:
   Hi @raulcd a
   
   I hope you're doing well.
   
   I've made progress on fixing issue #45394, but I've encountered some challenges with running the tests due to hardware and internet limitations. While I believe I've addressed the core of the problem, the tests are still not passing, and I think this might be an upstream issue.
   
   I've pushed my changes to the fix-45394 branch. Could you please take it from here and help with running the tests and finalizing any additional changes?
   
   I appreciate your assistance and look forward to your feedback.
   
   Thank you!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "raulcd (via GitHub)" <gi...@apache.org>.

raulcd commented on code in PR #45443:
URL: https://github.com/apache/arrow/pull/45443#discussion_r1944910066


##########
cpp/tests/test_file_json.cc:
##########
@@ -0,0 +1,81 @@
+#include <gtest/gtest.h>

Review Comment:
   no worries at all! Thanks for your contributions!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "JOBIN-SABU (via GitHub)" <gi...@apache.org>.

JOBIN-SABU commented on code in PR #45443:
URL: https://github.com/apache/arrow/pull/45443#discussion_r1944865504


##########
cpp/tests/test_file_json.cc:
##########
@@ -0,0 +1,81 @@
+#include <gtest/gtest.h>

Review Comment:
   Hi @raulcd,
   
   I sincerely apologize for overlooking the existing file_json_test.cc file and the associated infrastructure. I should have checked and integrated my changes into the existing test framework instead of introducing new CMake configurations.
   
   I’ll make the necessary corrections to align with the existing structure and update the PR accordingly. Thanks for pointing this out!
   
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] GH-45394: [C++] Handle Single-Line JSON Without Line Ending [arrow]

Posted by "JOBIN-SABU (via GitHub)" <gi...@apache.org>.

JOBIN-SABU commented on PR #45443:
URL: https://github.com/apache/arrow/pull/45443#issuecomment-2656135083

   Good morning Sir
   
   I hope you're doing well.
   
   I've made progress on fixing issue #45394, but I've encountered some
   challenges with running the tests due to hardware and internet limitations.
   While I believe I've addressed the core of the problem, the tests are still
   not passing, and I think this might be an upstream issue.
   
   I've pushed my changes to the fix-45394 branch. Could you please take it
   from here and help with running the tests and finalizing any additional
   changes?
   
   I appreciate your assistance and look forward to your feedback.
   
   Thank you!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org