You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "WillAyd (via GitHub)" <gi...@apache.org> on 2024/07/02 21:26:38 UTC

[PR] feat(c/driver/postgresql): Basic Support for Writing LIST types [arrow-adbc]

WillAyd opened a new pull request, #1962:
URL: https://github.com/apache/arrow-adbc/pull/1962

   This works towards https://github.com/apache/arrow-adbc/issues/1882
   
   It hasn't fully been implemented to generically support all LIST types and all children they may contain, but figured its far enough along to share any get feedback on approach


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(c/driver/postgresql): Implement LIST/LARGE_LIST Writer [arrow-adbc]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #1962:
URL: https://github.com/apache/arrow-adbc/pull/1962#discussion_r1664955780


##########
c/driver/postgresql/copy/writer.h:
##########
@@ -663,15 +766,15 @@ class PostgresCopyStreamWriter {
     return NANOARROW_OK;
   }
 
-  ArrowErrorCode InitFieldWriters(ArrowError* error) {
+  ArrowErrorCode InitFieldWriters(const PostgresConnection* conn, ArrowError* error) {

Review Comment:
   nit but should we pass in the whole connection or maybe just `const Resolver&`?



##########
c/driver/postgresql/copy/writer.h:
##########
@@ -663,15 +766,15 @@ class PostgresCopyStreamWriter {
     return NANOARROW_OK;
   }
 
-  ArrowErrorCode InitFieldWriters(ArrowError* error) {
+  ArrowErrorCode InitFieldWriters(const PostgresConnection* conn, ArrowError* error) {

Review Comment:
   That might also simplify the amount of setup you have to do in the unit test?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(c/driver/postgresql): Basic Support for Writing LIST types [arrow-adbc]

Posted by "WillAyd (via GitHub)" <gi...@apache.org>.
WillAyd commented on code in PR #1962:
URL: https://github.com/apache/arrow-adbc/pull/1962#discussion_r1663201965


##########
c/driver/postgresql/copy/writer.h:
##########
@@ -601,10 +667,46 @@ static inline ArrowErrorCode MakeCopyFieldWriter(
         case NANOARROW_TYPE_LARGE_BINARY:
         case NANOARROW_TYPE_LARGE_STRING:
           *out = std::make_unique<PostgresCopyBinaryDictFieldWriter>();
+          out->get()->Init(array_view);
           return NANOARROW_OK;
         default:
           break;
       }
+      break;
+    }
+    case NANOARROW_TYPE_LIST: {
+      // For now our implementation only supports primitive children types
+      // See PostgresCopyListFieldWriter::Write for limtiations
+      struct ArrowSchemaView child_schema_view;
+      NANOARROW_RETURN_NOT_OK(
+          ArrowSchemaViewInit(&child_schema_view, schema->children[0], error));
+      switch (child_schema_view.type) {
+        case NANOARROW_TYPE_INT32: {
+          // TODO: likely need to make type_resolver available here

Review Comment:
   Hinted at in current comment, but I _think_ we need a way to forward the resolver or connection to this part of the code to get this to work



##########
c/driver/postgresql/copy/writer.h:
##########
@@ -105,9 +105,7 @@ class PostgresCopyFieldWriter {
 class PostgresCopyFieldTupleWriter : public PostgresCopyFieldWriter {
  public:
   void AppendChild(std::unique_ptr<PostgresCopyFieldWriter> child) {
-    int64_t child_i = static_cast<int64_t>(children_.size());

Review Comment:
   The current design makes it so that the array_view is limited to being supplied to the writers through this `AppendChild` function; however, to support arbitrarily nested writers I think it makes more sense to be moved outside of this
   
   The flip side is now there are a  lot of `writer.Init()` calls placed in the factory function. I also tried to just make the ArrayView part of the constructor for each writer, but that became kind of strange when trying to create the root writer itself.
   
   Probably a better way to structure/refactor this, but decided to live with the code duplication for now to focus on the rest of implementation



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(c/driver/postgresql): Implement LIST/LARGE_LIST Writer [arrow-adbc]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm merged PR #1962:
URL: https://github.com/apache/arrow-adbc/pull/1962


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(c/driver/postgresql): Implement LIST/LARGE_LIST Writer [arrow-adbc]

Posted by "WillAyd (via GitHub)" <gi...@apache.org>.
WillAyd commented on code in PR #1962:
URL: https://github.com/apache/arrow-adbc/pull/1962#discussion_r1665000203


##########
c/driver/postgresql/copy/writer.h:
##########
@@ -663,15 +766,15 @@ class PostgresCopyStreamWriter {
     return NANOARROW_OK;
   }
 
-  ArrowErrorCode InitFieldWriters(ArrowError* error) {
+  ArrowErrorCode InitFieldWriters(const PostgresConnection* conn, ArrowError* error) {

Review Comment:
   Nice idea



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org