You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/09/18 06:47:20 UTC
[GitHub] [arrow] Zarca opened a new issue #8217: How to transform the Arrow data column to array of array efficiently?
Zarca opened a new issue #8217:
URL: https://github.com/apache/arrow/issues/8217
Hi, all:
In my case, I'm asking if there is any efficient way that i can take to transform the arrow format table into
the format such as Array[Array[Double]] in java or scala?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on issue #8217: How to transform the Arrow data column to array of array efficiently?
Posted by GitBox <gi...@apache.org>.
wesm commented on issue #8217:
URL: https://github.com/apache/arrow/issues/8217#issuecomment-696156173
This would be best discussed further on user@arrow.apache.org
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm closed issue #8217: How to transform the Arrow data column to array of array efficiently?
Posted by GitBox <gi...@apache.org>.
wesm closed issue #8217:
URL: https://github.com/apache/arrow/issues/8217
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorgecarleitao commented on issue #8217: How to transform the Arrow data column to array of array efficiently?
Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on issue #8217:
URL: https://github.com/apache/arrow/issues/8217#issuecomment-695783084
Hi @Zarca,
1. In any particular language?
2. Arrow is a columnar format. Thus, it is already formatted like you wrote. If you mean is the transpose (i.e. `array[i]` corresponds to an array on which entry is the value of a column), then AFAIK there is no way without a scan i.e.
```
for row in rows {
for column in table.columns() {
value = column[row]
}
}
```
as, like I mentioned, Arrow memory alignment is at the column level, no row level.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm commented on issue #8217: How to transform the Arrow data column to array of array efficiently?
Posted by GitBox <gi...@apache.org>.
wesm commented on issue #8217:
URL: https://github.com/apache/arrow/issues/8217#issuecomment-696156173
This would be best discussed further on user@arrow.apache.org
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] wesm closed issue #8217: How to transform the Arrow data column to array of array efficiently?
Posted by GitBox <gi...@apache.org>.
wesm closed issue #8217:
URL: https://github.com/apache/arrow/issues/8217
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorgecarleitao commented on issue #8217: How to transform the Arrow data column to array of array efficiently?
Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on issue #8217:
URL: https://github.com/apache/arrow/issues/8217#issuecomment-695783084
Hi @Zarca,
1. In any particular language?
2. Arrow is a columnar format. Thus, it is already formatted like you wrote. If you mean is the transpose (i.e. `array[i]` corresponds to an array on which entry is the value of a column), then AFAIK there is no way without a scan i.e.
```
for row in rows {
for column in table.columns() {
value = column[row]
}
}
```
as, like I mentioned, Arrow memory alignment is at the column level, no row level.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org