You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@calcite.apache.org by "Masayuki Takahashi (JIRA)" <ji...@apache.org> on 2018/04/30 14:21:00 UTC
[jira] [Comment Edited] (CALCITE-2173) Sample implementation of
ArrowAdapter
[ https://issues.apache.org/jira/browse/CALCITE-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458618#comment-16458618 ]
Masayuki Takahashi edited comment on CALCITE-2173 at 4/30/18 2:20 PM:
----------------------------------------------------------------------
It took me a long time, but now I can do filter + project with arrow. Aggregation is still not.
[https://github.com/masayuki038/calcite/tree/arrow/arrow/src/main/java/org/apache/calcite/adapter/arrow]
A test code(JdbcTest#filter) is below:
{code:java}
@Test
public void filter() throws SQLException, ClassNotFoundException {
Class.forName("org.apache.calcite.jdbc.Driver");
try(Connection conn = DriverManager.getConnection("jdbc:calcite:model=target/classes/samples/model.json", "admin", "admin")) {
PreparedStatement pstmt = conn.prepareStatement("select N_REGIONKEY, N_NATIONKEY, N_NAME from NATIONSSF WHERE N_REGIONKEY=?");
pstmt.setLong(1, 1L);
ResultSet rs = pstmt.executeQuery();
resultSetPrint(rs);
}
}
{code}
In this case, generated code is below:
{code:java}
/* 1 */ org.apache.calcite.DataContext root;
/* 2 */
/* 3 */ public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root0) {
/* 4 */ root = root0;
/* 5 */ final org.apache.calcite.linq4j.Enumerator _e0 = ((org.apache.calcite.adapter.arrow.ArrowTable) root.getRootSchema().getSubSchema("SAMPLES").getTable("NATIONSSF")).project(root, new int[] {
/* 6 */ 0,
/* 7 */ 1,
/* 8 */ 2}).enumerator();
/* 9 */ final org.apache.calcite.adapter.arrow.ArrowFilterEnumerator e1 = new org.apache.calcite.adapter.arrow.ArrowFilterEnumerator(
/* 10 */ _e0){
/* 11 */ public java.util.List filter(org.apache.calcite.adapter.arrow.VectorSchemaRootContainer container, int i) {
/* 12 */ final java.util.ArrayList list = new java.util.ArrayList();
/* 13 */ for (int j = 0; j < container.getRowCount(i); ++j) {
/* 14 */ if ((Long) container.getFieldVector(i, 2).getAccessor().getObject(j) != null && (Long) root.get("?0") != null && ((Long) container.getFieldVector(i, 2).getAccessor().getObject(j)).longValue() == ((Long) root.get("?0")).longValue()) {
/* 15 */ list.add((Object) Integer.valueOf(j));
/* 16 */ }
/* 17 */ }
/* 18 */ return list;
/* 19 */ }
/* 20 */
/* 21 */ };
/* 22 */ final org.apache.calcite.adapter.arrow.ArrowProjectEnumerator e2 = new org.apache.calcite.adapter.arrow.ArrowProjectEnumerator(
/* 23 */ e1){
/* 24 */ public int[] getProjectedIndexes() {
/* 25 */ return new int[] {
/* 26 */ 2,
/* 27 */ 0,
/* 28 */ 1};
/* 29 */ }
/* 30 */
/* 31 */ };
/* 32 */ return new org.apache.calcite.linq4j.AbstractEnumerable(){
/* 33 */ public org.apache.calcite.linq4j.Enumerator enumerator() {
/* 34 */ return e2;
/* 35 */ }
/* 36 */
/* 37 */ };
/* 38 */ }
/* 39 */
/* 40 */
/* 41 */ public Class getElementType() {
/* 42 */ return java.lang.Object[].class;
/* 43 */ }
/* 44 */
/* 45 */
{code}
was (Author: masayuki038):
It took me a long time, but now I can do filter + project with arrow. Aggregation. Aggregation is still not.
[https://github.com/masayuki038/calcite/tree/arrow/arrow/src/main/java/org/apache/calcite/adapter/arrow|https://github.com/masayuki038/calcite/tree/arrow/arrow/src/main/java/org/apache/calcite/adapter/arrow]
A test code(JdbcTest#filter) is below:
{code:java}
@Test
public void filter() throws SQLException, ClassNotFoundException {
Class.forName("org.apache.calcite.jdbc.Driver");
try(Connection conn = DriverManager.getConnection("jdbc:calcite:model=target/classes/samples/model.json", "admin", "admin")) {
PreparedStatement pstmt = conn.prepareStatement("select N_REGIONKEY, N_NATIONKEY, N_NAME from NATIONSSF WHERE N_REGIONKEY=?");
pstmt.setLong(1, 1L);
ResultSet rs = pstmt.executeQuery();
resultSetPrint(rs);
}
}
{code}
In this case, generated code is below:
{code:java}
/* 1 */ org.apache.calcite.DataContext root;
/* 2 */
/* 3 */ public org.apache.calcite.linq4j.Enumerable bind(final org.apache.calcite.DataContext root0) {
/* 4 */ root = root0;
/* 5 */ final org.apache.calcite.linq4j.Enumerator _e0 = ((org.apache.calcite.adapter.arrow.ArrowTable) root.getRootSchema().getSubSchema("SAMPLES").getTable("NATIONSSF")).project(root, new int[] {
/* 6 */ 0,
/* 7 */ 1,
/* 8 */ 2}).enumerator();
/* 9 */ final org.apache.calcite.adapter.arrow.ArrowFilterEnumerator e1 = new org.apache.calcite.adapter.arrow.ArrowFilterEnumerator(
/* 10 */ _e0){
/* 11 */ public java.util.List filter(org.apache.calcite.adapter.arrow.VectorSchemaRootContainer container, int i) {
/* 12 */ final java.util.ArrayList list = new java.util.ArrayList();
/* 13 */ for (int j = 0; j < container.getRowCount(i); ++j) {
/* 14 */ if ((Long) container.getFieldVector(i, 2).getAccessor().getObject(j) != null && (Long) root.get("?0") != null && ((Long) container.getFieldVector(i, 2).getAccessor().getObject(j)).longValue() == ((Long) root.get("?0")).longValue()) {
/* 15 */ list.add((Object) Integer.valueOf(j));
/* 16 */ }
/* 17 */ }
/* 18 */ return list;
/* 19 */ }
/* 20 */
/* 21 */ };
/* 22 */ final org.apache.calcite.adapter.arrow.ArrowProjectEnumerator e2 = new org.apache.calcite.adapter.arrow.ArrowProjectEnumerator(
/* 23 */ e1){
/* 24 */ public int[] getProjectedIndexes() {
/* 25 */ return new int[] {
/* 26 */ 2,
/* 27 */ 0,
/* 28 */ 1};
/* 29 */ }
/* 30 */
/* 31 */ };
/* 32 */ return new org.apache.calcite.linq4j.AbstractEnumerable(){
/* 33 */ public org.apache.calcite.linq4j.Enumerator enumerator() {
/* 34 */ return e2;
/* 35 */ }
/* 36 */
/* 37 */ };
/* 38 */ }
/* 39 */
/* 40 */
/* 41 */ public Class getElementType() {
/* 42 */ return java.lang.Object[].class;
/* 43 */ }
/* 44 */
/* 45 */
{code}
> Sample implementation of ArrowAdapter
> -------------------------------------
>
> Key: CALCITE-2173
> URL: https://issues.apache.org/jira/browse/CALCITE-2173
> Project: Calcite
> Issue Type: Improvement
> Reporter: Masayuki Takahashi
> Assignee: Julian Hyde
> Priority: Minor
>
> I try to implement Apache Arrow adaper.
> [https://github.com/masayuki038/calcite/tree/arrow]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)