[jira] [Commented] (CALCITE-2173) Sample implementation of ArrowAdapter

["Masayuki Takahashi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-2173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356943#comment-16356943 ] 

Masayuki Takahashi commented on CALCITE-2173:
---------------------------------------------

I don't know the right way to scan data with columnar-oriented style in Calcite. In particular, in order to adapt to the Enumerator interface, I think to generate the following code:

{code:java}
        return new AbstractEnumerable() {
            @Override
            public Enumerator enumerator() {
                return new ArrowFilterEnumerator(inputEnumerator) {
                    @Override
                    public int[] filter(VectorSchemaRootContainer container, int i) {
                        FieldVector field1 = container.getFieldVector(i, 0);
                        List<Integer> selected = Collections.emptyList();
                        for (int j = 0; j < container.getRowCount(i); j++) {
                            final Long inp2_ = (Long) field1.getAccessor().getObject(j);
                            if (inp2_ != null && (Long) dataContext.get("?0") != null && inp2_.longValue() == ((Long) dataContext.get("?0")).longValue()) {
                                selected.add(j);
                            }
                        }
                        return selected.stream().mapToInt(Integer::intValue).toArray();
                    }

                    @Override
                    public int[] getProjectedIndexes() {
                        return new int[]{0, 1};
                    }
                };
            }
        };
{code}

And base enumerator class is the following code:

{code:java}
public abstract class ArrowFilterEnumerator implements Enumerator {

    private int[][] indexes;
    private int rootIndex = 0;
    private int vectorIndex = 0;
    private Enumerator inputEnumerator;

    public ArrowFilterEnumerator(Enumerator inputEnumerator) {
        this.inputEnumerator = inputEnumerator;
    }

    @Override
    public Object current() {
        VectorSchemaRootContainer container = (VectorSchemaRootContainer)inputEnumerator;
        int[] fieldIndexes = getProjectedIndexes();
        Object[] current = new Object[fieldIndexes.length];
        for (int i = 0; i < fieldIndexes.length; i++) {
            FieldVector vector = container.getFieldVector(rootIndex, fieldIndexes[i]);
            current[i] = vector.getAccessor().getObject(vectorIndex);
        }
        return current;
    }

    @Override
    public boolean moveNext() {
        if (this.indexes == null) {
            this.indexes = filter();
            for (int rootIndex = 0; rootIndex < this.indexes.length; rootIndex++) {
                if (this.indexes[rootIndex].length > 0) {
                    return true;
                }
            }
            return false;
        }

        if (vectorIndex >= indexes[rootIndex].length) {
            if (rootIndex >= indexes.length) {
                return false;
            }
            rootIndex ++;
            vectorIndex = 0;
        } else {
            vectorIndex ++;
        }
        return true;
    }

    @Override
    public void reset() { inputEnumerator.reset(); }

    @Override
    public void close() { inputEnumerator.close(); }

    protected int[][] filter() {
        VectorSchemaRootContainer container = (VectorSchemaRootContainer)inputEnumerator;
        int size = container.getVectorSchemaRootCount();
        int index[][] = new int[size][];
        for (int i = 0; i < size; i++) {
            index[i] = filter(container, i);
        }
        return index;
    }

    abstract public int[] filter(VectorSchemaRootContainer container, int i);
    abstract public int[] getProjectedIndexes();
}
{code}

> Sample implementation of ArrowAdapter
> -------------------------------------
>
>                 Key: CALCITE-2173
>                 URL: https://issues.apache.org/jira/browse/CALCITE-2173
>             Project: Calcite
>          Issue Type: Improvement
>            Reporter: Masayuki Takahashi
>            Assignee: Julian Hyde
>            Priority: Minor
>
> I try to implement Apache Arrow adaper.
> [https://github.com/masayuki038/calcite/tree/arrow]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)