You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "xxlaykxx (via GitHub)" <gi...@apache.org> on 2025/04/01 08:14:59 UTC

[PR] GH-87: [Java][Vector] Add ExtensionWriter for List/Struct. [arrow-java]

xxlaykxx opened a new pull request, #697:
URL: https://github.com/apache/arrow-java/pull/697

   Based on changes from https://github.com/apache/arrow/pull/41731.
   
   ## What's Changed
   
   Added writer ExtensionWriter with 3 methods:
   - write method  for writing values from Extension holders;
   - writeExtensionType method for writing values (arguments is Object because we don't know exact type);
   - addExtensionTypeFactory method - because the exact vector and value type are unknown, the user should create their own extension type vector, write for it, and ExtensionTypeFactory, which should map the vector and writer.
   
   **This contains breaking changes.**  <!-- Remove this line if there are no breaking changes. -->
   
   Closes #NNN.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Vector] Add ExtensionWriter [arrow-java]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm merged PR #697:
URL: https://github.com/apache/arrow-java/pull/697


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Vector] Add ExtensionWriter [arrow-java]

Posted by "xxlaykxx (via GitHub)" <gi...@apache.org>.
xxlaykxx commented on code in PR #697:
URL: https://github.com/apache/arrow-java/pull/697#discussion_r2028753370


##########
vector/src/main/codegen/templates/BaseWriter.java:
##########
@@ -101,6 +103,13 @@ public interface MapWriter extends ListWriter {
     MapWriter value();
   }
 
+  public interface ExtensionWriter extends BaseWriter {
+    void writeNull();
+    <T extends ExtensionHolder> void write(T var1);

Review Comment:
   fixed



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Java][Vector] Add ExtensionWriter for List/Struct. [arrow-java]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #697:
URL: https://github.com/apache/arrow-java/pull/697#issuecomment-2768553352

   
   <!-- label_helper_comment -->
   Thank you for opening a pull request!
   
   Please label the PR with one or more of:
   
   - bug-fix
   - chore
   - dependencies
   - documentation
   - enhancement
   
   Also, add the 'breaking-change' label if appropriate.
   
   See [CONTRIBUTING.md](https://github.com/apache/arrow-java/blob/main/CONTRIBUTING.md) for details.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Java][Vector] Add ExtensionWriter for List/Struct. [arrow-java]

Posted by "xxlaykxx (via GitHub)" <gi...@apache.org>.
xxlaykxx commented on PR #697:
URL: https://github.com/apache/arrow-java/pull/697#issuecomment-2768578921

   @lidavidm, please take a look if this approach can be used


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Java][Vector] Add ExtensionWriter for List/Struct. [arrow-java]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #697:
URL: https://github.com/apache/arrow-java/pull/697#discussion_r2026159801


##########
vector/src/main/java/org/apache/arrow/vector/complex/impl/ExtensionTypeWriterFactory.java:
##########
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import org.apache.arrow.vector.ExtensionTypeVector;
+
+public interface ExtensionTypeWriterFactory<T extends AbstractFieldWriter> {

Review Comment:
   Can we document this?



##########
vector/src/main/java/org/apache/arrow/vector/holders/ExtensionHolder.java:
##########
@@ -0,0 +1,21 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.holders;
+
+public abstract class ExtensionHolder implements ValueHolder {

Review Comment:
   Can we document this?



##########
vector/src/test/java/org/apache/arrow/vector/types/pojo/TestExtensionType.java:
##########


Review Comment:
   We may want to just pull the UuidType into a toplevel class now that it's being used by multiple tests.



##########
vector/src/test/java/org/apache/arrow/vector/complex/impl/TestPromotableWriter.java:
##########
@@ -776,4 +780,95 @@ public void testPromoteToUnionFromDecimal() throws Exception {
       assertEquals(1, intHolder.value);
     }
   }
+
+  @Test
+  public void testExtensionType() throws Exception {
+    try (final NonNullableStructVector container =
+        NonNullableStructVector.empty(EMPTY_SCHEMA_PATH, allocator);
+        final UuidVector v =
+            container.addOrGet("uuid", FieldType.nullable(new UuidType()), UuidVector.class);
+        final PromotableWriter writer = new PromotableWriter(v, container)) {
+      UUID u1 = UUID.randomUUID();
+      UUID u2 = UUID.randomUUID();
+      container.allocateNew();
+      container.setValueCount(1);
+      writer.addExtensionTypeFactory(new UuidWriterFactory());
+
+      writer.setPosition(0);
+      writer.writeExtensionType(u1);
+      writer.setPosition(1);
+      writer.writeExtensionType(u2);
+
+      container.setValueCount(2);
+
+      UuidVector uuidVector = (UuidVector) container.getChild("uuid");
+      assertEquals(u1, uuidVector.getObject(0));
+      assertEquals(u2, uuidVector.getObject(1));
+    }
+  }
+
+  public class UuidWriterFactory implements ExtensionTypeWriterFactory {
+
+    @Override
+    public AbstractFieldWriter getWriterImpl(ExtensionTypeVector extensionTypeVector) {
+      if (extensionTypeVector instanceof UuidVector) {
+        return new UuidWriterImpl((UuidVector) extensionTypeVector);
+      }
+      return null;
+    }
+  }
+
+  public class UuidWriterImpl extends AbstractFieldWriter {
+    private final UuidVector vector;
+
+    public UuidWriterImpl(UuidVector vector) {
+      this.vector = vector;
+    }
+
+    @Override

Review Comment:
   Is there a way we can provide default implementations for these functions to reduce the boilerplate?



##########
vector/src/main/java/org/apache/arrow/vector/complex/impl/ExtensionTypeWriterFactory.java:
##########
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import org.apache.arrow.vector.ExtensionTypeVector;
+
+public interface ExtensionTypeWriterFactory<T extends AbstractFieldWriter> {

Review Comment:
   Does the type bound need to be `AbstractFieldWriter` or can it just be `FieldWriter` (the interface)?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Vector] Add ExtensionWriter [arrow-java]

Posted by "xxlaykxx (via GitHub)" <gi...@apache.org>.
xxlaykxx commented on code in PR #697:
URL: https://github.com/apache/arrow-java/pull/697#discussion_r2027399736


##########
vector/src/main/java/org/apache/arrow/vector/complex/impl/ExtensionTypeWriterFactory.java:
##########
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import org.apache.arrow.vector.ExtensionTypeVector;
+
+public interface ExtensionTypeWriterFactory<T extends AbstractFieldWriter> {

Review Comment:
   I chose **AbstractFieldWriter** just to make sure that the user will use some specific implementation of writer, but in general yes - it could be FieldWriter



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Vector] Add ExtensionWriter [arrow-java]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on PR #697:
URL: https://github.com/apache/arrow-java/pull/697#issuecomment-2789514431

   Thanks @xxlaykxx @FiV0!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Vector] Add ExtensionWriter [arrow-java]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #697:
URL: https://github.com/apache/arrow-java/pull/697#discussion_r2032494847


##########
vector/src/test/java/org/apache/arrow/vector/complex/impl/UuidWriterImpl.java:
##########
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import java.nio.ByteBuffer;
+import java.util.UUID;
+import org.apache.arrow.vector.UuidVector;
+import org.apache.arrow.vector.holder.UuidHolder;
+import org.apache.arrow.vector.holders.ExtensionHolder;
+
+public class UuidWriterImpl extends AbstractExtensionTypeWriter {

Review Comment:
   ```suggestion
   public class UuidWriterImpl extends AbstractExtensionTypeWriter<UuidVector> {
   ```



##########
vector/src/test/java/org/apache/arrow/vector/complex/impl/UuidWriterImpl.java:
##########
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import java.nio.ByteBuffer;
+import java.util.UUID;
+import org.apache.arrow.vector.UuidVector;
+import org.apache.arrow.vector.holder.UuidHolder;
+import org.apache.arrow.vector.holders.ExtensionHolder;
+
+public class UuidWriterImpl extends AbstractExtensionTypeWriter {
+
+  public UuidWriterImpl(UuidVector vector) {
+    super(vector);
+  }
+
+  @Override
+  public void writeExtensionType(Object value) {
+    UUID uuid = (UUID) value;
+    ByteBuffer bb = ByteBuffer.allocate(16);
+    bb.putLong(uuid.getMostSignificantBits());
+    bb.putLong(uuid.getLeastSignificantBits());
+    ((UuidVector) this.vector).setSafe(this.idx(), bb.array());

Review Comment:
   ```suggestion
       vector.setSafe(idx(), bb.array());
   ```



##########
vector/src/main/java/org/apache/arrow/vector/complex/impl/AbstractExtensionTypeWriter.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import org.apache.arrow.vector.ExtensionTypeVector;
+import org.apache.arrow.vector.types.pojo.Field;
+
+/**
+ * Base {@link AbstractFieldWriter} class for an {@link
+ * org.apache.arrow.vector.ExtensionTypeVector}.
+ *
+ * @param <T> a specific {@link ExtensionTypeVector}.
+ */
+public class AbstractExtensionTypeWriter<T extends ExtensionTypeVector>
+    extends AbstractFieldWriter {
+  protected final T vector;
+
+  public AbstractExtensionTypeWriter(T vector) {
+    this.vector = vector;
+  }
+
+  @Override
+  public Field getField() {
+    return this.vector.getField();
+  }
+
+  @Override
+  public int getValueCapacity() {
+    return this.vector.getValueCapacity();
+  }
+
+  @Override
+  public void allocate() {
+    this.vector.allocateNew();
+  }
+
+  @Override
+  public void close() {
+    this.vector.close();
+  }
+
+  @Override
+  public void clear() {
+    this.vector.clear();
+  }
+
+  @Override
+  protected int idx() {
+    return super.idx();

Review Comment:
   Hmm, is the only reason we override here to make it `protected` instead of package-private? (Should we just use `getPosition()` instead as it is already public and does the same thing?)



##########
vector/src/main/codegen/templates/BaseWriter.java:
##########
@@ -101,6 +103,35 @@ public interface MapWriter extends ListWriter {
     MapWriter value();
   }
 
+  public interface ExtensionWriter extends BaseWriter {
+    
+    /**
+     * Writes a null value.
+     */
+    void writeNull();
+
+    /**
+     * Writes vlaue from the given extension holder.

Review Comment:
   ```suggestion
        * Writes value from the given extension holder.
   ```



##########
vector/src/main/codegen/templates/BaseWriter.java:
##########
@@ -101,6 +103,35 @@ public interface MapWriter extends ListWriter {
     MapWriter value();
   }
 
+  public interface ExtensionWriter extends BaseWriter {
+    
+    /**
+     * Writes a null value.
+     */
+    void writeNull();
+
+    /**
+     * Writes vlaue from the given extension holder.
+     *
+     * @param holder the extension holder to write
+     */
+    void write(ExtensionHolder holder);
+
+    /**
+     * Writes the given extension type value.
+     *
+     * @param value the extension type value to write
+     */
+    void writeExtensionType(Object value);
+    
+    /**
+     * Adds the given extension type factory. This factory allows configuring writer implementations for specific ExtensionTypeVector.
+     *
+     * @param factory the extension type factory to add
+     */
+    void addExtensionTypeFactory(ExtensionTypeWriterFactory factory);

Review Comment:
   ```suggestion
       void addExtensionTypeWriterFactory(ExtensionTypeWriterFactory factory);
   ```
   for consistency as well



##########
vector/src/test/java/org/apache/arrow/vector/complex/impl/UuidWriterImpl.java:
##########
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import java.nio.ByteBuffer;
+import java.util.UUID;
+import org.apache.arrow.vector.UuidVector;
+import org.apache.arrow.vector.holder.UuidHolder;
+import org.apache.arrow.vector.holders.ExtensionHolder;
+
+public class UuidWriterImpl extends AbstractExtensionTypeWriter {
+
+  public UuidWriterImpl(UuidVector vector) {
+    super(vector);
+  }
+
+  @Override
+  public void writeExtensionType(Object value) {
+    UUID uuid = (UUID) value;
+    ByteBuffer bb = ByteBuffer.allocate(16);
+    bb.putLong(uuid.getMostSignificantBits());
+    bb.putLong(uuid.getLeastSignificantBits());
+    ((UuidVector) this.vector).setSafe(this.idx(), bb.array());
+    this.vector.setValueCount(this.idx() + 1);
+  }
+
+  @Override
+  public void write(ExtensionHolder holder) {
+    if (holder instanceof UuidHolder) {

Review Comment:
   Either delete the check to be consistent with writeExtensionType or explicitly error if the holder is the wrong type



##########
vector/src/main/codegen/templates/BaseWriter.java:
##########
@@ -101,6 +103,35 @@ public interface MapWriter extends ListWriter {
     MapWriter value();
   }
 
+  public interface ExtensionWriter extends BaseWriter {
+    
+    /**
+     * Writes a null value.
+     */
+    void writeNull();
+
+    /**
+     * Writes vlaue from the given extension holder.
+     *
+     * @param holder the extension holder to write
+     */
+    void write(ExtensionHolder holder);
+
+    /**
+     * Writes the given extension type value.
+     *
+     * @param value the extension type value to write
+     */
+    void writeExtensionType(Object value);

Review Comment:
   This is a nit but now I'm thinking it should be just `writeExtension` to parallel `writeVarChar`



##########
vector/src/test/java/org/apache/arrow/vector/complex/impl/UuidWriterImpl.java:
##########
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import java.nio.ByteBuffer;
+import java.util.UUID;
+import org.apache.arrow.vector.UuidVector;
+import org.apache.arrow.vector.holder.UuidHolder;
+import org.apache.arrow.vector.holders.ExtensionHolder;
+
+public class UuidWriterImpl extends AbstractExtensionTypeWriter {
+
+  public UuidWriterImpl(UuidVector vector) {
+    super(vector);
+  }
+
+  @Override
+  public void writeExtensionType(Object value) {
+    UUID uuid = (UUID) value;
+    ByteBuffer bb = ByteBuffer.allocate(16);
+    bb.putLong(uuid.getMostSignificantBits());
+    bb.putLong(uuid.getLeastSignificantBits());
+    ((UuidVector) this.vector).setSafe(this.idx(), bb.array());
+    this.vector.setValueCount(this.idx() + 1);
+  }
+
+  @Override
+  public void write(ExtensionHolder holder) {
+    if (holder instanceof UuidHolder) {
+      UuidHolder uuidHolder = (UuidHolder) holder;
+      ((UuidVector) this.vector).setSafe(this.idx(), uuidHolder.value);
+      this.vector.setValueCount(this.idx() + 1);

Review Comment:
   ```suggestion
         vector.setSafe(idx(), uuidHolder.value);
         vector.setValueCount(idx() + 1);
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Vector] Add ExtensionWriter [arrow-java]

Posted by "xxlaykxx (via GitHub)" <gi...@apache.org>.
xxlaykxx commented on code in PR #697:
URL: https://github.com/apache/arrow-java/pull/697#discussion_r2034575200


##########
vector/src/main/java/org/apache/arrow/vector/complex/impl/AbstractExtensionTypeWriter.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import org.apache.arrow.vector.ExtensionTypeVector;
+import org.apache.arrow.vector.types.pojo.Field;
+
+/**
+ * Base {@link AbstractFieldWriter} class for an {@link
+ * org.apache.arrow.vector.ExtensionTypeVector}.
+ *
+ * @param <T> a specific {@link ExtensionTypeVector}.
+ */
+public class AbstractExtensionTypeWriter<T extends ExtensionTypeVector>
+    extends AbstractFieldWriter {
+  protected final T vector;
+
+  public AbstractExtensionTypeWriter(T vector) {
+    this.vector = vector;
+  }
+
+  @Override
+  public Field getField() {
+    return this.vector.getField();
+  }
+
+  @Override
+  public int getValueCapacity() {
+    return this.vector.getValueCapacity();
+  }
+
+  @Override
+  public void allocate() {
+    this.vector.allocateNew();
+  }
+
+  @Override
+  public void close() {
+    this.vector.close();
+  }
+
+  @Override
+  public void clear() {
+    this.vector.clear();
+  }
+
+  @Override
+  protected int idx() {
+    return super.idx();

Review Comment:
   yep, makes sense - logic with idx() method added for all other writers.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Vector] Add ExtensionWriter [arrow-java]

Posted by "xxlaykxx (via GitHub)" <gi...@apache.org>.
xxlaykxx commented on PR #697:
URL: https://github.com/apache/arrow-java/pull/697#issuecomment-2777924844

   > Can we preserve the original committer's information if possible so they can get the credit for the original work? (A `Co-authored-by` tag in the PR should suffice I think)
   > 
   > Is there some way we could have a type-safe design instead of just `Object` everywhere?
   
   Because the type is unknown, we could allow only Holder impl for writing. Is this acceptable?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] GH-87: [Vector] Add ExtensionWriter [arrow-java]

Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on code in PR #697:
URL: https://github.com/apache/arrow-java/pull/697#discussion_r2028374926


##########
vector/src/test/java/org/apache/arrow/vector/types/pojo/TestUuidVector.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.types.pojo;
+
+import java.nio.ByteBuffer;
+import java.util.UUID;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.util.hash.ArrowBufHasher;
+import org.apache.arrow.vector.ExtensionTypeVector;
+import org.apache.arrow.vector.FieldVector;
+import org.apache.arrow.vector.FixedSizeBinaryVector;
+import org.apache.arrow.vector.ValueIterableVector;
+import org.apache.arrow.vector.ValueVector;
+import org.apache.arrow.vector.types.pojo.ArrowType.ExtensionType;
+import org.apache.arrow.vector.util.TransferPair;
+
+public class TestUuidVector {
+
+  public static class UuidType extends ExtensionType {

Review Comment:
   What I meant was, can we just make this a toplevel class? Not a nested class?



##########
vector/src/test/java/org/apache/arrow/vector/types/pojo/TestUuidVector.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.types.pojo;
+
+import java.nio.ByteBuffer;
+import java.util.UUID;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.util.hash.ArrowBufHasher;
+import org.apache.arrow.vector.ExtensionTypeVector;
+import org.apache.arrow.vector.FieldVector;
+import org.apache.arrow.vector.FixedSizeBinaryVector;
+import org.apache.arrow.vector.ValueIterableVector;
+import org.apache.arrow.vector.ValueVector;
+import org.apache.arrow.vector.types.pojo.ArrowType.ExtensionType;
+import org.apache.arrow.vector.util.TransferPair;
+
+public class TestUuidVector {
+
+  public static class UuidType extends ExtensionType {
+
+    @Override
+    public ArrowType storageType() {
+      return new ArrowType.FixedSizeBinary(16);
+    }
+
+    @Override
+    public String extensionName() {
+      return "uuid";
+    }
+
+    @Override
+    public boolean extensionEquals(ExtensionType other) {
+      return other instanceof UuidType;
+    }
+
+    @Override
+    public ArrowType deserialize(ArrowType storageType, String serializedData) {
+      if (!storageType.equals(storageType())) {
+        throw new UnsupportedOperationException(
+            "Cannot construct UuidType from underlying type " + storageType);
+      }
+      return new UuidType();
+    }
+
+    @Override
+    public String serialize() {
+      return "";
+    }
+
+    @Override
+    public FieldVector getNewVector(String name, FieldType fieldType, BufferAllocator allocator) {
+      return new UuidVector(name, allocator, new FixedSizeBinaryVector(name, allocator, 16));
+    }
+  }
+
+  public static class UuidVector extends ExtensionTypeVector<FixedSizeBinaryVector>

Review Comment:
   Same here



##########
vector/src/main/codegen/templates/BaseWriter.java:
##########
@@ -101,6 +103,13 @@ public interface MapWriter extends ListWriter {
     MapWriter value();
   }
 
+  public interface ExtensionWriter extends BaseWriter {
+    void writeNull();
+    <T extends ExtensionHolder> void write(T var1);
+    void writeExtensionType(Object var1);
+    <T extends ExtensionTypeWriterFactory> void addExtensionTypeFactory(T var1);

Review Comment:
   While the existing code is short on docstrings, can we add docstrings for new code going forward? In particular it's not clear what `addExtensionTypeFactory` is or how to use it



##########
vector/src/main/codegen/templates/BaseWriter.java:
##########
@@ -101,6 +103,13 @@ public interface MapWriter extends ListWriter {
     MapWriter value();
   }
 
+  public interface ExtensionWriter extends BaseWriter {
+    void writeNull();
+    <T extends ExtensionHolder> void write(T var1);
+    void writeExtensionType(Object var1);
+    <T extends ExtensionTypeWriterFactory> void addExtensionTypeFactory(T var1);

Review Comment:
   Are the generic bounds actually useful here? Why can't it just be `ExtensionTypeWriterFactory var1`?



##########
vector/src/main/codegen/templates/BaseWriter.java:
##########
@@ -101,6 +103,13 @@ public interface MapWriter extends ListWriter {
     MapWriter value();
   }
 
+  public interface ExtensionWriter extends BaseWriter {
+    void writeNull();
+    <T extends ExtensionHolder> void write(T var1);

Review Comment:
   Same here - why not just `void write(ExtensionHolder value)`?



##########
vector/src/test/java/org/apache/arrow/vector/complex/impl/TestPromotableWriter.java:
##########


Review Comment:
   I don't see any test of using ExtensionHolder?



##########
vector/src/main/java/org/apache/arrow/vector/complex/impl/ExtensionTypeWriterFactory.java:
##########
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import org.apache.arrow.vector.ExtensionTypeVector;
+
+/**
+ * A factory for {@link ExtensionTypeWriter} instances. The factory allow to configure writer
+ * implementation for specific ExtensionTypeVector.
+ *
+ * @param <T> writer implementation for specific {@link ExtensionTypeVector}.
+ */
+public interface ExtensionTypeWriterFactory<T extends AbstractFieldWriter> {
+  T getWriterImpl(ExtensionTypeVector vector);

Review Comment:
   Can we document the methods as well?



##########
vector/src/test/java/org/apache/arrow/vector/types/pojo/TestUuidVector.java:
##########
@@ -0,0 +1,152 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.types.pojo;
+
+import java.nio.ByteBuffer;
+import java.util.UUID;
+import org.apache.arrow.memory.BufferAllocator;
+import org.apache.arrow.memory.util.hash.ArrowBufHasher;
+import org.apache.arrow.vector.ExtensionTypeVector;
+import org.apache.arrow.vector.FieldVector;
+import org.apache.arrow.vector.FixedSizeBinaryVector;
+import org.apache.arrow.vector.ValueIterableVector;
+import org.apache.arrow.vector.ValueVector;
+import org.apache.arrow.vector.types.pojo.ArrowType.ExtensionType;
+import org.apache.arrow.vector.util.TransferPair;
+
+public class TestUuidVector {

Review Comment:
   In fact why is this class here at all? All it does is wrap the two inner classes



##########
vector/src/main/codegen/templates/BaseWriter.java:
##########
@@ -101,6 +103,13 @@ public interface MapWriter extends ListWriter {
     MapWriter value();
   }
 
+  public interface ExtensionWriter extends BaseWriter {
+    void writeNull();
+    <T extends ExtensionHolder> void write(T var1);
+    void writeExtensionType(Object var1);
+    <T extends ExtensionTypeWriterFactory> void addExtensionTypeFactory(T var1);

Review Comment:
   Can we replace `var1` with meaningful parameter names?



##########
vector/src/main/java/org/apache/arrow/vector/complex/impl/ExtensionTypeWriterFactory.java:
##########
@@ -0,0 +1,23 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.arrow.vector.complex.impl;
+
+import org.apache.arrow.vector.ExtensionTypeVector;
+
+public interface ExtensionTypeWriterFactory<T extends AbstractFieldWriter> {

Review Comment:
   IMO abstract implementation classes should not go in generic bounds - it should be the interface type



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org