You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "tilgalas (via GitHub)" <gi...@apache.org> on 2024/08/05 15:18:30 UTC

[PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

tilgalas opened a new pull request, #32081:
URL: https://github.com/apache/beam/pull/32081

   This is a 2nd PR in the series of PRs (based on the now closed #31648) whose ultimate goal is to add support for generic classes to schema providers. 
   
   `FieldValueTypeInformation` creators will now accept a `TypeDescriptor` parameter describing the field's containing class, that will let them infer more accurate type information about that field. For example - consider a `MyClass<T>` class - now with a `TypeDescriptor` of `MyClass<String>` and the getter of type `T` the type resolver can infer the type of field to be `String` , see the added test class that shows the new functionality
   
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2438736706

   Fixes #32081


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2271079740

   Run Java_GCP_IO_Direct PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2269540582

   Run Java PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1819075470


##########
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:
##########
@@ -486,10 +518,17 @@ public static Schema toBeamSchema(org.apache.avro.Schema schema) {
     return builder.build();
   }
 
+  @EnsuresNonNullIf(
+      expression = {"#1"},
+      result = false)
+  private static boolean isNullOrEmpty(@Nullable String str) {
+    return str == null || str.isEmpty();
+  }

Review Comment:
   in other words, checkerframework doesn't understand that by using this method (and getting a false returned from it) we've just asserted, that the variable is `@NonNull` - my "polyfill" method uses the checkerframework's annotation:
   ```
     @EnsuresNonNullIf(
         expression = {"#1"},
         result = false)
   ``` 
   which tells it that if it returns false the first argument is verified to be `@NonNull` - I don't know why checkerframwork doesn't understand guava method's postconditions here (maybe it's fixed in some newer version?)    



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1817572175


##########
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:
##########
@@ -486,10 +518,17 @@ public static Schema toBeamSchema(org.apache.avro.Schema schema) {
     return builder.build();
   }
 
+  @EnsuresNonNullIf(
+      expression = {"#1"},
+      result = false)
+  private static boolean isNullOrEmpty(@Nullable String str) {
+    return str == null || str.isEmpty();
+  }

Review Comment:
   I don't understand what this example has to do with isNullOrEmpty (which returns a boolean)?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1816516273


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/values/RowWithGetters.java:
##########
@@ -42,45 +42,43 @@
  * the appropriate fields from the POJO.
  */
 @SuppressWarnings("rawtypes")
-public class RowWithGetters extends Row {
-  private final Object getterTarget;
-  private final List<FieldValueGetter> getters;
+public class RowWithGetters<T extends @NonNull Object> extends Row {
+  private final T getterTarget;
+  private final List<FieldValueGetter<T, ?>> getters;
   private @Nullable Map<Integer, @Nullable Object> cache = null;
 
   RowWithGetters(
-      Schema schema, Factory<List<FieldValueGetter>> getterFactory, Object getterTarget) {
+      Schema schema, Factory<List<FieldValueGetter<T, ?>>> getterFactory, T getterTarget) {
     super(schema);
     this.getterTarget = getterTarget;
     this.getters = getterFactory.create(TypeDescriptor.of(getterTarget.getClass()), schema);
   }
 
   @Override
   @SuppressWarnings({"TypeParameterUnusedInFormals", "unchecked"})
-  public <T> @Nullable T getValue(int fieldIdx) {
+  public @Nullable Object getValue(int fieldIdx) {

Review Comment:
   I guess it's probably a matter of taste - my view is that this class can't possibly know what type of value a particular getter is going to return, so it's not commiting to anything by making the method generic - "I only know it's an Object, if you need a specific type, you'll have to cast it yourself" - I should probably also change the type of the getters list to `List<FieldValueGetter<T, Object>>` but then it would probably open another can of worms of covariant types, that is whether a, say, `FieldValueGetter<T, String>` is a `FieldValueGetter<T, Object>` - the current type of `List<FieldValueGetter<T, ?>>` is slightly incorrect as the `?` wildcard means an unknown type, but which is at least shared by all members of the list (which is not true)
   
   I can understand the approach where the caller requests a specific type via a generic type parameter and is happy to accept the method throwing a ClassCastException if they had wrong assumptions about the getter - let me know if you want me to make it back a generic method



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1816664202


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/values/Row.java:
##########
@@ -836,10 +838,10 @@ public int nextFieldId() {
     }
 
     @Internal
-    public Row withFieldValueGetters(
-        Factory<List<FieldValueGetter>> fieldValueGetterFactory, Object getterTarget) {
+    public <T extends @NonNull Object> Row withFieldValueGetters(

Review Comment:
   I could change it to `<@NonNull T>` I suppose - it compiles, but then my IDE complains that "Non-null type argument is expected" on the `T` parametor of the `FieldValueGetter<T, ?>` type 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1816794102


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/CachingFactory.java:
##########
@@ -39,9 +41,9 @@
 public class CachingFactory<CreatedT> implements Factory<CreatedT> {
   private transient @Nullable ConcurrentHashMap<TypeDescriptor<?>, CreatedT> cache = null;
 
-  private final Factory<CreatedT> innerFactory;
+  private final @NotOnlyInitialized Factory<CreatedT> innerFactory;

Review Comment:
   so in `RowValueGettersFactory` there's this code:
   ```
       RowValueGettersFactory(Factory<List<FieldValueGetter<T, Object>>> gettersFactory) {
         this.gettersFactory = gettersFactory;
         this.cachingGettersFactory = new CachingFactory<>(this);
       }
   ```
   since we're in the constructor, the `this` parameter is still not fully initialized, so by using this annotation we can temporarily (for the duration of the initalization of the CachingFactory object) allow assigning the `innerFactory` field a value which itself is under initialization (or indeed whose initialization status is unknown, as the signature of the CachingFactory constructor tells us)
   
   see also https://checkerframework.org/manual/#circular-initialization



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2271191563

   Run Java PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-3480797159

   > This whole thread of work is interesting. FWIW the whole point of `TypeDescriptor` is to carry the generic information. Otherwise we could just use Java's reflective capabilities. Is there a TL;DR document about the limitation that you are improving?
   
   Ah, I see there is a doc linked from https://github.com/apache/beam/pull/31785


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1816518505


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/values/RowWithGetters.java:
##########
@@ -42,45 +42,43 @@
  * the appropriate fields from the POJO.
  */
 @SuppressWarnings("rawtypes")
-public class RowWithGetters extends Row {
-  private final Object getterTarget;
-  private final List<FieldValueGetter> getters;
+public class RowWithGetters<T extends @NonNull Object> extends Row {
+  private final T getterTarget;
+  private final List<FieldValueGetter<T, ?>> getters;
   private @Nullable Map<Integer, @Nullable Object> cache = null;
 
   RowWithGetters(
-      Schema schema, Factory<List<FieldValueGetter>> getterFactory, Object getterTarget) {
+      Schema schema, Factory<List<FieldValueGetter<T, ?>>> getterFactory, T getterTarget) {
     super(schema);
     this.getterTarget = getterTarget;
     this.getters = getterFactory.create(TypeDescriptor.of(getterTarget.getClass()), schema);
   }
 
   @Override
   @SuppressWarnings({"TypeParameterUnusedInFormals", "unchecked"})
-  public <T> @Nullable T getValue(int fieldIdx) {
+  public @Nullable Object getValue(int fieldIdx) {

Review Comment:
   I feel like I will change the type of the list elements to `FieldValueGetter<T, Object>` anyway, as I find it more correct



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1816664202


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/values/Row.java:
##########
@@ -836,10 +838,10 @@ public int nextFieldId() {
     }
 
     @Internal
-    public Row withFieldValueGetters(
-        Factory<List<FieldValueGetter>> fieldValueGetterFactory, Object getterTarget) {
+    public <T extends @NonNull Object> Row withFieldValueGetters(

Review Comment:
   I could change it to `<@NonNull T>` I suppose - it compiles, but then my IDE complains that "Non-null type argument is expected" on the `T` parameter of the `FieldValueGetter<T, ?>` type 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1819066036


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/FieldValueGetter.java:
##########
@@ -29,7 +30,7 @@
  * <p>Implementations of this interface are generated at runtime to map object fields to Row fields.
  */
 @Internal
-public interface FieldValueGetter<ObjectT, ValueT> extends Serializable {
+public interface FieldValueGetter<ObjectT extends @NonNull Object, ValueT> extends Serializable {

Review Comment:
   yeah, at least not when we're talking type parameters, for example:
   ```
     public interface I<T> {
         T getT();
     }
   
     public static <T> void f(I<T> i) {
       System.out.println(i.getT().getClass());
     }
     ```
     gives me 
     ```
   error: [dereference.of.nullable] dereference of possibly-null reference i.getT()
       System.out.println(i.getT().getClass());
                                ^
     ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2447047378

   Run Java PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2447826675

   lgtm


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1811391760


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/FieldValueTypeInformation.java:
##########
@@ -126,17 +123,26 @@ public static FieldValueTypeInformation forOneOf(
   }
 
   public static FieldValueTypeInformation forField(Field field, int index) {
-    TypeDescriptor<?> type = TypeDescriptor.of(field.getGenericType());
+    return forField(null, field, index);
+  }

Review Comment:
   This class wasn't really meant to be public. I would add an @Internal tag to the class, and not bother keeping this backwards-compatible method



##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/GetterBasedSchemaProvider.java:
##########
@@ -112,22 +111,28 @@ public SchemaUserTypeCreator schemaTypeCreator(
     return schemaTypeCreator(targetTypeDescriptor.getRawType(), schema);
   }
 
-  private class ToRowWithValueGetters<T> implements SerializableFunction<T, Row> {
+  private class ToRowWithValueGetters<T extends @NonNull Object>
+      implements SerializableFunction<T, Row> {
     private final Schema schema;
-    private final Factory<List<FieldValueGetter>> getterFactory;
+    private final Factory<List<FieldValueGetter<?, ?>>> getterFactory;
 
     public ToRowWithValueGetters(Schema schema) {
       this.schema = schema;
       // Since we know that this factory is always called from inside the lambda with the same
       // schema, return a caching factory that caches the first value seen for each class. This
       // prevents having to lookup the getter list each time createGetters is called.
       this.getterFactory =
-          RowValueGettersFactory.of(GetterBasedSchemaProvider.this::fieldValueGetters);
+          RowValueGettersFactory.of(
+              (Factory<List<FieldValueGetter<?, ?>>>)
+                  (typeDescriptor, schema1) ->
+                      (List)
+                          GetterBasedSchemaProvider.this.fieldValueGetters(
+                              typeDescriptor, schema1));
     }
 
     @Override
     public Row apply(T input) {
-      return Row.withSchema(schema).withFieldValueGetters(getterFactory, input);
+      return Row.withSchema(schema).withFieldValueGetters((Factory) getterFactory, input);

Review Comment:
   why do you need this cast?



##########
sdks/java/core/src/main/java/org/apache/beam/sdk/values/RowWithGetters.java:
##########
@@ -42,45 +42,43 @@
  * the appropriate fields from the POJO.
  */
 @SuppressWarnings("rawtypes")
-public class RowWithGetters extends Row {
-  private final Object getterTarget;
-  private final List<FieldValueGetter> getters;
+public class RowWithGetters<T extends @NonNull Object> extends Row {
+  private final T getterTarget;
+  private final List<FieldValueGetter<T, ?>> getters;
   private @Nullable Map<Integer, @Nullable Object> cache = null;
 
   RowWithGetters(
-      Schema schema, Factory<List<FieldValueGetter>> getterFactory, Object getterTarget) {
+      Schema schema, Factory<List<FieldValueGetter<T, ?>>> getterFactory, T getterTarget) {
     super(schema);
     this.getterTarget = getterTarget;
     this.getters = getterFactory.create(TypeDescriptor.of(getterTarget.getClass()), schema);
   }
 
   @Override
   @SuppressWarnings({"TypeParameterUnusedInFormals", "unchecked"})
-  public <T> @Nullable T getValue(int fieldIdx) {
+  public @Nullable Object getValue(int fieldIdx) {

Review Comment:
   why remove type parameter?



##########
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:
##########
@@ -486,10 +518,17 @@ public static Schema toBeamSchema(org.apache.avro.Schema schema) {
     return builder.build();
   }
 
+  @EnsuresNonNullIf(
+      expression = {"#1"},
+      result = false)
+  private static boolean isNullOrEmpty(@Nullable String str) {
+    return str == null || str.isEmpty();
+  }

Review Comment:
   what is wrong with Strings.isNullOrEmpty?



##########
sdks/java/core/src/main/java/org/apache/beam/sdk/values/Row.java:
##########
@@ -836,10 +838,10 @@ public int nextFieldId() {
     }
 
     @Internal
-    public Row withFieldValueGetters(
-        Factory<List<FieldValueGetter>> fieldValueGetterFactory, Object getterTarget) {
+    public <T extends @NonNull Object> Row withFieldValueGetters(

Review Comment:
   unclear on this bound as well



##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/CachingFactory.java:
##########
@@ -39,9 +41,9 @@
 public class CachingFactory<CreatedT> implements Factory<CreatedT> {
   private transient @Nullable ConcurrentHashMap<TypeDescriptor<?>, CreatedT> cache = null;
 
-  private final Factory<CreatedT> innerFactory;
+  private final @NotOnlyInitialized Factory<CreatedT> innerFactory;

Review Comment:
   what are these new tags for?



##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/FieldValueGetter.java:
##########
@@ -29,7 +30,7 @@
  * <p>Implementations of this interface are generated at runtime to map object fields to Row fields.
  */
 @Internal
-public interface FieldValueGetter<ObjectT, ValueT> extends Serializable {
+public interface FieldValueGetter<ObjectT extends @NonNull Object, ValueT> extends Serializable {

Review Comment:
   why do you have ObjectT extends Object?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1816634987


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/FieldValueGetter.java:
##########
@@ -29,7 +30,7 @@
  * <p>Implementations of this interface are generated at runtime to map object fields to Row fields.
  */
 @Internal
-public interface FieldValueGetter<ObjectT, ValueT> extends Serializable {
+public interface FieldValueGetter<ObjectT extends @NonNull Object, ValueT> extends Serializable {

Review Comment:
   so apparently there's a difference (I don't understand why it would be so, though) between a `@NonNull T` and `T extends @NonNull Object` - here's an example, consider:
   ```
     public interface I<T extends @NonNull Object> {
         T getT();
     }
   ```
   with a signature as above, we can only write:
   ```
     public static <T extends @NonNull Object> void f(I<T> i) {
       System.out.println(i.getT().getClass());
     }
   ```
   or alternatively:
   ```
     public static <T> void f(I<@NonNull T> i) {
       System.out.println(i.getT().getClass());
     }
   ``` 
   The checkerframework will make sure that we propagate the non-nullness of the parameter. 
   This won't work for example:
   ```
     public static <@Nullable T> void f(I<T> i) {
       T t = i.getT();
       if (t != null) {
         System.out.println(t.getClass());
       }
     }
   ```
   nor will this:
   ```
     public static <T> void f(I<@Nullable T> i) {
       T t = i.getT();
       if (t != null) {
         System.out.println(t.getClass());
       }
     }
   ```
   but it will work if I change the signature of `I` to:
   ```
     public interface I<@NonNull T> {
         T getT();
     }
   ```
   in other words, the `@NonNull T` annotation on the interface is not really enforced anywhere, it's more like a suggestion, and so it's not really expressing the concept of the interface only works with `@NonNull` types 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2310062282

   Reminder, please take a look at this pr: @m-trieu @damondouglas 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1819437330


##########
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:
##########
@@ -486,10 +518,17 @@ public static Schema toBeamSchema(org.apache.avro.Schema schema) {
     return builder.build();
   }
 
+  @EnsuresNonNullIf(
+      expression = {"#1"},
+      result = false)
+  private static boolean isNullOrEmpty(@Nullable String str) {
+    return str == null || str.isEmpty();
+  }

Review Comment:
   understood



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2447050777

   Run Java_GCP_IO_Direct PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1841033733


##########
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:
##########
@@ -451,8 +484,7 @@ public static Field toBeamField(org.apache.avro.Schema.Field field) {
   public static org.apache.avro.Schema.Field toAvroField(Field field, String namespace) {
     org.apache.avro.Schema fieldSchema =
         getFieldSchema(field.getType(), field.getName(), namespace);
-    return new org.apache.avro.Schema.Field(
-        field.getName(), fieldSchema, field.getDescription(), (Object) null);
+    return new org.apache.avro.Schema.Field(field.getName(), fieldSchema, field.getDescription());

Review Comment:
   This seems to break https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_Avro_Versions.yml
   
   I think it is because in some older versions of avro, the constructor is different - https://avro.apache.org/docs/1.8.2/api/java/org/apache/avro/Schema.Field.html



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2416622452

   since there's a #32757, I'm closing this PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax merged PR #32081:
URL: https://github.com/apache/beam/pull/32081


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1816634987


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/FieldValueGetter.java:
##########
@@ -29,7 +30,7 @@
  * <p>Implementations of this interface are generated at runtime to map object fields to Row fields.
  */
 @Internal
-public interface FieldValueGetter<ObjectT, ValueT> extends Serializable {
+public interface FieldValueGetter<ObjectT extends @NonNull Object, ValueT> extends Serializable {

Review Comment:
   so apparently there's a difference (I don't understand why it would be so, though) between a `@NonNull T` and `T extends @NonNull Object` - here's an example, consider:
   ```
     public interface I<T extends @NonNull Object> {
         T getT();
     }
   ```
   with a signature as above, we can only write:
   ```
     public static <T extends @NonNull Object> void f(I<T> i) {
       System.out.println(i.getT().getClass());
     }
   ```
   or alternatively:
   ```
     public static <T> void f(I<@NonNull T> i) {
       System.out.println(i.getT().getClass());
     }
   ``` 
   The checkerframework will make sure that we propagate the non-nullness of the parameter. 
   This won't work for example:
   ```
     public static <@Nullable T> void f(I<T> i) {
       T t = i.getT();
       if (t != null) {
         System.out.println(t.getClass());
       }
     }
   ```
   nor won't this:
   ```
     public static <T> void f(I<@Nullable T> i) {
       T t = i.getT();
       if (t != null) {
         System.out.println(t.getClass());
       }
     }
   ```
   but it will work if I change the signature of `I` to:
   ```
     public interface I<@NonNull T> {
         T getT();
     }
   ```
   in other words, the `@NonNull T` annotation on the interface is not really enforced anywhere, it's more like a suggestion, and so it's not really expressing the concept of the interface only works with `@NonNull` types 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1819436260


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/values/Row.java:
##########
@@ -836,10 +838,10 @@ public int nextFieldId() {
     }
 
     @Internal
-    public Row withFieldValueGetters(
-        Factory<List<FieldValueGetter>> fieldValueGetterFactory, Object getterTarget) {
+    public <T extends @NonNull Object> Row withFieldValueGetters(

Review Comment:
   I think I prefer @NonNull T (is this simply a weakness in the IDE?), but I guess I don't have strong feelings here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2442159654

   A bunch of tests are failing. PTAL to see if this is due to your PR or just flaky tests.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2444688645

   Run Java PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2296256012

   > @tilgalas Thank you for working on this! Would you mind to first remove the `"nullness"` value in the `@SuppressWarnings` annotation for the involved classes in this PR? Please consider this suggestion as non-blocking but I would like to see if this is possible with your helpful changes. As a second phase, may we also consider removing the `"rawtypes"`? Again, non-blocking but it's something we would like to strive for to improve the code quality.
   
   sure, with pleasure!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2479607653

   This likely breaks DataflowTemplate unit test: https://github.com/GoogleCloudPlatform/DataflowTemplates/pull/2014/checks?check_run_id=33057418339
   
   I'm trying to understand the scope of breaking change


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2348950115

   added a refactoring commit on top of the main one, which deals with most of the nullness and rawtypes warnings in the classes involved - I'm happy to move it to a PR of its own if the reviewers find the resulting changes too large to safely review


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2288296936

   @damondouglas could you please take a look at this one?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1816690001


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/GetterBasedSchemaProvider.java:
##########
@@ -112,22 +111,28 @@ public SchemaUserTypeCreator schemaTypeCreator(
     return schemaTypeCreator(targetTypeDescriptor.getRawType(), schema);
   }
 
-  private class ToRowWithValueGetters<T> implements SerializableFunction<T, Row> {
+  private class ToRowWithValueGetters<T extends @NonNull Object>
+      implements SerializableFunction<T, Row> {
     private final Schema schema;
-    private final Factory<List<FieldValueGetter>> getterFactory;
+    private final Factory<List<FieldValueGetter<?, ?>>> getterFactory;
 
     public ToRowWithValueGetters(Schema schema) {
       this.schema = schema;
       // Since we know that this factory is always called from inside the lambda with the same
       // schema, return a caching factory that caches the first value seen for each class. This
       // prevents having to lookup the getter list each time createGetters is called.
       this.getterFactory =
-          RowValueGettersFactory.of(GetterBasedSchemaProvider.this::fieldValueGetters);
+          RowValueGettersFactory.of(
+              (Factory<List<FieldValueGetter<?, ?>>>)
+                  (typeDescriptor, schema1) ->
+                      (List)
+                          GetterBasedSchemaProvider.this.fieldValueGetters(
+                              typeDescriptor, schema1));
     }
 
     @Override
     public Row apply(T input) {
-      return Row.withSchema(schema).withFieldValueGetters(getterFactory, input);
+      return Row.withSchema(schema).withFieldValueGetters((Factory) getterFactory, input);

Review Comment:
   yeah, good question - I'm playing with that code right now and I think I could get rid of it actually



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1817565801


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/FieldValueGetter.java:
##########
@@ -29,7 +30,7 @@
  * <p>Implementations of this interface are generated at runtime to map object fields to Row fields.
  */
 @Internal
-public interface FieldValueGetter<ObjectT, ValueT> extends Serializable {
+public interface FieldValueGetter<ObjectT extends @NonNull Object, ValueT> extends Serializable {

Review Comment:
   Generally in Beam I've mostly seen us using @Nullable, not @NonNull. I believe that NonNull is assumed unless @Nullable is specified. Are you not finding that to be the case?



##########
sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/GetterBasedSchemaProvider.java:
##########
@@ -112,22 +111,28 @@ public SchemaUserTypeCreator schemaTypeCreator(
     return schemaTypeCreator(targetTypeDescriptor.getRawType(), schema);
   }
 
-  private class ToRowWithValueGetters<T> implements SerializableFunction<T, Row> {
+  private class ToRowWithValueGetters<T extends @NonNull Object>
+      implements SerializableFunction<T, Row> {
     private final Schema schema;
-    private final Factory<List<FieldValueGetter>> getterFactory;
+    private final Factory<List<FieldValueGetter<?, ?>>> getterFactory;
 
     public ToRowWithValueGetters(Schema schema) {
       this.schema = schema;
       // Since we know that this factory is always called from inside the lambda with the same
       // schema, return a caching factory that caches the first value seen for each class. This
       // prevents having to lookup the getter list each time createGetters is called.
       this.getterFactory =
-          RowValueGettersFactory.of(GetterBasedSchemaProvider.this::fieldValueGetters);
+          RowValueGettersFactory.of(
+              (Factory<List<FieldValueGetter<?, ?>>>)
+                  (typeDescriptor, schema1) ->
+                      (List)
+                          GetterBasedSchemaProvider.this.fieldValueGetters(
+                              typeDescriptor, schema1));
     }
 
     @Override
     public Row apply(T input) {
-      return Row.withSchema(schema).withFieldValueGetters(getterFactory, input);
+      return Row.withSchema(schema).withFieldValueGetters((Factory) getterFactory, input);

Review Comment:
   ok



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1816501746


##########
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:
##########
@@ -486,10 +518,17 @@ public static Schema toBeamSchema(org.apache.avro.Schema schema) {
     return builder.build();
   }
 
+  @EnsuresNonNullIf(
+      expression = {"#1"},
+      result = false)
+  private static boolean isNullOrEmpty(@Nullable String str) {
+    return str == null || str.isEmpty();
+  }

Review Comment:
   for some reason the checkerframework doesn't understand the nullness assertion when using it:
   ```
   /usr/local/google/home/mszwaja/src/beam/sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:538: error: [argument] incompatible argument for parameter namespace of AvroUtils.toAvroField.
         org.apache.avro.Schema.Field recordField = toAvroField(field, childNamespace);
                                                                       ^
     found   : @Initialized @Nullable String
     required: @Initialized @NonNull String
   /usr/local/google/home/mszwaja/src/beam/sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:541: error: [argument] incompatible argument for parameter name of Schema.createRecord.
       return org.apache.avro.Schema.createRecord(schemaName, "", schemaNamespace, false, fields);
                                                  ^
     found   : @Initialized @Nullable String
     required: @Initialized @NonNull String
     ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-3480808033

   (also worth noting that `TypeDescriptor` is forked from Guava's TypeToken circa 2014 or 2015 in case that wasn't known or obvious)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2271440459

   Run Java_Hadoop_IO_Direct PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2315160919

   Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`:
   
   R: @Abacn for label java.
   R: @Abacn for label io.
   
   Available commands:
   - `stop reviewer notifications` - opt out of the automated review tooling
   - `remind me after tests pass` - tag the comment author after tests pass
   - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2269477770

   Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2365165638

   Reminder, please take a look at this pr: @Abacn @Abacn 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2373916559

   Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment `assign to next reviewer`:
   
   R: @damondouglas for label java.
   R: @damondouglas for label io.
   
   Available commands:
   - `stop reviewer notifications` - opt out of the automated review tooling
   - `remind me after tests pass` - tag the comment author after tests pass
   - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1819059036


##########
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:
##########
@@ -486,10 +518,17 @@ public static Schema toBeamSchema(org.apache.avro.Schema schema) {
     return builder.build();
   }
 
+  @EnsuresNonNullIf(
+      expression = {"#1"},
+      result = false)
+  private static boolean isNullOrEmpty(@Nullable String str) {
+    return str == null || str.isEmpty();
+  }

Review Comment:
   this is the error that appears when I use Strings.isNullOrEmpty



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2429475617

   reopening after discussing offline


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1841160132


##########
sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:
##########
@@ -451,8 +484,7 @@ public static Field toBeamField(org.apache.avro.Schema.Field field) {
   public static org.apache.avro.Schema.Field toAvroField(Field field, String namespace) {
     org.apache.avro.Schema fieldSchema =
         getFieldSchema(field.getType(), field.getName(), namespace);
-    return new org.apache.avro.Schema.Field(
-        field.getName(), fieldSchema, field.getDescription(), (Object) null);
+    return new org.apache.avro.Schema.Field(field.getName(), fieldSchema, field.getDescription());

Review Comment:
   I think https://github.com/apache/beam/pull/33102 should fix this (I cherry-picked it to the current release branch as well)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2349280449

   Run Java PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2410794776

   hi reviewers, I'm going to split this PR and put the nullness and rawness refactoring commit into its own PR


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2332067911

   waiting on author


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2331365814

   Reminder, please take a look at this pr: @Abacn @Abacn 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas closed pull request #32081: make FieldValueTypeInformation creators take a TypeDescriptor parameter
URL: https://github.com/apache/beam/pull/32081


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "reuvenlax (via GitHub)" <gi...@apache.org>.
reuvenlax commented on code in PR #32081:
URL: https://github.com/apache/beam/pull/32081#discussion_r1817567455


##########
sdks/java/core/src/main/java/org/apache/beam/sdk/values/RowWithGetters.java:
##########
@@ -42,45 +42,43 @@
  * the appropriate fields from the POJO.
  */
 @SuppressWarnings("rawtypes")
-public class RowWithGetters extends Row {
-  private final Object getterTarget;
-  private final List<FieldValueGetter> getters;
+public class RowWithGetters<T extends @NonNull Object> extends Row {
+  private final T getterTarget;
+  private final List<FieldValueGetter<T, ?>> getters;
   private @Nullable Map<Integer, @Nullable Object> cache = null;
 
   RowWithGetters(
-      Schema schema, Factory<List<FieldValueGetter>> getterFactory, Object getterTarget) {
+      Schema schema, Factory<List<FieldValueGetter<T, ?>>> getterFactory, T getterTarget) {
     super(schema);
     this.getterTarget = getterTarget;
     this.getters = getterFactory.create(TypeDescriptor.of(getterTarget.getClass()), schema);
   }
 
   @Override
   @SuppressWarnings({"TypeParameterUnusedInFormals", "unchecked"})
-  public <T> @Nullable T getValue(int fieldIdx) {
+  public @Nullable Object getValue(int fieldIdx) {

Review Comment:
   The base class uses a parameter here, so I think for now we should just match the base class.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2271663495

   Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
   
   R: @m-trieu for label java.
   R: @damondouglas for label io.
   
   Available commands:
   - `stop reviewer notifications` - opt out of the automated review tooling
   - `remind me after tests pass` - tag the comment author after tests pass
   - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
   
   The PR bot will only process comments in the main thread (not review comments).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "tilgalas (via GitHub)" <gi...@apache.org>.
tilgalas commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-2271661226

   assign set of reviewers


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] make FieldValueTypeInformation creators take a TypeDescriptor parameter [beam]

Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on PR #32081:
URL: https://github.com/apache/beam/pull/32081#issuecomment-3480788985

   This whole thread of work is interesting. FWIW the whole point of `TypeDescriptor` is to carry the generic information. Otherwise we could just use Java's reflective capabilities. Is there a TL;DR document about the limitation that you are improving?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org