You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Matthew Ouyang <ma...@gmail.com> on 2021/10/15 20:28:07 UTC
Apply string trim to all fields including structs and arrays
I have a Row that has a struct that has string fields and an array of
structs which also has string fields. Is there any simple way to apply a
whitespace trim to all string fields (like a ** wildcard)? I know
FieldAccessDescriptor.withAllFields exists but will that work for anything
beyond the top level? I'm hoping that I don't have to traverse the Row in
code.
Re: Apply string trim to all fields including structs and arrays
Posted by Matthew Ouyang <ma...@gmail.com>.
By chance, I found a package-private class RowUtils.RowCases. I'm going to
give this a try.
https://github.com/apache/beam/blob/7418c84de2b68b06e0318766ca6c69f4d2144390/sdks/java/core/src/main/java/org/apache/beam/sdk/values/RowUtils.java#L80
On Fri, Oct 15, 2021 at 5:09 PM Andrew Pilloud <ap...@google.com> wrote:
> I don't think FieldAccessDescriptor will help you here. You don't want to
> access fields, you want to apply a method to them. I took a quick look and
> didn't find anything. If we had something like this it would probably live
> in one of these two folders, you might be able to find some examples to
> start from:
>
> https://github.com/apache/beam/tree/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms
>
> https://github.com/apache/beam/tree/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils
>
>
>
>
> On Fri, Oct 15, 2021 at 1:28 PM Matthew Ouyang <ma...@gmail.com>
> wrote:
>
>> I have a Row that has a struct that has string fields and an array of
>> structs which also has string fields. Is there any simple way to apply a
>> whitespace trim to all string fields (like a ** wildcard)? I know
>> FieldAccessDescriptor.withAllFields exists but will that work for anything
>> beyond the top level? I'm hoping that I don't have to traverse the Row in
>> code.
>>
>
Re: Apply string trim to all fields including structs and arrays
Posted by Brian Hulette <bh...@google.com>.
Yeah I don't think anything like this exists already, it's an interesting
idea for a transform to add to core though.
On Fri, Oct 15, 2021 at 2:09 PM Andrew Pilloud <ap...@google.com> wrote:
> I don't think FieldAccessDescriptor will help you here. You don't want to
> access fields, you want to apply a method to them. I took a quick look and
> didn't find anything. If we had something like this it would probably live
> in one of these two folders, you might be able to find some examples to
> start from:
>
> https://github.com/apache/beam/tree/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms
>
> https://github.com/apache/beam/tree/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils
>
>
>
>
> On Fri, Oct 15, 2021 at 1:28 PM Matthew Ouyang <ma...@gmail.com>
> wrote:
>
>> I have a Row that has a struct that has string fields and an array of
>> structs which also has string fields. Is there any simple way to apply a
>> whitespace trim to all string fields (like a ** wildcard)? I know
>> FieldAccessDescriptor.withAllFields exists but will that work for anything
>> beyond the top level? I'm hoping that I don't have to traverse the Row in
>> code.
>>
>
Re: Apply string trim to all fields including structs and arrays
Posted by Andrew Pilloud <ap...@google.com>.
I don't think FieldAccessDescriptor will help you here. You don't want to
access fields, you want to apply a method to them. I took a quick look and
didn't find anything. If we had something like this it would probably live
in one of these two folders, you might be able to find some examples to
start from:
https://github.com/apache/beam/tree/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms
https://github.com/apache/beam/tree/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/utils
On Fri, Oct 15, 2021 at 1:28 PM Matthew Ouyang <ma...@gmail.com>
wrote:
> I have a Row that has a struct that has string fields and an array of
> structs which also has string fields. Is there any simple way to apply a
> whitespace trim to all string fields (like a ** wildcard)? I know
> FieldAccessDescriptor.withAllFields exists but will that work for anything
> beyond the top level? I'm hoping that I don't have to traverse the Row in
> code.
>