Using record path API to remove field values

[Mike Thomsen <mi...@gmail.com>]

Is there an example of how to use the record path API to remove an element
from a record? Ideally, if the field is nullable, I'd like to be able to
completely remove the value rather than send a null value (to Elastic in
this case).

Thanks,

Mike

Re: Using record path API to remove field values

[Bryan Bende <bb...@gmail.com>]

I'm not sure that removing a field from record path makes sense since
technically removing the field all together is a different schema.

You could easily make two versions of the schema, one with all fields,
and a second with a subset, then just use any record processor with a
reader using schema 1 and a writer using schema 2.
On Tue, Oct 30, 2018 at 9:28 AM Mike Thomsen <mi...@gmail.com> wrote:
>
> Something more aggressive than that. I have a new ES bulk operation
> processor that uses the record API to specify operations. I want to be able
> to remove the record paths that relate to things like the index and type.
>
> In StandardFieldValue it also looks like we ignore nulls altogether:
>
>     @Override
>     public void updateValue(final Object newValue) {
>         final Optional<Record> parentRecord = getParentRecord();
>         if (!parentRecord.isPresent()) {
>             if (value instanceof Record) {
>                 ((Record) value).setValue(getField().getFieldName(),
> newValue);
>                 return;
>             } else if (value == null) {
>                 return; // value is null, nothing to update
>             } else {
>                 throw new UnsupportedOperationException("Cannot update the
> field value because the value is not associated with any record");
>             }
>         }
>
>         parentRecord.get().setValue(getField().getFieldName(), newValue);
>     }
>
> Not sure what the rationale there is for not checking the value's field to
> see if isNullable() is true or not and then acting from what. If no one
> knows any gotchas, I can expand that out to allow null on nullable fields.
> Then I could implement null suppression in a similar way as the json
> writer, I suppose.
>
> Thanks,
>
> Mike
>
> On Tue, Oct 30, 2018 at 8:51 AM Pierre Villard <pi...@gmail.com>
> wrote:
>
> > Hey Mike,
> >
> > In the JSON Record Set Writer you have a "Suppress Null Values" parameter
> > to do just that.
> > Are you thinking about something else?
> >
> > Pierre
> >
> > Le mar. 30 oct. 2018 à 13:44, Mike Thomsen <mi...@gmail.com> a
> > écrit :
> >
> > > Is there an example of how to use the record path API to remove an
> > element
> > > from a record? Ideally, if the field is nullable, I'd like to be able to
> > > completely remove the value rather than send a null value (to Elastic in
> > > this case).
> > >
> > > Thanks,
> > >
> > > Mike
> > >
> >

Re: Using record path API to remove field values

[Mike Thomsen <mi...@gmail.com>]

Something more aggressive than that. I have a new ES bulk operation
processor that uses the record API to specify operations. I want to be able
to remove the record paths that relate to things like the index and type.

In StandardFieldValue it also looks like we ignore nulls altogether:

    @Override
    public void updateValue(final Object newValue) {
        final Optional<Record> parentRecord = getParentRecord();
        if (!parentRecord.isPresent()) {
            if (value instanceof Record) {
                ((Record) value).setValue(getField().getFieldName(),
newValue);
                return;
            } else if (value == null) {
                return; // value is null, nothing to update
            } else {
                throw new UnsupportedOperationException("Cannot update the
field value because the value is not associated with any record");
            }
        }

        parentRecord.get().setValue(getField().getFieldName(), newValue);
    }

Not sure what the rationale there is for not checking the value's field to
see if isNullable() is true or not and then acting from what. If no one
knows any gotchas, I can expand that out to allow null on nullable fields.
Then I could implement null suppression in a similar way as the json
writer, I suppose.

Thanks,

Mike

On Tue, Oct 30, 2018 at 8:51 AM Pierre Villard <pi...@gmail.com>
wrote:

> Hey Mike,
>
> In the JSON Record Set Writer you have a "Suppress Null Values" parameter
> to do just that.
> Are you thinking about something else?
>
> Pierre
>
> Le mar. 30 oct. 2018 à 13:44, Mike Thomsen <mi...@gmail.com> a
> écrit :
>
> > Is there an example of how to use the record path API to remove an
> element
> > from a record? Ideally, if the field is nullable, I'd like to be able to
> > completely remove the value rather than send a null value (to Elastic in
> > this case).
> >
> > Thanks,
> >
> > Mike
> >
>

Re: Using record path API to remove field values

[Pierre Villard <pi...@gmail.com>]

Hey Mike,

In the JSON Record Set Writer you have a "Suppress Null Values" parameter
to do just that.
Are you thinking about something else?

Pierre

Le mar. 30 oct. 2018 à 13:44, Mike Thomsen <mi...@gmail.com> a
écrit :

> Is there an example of how to use the record path API to remove an element
> from a record? Ideally, if the field is nullable, I'd like to be able to
> completely remove the value rather than send a null value (to Elastic in
> this case).
>
> Thanks,
>
> Mike
>