You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@nifi.apache.org by Mike Thomsen <mi...@gmail.com> on 2017/11/01 19:27:29 UTC

Creating a record schema that has a date or timestamp field

According to the Avro documentation, this should define a date field:

{
    "name": "DateTest",
    "type": "record",
    "fields": [
        {"name": "date", "type": "long", "logicalType": "date"}
    ]
}

NiFi 1.5.0-SNAPSHOT treats that as a Long and writes it to Mongo that way.
What am I doing wrong here?

Thanks,

Mike

Re: Creating a record schema that has a date or timestamp field

Posted by Mike Thomsen <mi...@gmail.com>.

Thanks. That got me further. The final piece was I was using the wrong
type. I was annotating a long instead of an int.

Thanks,

Mike

On Wed, Nov 1, 2017 at 3:35 PM, Matt Burgess <ma...@apache.org> wrote:

> Mike,
>
> Which Record Readers/Writers are you using? Do they have the option
> for "Date Format", and if so, are they filled in? Date Format defaults
> to empty, and the doc says it "[s]pecifies the format to use when
> reading/writing Date fields. If not specified, Date fields will be
> assumed to be number of milliseconds since epoch (Midnight, Jan 1,
> 1970 GMT). If specified, the value must match the Java Simple Date
> Format (for example, MM/dd/yyyy for a two-digit month, followed by a
> two-digit day, followed by a four-digit year, all separated by '/'
> characters, as in 01/01/2017)."
>
> I think at least for the Reader you specify in PutMongoRecord, you'll
> need to configure it with a Date Format string.
>
> Regards,
> Matt
>
>
> On Wed, Nov 1, 2017 at 3:27 PM, Mike Thomsen <mi...@gmail.com>
> wrote:
> > According to the Avro documentation, this should define a date field:
> >
> > {
> >     "name": "DateTest",
> >     "type": "record",
> >     "fields": [
> >         {"name": "date", "type": "long", "logicalType": "date"}
> >     ]
> > }
> >
> > NiFi 1.5.0-SNAPSHOT treats that as a Long and writes it to Mongo that
> way.
> > What am I doing wrong here?
> >
> > Thanks,
> >
> > Mike
>

Re: Creating a record schema that has a date or timestamp field

Posted by Matt Burgess <ma...@apache.org>.

Mike,

Which Record Readers/Writers are you using? Do they have the option
for "Date Format", and if so, are they filled in? Date Format defaults
to empty, and the doc says it "[s]pecifies the format to use when
reading/writing Date fields. If not specified, Date fields will be
assumed to be number of milliseconds since epoch (Midnight, Jan 1,
1970 GMT). If specified, the value must match the Java Simple Date
Format (for example, MM/dd/yyyy for a two-digit month, followed by a
two-digit day, followed by a four-digit year, all separated by '/'
characters, as in 01/01/2017)."

I think at least for the Reader you specify in PutMongoRecord, you'll
need to configure it with a Date Format string.

Regards,
Matt

On Wed, Nov 1, 2017 at 3:27 PM, Mike Thomsen <mi...@gmail.com> wrote:
> According to the Avro documentation, this should define a date field:
>
> {
>     "name": "DateTest",
>     "type": "record",
>     "fields": [
>         {"name": "date", "type": "long", "logicalType": "date"}
>     ]
> }
>
> NiFi 1.5.0-SNAPSHOT treats that as a Long and writes it to Mongo that way.
> What am I doing wrong here?
>
> Thanks,
>
> Mike