Lookup Based Schema Registry

[Shawn Weeks <sw...@weeksconsulting.us>]

I’m looking at building a version of the Avro schema registry that uses a lookup service. This would be for cases where you have too many schemas to easily manage from the property based avro schema registry but don’t need or want the complexities of the Confluent or Hortonworks registries. Technically you could use lookup attribute to do this but then you’re adding the entire schema to the flow file attributes. For some schemas that can be fairly large.

Does anyone else think this might be useful?

Thanks
Shawn

Re: Lookup Based Schema Registry

[Mike Thomsen <mi...@gmail.com>]

This might scratch most of your itch:

https://github.com/MikeThomsen/nifi-fs-schema-registry

It basically just maps ${schema.name} to a file name in a folder. A few
teams at my client site use it because they can just SCP a new copy of
their schema to the folder and there's no reloading of the controller
service required to make it go live.

On Mon, Dec 30, 2019 at 11:30 AM Shawn Weeks <sw...@weeksconsulting.us>
wrote:

> I’m looking at building a version of the Avro schema registry that uses a
> lookup service. This would be for cases where you have too many schemas to
> easily manage from the property based avro schema registry but don’t need
> or want the complexities of the Confluent or Hortonworks registries.
> Technically you could use lookup attribute to do this but then you’re
> adding the entire schema to the flow file attributes. For some schemas that
> can be fairly large.
>
> Does anyone else think this might be useful?
>
> Thanks
> Shawn
>