Ideas around making UDFs simpler: better import support

[Jacques Nadeau <ja...@dremio.com>]

Hey Guys,

I just filed DRILL-3834 [1]. This a task a relatively new developer could
probably pick up to better understand the Drill compilation and generation
code paths. Also up for other ideas around this. This is a something that
makes UDF creation much less pleasant than a user should have to endure.

[1] https://issues.apache.org/jira/browse/DRILL-3834


--
Jacques Nadeau
CTO and Co-Founder, Dremio

Re: Ideas around making UDFs simpler: better import support

[Edmon Begoli <eb...@gmail.com>]

First, this idea is something that makes not only total sense but it is
also highly needed if Drill is to see broader contribution.

Second, I think this "reform" is needed in other aspects of the
architecture as much if not even more than for UDFs.

At this point it is really hard to get your head around the API and how to
write extensions for otherwise extensible parts unless you are working with
the guys who wrote the original APIs.

That would not be as much of an issue that Drill is not a tool that is
supposed to thrive on the broadness and diversity of (contributed) handlers
for different data formats, data storage and data management tools ...

I would love to see one day Drill being capable of querying image metadata,
NetCDF files, hierarchical databases, yaml files, Turtle RDFs, etc.

Maybe have "kits" for writing extensions - documentation, template-oriented
generators, well documented examples.

Thank you,
Edmon

On Thursday, September 24, 2015, Jacques Nadeau <ja...@dremio.com> wrote:

> Hey Guys,
>
> I just filed DRILL-3834 [1]. This a task a relatively new developer could
> probably pick up to better understand the Drill compilation and generation
> code paths. Also up for other ideas around this. This is a something that
> makes UDF creation much less pleasant than a user should have to endure.
>
> [1] https://issues.apache.org/jira/browse/DRILL-3834
>
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>