You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Julian Hyde <jh...@apache.org> on 2016/06/06 19:27:12 UTC
[DISCUSS] Allow HOCON as a format for Calcite models and config files
HOCON is a superset of JSON that makes it easier for humans to edit JSON files. For example, it supports multi-line strings, file inclusion, comments.
Should Calcite support HOCON in addition to strict JSON for model files any other config files? What should be the suffix of such files?
https://github.com/typesafehub/config <https://github.com/typesafehub/config>
Julian
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files
Posted by Julian Hyde <jh...@apache.org>.
I have logged https://issues.apache.org/jira/browse/CALCITE-1280 <https://issues.apache.org/jira/browse/CALCITE-1280> to capture this discussion. I think HOCON and YAML are both strong candidates if someone wanted to contribute them (in addition to JSON of course).
> On Jun 6, 2016, at 6:36 PM, Josh Elser <jo...@gmail.com> wrote:
>
> +1
>
> Michael Mior wrote:
>> I think this would be the ideal approach assuming there isn't too much
>> overhead in maintaining different parsers and keeping them in sync. We
>> wouldn't want to end up in the situation where the parsers start drifting
>> and it could end up being a lot of work to maintain test cases for all the
>> different permutations.
>>
>> --
>> Michael Mior
>> michael.mior@gmail.com
>>
>> 2016-06-06 19:55 GMT-04:00 Julian Hyde<jh...@apache.org>:
>>
>>> YAML isn’t an aside — thanks for raising it. Along with JSON and HOCON,
>>> YAML is another viable language for writing model files.
>>>
>>> I think any of these formats would plug into ModelHandler quite easily,
>>> because they all produce data in the JSON model. And performance isn’t
>>> important, because the model is only read once. The real point is what
>>> format our users prefer using.
>>>
>>> Maybe we don’t even have to choose: we could allow models with .json,
>>> .yaml and .conf suffixes and just invoke the appropriate parser.
>>>
>>> Julian
>>>
>>>
>>>> On Jun 6, 2016, at 1:55 PM, Michael Mior<mm...@uwaterloo.ca> wrote:
>>>>
>>>> As an aside if we want a human-readable configuration format that's more
>>>> flexible than JSON I would personally prefer YAML. It's much more
>>> standard
>>>> than any of the JSON variants and can encode all the necessary
>>> structures.
>>>> --
>>>> Michael Mior
>>>> michael.mior@gmail.com
>>>>
>>>> 2016-06-06 16:51 GMT-04:00 Michael Mior<mm...@uwaterloo.ca>:
>>>>
>>>>> Right now none of the examples are strictly JSON anyway. There are
>>> several
>>>>> different JSON supersets that we could potentially use. It might be
>>> nice to
>>>>> pick one so there's at least some kind of standard.
>>>>>
>>>>> https://hjson.org/
>>>>> https://github.com/timjansen/hanson
>>>>> http://json5.org/
>>>>>
>>>>> --
>>>>> Michael Mior
>>>>> michael.mior@gmail.com
>>>>>
>>>>> 2016-06-06 15:27 GMT-04:00 Julian Hyde<jh...@apache.org>:
>>>>>
>>>>>> HOCON is a superset of JSON that makes it easier for humans to edit
>>> JSON
>>>>>> files. For example, it supports multi-line strings, file inclusion,
>>>>>> comments.
>>>>>>
>>>>>> Should Calcite support HOCON in addition to strict JSON for model files
>>>>>> any other config files? What should be the suffix of such files?
>>>>>>
>>>>>> https://github.com/typesafehub/config<
>>>>>> https://github.com/typesafehub/config>
>>>>>>
>>>>>> Julian
>>>>>>
>>>>>>
>>>
>>
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config
files
Posted by Josh Elser <jo...@gmail.com>.
+1
Michael Mior wrote:
> I think this would be the ideal approach assuming there isn't too much
> overhead in maintaining different parsers and keeping them in sync. We
> wouldn't want to end up in the situation where the parsers start drifting
> and it could end up being a lot of work to maintain test cases for all the
> different permutations.
>
> --
> Michael Mior
> michael.mior@gmail.com
>
> 2016-06-06 19:55 GMT-04:00 Julian Hyde<jh...@apache.org>:
>
>> YAML isn\u2019t an aside \u2014 thanks for raising it. Along with JSON and HOCON,
>> YAML is another viable language for writing model files.
>>
>> I think any of these formats would plug into ModelHandler quite easily,
>> because they all produce data in the JSON model. And performance isn\u2019t
>> important, because the model is only read once. The real point is what
>> format our users prefer using.
>>
>> Maybe we don\u2019t even have to choose: we could allow models with .json,
>> .yaml and .conf suffixes and just invoke the appropriate parser.
>>
>> Julian
>>
>>
>>> On Jun 6, 2016, at 1:55 PM, Michael Mior<mm...@uwaterloo.ca> wrote:
>>>
>>> As an aside if we want a human-readable configuration format that's more
>>> flexible than JSON I would personally prefer YAML. It's much more
>> standard
>>> than any of the JSON variants and can encode all the necessary
>> structures.
>>> --
>>> Michael Mior
>>> michael.mior@gmail.com
>>>
>>> 2016-06-06 16:51 GMT-04:00 Michael Mior<mm...@uwaterloo.ca>:
>>>
>>>> Right now none of the examples are strictly JSON anyway. There are
>> several
>>>> different JSON supersets that we could potentially use. It might be
>> nice to
>>>> pick one so there's at least some kind of standard.
>>>>
>>>> https://hjson.org/
>>>> https://github.com/timjansen/hanson
>>>> http://json5.org/
>>>>
>>>> --
>>>> Michael Mior
>>>> michael.mior@gmail.com
>>>>
>>>> 2016-06-06 15:27 GMT-04:00 Julian Hyde<jh...@apache.org>:
>>>>
>>>>> HOCON is a superset of JSON that makes it easier for humans to edit
>> JSON
>>>>> files. For example, it supports multi-line strings, file inclusion,
>>>>> comments.
>>>>>
>>>>> Should Calcite support HOCON in addition to strict JSON for model files
>>>>> any other config files? What should be the suffix of such files?
>>>>>
>>>>> https://github.com/typesafehub/config<
>>>>> https://github.com/typesafehub/config>
>>>>>
>>>>> Julian
>>>>>
>>>>>
>>
>
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files
Posted by Michael Mior <mm...@uwaterloo.ca>.
I think this would be the ideal approach assuming there isn't too much
overhead in maintaining different parsers and keeping them in sync. We
wouldn't want to end up in the situation where the parsers start drifting
and it could end up being a lot of work to maintain test cases for all the
different permutations.
--
Michael Mior
michael.mior@gmail.com
2016-06-06 19:55 GMT-04:00 Julian Hyde <jh...@apache.org>:
> YAML isn’t an aside — thanks for raising it. Along with JSON and HOCON,
> YAML is another viable language for writing model files.
>
> I think any of these formats would plug into ModelHandler quite easily,
> because they all produce data in the JSON model. And performance isn’t
> important, because the model is only read once. The real point is what
> format our users prefer using.
>
> Maybe we don’t even have to choose: we could allow models with .json,
> .yaml and .conf suffixes and just invoke the appropriate parser.
>
> Julian
>
>
> > On Jun 6, 2016, at 1:55 PM, Michael Mior <mm...@uwaterloo.ca> wrote:
> >
> > As an aside if we want a human-readable configuration format that's more
> > flexible than JSON I would personally prefer YAML. It's much more
> standard
> > than any of the JSON variants and can encode all the necessary
> structures.
> >
> > --
> > Michael Mior
> > michael.mior@gmail.com
> >
> > 2016-06-06 16:51 GMT-04:00 Michael Mior <mm...@uwaterloo.ca>:
> >
> >> Right now none of the examples are strictly JSON anyway. There are
> several
> >> different JSON supersets that we could potentially use. It might be
> nice to
> >> pick one so there's at least some kind of standard.
> >>
> >> https://hjson.org/
> >> https://github.com/timjansen/hanson
> >> http://json5.org/
> >>
> >> --
> >> Michael Mior
> >> michael.mior@gmail.com
> >>
> >> 2016-06-06 15:27 GMT-04:00 Julian Hyde <jh...@apache.org>:
> >>
> >>> HOCON is a superset of JSON that makes it easier for humans to edit
> JSON
> >>> files. For example, it supports multi-line strings, file inclusion,
> >>> comments.
> >>>
> >>> Should Calcite support HOCON in addition to strict JSON for model files
> >>> any other config files? What should be the suffix of such files?
> >>>
> >>> https://github.com/typesafehub/config <
> >>> https://github.com/typesafehub/config>
> >>>
> >>> Julian
> >>>
> >>>
> >>
>
>
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files
Posted by Julian Hyde <jh...@apache.org>.
YAML isn’t an aside — thanks for raising it. Along with JSON and HOCON, YAML is another viable language for writing model files.
I think any of these formats would plug into ModelHandler quite easily, because they all produce data in the JSON model. And performance isn’t important, because the model is only read once. The real point is what format our users prefer using.
Maybe we don’t even have to choose: we could allow models with .json, .yaml and .conf suffixes and just invoke the appropriate parser.
Julian
> On Jun 6, 2016, at 1:55 PM, Michael Mior <mm...@uwaterloo.ca> wrote:
>
> As an aside if we want a human-readable configuration format that's more
> flexible than JSON I would personally prefer YAML. It's much more standard
> than any of the JSON variants and can encode all the necessary structures.
>
> --
> Michael Mior
> michael.mior@gmail.com
>
> 2016-06-06 16:51 GMT-04:00 Michael Mior <mm...@uwaterloo.ca>:
>
>> Right now none of the examples are strictly JSON anyway. There are several
>> different JSON supersets that we could potentially use. It might be nice to
>> pick one so there's at least some kind of standard.
>>
>> https://hjson.org/
>> https://github.com/timjansen/hanson
>> http://json5.org/
>>
>> --
>> Michael Mior
>> michael.mior@gmail.com
>>
>> 2016-06-06 15:27 GMT-04:00 Julian Hyde <jh...@apache.org>:
>>
>>> HOCON is a superset of JSON that makes it easier for humans to edit JSON
>>> files. For example, it supports multi-line strings, file inclusion,
>>> comments.
>>>
>>> Should Calcite support HOCON in addition to strict JSON for model files
>>> any other config files? What should be the suffix of such files?
>>>
>>> https://github.com/typesafehub/config <
>>> https://github.com/typesafehub/config>
>>>
>>> Julian
>>>
>>>
>>
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files
Posted by Michael Mior <mm...@uwaterloo.ca>.
As an aside if we want a human-readable configuration format that's more
flexible than JSON I would personally prefer YAML. It's much more standard
than any of the JSON variants and can encode all the necessary structures.
--
Michael Mior
michael.mior@gmail.com
2016-06-06 16:51 GMT-04:00 Michael Mior <mm...@uwaterloo.ca>:
> Right now none of the examples are strictly JSON anyway. There are several
> different JSON supersets that we could potentially use. It might be nice to
> pick one so there's at least some kind of standard.
>
> https://hjson.org/
> https://github.com/timjansen/hanson
> http://json5.org/
>
> --
> Michael Mior
> michael.mior@gmail.com
>
> 2016-06-06 15:27 GMT-04:00 Julian Hyde <jh...@apache.org>:
>
>> HOCON is a superset of JSON that makes it easier for humans to edit JSON
>> files. For example, it supports multi-line strings, file inclusion,
>> comments.
>>
>> Should Calcite support HOCON in addition to strict JSON for model files
>> any other config files? What should be the suffix of such files?
>>
>> https://github.com/typesafehub/config <
>> https://github.com/typesafehub/config>
>>
>> Julian
>>
>>
>
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files
Posted by Michael Mior <mm...@uwaterloo.ca>.
Right now none of the examples are strictly JSON anyway. There are several
different JSON supersets that we could potentially use. It might be nice to
pick one so there's at least some kind of standard.
https://hjson.org/
https://github.com/timjansen/hanson
http://json5.org/
--
Michael Mior
michael.mior@gmail.com
2016-06-06 15:27 GMT-04:00 Julian Hyde <jh...@apache.org>:
> HOCON is a superset of JSON that makes it easier for humans to edit JSON
> files. For example, it supports multi-line strings, file inclusion,
> comments.
>
> Should Calcite support HOCON in addition to strict JSON for model files
> any other config files? What should be the suffix of such files?
>
> https://github.com/typesafehub/config <
> https://github.com/typesafehub/config>
>
> Julian
>
>
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config
files
Posted by Josh Elser <jo...@gmail.com>.
Cool :)
Justification enough for me.
Jacques Nadeau wrote:
> Just to clarify, my responses were in regards to HOCON.
>
>
> On Mon, Jun 6, 2016 at 1:53 PM, Jacques Nadeau<ja...@apache.org> wrote:
>
>> We use it a lot in Drill for configuration files. It serves this purpose
>> well. (We especially leverage the caoncatenation/overlay capbilities.)
>>
>> I haven't thought enough about its use in this case to have an opinion one
>> way or the other.
>>
>> On Mon, Jun 6, 2016 at 1:33 PM, Josh Elser<jo...@gmail.com> wrote:
>>
>>> At a glance, it looks like it would be a nice improvement over JSON with
>>> the ability to still just provide json. Haven't had any practical
>>> experience with it though (e.g. does it work as advertised).
>>>
>>> Looks like they recommend ".conf" as the suffix
>>>
>>> <snip>The JSON superset is called "Human-Optimized Config Object
>>> Notation" or HOCON, and files use the suffix .conf.</snip>
>>>
>>>
>>> Julian Hyde wrote:
>>>
>>>> HOCON is a superset of JSON that makes it easier for humans to edit JSON
>>>> files. For example, it supports multi-line strings, file inclusion,
>>>> comments.
>>>>
>>>> Should Calcite support HOCON in addition to strict JSON for model files
>>>> any other config files? What should be the suffix of such files?
>>>>
>>>> https://github.com/typesafehub/config<
>>>> https://github.com/typesafehub/config>
>>>>
>>>> Julian
>>>>
>>>>
>>>>
>
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files
Posted by Jacques Nadeau <ja...@apache.org>.
Just to clarify, my responses were in regards to HOCON.
On Mon, Jun 6, 2016 at 1:53 PM, Jacques Nadeau <ja...@apache.org> wrote:
> We use it a lot in Drill for configuration files. It serves this purpose
> well. (We especially leverage the caoncatenation/overlay capbilities.)
>
> I haven't thought enough about its use in this case to have an opinion one
> way or the other.
>
> On Mon, Jun 6, 2016 at 1:33 PM, Josh Elser <jo...@gmail.com> wrote:
>
>> At a glance, it looks like it would be a nice improvement over JSON with
>> the ability to still just provide json. Haven't had any practical
>> experience with it though (e.g. does it work as advertised).
>>
>> Looks like they recommend ".conf" as the suffix
>>
>> <snip>The JSON superset is called "Human-Optimized Config Object
>> Notation" or HOCON, and files use the suffix .conf.</snip>
>>
>>
>> Julian Hyde wrote:
>>
>>> HOCON is a superset of JSON that makes it easier for humans to edit JSON
>>> files. For example, it supports multi-line strings, file inclusion,
>>> comments.
>>>
>>> Should Calcite support HOCON in addition to strict JSON for model files
>>> any other config files? What should be the suffix of such files?
>>>
>>> https://github.com/typesafehub/config<
>>> https://github.com/typesafehub/config>
>>>
>>> Julian
>>>
>>>
>>>
>
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config files
Posted by Jacques Nadeau <ja...@apache.org>.
We use it a lot in Drill for configuration files. It serves this purpose
well. (We especially leverage the caoncatenation/overlay capbilities.)
I haven't thought enough about its use in this case to have an opinion one
way or the other.
On Mon, Jun 6, 2016 at 1:33 PM, Josh Elser <jo...@gmail.com> wrote:
> At a glance, it looks like it would be a nice improvement over JSON with
> the ability to still just provide json. Haven't had any practical
> experience with it though (e.g. does it work as advertised).
>
> Looks like they recommend ".conf" as the suffix
>
> <snip>The JSON superset is called "Human-Optimized Config Object Notation"
> or HOCON, and files use the suffix .conf.</snip>
>
>
> Julian Hyde wrote:
>
>> HOCON is a superset of JSON that makes it easier for humans to edit JSON
>> files. For example, it supports multi-line strings, file inclusion,
>> comments.
>>
>> Should Calcite support HOCON in addition to strict JSON for model files
>> any other config files? What should be the suffix of such files?
>>
>> https://github.com/typesafehub/config<
>> https://github.com/typesafehub/config>
>>
>> Julian
>>
>>
>>
Re: [DISCUSS] Allow HOCON as a format for Calcite models and config
files
Posted by Josh Elser <jo...@gmail.com>.
At a glance, it looks like it would be a nice improvement over JSON with
the ability to still just provide json. Haven't had any practical
experience with it though (e.g. does it work as advertised).
Looks like they recommend ".conf" as the suffix
<snip>The JSON superset is called "Human-Optimized Config Object
Notation" or HOCON, and files use the suffix .conf.</snip>
Julian Hyde wrote:
> HOCON is a superset of JSON that makes it easier for humans to edit JSON files. For example, it supports multi-line strings, file inclusion, comments.
>
> Should Calcite support HOCON in addition to strict JSON for model files any other config files? What should be the suffix of such files?
>
> https://github.com/typesafehub/config<https://github.com/typesafehub/config>
>
> Julian
>
>