Drill1.7 Feedback - Logs in Web UI

[John Omernik <jo...@omernik.com>]

I like the concept of logs in the web UI, however at this time, it assumes
that there will only be one directory for logfiles. The way I've set mine
up is to have different directories for logs, dcplogs, profiles, etc.  That
way, I can organize them out a bit, and for those logs that are in json
format, actually use drill to query them (awesome).   So to that end, here
are some observations/suggestions.


1. Create an option that will specify what the extensions of valid log
files will be (perhaps default to .json,.log) that way, you don't have a
web server trying to render things that perhaps should not be rendered
(only one of many protections that may need to be here). For example, I
tared up some logfiles, the UI shouldn't try to render that, or show that
it exists.

2. Allow traversal from the log directory to subdirectories. (But not up
pas the log dir root!)

3.  Provide sortable (Name, Size, Last Modified)

4. Show permissions on the list page

Re: Drill1.7 Feedback - Logs in Web UI

[Arina Yelchiyeva <ar...@gmail.com>]

I have created Jira to address enhancements mentioned by John -
https://issues.apache.org/jira/browse/DRILL-4775.

On Fri, Jul 8, 2016 at 3:22 PM, John Omernik <jo...@omernik.com> wrote:

> 1. Do to how things could be rendered, I think it's a nice way to ensure
> we don't open say binary files, or extremely large files in the web
> browser.  My worry is actions taken here could also be a point of malicious
> attack, i.e. rendering things that aren't text files and triggering a
> vulnerability in a browser or even the JVM.  By limiting down to say
> ".txt,.log,.json" initially, but allowing users to expand that if needed,
> we put a few protections in place and ensure the browser doesn't get to a
> state where it's trying to render a 100mb binary file of some sort.   (I
> wonder if we should do some sanity checking on file sizes in addition... a
> setting of "max display log file" or something like that.
>
> 4.  They aren't, I am thinking about when I use Mesos, and it shows the
> file permissions and ownership. This would not be needed here, I just fine
> helpful when I am looking at sandbox logs in Mesos, so I suggested it. I
> guess I can't really come up with a use case in Drill other than "I like
> it" :)
>
>
>
> On Thu, Jul 7, 2016 at 10:53 AM, Arina Yelchiyeva <
> arina.yelchiyeva@gmail.com> wrote:
>
>> Hi John!
>>
>> Thanks a lot for your feedback!
>> Please see my comments inline.
>>
>> On Tue, Jul 5, 2016 at 8:30 PM, John Omernik <jo...@omernik.com> wrote:
>>
>>> I like the concept of logs in the web UI, however at this time, it
>>> assumes
>>> that there will only be one directory for logfiles.
>>
>> The way I've set mine
>>> up is to have different directories for logs, dcplogs, profiles, etc.
>>> That
>>> way, I can organize them out a bit, and for those logs that are in json
>>> format, actually use drill to query them (awesome).   So to that end,
>>> here
>>> are some observations/suggestions.
>>>
>>>
>>> 1. Create an option that will specify what the extensions of valid log
>>> files will be (perhaps default to .json,.log) that way, you don't have a
>>> web server trying to render things that perhaps should not be rendered
>>> (only one of many protections that may need to be here). For example, I
>>> tared up some logfiles, the UI shouldn't try to render that, or show that
>>> it exists.
>>
>>
>> I suggest to show all files by default. If user wants to exclude some
>> extensions, he may modify the option.
>>
>>
>>> 2. Allow traversal from the log directory to subdirectories. (But not up
>>> pas the log dir root!)
>>
>>
>> Agree.
>>
>> 3.  Provide sortable (Name, Size, Last Modified)
>>>
>>
>> Agree.
>>
>>
>>> 4. Show permissions on the list page
>>>
>>
>> May I ask why permissions are needed?
>>
>>
>

Re: Drill1.7 Feedback - Logs in Web UI

[Arina Yelchiyeva <ar...@gmail.com>]

Hi John!

Thanks a lot for your feedback!
Please see my comments inline.

On Tue, Jul 5, 2016 at 8:30 PM, John Omernik <jo...@omernik.com> wrote:

> I like the concept of logs in the web UI, however at this time, it assumes
> that there will only be one directory for logfiles.

The way I've set mine
> up is to have different directories for logs, dcplogs, profiles, etc.  That
> way, I can organize them out a bit, and for those logs that are in json
> format, actually use drill to query them (awesome).   So to that end, here
> are some observations/suggestions.
>
>
> 1. Create an option that will specify what the extensions of valid log
> files will be (perhaps default to .json,.log) that way, you don't have a
> web server trying to render things that perhaps should not be rendered
> (only one of many protections that may need to be here). For example, I
> tared up some logfiles, the UI shouldn't try to render that, or show that
> it exists.


I suggest to show all files by default. If user wants to exclude some
extensions, he may modify the option.


> 2. Allow traversal from the log directory to subdirectories. (But not up
> pas the log dir root!)


Agree.

3.  Provide sortable (Name, Size, Last Modified)
>

Agree.


> 4. Show permissions on the list page
>

May I ask why permissions are needed?