You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@iceberg.apache.org by Filip <fi...@gmail.com> on 2021/03/02 16:18:22 UTC

Basic iceberg metrics viz tool

Hi devs,

With a lot of help from plotly.js [1] and some very basic vanilla
javascript (sorry, I peaked in javascript back in the days of prototype and
scriptaculous) we managed to craft a tool to visually analyze rows and data
files metrics of an Iceberg metadata file.

Just drag and drop an Iceberg metadata file and you get something looking
like this.
[image: sample.iceberg.viz.png]

Best part is we can host this on github pages cause it's just a javascript
file and one lonely html file (no CSS stylesheet, sorry :| I peaked around
the emergence of reset.css files).
Let me know if you think this would make a good addition to the repo.

[1] https://github.com/plotly/plotly.js/blob/master/LICENSE

-- 
Filip Bocse

Re: Basic iceberg metrics viz tool

Posted by Tianyi Wang <ca...@gmail.com>.

I did something similar to visualize the snapshots and files. But instead
of using the static website, I was using the Java API to get the metadata
from HDFS and send it back to the frontend.
Something like this:
https://observablehq.com/@capkurmagati/iceberg-metadata-visualization
My actual implementation does not use vega for visualization but you get
the idea.

My team plans to integrate Iceberg table into our existing Hive-based
catalog system. I think my team can work with the community.

Bests,

On Wed, Mar 3, 2021 at 10:29 AM Ryan Blue <rb...@netflix.com.invalid> wrote:

> This is great! If there's an easy way to make this available to other
> people I think that's a great idea.
>
> On Tue, Mar 2, 2021 at 9:24 AM John Zhuge <jz...@apache.org> wrote:
>
>> Nice!
>>
>> On Tue, Mar 2, 2021 at 8:39 AM Filip <fi...@gmail.com> wrote:
>>
>>> Ooops, sorry, forgot to post a link to the tools screenshot sample
>>> https://i.imgur.com/FERzd8X.png
>>>
>>> On 2021/03/02 16:18:22, Filip <fi...@gmail.com> wrote:
>>> > Hi devs,
>>> >
>>> > With a lot of help from plotly.js [1] and some very basic vanilla
>>> > javascript (sorry, I peaked in javascript back in the days of
>>> prototype and
>>> > scriptaculous) we managed to craft a tool to visually analyze rows and
>>> data
>>> > files metrics of an Iceberg metadata file.
>>> >
>>> > Just drag and drop an Iceberg metadata file and you get something
>>> looking
>>> > like this.
>>> > [image: sample.iceberg.viz.png]
>>> >
>>> > Best part is we can host this on github pages cause it's just a
>>> javascript
>>> > file and one lonely html file (no CSS stylesheet, sorry :| I peaked
>>> around
>>> > the emergence of reset.css files).
>>> > Let me know if you think this would make a good addition to the repo.
>>> >
>>> > [1] https://github.com/plotly/plotly.js/blob/master/LICENSE
>>> >
>>> > --
>>> > Filip Bocse
>>> >
>>>
>>
>>
>> --
>> John Zhuge
>>
>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>

Re: Basic iceberg metrics viz tool

Posted by Ryan Blue <rb...@netflix.com.INVALID>.

This is great! If there's an easy way to make this available to other
people I think that's a great idea.

On Tue, Mar 2, 2021 at 9:24 AM John Zhuge <jz...@apache.org> wrote:

> Nice!
>
> On Tue, Mar 2, 2021 at 8:39 AM Filip <fi...@gmail.com> wrote:
>
>> Ooops, sorry, forgot to post a link to the tools screenshot sample
>> https://i.imgur.com/FERzd8X.png
>>
>> On 2021/03/02 16:18:22, Filip <fi...@gmail.com> wrote:
>> > Hi devs,
>> >
>> > With a lot of help from plotly.js [1] and some very basic vanilla
>> > javascript (sorry, I peaked in javascript back in the days of prototype
>> and
>> > scriptaculous) we managed to craft a tool to visually analyze rows and
>> data
>> > files metrics of an Iceberg metadata file.
>> >
>> > Just drag and drop an Iceberg metadata file and you get something
>> looking
>> > like this.
>> > [image: sample.iceberg.viz.png]
>> >
>> > Best part is we can host this on github pages cause it's just a
>> javascript
>> > file and one lonely html file (no CSS stylesheet, sorry :| I peaked
>> around
>> > the emergence of reset.css files).
>> > Let me know if you think this would make a good addition to the repo.
>> >
>> > [1] https://github.com/plotly/plotly.js/blob/master/LICENSE
>> >
>> > --
>> > Filip Bocse
>> >
>>
>
>
> --
> John Zhuge
>


-- 
Ryan Blue
Software Engineer
Netflix

Re: Basic iceberg metrics viz tool

Posted by John Zhuge <jz...@apache.org>.

Nice!

On Tue, Mar 2, 2021 at 8:39 AM Filip <fi...@gmail.com> wrote:

> Ooops, sorry, forgot to post a link to the tools screenshot sample
> https://i.imgur.com/FERzd8X.png
>
> On 2021/03/02 16:18:22, Filip <fi...@gmail.com> wrote:
> > Hi devs,
> >
> > With a lot of help from plotly.js [1] and some very basic vanilla
> > javascript (sorry, I peaked in javascript back in the days of prototype
> and
> > scriptaculous) we managed to craft a tool to visually analyze rows and
> data
> > files metrics of an Iceberg metadata file.
> >
> > Just drag and drop an Iceberg metadata file and you get something looking
> > like this.
> > [image: sample.iceberg.viz.png]
> >
> > Best part is we can host this on github pages cause it's just a
> javascript
> > file and one lonely html file (no CSS stylesheet, sorry :| I peaked
> around
> > the emergence of reset.css files).
> > Let me know if you think this would make a good addition to the repo.
> >
> > [1] https://github.com/plotly/plotly.js/blob/master/LICENSE
> >
> > --
> > Filip Bocse
> >
>


-- 
John Zhuge

Re: Basic iceberg metrics viz tool

Posted by Filip <fi...@gmail.com>.

Ooops, sorry, forgot to post a link to the tools screenshot sample
https://i.imgur.com/FERzd8X.png

On 2021/03/02 16:18:22, Filip <fi...@gmail.com> wrote: 
> Hi devs,
> 
> With a lot of help from plotly.js [1] and some very basic vanilla
> javascript (sorry, I peaked in javascript back in the days of prototype and
> scriptaculous) we managed to craft a tool to visually analyze rows and data
> files metrics of an Iceberg metadata file.
> 
> Just drag and drop an Iceberg metadata file and you get something looking
> like this.
> [image: sample.iceberg.viz.png]
> 
> Best part is we can host this on github pages cause it's just a javascript
> file and one lonely html file (no CSS stylesheet, sorry :| I peaked around
> the emergence of reset.css files).
> Let me know if you think this would make a good addition to the repo.
> 
> [1] https://github.com/plotly/plotly.js/blob/master/LICENSE
> 
> -- 
> Filip Bocse
>