You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Gerard Toonstra <gt...@gmail.com> on 2017/12/03 07:42:09 UTC

Re: Data lineage and data portal

Good morning,

The meeting has been scheduled for wednesday 6th december:

London 17:00:00 UTC, Amsterdam 18:00:00 UTC+1, San Francisco 09:00:00 PST,
New York 12:00:00 EST

--------

Gerard Toonstra is inviting you to a scheduled Zoom meeting.

Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/789615759
<https://www.google.com/url?q=https%3A%2F%2Fzoom.us%2Fj%2F789615759&sa=D&usd=2&usg=AFQjCNFfEIitKjiD6AjsxPB5ddfMcyLv5Q>


Or iPhone one-tap :
    US: +16699006833,,789615759# or +16468769923,,789615759#
Or Telephone:
    Dial(for higher quality, dial a number based on your current location):
        US: +1 669 900 6833 or +1 646 876 9923
    Meeting ID: 789 615 759
    International numbers available:
https://zoom.us/zoomconference?m=5eYALpFJGSSZWrP577vUpbK_0dP7WPfp
<https://www.google.com/url?q=https%3A%2F%2Fzoom.us%2Fzoomconference%3Fm%3D5eYALpFJGSSZWrP577vUpbK_0dP7WPfp&sa=D&usd=2&usg=AFQjCNE41pjZbgmI9NNAqOi_6ah3BufM1A>


-------

Best regards,

Gerard


On Thu, Nov 30, 2017 at 9:05 PM, Gerard Toonstra <gt...@gmail.com>
wrote:

> Hi all,
>
> Nice overwhelming response! :)
>
> I'd like to host a meeting on zoom to discuss. I'm on the free plan there,
> so we are limited to 40 minutes! :)  (msg me priv if you have better
> hosting options).
> Here's a way to let me know your availability without polluting the
> mailing list:
>
> https://docs.google.com/forms/d/e/1FAIpQLSfsvfM6mUVwD_
> qV9EXdUB1aLcdKt55toYKRp3JDanonoFRSLQ/viewform?usp=sf_link
>
> I'll record the meeting and share the video for those who cannot attend.
>
> Agenda:
> - What specifically is so useful about the data portal?
> - ( If anyone from airbnb attends, they get the floor to give us more
> context and info )
> - Is there sufficient interest and value creation through a collaboration?
> - How would a data portal complement airflow and vice versa?
> - How do we progress this further and which actions can I take here?
>
> You can prepare for the meeting by reading the blog post or watching the
> video (which has even more detail):
>
> https://medium.com/airbnb-engineering/democratizing-data-at-
> airbnb-852d76c51770
>
> https://www.youtube.com/watch?v=gayXC2FDSiA&t=4s
>
> Looking forward to your availability responses.
>
> Best Regards,
>
> Gerard
>
>
> On Wed, Nov 29, 2017 at 7:31 PM, Nathan McIntyre <nw...@gmail.com>
> wrote:
>
>> +1
>>
>> > On Nov 29, 2017, at 09:48, Alek Storm <al...@gmail.com> wrote:
>> >
>> > +1
>> >
>> > On Wed, Nov 29, 2017 at 9:50 AM, Igors Vaitkus <
>> litdeviant@protonmail.com>
>> > wrote:
>> >
>> >> +1
>> >>
>> >> On 11/29/17, 3:48 PM, "Kate-Laurel Agnew" <ka...@signal.co> wrote:
>> >>
>> >>    +1
>> >>
>> >>    On Wed, Nov 29, 2017 at 12:09 AM, Koen Mevissen <
>> kmevissen@travix.com>
>> >>    wrote:
>> >>
>> >>> +1
>> >>>
>> >>> I'm interested as well!
>> >>>
>> >>>
>> >>>
>> >>> Op di 28 nov. 2017 om 14:04 schreef Marc Bollinger <
>> >> marc@lumoslabs.com>
>> >>>
>> >>>> +1
>> >>>>
>> >>>> On Mon, Nov 27, 2017 at 6:18 PM, Ruslan Dautkhanov <
>> >> dautkhanov@gmail.com
>> >>>>
>> >>>> wrote:
>> >>>>
>> >>>>> ‘’’
>> >>>>> I'm
>> >>>>> now working on sql scanners, extractors and other tools that
>> >> allow me
>> >>> to
>> >>>>> populate the database
>> >>>>> ‘’’
>> >>>>>
>> >>>>> Very cool. Cloudera Navigator ( not an open source product) does
>> >> this
>> >>> too
>> >>>>> to some extent - collect metadata and create data lineage
>> >>> automatically (
>> >>>>> stored as a Solr collection) by parsing sql queries.
>> >>>>>
>> >>>>> https://www.cloudera.com/documentation/enterprise/5-12-
>> >>>>> x/topics/datamgmt_extraction_indexing.html
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Mon, Nov 27, 2017 at 12:38 PM Gerard Toonstra <
>> >> gtoonstra@gmail.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> Hi all,
>> >>>>>>
>> >>>>>> So something that really drew my attention recently is a "data
>> >>> portal"
>> >>>>> as
>> >>>>>> described by a team from airbnb somewhere in May. The idea is
>> >>>> basically a
>> >>>>>> "facebook of data":
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> https://medium.com/airbnb-engineering/democratizing-
>> >>>>> data-at-airbnb-852d76c51770
>> >>>>>>
>> >>>>>>
>> >>>>>> Unfortunately it looks like it's not going to be opensourced
>> >> due to
>> >>> how
>> >>>>>> heavily integrated it is with their specific infrastructure;
>> >> but the
>> >>>> idea
>> >>>>>> itself to me sounds like it's something every organization of a
>> >>> certain
>> >>>>>> size should have to keep track of data and stay informed as an
>> >>>>>> organization.
>> >>>>>>
>> >>>>>> Based on the descriptions, I prototyped some things away and
>> >> am happy
>> >>>>> with
>> >>>>>> the results and the speed that something like this can be
>> >>> constructed.
>> >>>>> I'm
>> >>>>>> now working on sql scanners, extractors and other tools that
>> >> allow me
>> >>>> to
>> >>>>>> populate the database and put a poc together on some real data.
>> >>>>>>
>> >>>>>> If other people have similar concerns in their organization
>> >> and think
>> >>>>> this
>> >>>>>> would be a great thing to have, reply to me or the list; with
>> >>>> sufficient
>> >>>>>> interest I may set up a web chat/meet session so this can be
>> >>> discussed
>> >>>> in
>> >>>>>> more detail and find ways to progress this.
>> >>>>>>
>> >>>>>>
>> >>>>>> Best regards,
>> >>>>>>
>> >>>>>> Gerard
>> >>>>>>
>> >>>>>
>> >>>>
>> >>> --
>> >>> Kind regards,
>> >>> Met vriendelijke groet,
>> >>>
>> >>> *Koen Mevissen*
>> >>> Principal BI Developer
>> >>>
>> >>>
>> >>> *Travix Nederland B.V.*
>> >>> Piet Heinkade 55
>> >>> 1019 GM Amsterdam
>> >>> The Netherlands
>> >>>
>> >>> T. +31 (0)20 203 3241
>> >>> E: KMevissen@travix.com
>> >>> www.travix.com
>> >>>
>> >>> *Brands: * CheapTickets  |  Vliegwinkel  |  Vayama  |  BudgetAir  |
>> >>> Flugladen
>> >>>
>> >>
>> >>
>> >>
>> >>    --
>> >>
>> >>
>> >>
>> >>
>> >>    *Kate-Laurel AgnewData Engineerm: 503-741-9207
>> >>    <503%20741%209207>e: kagnew@signal.co
>> >>    <https://mail.google.com/mail/?view=cm&fs=1&tf=1&to=kagnew@s
>> ignal.co>
>> >> signal.co
>> >>    <http://www.google.com/url?q=http%3A%2F%2Fsignal.co%2F&sa=
>> >> D&sntz=1&usg=AFQjCNGG8nAqplB1u72dXPbYfPYRcfopNQ>____________
>> ____________
>> >> Cut
>> >>    Through the NoiseThis e-mail and any files transmitted with it are
>> for
>> >> the
>> >>    sole use of the intended recipient(s) and may contain confidential
>> and
>> >>    privileged information. Any unauthorized use of this email is
>> strictly
>> >>    prohibited. ©2015 Signal. All rights reserved.*
>> >>
>> >>
>> >>
>> >>
>> >>
>>
>
>

Re: Data lineage and data portal

Posted by Gerard Toonstra <gt...@gmail.com>.
Hi all,

So we held a short meeting to explore cooperation a bit and get ideas what
people expect.
You can view the short meeting minutes here:

https://docs.google.com/document/d/16yyS07a_i7qpac4B5Xqec_S6j7kVwHcjUaPTdcnLih0/edit?usp=sharing

The mp3 audio:
https://drive.google.com/open?id=1kCmd8I5X8L4T0hLkN-9bIH3FCx1NKIIK
<https://drive.google.com/open?id=1kCmd8I5X8L4T0hLkN-9bIH3FCx1NKIIK>


I set up a google group to discuss the data portal further and kick it off
from there:

https://groups.google.com/forum/#!forum/dataportalkickoff


You can request to join that group, I'll accept everyone and then we can
start writing your ideas, references,
things you looked at, tried, etc.

See you there!

Gerard


On Sun, Dec 3, 2017 at 11:40 AM, Sam Elamin <hu...@gmail.com> wrote:

> I'm def in.
>
> Thanks for organising Gerard!
>
> On Sun, 3 Dec 2017 at 07:42, Gerard Toonstra <gt...@gmail.com> wrote:
>
> > Good morning,
> >
> > The meeting has been scheduled for wednesday 6th december:
> >
> > London 17:00:00 UTC, Amsterdam 18:00:00 UTC+1, San Francisco 09:00:00
> PST,
> > New York 12:00:00 EST
> >
> > --------
> >
> > Gerard Toonstra is inviting you to a scheduled Zoom meeting.
> >
> > Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/789615759
> > <
> > https://www.google.com/url?q=https%3A%2F%2Fzoom.us%2Fj%
> 2F789615759&sa=D&usd=2&usg=AFQjCNFfEIitKjiD6AjsxPB5ddfMcyLv5Q
> > >
> >
> >
> > Or iPhone one-tap :
> >     US: +16699006833,,789615759# or +16468769923,,789615759#
> > Or Telephone:
> >     Dial(for higher quality, dial a number based on your current
> location):
> >         US: +1 669 900 6833 or +1 646 876 9923
> >     Meeting ID: 789 615 759
> >     International numbers available:
> > https://zoom.us/zoomconference?m=5eYALpFJGSSZWrP577vUpbK_0dP7WPfp
> > <
> > https://www.google.com/url?q=https%3A%2F%2Fzoom.us%2Fzoomconference%3Fm%
> 3D5eYALpFJGSSZWrP577vUpbK_0dP7WPfp&sa=D&usd=2&usg=AFQjCNE41pjZbgmI9NNAqOi_
> 6ah3BufM1A
> > >
> >
> >
> > -------
> >
> > Best regards,
> >
> > Gerard
> >
> >
> > On Thu, Nov 30, 2017 at 9:05 PM, Gerard Toonstra <gt...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > Nice overwhelming response! :)
> > >
> > > I'd like to host a meeting on zoom to discuss. I'm on the free plan
> > there,
> > > so we are limited to 40 minutes! :)  (msg me priv if you have better
> > > hosting options).
> > > Here's a way to let me know your availability without polluting the
> > > mailing list:
> > >
> > > https://docs.google.com/forms/d/e/1FAIpQLSfsvfM6mUVwD_
> > > qV9EXdUB1aLcdKt55toYKRp3JDanonoFRSLQ/viewform?usp=sf_link
> > >
> > > I'll record the meeting and share the video for those who cannot
> attend.
> > >
> > > Agenda:
> > > - What specifically is so useful about the data portal?
> > > - ( If anyone from airbnb attends, they get the floor to give us more
> > > context and info )
> > > - Is there sufficient interest and value creation through a
> > collaboration?
> > > - How would a data portal complement airflow and vice versa?
> > > - How do we progress this further and which actions can I take here?
> > >
> > > You can prepare for the meeting by reading the blog post or watching
> the
> > > video (which has even more detail):
> > >
> > > https://medium.com/airbnb-engineering/democratizing-data-at-
> > > airbnb-852d76c51770
> > >
> > > https://www.youtube.com/watch?v=gayXC2FDSiA&t=4s
> > >
> > > Looking forward to your availability responses.
> > >
> > > Best Regards,
> > >
> > > Gerard
> > >
> > >
> > > On Wed, Nov 29, 2017 at 7:31 PM, Nathan McIntyre <nwmcintyre@gmail.com
> >
> > > wrote:
> > >
> > >> +1
> > >>
> > >> > On Nov 29, 2017, at 09:48, Alek Storm <al...@gmail.com> wrote:
> > >> >
> > >> > +1
> > >> >
> > >> > On Wed, Nov 29, 2017 at 9:50 AM, Igors Vaitkus <
> > >> litdeviant@protonmail.com>
> > >> > wrote:
> > >> >
> > >> >> +1
> > >> >>
> > >> >> On 11/29/17, 3:48 PM, "Kate-Laurel Agnew" <ka...@signal.co>
> wrote:
> > >> >>
> > >> >>    +1
> > >> >>
> > >> >>    On Wed, Nov 29, 2017 at 12:09 AM, Koen Mevissen <
> > >> kmevissen@travix.com>
> > >> >>    wrote:
> > >> >>
> > >> >>> +1
> > >> >>>
> > >> >>> I'm interested as well!
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>> Op di 28 nov. 2017 om 14:04 schreef Marc Bollinger <
> > >> >> marc@lumoslabs.com>
> > >> >>>
> > >> >>>> +1
> > >> >>>>
> > >> >>>> On Mon, Nov 27, 2017 at 6:18 PM, Ruslan Dautkhanov <
> > >> >> dautkhanov@gmail.com
> > >> >>>>
> > >> >>>> wrote:
> > >> >>>>
> > >> >>>>> ‘’’
> > >> >>>>> I'm
> > >> >>>>> now working on sql scanners, extractors and other tools that
> > >> >> allow me
> > >> >>> to
> > >> >>>>> populate the database
> > >> >>>>> ‘’’
> > >> >>>>>
> > >> >>>>> Very cool. Cloudera Navigator ( not an open source product) does
> > >> >> this
> > >> >>> too
> > >> >>>>> to some extent - collect metadata and create data lineage
> > >> >>> automatically (
> > >> >>>>> stored as a Solr collection) by parsing sql queries.
> > >> >>>>>
> > >> >>>>> https://www.cloudera.com/documentation/enterprise/5-12-
> > >> >>>>> x/topics/datamgmt_extraction_indexing.html
> > >> >>>>>
> > >> >>>>>
> > >> >>>>>
> > >> >>>>> On Mon, Nov 27, 2017 at 12:38 PM Gerard Toonstra <
> > >> >> gtoonstra@gmail.com>
> > >> >>>>> wrote:
> > >> >>>>>
> > >> >>>>>> Hi all,
> > >> >>>>>>
> > >> >>>>>> So something that really drew my attention recently is a "data
> > >> >>> portal"
> > >> >>>>> as
> > >> >>>>>> described by a team from airbnb somewhere in May. The idea is
> > >> >>>> basically a
> > >> >>>>>> "facebook of data":
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> https://medium.com/airbnb-engineering/democratizing-
> > >> >>>>> data-at-airbnb-852d76c51770
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> Unfortunately it looks like it's not going to be opensourced
> > >> >> due to
> > >> >>> how
> > >> >>>>>> heavily integrated it is with their specific infrastructure;
> > >> >> but the
> > >> >>>> idea
> > >> >>>>>> itself to me sounds like it's something every organization of a
> > >> >>> certain
> > >> >>>>>> size should have to keep track of data and stay informed as an
> > >> >>>>>> organization.
> > >> >>>>>>
> > >> >>>>>> Based on the descriptions, I prototyped some things away and
> > >> >> am happy
> > >> >>>>> with
> > >> >>>>>> the results and the speed that something like this can be
> > >> >>> constructed.
> > >> >>>>> I'm
> > >> >>>>>> now working on sql scanners, extractors and other tools that
> > >> >> allow me
> > >> >>>> to
> > >> >>>>>> populate the database and put a poc together on some real data.
> > >> >>>>>>
> > >> >>>>>> If other people have similar concerns in their organization
> > >> >> and think
> > >> >>>>> this
> > >> >>>>>> would be a great thing to have, reply to me or the list; with
> > >> >>>> sufficient
> > >> >>>>>> interest I may set up a web chat/meet session so this can be
> > >> >>> discussed
> > >> >>>> in
> > >> >>>>>> more detail and find ways to progress this.
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> Best regards,
> > >> >>>>>>
> > >> >>>>>> Gerard
> > >> >>>>>>
> > >> >>>>>
> > >> >>>>
> > >> >>> --
> > >> >>> Kind regards,
> > >> >>> Met vriendelijke groet,
> > >> >>>
> > >> >>> *Koen Mevissen*
> > >> >>> Principal BI Developer
> > >> >>>
> > >> >>>
> > >> >>> *Travix Nederland B.V.*
> > >> >>> Piet Heinkade 55
> > >> >>> 1019 GM Amsterdam
> > >> >>> The Netherlands
> > >> >>>
> > >> >>> T. +31 (0)20 203 3241
> > >> >>> E: KMevissen@travix.com
> > >> >>> www.travix.com
> > >> >>>
> > >> >>> *Brands: * CheapTickets  |  Vliegwinkel  |  Vayama  |  BudgetAir
> |
> > >> >>> Flugladen
> > >> >>>
> > >> >>
> > >> >>
> > >> >>
> > >> >>    --
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > >> >>    *Kate-Laurel AgnewData Engineerm: 503-741-9207
> > >> >>    <503%20741%209207>e: kagnew@signal.co
> > >> >>    <https://mail.google.com/mail/?view=cm&fs=1&tf=1&to=kagnew@s
> > >> ignal.co>
> > >> >> signal.co
> > >> >>    <http://www.google.com/url?q=http%3A%2F%2Fsignal.co%2F&sa=
> > >> >> D&sntz=1&usg=AFQjCNGG8nAqplB1u72dXPbYfPYRcfopNQ>____________
> > >> ____________
> > >> >> Cut
> > >> >>    Through the NoiseThis e-mail and any files transmitted with it
> are
> > >> for
> > >> >> the
> > >> >>    sole use of the intended recipient(s) and may contain
> confidential
> > >> and
> > >> >>    privileged information. Any unauthorized use of this email is
> > >> strictly
> > >> >>    prohibited. ©2015 Signal. All rights reserved.*
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > >>
> > >
> > >
> >
>

Re: Data lineage and data portal

Posted by Sam Elamin <hu...@gmail.com>.
I'm def in.

Thanks for organising Gerard!

On Sun, 3 Dec 2017 at 07:42, Gerard Toonstra <gt...@gmail.com> wrote:

> Good morning,
>
> The meeting has been scheduled for wednesday 6th december:
>
> London 17:00:00 UTC, Amsterdam 18:00:00 UTC+1, San Francisco 09:00:00 PST,
> New York 12:00:00 EST
>
> --------
>
> Gerard Toonstra is inviting you to a scheduled Zoom meeting.
>
> Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/789615759
> <
> https://www.google.com/url?q=https%3A%2F%2Fzoom.us%2Fj%2F789615759&sa=D&usd=2&usg=AFQjCNFfEIitKjiD6AjsxPB5ddfMcyLv5Q
> >
>
>
> Or iPhone one-tap :
>     US: +16699006833,,789615759# or +16468769923,,789615759#
> Or Telephone:
>     Dial(for higher quality, dial a number based on your current location):
>         US: +1 669 900 6833 or +1 646 876 9923
>     Meeting ID: 789 615 759
>     International numbers available:
> https://zoom.us/zoomconference?m=5eYALpFJGSSZWrP577vUpbK_0dP7WPfp
> <
> https://www.google.com/url?q=https%3A%2F%2Fzoom.us%2Fzoomconference%3Fm%3D5eYALpFJGSSZWrP577vUpbK_0dP7WPfp&sa=D&usd=2&usg=AFQjCNE41pjZbgmI9NNAqOi_6ah3BufM1A
> >
>
>
> -------
>
> Best regards,
>
> Gerard
>
>
> On Thu, Nov 30, 2017 at 9:05 PM, Gerard Toonstra <gt...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > Nice overwhelming response! :)
> >
> > I'd like to host a meeting on zoom to discuss. I'm on the free plan
> there,
> > so we are limited to 40 minutes! :)  (msg me priv if you have better
> > hosting options).
> > Here's a way to let me know your availability without polluting the
> > mailing list:
> >
> > https://docs.google.com/forms/d/e/1FAIpQLSfsvfM6mUVwD_
> > qV9EXdUB1aLcdKt55toYKRp3JDanonoFRSLQ/viewform?usp=sf_link
> >
> > I'll record the meeting and share the video for those who cannot attend.
> >
> > Agenda:
> > - What specifically is so useful about the data portal?
> > - ( If anyone from airbnb attends, they get the floor to give us more
> > context and info )
> > - Is there sufficient interest and value creation through a
> collaboration?
> > - How would a data portal complement airflow and vice versa?
> > - How do we progress this further and which actions can I take here?
> >
> > You can prepare for the meeting by reading the blog post or watching the
> > video (which has even more detail):
> >
> > https://medium.com/airbnb-engineering/democratizing-data-at-
> > airbnb-852d76c51770
> >
> > https://www.youtube.com/watch?v=gayXC2FDSiA&t=4s
> >
> > Looking forward to your availability responses.
> >
> > Best Regards,
> >
> > Gerard
> >
> >
> > On Wed, Nov 29, 2017 at 7:31 PM, Nathan McIntyre <nw...@gmail.com>
> > wrote:
> >
> >> +1
> >>
> >> > On Nov 29, 2017, at 09:48, Alek Storm <al...@gmail.com> wrote:
> >> >
> >> > +1
> >> >
> >> > On Wed, Nov 29, 2017 at 9:50 AM, Igors Vaitkus <
> >> litdeviant@protonmail.com>
> >> > wrote:
> >> >
> >> >> +1
> >> >>
> >> >> On 11/29/17, 3:48 PM, "Kate-Laurel Agnew" <ka...@signal.co> wrote:
> >> >>
> >> >>    +1
> >> >>
> >> >>    On Wed, Nov 29, 2017 at 12:09 AM, Koen Mevissen <
> >> kmevissen@travix.com>
> >> >>    wrote:
> >> >>
> >> >>> +1
> >> >>>
> >> >>> I'm interested as well!
> >> >>>
> >> >>>
> >> >>>
> >> >>> Op di 28 nov. 2017 om 14:04 schreef Marc Bollinger <
> >> >> marc@lumoslabs.com>
> >> >>>
> >> >>>> +1
> >> >>>>
> >> >>>> On Mon, Nov 27, 2017 at 6:18 PM, Ruslan Dautkhanov <
> >> >> dautkhanov@gmail.com
> >> >>>>
> >> >>>> wrote:
> >> >>>>
> >> >>>>> ‘’’
> >> >>>>> I'm
> >> >>>>> now working on sql scanners, extractors and other tools that
> >> >> allow me
> >> >>> to
> >> >>>>> populate the database
> >> >>>>> ‘’’
> >> >>>>>
> >> >>>>> Very cool. Cloudera Navigator ( not an open source product) does
> >> >> this
> >> >>> too
> >> >>>>> to some extent - collect metadata and create data lineage
> >> >>> automatically (
> >> >>>>> stored as a Solr collection) by parsing sql queries.
> >> >>>>>
> >> >>>>> https://www.cloudera.com/documentation/enterprise/5-12-
> >> >>>>> x/topics/datamgmt_extraction_indexing.html
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> On Mon, Nov 27, 2017 at 12:38 PM Gerard Toonstra <
> >> >> gtoonstra@gmail.com>
> >> >>>>> wrote:
> >> >>>>>
> >> >>>>>> Hi all,
> >> >>>>>>
> >> >>>>>> So something that really drew my attention recently is a "data
> >> >>> portal"
> >> >>>>> as
> >> >>>>>> described by a team from airbnb somewhere in May. The idea is
> >> >>>> basically a
> >> >>>>>> "facebook of data":
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> https://medium.com/airbnb-engineering/democratizing-
> >> >>>>> data-at-airbnb-852d76c51770
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> Unfortunately it looks like it's not going to be opensourced
> >> >> due to
> >> >>> how
> >> >>>>>> heavily integrated it is with their specific infrastructure;
> >> >> but the
> >> >>>> idea
> >> >>>>>> itself to me sounds like it's something every organization of a
> >> >>> certain
> >> >>>>>> size should have to keep track of data and stay informed as an
> >> >>>>>> organization.
> >> >>>>>>
> >> >>>>>> Based on the descriptions, I prototyped some things away and
> >> >> am happy
> >> >>>>> with
> >> >>>>>> the results and the speed that something like this can be
> >> >>> constructed.
> >> >>>>> I'm
> >> >>>>>> now working on sql scanners, extractors and other tools that
> >> >> allow me
> >> >>>> to
> >> >>>>>> populate the database and put a poc together on some real data.
> >> >>>>>>
> >> >>>>>> If other people have similar concerns in their organization
> >> >> and think
> >> >>>>> this
> >> >>>>>> would be a great thing to have, reply to me or the list; with
> >> >>>> sufficient
> >> >>>>>> interest I may set up a web chat/meet session so this can be
> >> >>> discussed
> >> >>>> in
> >> >>>>>> more detail and find ways to progress this.
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> Best regards,
> >> >>>>>>
> >> >>>>>> Gerard
> >> >>>>>>
> >> >>>>>
> >> >>>>
> >> >>> --
> >> >>> Kind regards,
> >> >>> Met vriendelijke groet,
> >> >>>
> >> >>> *Koen Mevissen*
> >> >>> Principal BI Developer
> >> >>>
> >> >>>
> >> >>> *Travix Nederland B.V.*
> >> >>> Piet Heinkade 55
> >> >>> 1019 GM Amsterdam
> >> >>> The Netherlands
> >> >>>
> >> >>> T. +31 (0)20 203 3241
> >> >>> E: KMevissen@travix.com
> >> >>> www.travix.com
> >> >>>
> >> >>> *Brands: * CheapTickets  |  Vliegwinkel  |  Vayama  |  BudgetAir  |
> >> >>> Flugladen
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >>    --
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>    *Kate-Laurel AgnewData Engineerm: 503-741-9207
> >> >>    <503%20741%209207>e: kagnew@signal.co
> >> >>    <https://mail.google.com/mail/?view=cm&fs=1&tf=1&to=kagnew@s
> >> ignal.co>
> >> >> signal.co
> >> >>    <http://www.google.com/url?q=http%3A%2F%2Fsignal.co%2F&sa=
> >> >> D&sntz=1&usg=AFQjCNGG8nAqplB1u72dXPbYfPYRcfopNQ>____________
> >> ____________
> >> >> Cut
> >> >>    Through the NoiseThis e-mail and any files transmitted with it are
> >> for
> >> >> the
> >> >>    sole use of the intended recipient(s) and may contain confidential
> >> and
> >> >>    privileged information. Any unauthorized use of this email is
> >> strictly
> >> >>    prohibited. ©2015 Signal. All rights reserved.*
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >>
> >
> >
>