You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@superset.apache.org by Andy Clapson <an...@slice.is> on 2020/06/18 16:13:45 UTC
Re: [EXTERNAL] Re: Addressing Charts directly / how to handle
embedding in dev, stage, prod cycle?
A big +1 on this one đź‘Ť.
UUIDs would solve some definite deployment challenges on this one -
mainly importing a pre-built set of Dashboards into many different
environments.
Andy
On 2020-06-18 12:11, Maxime Beauchemin wrote:
> I meant uuid in its pure form (just a universal unique identifier), not a
> fingerprint/hash of the object.
>
> Currently the importer has to keep track of the "export_object_id" to try
> to upsert properly if the same object is re-imported. It can have collision
> issue when multiple systems are involved and has other disadvantages.
>
> Max
>
> On Thu, Jun 18, 2020 at 6:35 AM Dennis Meyer <de...@jdeluxe.org> wrote:
>
>> @Eric:
>> Thanks for the feedback Eric. It’s good to know. We’ll keep it in the
>> options list for now as this would also have some advantages :-)
>>
>> @Max:
>> Removing the slice id and passing all parameters works, that’s really
>> useful information. We might stick with this until slugs are available.
>> Thanks very much!
>> The charts can be static, but we need filtering on certain IDs. Something
>> like a product ID filter for example. But this seem to work ok with the
>> approach by adding a simple filter :-)
>>
>> Rison would be awesome for debugging and developing for sure!Even though
>> UUIDs would solve our problem they can get a pain pretty quick if not
>> architected very thoroughly. You have to use hashes in a consistent way
>> (which data/column do you use? What happens with added columns on DDL
>> updates etc) and dealing with those is non trivial when there’s no good
>> transportable way in generating those across application and DBs. Slug
>> seems to abstract the problem [at least the one we have] a lot better and
>> simpler at the same time.
>>
>> Dennis
>>
>>> On 18. Jun 2020, at 02:38, Erik Ritter <er...@gmail.com> wrote:
>>>
>>> Hi Dennis,
>>>
>>> You mention that cloning the complete Superset database seems overkill
>>> here, but for our use case, we've found that copying almost the entire
>>> Superset db into our dev and staging environments solves these types of
>>> issues for us. To ensure the process doesn't take too long, we exclude
>> the
>>> `logs` and the `queries` tables from our DB dump, but everything else we
>>> copy straight from prod to dev and staging for testing purposes.
>>>
>>> I know this doesn't address a case where you build a chart in dev and
>> want
>>> to move it to prod, but that's not a pattern we've really seen or needed
>> in
>>> our deployment.
>>>
>>> Hope this helps!
>>> Erik
>>>
>>> On Wed, Jun 17, 2020 at 7:46 AM Maxime Beauchemin <
>>> maximebeauchemin@gmail.com> wrote:
>>>
>>>> Few thoughts:
>>>> - did you try removing the id? this should work without the id since
>> it's
>>>> totally possible to have an explore session without a saved chart
>>>> - we spoke in the past about introducing *uuids* for
>> exportable/importable
>>>> objects that would be consistent across superset instances
>>>> - slugs for chart would help here, and should be easy to implement - we
>>>> need that regardless
>>>> - side note: it'd be great to userison here for more readable URL
>> encoding
>>>> - more generally - should an embeddable snippet point to a chart or a
>> state
>>>> of an explore session? I think both use-cases make sense
>>>>
>>>> On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org>
>> wrote:
>>>>> Sorry for broken formatting. The URL contains an “slice_id”, see edited
>>>>> mail below for better readability.
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> In the charts there’s an explicit embedding functionality generating
>>>>> Iframe code:
>>>>>> <iframe
>>>>>> width="600"
>>>>>> height="400"
>>>>>> seamless
>>>>>> frameBorder="0"
>>>>>> scrolling="no"
>>>>> src="…slice_id=306…”
>>>>>
>>>>>
>>>>>> </iframe>
>>>>>>
>>>>>> It seems like there’s an ID hardcoded (see above) - which is very
>>>>> unfortunate when exporting and importing again and the embedding is in
>>>>> another application. In this case ids are different after import.
>>>>>> I found a ticket about adding Slug to a chart, but it didn’t get too
>>>>> much attention.
>>>>>> https://github.com/apache/incubator-superset/issues/6537 <
>>>>> https://github.com/apache/incubator-superset/issues/6537>
>>>>>> Is there currently a way to address this better if I wanted to use
>>>>> embedding code in dev, staging and prod environments?
>>>>>> Cloning the complete Superset database seems overkill for this. Also
>> it
>>>>> won’t really work if you let production add charts by users and create
>>>> some
>>>>> major charts in dev you want to move to prod after some thorough
>> testing.
>>>>>> I’m glad for any suggestion.
>>>>>>
>>>>>> Thanks,
>>>>>> Dennis
>>>>>>
>>>>>>
>>>>>
>>
Re: [EXTERNAL] Re: Addressing Charts directly / how to handle
embedding in dev, stage, prod cycle?
Posted by Maxime Beauchemin <ma...@preset.io>.
For reference, I did some work around introducing uuids in the past, but
never got it through https://github.com/apache/incubator-superset/pull/7829.
On Thu, Jun 18, 2020 at 9:16 AM Dennis Meyer <de...@jdeluxe.org> wrote:
> Ok, I understand, this definitely makes good sense!
> This would definitely be a great help for different envs.
>
> Dennis
>
> > On 18. Jun 2020, at 18:13, Andy Clapson <an...@slice.is> wrote:
> >
> > A big +1 on this one đź‘Ť.
> >
> > UUIDs would solve some definite deployment challenges on this one -
> mainly importing a pre-built set of Dashboards into many different
> environments.
> >
> > Andy
> >
> > On 2020-06-18 12:11, Maxime Beauchemin wrote:
> >> I meant uuid in its pure form (just a universal unique identifier), not
> a
> >> fingerprint/hash of the object.
> >>
> >> Currently the importer has to keep track of the "export_object_id" to
> try
> >> to upsert properly if the same object is re-imported. It can have
> collision
> >> issue when multiple systems are involved and has other disadvantages.
> >>
> >> Max
> >>
> >> On Thu, Jun 18, 2020 at 6:35 AM Dennis Meyer <de...@jdeluxe.org>
> wrote:
> >>
> >>> @Eric:
> >>> Thanks for the feedback Eric. It’s good to know. We’ll keep it in the
> >>> options list for now as this would also have some advantages :-)
> >>>
> >>> @Max:
> >>> Removing the slice id and passing all parameters works, that’s really
> >>> useful information. We might stick with this until slugs are available.
> >>> Thanks very much!
> >>> The charts can be static, but we need filtering on certain IDs.
> Something
> >>> like a product ID filter for example. But this seem to work ok with the
> >>> approach by adding a simple filter :-)
> >>>
> >>> Rison would be awesome for debugging and developing for sure!Even
> though
> >>> UUIDs would solve our problem they can get a pain pretty quick if not
> >>> architected very thoroughly. You have to use hashes in a consistent way
> >>> (which data/column do you use? What happens with added columns on DDL
> >>> updates etc) and dealing with those is non trivial when there’s no good
> >>> transportable way in generating those across application and DBs. Slug
> >>> seems to abstract the problem [at least the one we have] a lot better
> and
> >>> simpler at the same time.
> >>>
> >>> Dennis
> >>>
> >>>> On 18. Jun 2020, at 02:38, Erik Ritter <er...@gmail.com>
> wrote:
> >>>>
> >>>> Hi Dennis,
> >>>>
> >>>> You mention that cloning the complete Superset database seems overkill
> >>>> here, but for our use case, we've found that copying almost the entire
> >>>> Superset db into our dev and staging environments solves these types
> of
> >>>> issues for us. To ensure the process doesn't take too long, we exclude
> >>> the
> >>>> `logs` and the `queries` tables from our DB dump, but everything else
> we
> >>>> copy straight from prod to dev and staging for testing purposes.
> >>>>
> >>>> I know this doesn't address a case where you build a chart in dev and
> >>> want
> >>>> to move it to prod, but that's not a pattern we've really seen or
> needed
> >>> in
> >>>> our deployment.
> >>>>
> >>>> Hope this helps!
> >>>> Erik
> >>>>
> >>>> On Wed, Jun 17, 2020 at 7:46 AM Maxime Beauchemin <
> >>>> maximebeauchemin@gmail.com> wrote:
> >>>>
> >>>>> Few thoughts:
> >>>>> - did you try removing the id? this should work without the id since
> >>> it's
> >>>>> totally possible to have an explore session without a saved chart
> >>>>> - we spoke in the past about introducing *uuids* for
> >>> exportable/importable
> >>>>> objects that would be consistent across superset instances
> >>>>> - slugs for chart would help here, and should be easy to implement -
> we
> >>>>> need that regardless
> >>>>> - side note: it'd be great to userison here for more readable URL
> >>> encoding
> >>>>> - more generally - should an embeddable snippet point to a chart or a
> >>> state
> >>>>> of an explore session? I think both use-cases make sense
> >>>>>
> >>>>> On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org>
> >>> wrote:
> >>>>>> Sorry for broken formatting. The URL contains an “slice_id”, see
> edited
> >>>>>> mail below for better readability.
> >>>>>>
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> In the charts there’s an explicit embedding functionality
> generating
> >>>>>> Iframe code:
> >>>>>>> <iframe
> >>>>>>> width="600"
> >>>>>>> height="400"
> >>>>>>> seamless
> >>>>>>> frameBorder="0"
> >>>>>>> scrolling="no"
> >>>>>> src="…slice_id=306…”
> >>>>>>
> >>>>>>
> >>>>>>> </iframe>
> >>>>>>>
> >>>>>>> It seems like there’s an ID hardcoded (see above) - which is very
> >>>>>> unfortunate when exporting and importing again and the embedding is
> in
> >>>>>> another application. In this case ids are different after import.
> >>>>>>> I found a ticket about adding Slug to a chart, but it didn’t get
> too
> >>>>>> much attention.
> >>>>>>> https://github.com/apache/incubator-superset/issues/6537 <
> >>>>>> https://github.com/apache/incubator-superset/issues/6537>
> >>>>>>> Is there currently a way to address this better if I wanted to use
> >>>>>> embedding code in dev, staging and prod environments?
> >>>>>>> Cloning the complete Superset database seems overkill for this.
> Also
> >>> it
> >>>>>> won’t really work if you let production add charts by users and
> create
> >>>>> some
> >>>>>> major charts in dev you want to move to prod after some thorough
> >>> testing.
> >>>>>>> I’m glad for any suggestion.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Dennis
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>
>
>
Re: [EXTERNAL] Re: Addressing Charts directly / how to handle
embedding in dev, stage, prod cycle?
Posted by Dennis Meyer <de...@jdeluxe.org>.
Ok, I understand, this definitely makes good sense!
This would definitely be a great help for different envs.
Dennis
> On 18. Jun 2020, at 18:13, Andy Clapson <an...@slice.is> wrote:
>
> A big +1 on this one đź‘Ť.
>
> UUIDs would solve some definite deployment challenges on this one - mainly importing a pre-built set of Dashboards into many different environments.
>
> Andy
>
> On 2020-06-18 12:11, Maxime Beauchemin wrote:
>> I meant uuid in its pure form (just a universal unique identifier), not a
>> fingerprint/hash of the object.
>>
>> Currently the importer has to keep track of the "export_object_id" to try
>> to upsert properly if the same object is re-imported. It can have collision
>> issue when multiple systems are involved and has other disadvantages.
>>
>> Max
>>
>> On Thu, Jun 18, 2020 at 6:35 AM Dennis Meyer <de...@jdeluxe.org> wrote:
>>
>>> @Eric:
>>> Thanks for the feedback Eric. It’s good to know. We’ll keep it in the
>>> options list for now as this would also have some advantages :-)
>>>
>>> @Max:
>>> Removing the slice id and passing all parameters works, that’s really
>>> useful information. We might stick with this until slugs are available.
>>> Thanks very much!
>>> The charts can be static, but we need filtering on certain IDs. Something
>>> like a product ID filter for example. But this seem to work ok with the
>>> approach by adding a simple filter :-)
>>>
>>> Rison would be awesome for debugging and developing for sure!Even though
>>> UUIDs would solve our problem they can get a pain pretty quick if not
>>> architected very thoroughly. You have to use hashes in a consistent way
>>> (which data/column do you use? What happens with added columns on DDL
>>> updates etc) and dealing with those is non trivial when there’s no good
>>> transportable way in generating those across application and DBs. Slug
>>> seems to abstract the problem [at least the one we have] a lot better and
>>> simpler at the same time.
>>>
>>> Dennis
>>>
>>>> On 18. Jun 2020, at 02:38, Erik Ritter <er...@gmail.com> wrote:
>>>>
>>>> Hi Dennis,
>>>>
>>>> You mention that cloning the complete Superset database seems overkill
>>>> here, but for our use case, we've found that copying almost the entire
>>>> Superset db into our dev and staging environments solves these types of
>>>> issues for us. To ensure the process doesn't take too long, we exclude
>>> the
>>>> `logs` and the `queries` tables from our DB dump, but everything else we
>>>> copy straight from prod to dev and staging for testing purposes.
>>>>
>>>> I know this doesn't address a case where you build a chart in dev and
>>> want
>>>> to move it to prod, but that's not a pattern we've really seen or needed
>>> in
>>>> our deployment.
>>>>
>>>> Hope this helps!
>>>> Erik
>>>>
>>>> On Wed, Jun 17, 2020 at 7:46 AM Maxime Beauchemin <
>>>> maximebeauchemin@gmail.com> wrote:
>>>>
>>>>> Few thoughts:
>>>>> - did you try removing the id? this should work without the id since
>>> it's
>>>>> totally possible to have an explore session without a saved chart
>>>>> - we spoke in the past about introducing *uuids* for
>>> exportable/importable
>>>>> objects that would be consistent across superset instances
>>>>> - slugs for chart would help here, and should be easy to implement - we
>>>>> need that regardless
>>>>> - side note: it'd be great to userison here for more readable URL
>>> encoding
>>>>> - more generally - should an embeddable snippet point to a chart or a
>>> state
>>>>> of an explore session? I think both use-cases make sense
>>>>>
>>>>> On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org>
>>> wrote:
>>>>>> Sorry for broken formatting. The URL contains an “slice_id”, see edited
>>>>>> mail below for better readability.
>>>>>>
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> In the charts there’s an explicit embedding functionality generating
>>>>>> Iframe code:
>>>>>>> <iframe
>>>>>>> width="600"
>>>>>>> height="400"
>>>>>>> seamless
>>>>>>> frameBorder="0"
>>>>>>> scrolling="no"
>>>>>> src="…slice_id=306…”
>>>>>>
>>>>>>
>>>>>>> </iframe>
>>>>>>>
>>>>>>> It seems like there’s an ID hardcoded (see above) - which is very
>>>>>> unfortunate when exporting and importing again and the embedding is in
>>>>>> another application. In this case ids are different after import.
>>>>>>> I found a ticket about adding Slug to a chart, but it didn’t get too
>>>>>> much attention.
>>>>>>> https://github.com/apache/incubator-superset/issues/6537 <
>>>>>> https://github.com/apache/incubator-superset/issues/6537>
>>>>>>> Is there currently a way to address this better if I wanted to use
>>>>>> embedding code in dev, staging and prod environments?
>>>>>>> Cloning the complete Superset database seems overkill for this. Also
>>> it
>>>>>> won’t really work if you let production add charts by users and create
>>>>> some
>>>>>> major charts in dev you want to move to prod after some thorough
>>> testing.
>>>>>>> I’m glad for any suggestion.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dennis
>>>>>>>
>>>>>>>
>>>>>>
>>>