You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@superset.apache.org by Dennis Meyer <de...@jdeluxe.org> on 2020/06/16 19:32:25 UTC
Addressing Charts directly / how to handle embedding in dev, stage,
prod cycle?
Hi,
In the charts there’s an explicit embedding functionality generating Iframe code:
<iframe
width="600"
height="400"
seamless
frameBorder="0"
scrolling="no"
src="http://www.my.url.com/superset/explore/?form_data=%7B%22datasource%22%3A%222__table%22%2C%22viz_type%22%3A%22big_number%22%2C%22 <http://superset.internal.us-east-1.production.spearad.video/superset/explore/?form_data={%22datasource%22:%222__table%22,%22viz_type%22:%22big_number%22,%22slice_id%22:306,%22url_params%22:{},%22time_range_endpoints%22:[%22unknown%22,%22inclusive%22],%22granularity_sqla%22:%22delivery_date%22,%22time_grain_sqla%22:%22P1Y%22,%22time_range%22:%22No+filter%22,%22metric%22:{%22aggregate%22:null,%22column%22:null,%22expressionType%22:%22SQL%22,%22fromFormData%22:true,%22hasCustomLabel%22:true,%22label%22:%22Revenue%22,%22optionName%22:%22metric_p07aort2umb_gyzmhz31uns%22,%22sqlExpression%22:%22SUM(revenue)/1000000%22},%22adhoc_filters%22:[],%22y_axis_format%22:%22$,.2f%22,%22show_trend_line%22:false,%22start_y_axis_at_zero%22:false,%22color_picker%22:{%22a%22:1,%22b%22:212,%22g%22:184,%22r%22:186},%22header_font_size%22:0.5,%22subheader_font_size%22:0.125,%22rolling_type%22:%22None%22}&standalone=true&height=400>slice_id <http://superset.internal.us-east-1.production.spearad.video/superset/explore/?form_data={%22datasource%22:%222__table%22,%22viz_type%22:%22big_number%22,%22slice_id%22:306,%22url_params%22:{},%22time_range_endpoints%22:[%22unknown%22,%22inclusive%22],%22granularity_sqla%22:%22delivery_date%22,%22time_grain_sqla%22:%22P1Y%22,%22time_range%22:%22No+filter%22,%22metric%22:{%22aggregate%22:null,%22column%22:null,%22expressionType%22:%22SQL%22,%22fromFormData%22:true,%22hasCustomLabel%22:true,%22label%22:%22Revenue%22,%22optionName%22:%22metric_p07aort2umb_gyzmhz31uns%22,%22sqlExpression%22:%22SUM(revenue)/1000000%22},%22adhoc_filters%22:[],%22y_axis_format%22:%22$,.2f%22,%22show_trend_line%22:false,%22start_y_axis_at_zero%22:false,%22color_picker%22:{%22a%22:1,%22b%22:212,%22g%22:184,%22r%22:186},%22header_font_size%22:0.5,%22subheader_font_size%22:0.125,%22rolling_type%22:%22None%22}&standalone=true&height=400>%22%3A <http://superset.internal.us-east-1.production.spearad.video/superset/explore/?form_data={%22datasource%22:%222__table%22,%22viz_type%22:%22big_number%22,%22slice_id%22:306,%22url_params%22:{},%22time_range_endpoints%22:[%22unknown%22,%22inclusive%22],%22granularity_sqla%22:%22delivery_date%22,%22time_grain_sqla%22:%22P1Y%22,%22time_range%22:%22No+filter%22,%22metric%22:{%22aggregate%22:null,%22column%22:null,%22expressionType%22:%22SQL%22,%22fromFormData%22:true,%22hasCustomLabel%22:true,%22label%22:%22Revenue%22,%22optionName%22:%22metric_p07aort2umb_gyzmhz31uns%22,%22sqlExpression%22:%22SUM(revenue)/1000000%22},%22adhoc_filters%22:[],%22y_axis_format%22:%22$,.2f%22,%22show_trend_line%22:false,%22start_y_axis_at_zero%22:false,%22color_picker%22:{%22a%22:1,%22b%22:212,%22g%22:184,%22r%22:186},%22header_font_size%22:0.5,%22subheader_font_size%22:0.125,%22rolling_type%22:%22None%22}&standalone=true&height=400>306 <http://superset.internal.us-east-1.production.spearad.video/superset/explore/?form_data={%22datasource%22:%222__table%22,%22viz_type%22:%22big_number%22,%22slice_id%22:306,%22url_params%22:{},%22time_range_endpoints%22:[%22unknown%22,%22inclusive%22],%22granularity_sqla%22:%22delivery_date%22,%22time_grain_sqla%22:%22P1Y%22,%22time_range%22:%22No+filter%22,%22metric%22:{%22aggregate%22:null,%22column%22:null,%22expressionType%22:%22SQL%22,%22fromFormData%22:true,%22hasCustomLabel%22:true,%22label%22:%22Revenue%22,%22optionName%22:%22metric_p07aort2umb_gyzmhz31uns%22,%22sqlExpression%22:%22SUM(revenue)/1000000%22},%22adhoc_filters%22:[],%22y_axis_format%22:%22$,.2f%22,%22show_trend_line%22:false,%22start_y_axis_at_zero%22:false,%22color_picker%22:{%22a%22:1,%22b%22:212,%22g%22:184,%22r%22:186},%22header_font_size%22:0.5,%22subheader_font_size%22:0.125,%22rolling_type%22:%22None%22}&standalone=true&height=400>%2C%22url_params%22%3A%7B%7D%2C%22time_range_endpoints%22%3A%5B%22unknown%22%2C%22inclusive%22%5D%2C%22granularity_sqla%22%3A%22delivery_date%22%2C%22time_grain_sqla%22%3A%22P1Y%22%2C%22time_range%22%3A%22No+filter%22%2C%22metric%22%3A%7B%22aggregate%22%3Anull%2C%22column%22%3Anull%2C%22expressionType%22%3A%22SQL%22%2C%22fromFormData%22%3Atrue%2C%22hasCustomLabel%22%3Atrue%2C%22label%22%3A%22Revenue%22%2C%22optionName%22%3A%22metric_p07aort2umb_gyzmhz31uns%22%2C%22sqlExpression%22%3A%22SUM%28revenue%29%2F1000000%22%7D%2C%22adhoc_filters%22%3A%5B%5D%2C%22y_axis_format%22%3A%22%24%2C.2f%22%2C%22show_trend_line%22%3Afalse%2C%22start_y_axis_at_zero%22%3Afalse%2C%22color_picker%22%3A%7B%22a%22%3A1%2C%22b%22%3A212%2C%22g%22%3A184%2C%22r%22%3A186%7D%2C%22header_font_size%22%3A0.5%2C%22subheader_font_size%22%3A0.125%2C%22rolling_type%22%3A%22None%22%7D&standalone=true&height=400 <http://superset.internal.us-east-1.production.spearad.video/superset/explore/?form_data={%22datasource%22:%222__table%22,%22viz_type%22:%22big_number%22,%22slice_id%22:306,%22url_params%22:{},%22time_range_endpoints%22:[%22unknown%22,%22inclusive%22],%22granularity_sqla%22:%22delivery_date%22,%22time_grain_sqla%22:%22P1Y%22,%22time_range%22:%22No+filter%22,%22metric%22:{%22aggregate%22:null,%22column%22:null,%22expressionType%22:%22SQL%22,%22fromFormData%22:true,%22hasCustomLabel%22:true,%22label%22:%22Revenue%22,%22optionName%22:%22metric_p07aort2umb_gyzmhz31uns%22,%22sqlExpression%22:%22SUM(revenue)/1000000%22},%22adhoc_filters%22:[],%22y_axis_format%22:%22$,.2f%22,%22show_trend_line%22:false,%22start_y_axis_at_zero%22:false,%22color_picker%22:{%22a%22:1,%22b%22:212,%22g%22:184,%22r%22:186},%22header_font_size%22:0.5,%22subheader_font_size%22:0.125,%22rolling_type%22:%22None%22}&standalone=true&height=400>"
>
</iframe>
It seems like there’s an ID hardcoded (marked red above) - which is very unfortunate when exporting and importing again and the embedding is in another application. In this case ids are different after import.
I found a ticket about adding Slug to a chart, but it didn’t get too much attention.
https://github.com/apache/incubator-superset/issues/6537 <https://github.com/apache/incubator-superset/issues/6537>
Is there currently a way to address this better if I wanted to use embedding code in dev, staging and prod environments?
Cloning the complete Superset database seems overkill for this. Also it won’t really work if you let production add charts by users and create some major charts in dev you want to move to prod after some thorough testing.
I’m glad for any suggestion.
Thanks,
Dennis
Re: [EXTERNAL] Re: Addressing Charts directly / how to handle
embedding in dev, stage, prod cycle?
Posted by Maxime Beauchemin <ma...@preset.io>.
For reference, I did some work around introducing uuids in the past, but
never got it through https://github.com/apache/incubator-superset/pull/7829.
On Thu, Jun 18, 2020 at 9:16 AM Dennis Meyer <de...@jdeluxe.org> wrote:
> Ok, I understand, this definitely makes good sense!
> This would definitely be a great help for different envs.
>
> Dennis
>
> > On 18. Jun 2020, at 18:13, Andy Clapson <an...@slice.is> wrote:
> >
> > A big +1 on this one 👍.
> >
> > UUIDs would solve some definite deployment challenges on this one -
> mainly importing a pre-built set of Dashboards into many different
> environments.
> >
> > Andy
> >
> > On 2020-06-18 12:11, Maxime Beauchemin wrote:
> >> I meant uuid in its pure form (just a universal unique identifier), not
> a
> >> fingerprint/hash of the object.
> >>
> >> Currently the importer has to keep track of the "export_object_id" to
> try
> >> to upsert properly if the same object is re-imported. It can have
> collision
> >> issue when multiple systems are involved and has other disadvantages.
> >>
> >> Max
> >>
> >> On Thu, Jun 18, 2020 at 6:35 AM Dennis Meyer <de...@jdeluxe.org>
> wrote:
> >>
> >>> @Eric:
> >>> Thanks for the feedback Eric. It’s good to know. We’ll keep it in the
> >>> options list for now as this would also have some advantages :-)
> >>>
> >>> @Max:
> >>> Removing the slice id and passing all parameters works, that’s really
> >>> useful information. We might stick with this until slugs are available.
> >>> Thanks very much!
> >>> The charts can be static, but we need filtering on certain IDs.
> Something
> >>> like a product ID filter for example. But this seem to work ok with the
> >>> approach by adding a simple filter :-)
> >>>
> >>> Rison would be awesome for debugging and developing for sure!Even
> though
> >>> UUIDs would solve our problem they can get a pain pretty quick if not
> >>> architected very thoroughly. You have to use hashes in a consistent way
> >>> (which data/column do you use? What happens with added columns on DDL
> >>> updates etc) and dealing with those is non trivial when there’s no good
> >>> transportable way in generating those across application and DBs. Slug
> >>> seems to abstract the problem [at least the one we have] a lot better
> and
> >>> simpler at the same time.
> >>>
> >>> Dennis
> >>>
> >>>> On 18. Jun 2020, at 02:38, Erik Ritter <er...@gmail.com>
> wrote:
> >>>>
> >>>> Hi Dennis,
> >>>>
> >>>> You mention that cloning the complete Superset database seems overkill
> >>>> here, but for our use case, we've found that copying almost the entire
> >>>> Superset db into our dev and staging environments solves these types
> of
> >>>> issues for us. To ensure the process doesn't take too long, we exclude
> >>> the
> >>>> `logs` and the `queries` tables from our DB dump, but everything else
> we
> >>>> copy straight from prod to dev and staging for testing purposes.
> >>>>
> >>>> I know this doesn't address a case where you build a chart in dev and
> >>> want
> >>>> to move it to prod, but that's not a pattern we've really seen or
> needed
> >>> in
> >>>> our deployment.
> >>>>
> >>>> Hope this helps!
> >>>> Erik
> >>>>
> >>>> On Wed, Jun 17, 2020 at 7:46 AM Maxime Beauchemin <
> >>>> maximebeauchemin@gmail.com> wrote:
> >>>>
> >>>>> Few thoughts:
> >>>>> - did you try removing the id? this should work without the id since
> >>> it's
> >>>>> totally possible to have an explore session without a saved chart
> >>>>> - we spoke in the past about introducing *uuids* for
> >>> exportable/importable
> >>>>> objects that would be consistent across superset instances
> >>>>> - slugs for chart would help here, and should be easy to implement -
> we
> >>>>> need that regardless
> >>>>> - side note: it'd be great to userison here for more readable URL
> >>> encoding
> >>>>> - more generally - should an embeddable snippet point to a chart or a
> >>> state
> >>>>> of an explore session? I think both use-cases make sense
> >>>>>
> >>>>> On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org>
> >>> wrote:
> >>>>>> Sorry for broken formatting. The URL contains an “slice_id”, see
> edited
> >>>>>> mail below for better readability.
> >>>>>>
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> In the charts there’s an explicit embedding functionality
> generating
> >>>>>> Iframe code:
> >>>>>>> <iframe
> >>>>>>> width="600"
> >>>>>>> height="400"
> >>>>>>> seamless
> >>>>>>> frameBorder="0"
> >>>>>>> scrolling="no"
> >>>>>> src="…slice_id=306…”
> >>>>>>
> >>>>>>
> >>>>>>> </iframe>
> >>>>>>>
> >>>>>>> It seems like there’s an ID hardcoded (see above) - which is very
> >>>>>> unfortunate when exporting and importing again and the embedding is
> in
> >>>>>> another application. In this case ids are different after import.
> >>>>>>> I found a ticket about adding Slug to a chart, but it didn’t get
> too
> >>>>>> much attention.
> >>>>>>> https://github.com/apache/incubator-superset/issues/6537 <
> >>>>>> https://github.com/apache/incubator-superset/issues/6537>
> >>>>>>> Is there currently a way to address this better if I wanted to use
> >>>>>> embedding code in dev, staging and prod environments?
> >>>>>>> Cloning the complete Superset database seems overkill for this.
> Also
> >>> it
> >>>>>> won’t really work if you let production add charts by users and
> create
> >>>>> some
> >>>>>> major charts in dev you want to move to prod after some thorough
> >>> testing.
> >>>>>>> I’m glad for any suggestion.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Dennis
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>
>
>
Re: [EXTERNAL] Re: Addressing Charts directly / how to handle
embedding in dev, stage, prod cycle?
Posted by Dennis Meyer <de...@jdeluxe.org>.
Ok, I understand, this definitely makes good sense!
This would definitely be a great help for different envs.
Dennis
> On 18. Jun 2020, at 18:13, Andy Clapson <an...@slice.is> wrote:
>
> A big +1 on this one 👍.
>
> UUIDs would solve some definite deployment challenges on this one - mainly importing a pre-built set of Dashboards into many different environments.
>
> Andy
>
> On 2020-06-18 12:11, Maxime Beauchemin wrote:
>> I meant uuid in its pure form (just a universal unique identifier), not a
>> fingerprint/hash of the object.
>>
>> Currently the importer has to keep track of the "export_object_id" to try
>> to upsert properly if the same object is re-imported. It can have collision
>> issue when multiple systems are involved and has other disadvantages.
>>
>> Max
>>
>> On Thu, Jun 18, 2020 at 6:35 AM Dennis Meyer <de...@jdeluxe.org> wrote:
>>
>>> @Eric:
>>> Thanks for the feedback Eric. It’s good to know. We’ll keep it in the
>>> options list for now as this would also have some advantages :-)
>>>
>>> @Max:
>>> Removing the slice id and passing all parameters works, that’s really
>>> useful information. We might stick with this until slugs are available.
>>> Thanks very much!
>>> The charts can be static, but we need filtering on certain IDs. Something
>>> like a product ID filter for example. But this seem to work ok with the
>>> approach by adding a simple filter :-)
>>>
>>> Rison would be awesome for debugging and developing for sure!Even though
>>> UUIDs would solve our problem they can get a pain pretty quick if not
>>> architected very thoroughly. You have to use hashes in a consistent way
>>> (which data/column do you use? What happens with added columns on DDL
>>> updates etc) and dealing with those is non trivial when there’s no good
>>> transportable way in generating those across application and DBs. Slug
>>> seems to abstract the problem [at least the one we have] a lot better and
>>> simpler at the same time.
>>>
>>> Dennis
>>>
>>>> On 18. Jun 2020, at 02:38, Erik Ritter <er...@gmail.com> wrote:
>>>>
>>>> Hi Dennis,
>>>>
>>>> You mention that cloning the complete Superset database seems overkill
>>>> here, but for our use case, we've found that copying almost the entire
>>>> Superset db into our dev and staging environments solves these types of
>>>> issues for us. To ensure the process doesn't take too long, we exclude
>>> the
>>>> `logs` and the `queries` tables from our DB dump, but everything else we
>>>> copy straight from prod to dev and staging for testing purposes.
>>>>
>>>> I know this doesn't address a case where you build a chart in dev and
>>> want
>>>> to move it to prod, but that's not a pattern we've really seen or needed
>>> in
>>>> our deployment.
>>>>
>>>> Hope this helps!
>>>> Erik
>>>>
>>>> On Wed, Jun 17, 2020 at 7:46 AM Maxime Beauchemin <
>>>> maximebeauchemin@gmail.com> wrote:
>>>>
>>>>> Few thoughts:
>>>>> - did you try removing the id? this should work without the id since
>>> it's
>>>>> totally possible to have an explore session without a saved chart
>>>>> - we spoke in the past about introducing *uuids* for
>>> exportable/importable
>>>>> objects that would be consistent across superset instances
>>>>> - slugs for chart would help here, and should be easy to implement - we
>>>>> need that regardless
>>>>> - side note: it'd be great to userison here for more readable URL
>>> encoding
>>>>> - more generally - should an embeddable snippet point to a chart or a
>>> state
>>>>> of an explore session? I think both use-cases make sense
>>>>>
>>>>> On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org>
>>> wrote:
>>>>>> Sorry for broken formatting. The URL contains an “slice_id”, see edited
>>>>>> mail below for better readability.
>>>>>>
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> In the charts there’s an explicit embedding functionality generating
>>>>>> Iframe code:
>>>>>>> <iframe
>>>>>>> width="600"
>>>>>>> height="400"
>>>>>>> seamless
>>>>>>> frameBorder="0"
>>>>>>> scrolling="no"
>>>>>> src="…slice_id=306…”
>>>>>>
>>>>>>
>>>>>>> </iframe>
>>>>>>>
>>>>>>> It seems like there’s an ID hardcoded (see above) - which is very
>>>>>> unfortunate when exporting and importing again and the embedding is in
>>>>>> another application. In this case ids are different after import.
>>>>>>> I found a ticket about adding Slug to a chart, but it didn’t get too
>>>>>> much attention.
>>>>>>> https://github.com/apache/incubator-superset/issues/6537 <
>>>>>> https://github.com/apache/incubator-superset/issues/6537>
>>>>>>> Is there currently a way to address this better if I wanted to use
>>>>>> embedding code in dev, staging and prod environments?
>>>>>>> Cloning the complete Superset database seems overkill for this. Also
>>> it
>>>>>> won’t really work if you let production add charts by users and create
>>>>> some
>>>>>> major charts in dev you want to move to prod after some thorough
>>> testing.
>>>>>>> I’m glad for any suggestion.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dennis
>>>>>>>
>>>>>>>
>>>>>>
>>>
Re: [EXTERNAL] Re: Addressing Charts directly / how to handle
embedding in dev, stage, prod cycle?
Posted by Andy Clapson <an...@slice.is>.
A big +1 on this one 👍.
UUIDs would solve some definite deployment challenges on this one -
mainly importing a pre-built set of Dashboards into many different
environments.
Andy
On 2020-06-18 12:11, Maxime Beauchemin wrote:
> I meant uuid in its pure form (just a universal unique identifier), not a
> fingerprint/hash of the object.
>
> Currently the importer has to keep track of the "export_object_id" to try
> to upsert properly if the same object is re-imported. It can have collision
> issue when multiple systems are involved and has other disadvantages.
>
> Max
>
> On Thu, Jun 18, 2020 at 6:35 AM Dennis Meyer <de...@jdeluxe.org> wrote:
>
>> @Eric:
>> Thanks for the feedback Eric. It’s good to know. We’ll keep it in the
>> options list for now as this would also have some advantages :-)
>>
>> @Max:
>> Removing the slice id and passing all parameters works, that’s really
>> useful information. We might stick with this until slugs are available.
>> Thanks very much!
>> The charts can be static, but we need filtering on certain IDs. Something
>> like a product ID filter for example. But this seem to work ok with the
>> approach by adding a simple filter :-)
>>
>> Rison would be awesome for debugging and developing for sure!Even though
>> UUIDs would solve our problem they can get a pain pretty quick if not
>> architected very thoroughly. You have to use hashes in a consistent way
>> (which data/column do you use? What happens with added columns on DDL
>> updates etc) and dealing with those is non trivial when there’s no good
>> transportable way in generating those across application and DBs. Slug
>> seems to abstract the problem [at least the one we have] a lot better and
>> simpler at the same time.
>>
>> Dennis
>>
>>> On 18. Jun 2020, at 02:38, Erik Ritter <er...@gmail.com> wrote:
>>>
>>> Hi Dennis,
>>>
>>> You mention that cloning the complete Superset database seems overkill
>>> here, but for our use case, we've found that copying almost the entire
>>> Superset db into our dev and staging environments solves these types of
>>> issues for us. To ensure the process doesn't take too long, we exclude
>> the
>>> `logs` and the `queries` tables from our DB dump, but everything else we
>>> copy straight from prod to dev and staging for testing purposes.
>>>
>>> I know this doesn't address a case where you build a chart in dev and
>> want
>>> to move it to prod, but that's not a pattern we've really seen or needed
>> in
>>> our deployment.
>>>
>>> Hope this helps!
>>> Erik
>>>
>>> On Wed, Jun 17, 2020 at 7:46 AM Maxime Beauchemin <
>>> maximebeauchemin@gmail.com> wrote:
>>>
>>>> Few thoughts:
>>>> - did you try removing the id? this should work without the id since
>> it's
>>>> totally possible to have an explore session without a saved chart
>>>> - we spoke in the past about introducing *uuids* for
>> exportable/importable
>>>> objects that would be consistent across superset instances
>>>> - slugs for chart would help here, and should be easy to implement - we
>>>> need that regardless
>>>> - side note: it'd be great to userison here for more readable URL
>> encoding
>>>> - more generally - should an embeddable snippet point to a chart or a
>> state
>>>> of an explore session? I think both use-cases make sense
>>>>
>>>> On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org>
>> wrote:
>>>>> Sorry for broken formatting. The URL contains an “slice_id”, see edited
>>>>> mail below for better readability.
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> In the charts there’s an explicit embedding functionality generating
>>>>> Iframe code:
>>>>>> <iframe
>>>>>> width="600"
>>>>>> height="400"
>>>>>> seamless
>>>>>> frameBorder="0"
>>>>>> scrolling="no"
>>>>> src="…slice_id=306…”
>>>>>
>>>>>
>>>>>> </iframe>
>>>>>>
>>>>>> It seems like there’s an ID hardcoded (see above) - which is very
>>>>> unfortunate when exporting and importing again and the embedding is in
>>>>> another application. In this case ids are different after import.
>>>>>> I found a ticket about adding Slug to a chart, but it didn’t get too
>>>>> much attention.
>>>>>> https://github.com/apache/incubator-superset/issues/6537 <
>>>>> https://github.com/apache/incubator-superset/issues/6537>
>>>>>> Is there currently a way to address this better if I wanted to use
>>>>> embedding code in dev, staging and prod environments?
>>>>>> Cloning the complete Superset database seems overkill for this. Also
>> it
>>>>> won’t really work if you let production add charts by users and create
>>>> some
>>>>> major charts in dev you want to move to prod after some thorough
>> testing.
>>>>>> I’m glad for any suggestion.
>>>>>>
>>>>>> Thanks,
>>>>>> Dennis
>>>>>>
>>>>>>
>>>>>
>>
Re: Addressing Charts directly / how to handle embedding in dev,
stage, prod cycle?
Posted by Maxime Beauchemin <ma...@gmail.com>.
I meant uuid in its pure form (just a universal unique identifier), not a
fingerprint/hash of the object.
Currently the importer has to keep track of the "export_object_id" to try
to upsert properly if the same object is re-imported. It can have collision
issue when multiple systems are involved and has other disadvantages.
Max
On Thu, Jun 18, 2020 at 6:35 AM Dennis Meyer <de...@jdeluxe.org> wrote:
> @Eric:
> Thanks for the feedback Eric. It’s good to know. We’ll keep it in the
> options list for now as this would also have some advantages :-)
>
> @Max:
> Removing the slice id and passing all parameters works, that’s really
> useful information. We might stick with this until slugs are available.
> Thanks very much!
> The charts can be static, but we need filtering on certain IDs. Something
> like a product ID filter for example. But this seem to work ok with the
> approach by adding a simple filter :-)
>
> Rison would be awesome for debugging and developing for sure!Even though
> UUIDs would solve our problem they can get a pain pretty quick if not
> architected very thoroughly. You have to use hashes in a consistent way
> (which data/column do you use? What happens with added columns on DDL
> updates etc) and dealing with those is non trivial when there’s no good
> transportable way in generating those across application and DBs. Slug
> seems to abstract the problem [at least the one we have] a lot better and
> simpler at the same time.
>
> Dennis
>
> > On 18. Jun 2020, at 02:38, Erik Ritter <er...@gmail.com> wrote:
> >
> > Hi Dennis,
> >
> > You mention that cloning the complete Superset database seems overkill
> > here, but for our use case, we've found that copying almost the entire
> > Superset db into our dev and staging environments solves these types of
> > issues for us. To ensure the process doesn't take too long, we exclude
> the
> > `logs` and the `queries` tables from our DB dump, but everything else we
> > copy straight from prod to dev and staging for testing purposes.
> >
> > I know this doesn't address a case where you build a chart in dev and
> want
> > to move it to prod, but that's not a pattern we've really seen or needed
> in
> > our deployment.
> >
> > Hope this helps!
> > Erik
> >
> > On Wed, Jun 17, 2020 at 7:46 AM Maxime Beauchemin <
> > maximebeauchemin@gmail.com> wrote:
> >
> >> Few thoughts:
> >> - did you try removing the id? this should work without the id since
> it's
> >> totally possible to have an explore session without a saved chart
> >> - we spoke in the past about introducing *uuids* for
> exportable/importable
> >> objects that would be consistent across superset instances
> >> - slugs for chart would help here, and should be easy to implement - we
> >> need that regardless
> >> - side note: it'd be great to userison here for more readable URL
> encoding
> >> - more generally - should an embeddable snippet point to a chart or a
> state
> >> of an explore session? I think both use-cases make sense
> >>
> >> On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org>
> wrote:
> >>
> >>> Sorry for broken formatting. The URL contains an “slice_id”, see edited
> >>> mail below for better readability.
> >>>
> >>>
> >>>>
> >>>> Hi,
> >>>>
> >>>> In the charts there’s an explicit embedding functionality generating
> >>> Iframe code:
> >>>>
> >>>> <iframe
> >>>> width="600"
> >>>> height="400"
> >>>> seamless
> >>>> frameBorder="0"
> >>>> scrolling="no"
> >>>
> >>> src="…slice_id=306…”
> >>>
> >>>
> >>>> </iframe>
> >>>>
> >>>> It seems like there’s an ID hardcoded (see above) - which is very
> >>> unfortunate when exporting and importing again and the embedding is in
> >>> another application. In this case ids are different after import.
> >>>>
> >>>> I found a ticket about adding Slug to a chart, but it didn’t get too
> >>> much attention.
> >>>> https://github.com/apache/incubator-superset/issues/6537 <
> >>> https://github.com/apache/incubator-superset/issues/6537>
> >>>>
> >>>> Is there currently a way to address this better if I wanted to use
> >>> embedding code in dev, staging and prod environments?
> >>>> Cloning the complete Superset database seems overkill for this. Also
> it
> >>> won’t really work if you let production add charts by users and create
> >> some
> >>> major charts in dev you want to move to prod after some thorough
> testing.
> >>>>
> >>>> I’m glad for any suggestion.
> >>>>
> >>>> Thanks,
> >>>> Dennis
> >>>>
> >>>>
> >>>
> >>>
> >>
>
>
Re: Addressing Charts directly / how to handle embedding in dev,
stage, prod cycle?
Posted by Dennis Meyer <de...@jdeluxe.org>.
@Eric:
Thanks for the feedback Eric. It’s good to know. We’ll keep it in the options list for now as this would also have some advantages :-)
@Max:
Removing the slice id and passing all parameters works, that’s really useful information. We might stick with this until slugs are available. Thanks very much!
The charts can be static, but we need filtering on certain IDs. Something like a product ID filter for example. But this seem to work ok with the approach by adding a simple filter :-)
Rison would be awesome for debugging and developing for sure!Even though UUIDs would solve our problem they can get a pain pretty quick if not architected very thoroughly. You have to use hashes in a consistent way (which data/column do you use? What happens with added columns on DDL updates etc) and dealing with those is non trivial when there’s no good transportable way in generating those across application and DBs. Slug seems to abstract the problem [at least the one we have] a lot better and simpler at the same time.
Dennis
> On 18. Jun 2020, at 02:38, Erik Ritter <er...@gmail.com> wrote:
>
> Hi Dennis,
>
> You mention that cloning the complete Superset database seems overkill
> here, but for our use case, we've found that copying almost the entire
> Superset db into our dev and staging environments solves these types of
> issues for us. To ensure the process doesn't take too long, we exclude the
> `logs` and the `queries` tables from our DB dump, but everything else we
> copy straight from prod to dev and staging for testing purposes.
>
> I know this doesn't address a case where you build a chart in dev and want
> to move it to prod, but that's not a pattern we've really seen or needed in
> our deployment.
>
> Hope this helps!
> Erik
>
> On Wed, Jun 17, 2020 at 7:46 AM Maxime Beauchemin <
> maximebeauchemin@gmail.com> wrote:
>
>> Few thoughts:
>> - did you try removing the id? this should work without the id since it's
>> totally possible to have an explore session without a saved chart
>> - we spoke in the past about introducing *uuids* for exportable/importable
>> objects that would be consistent across superset instances
>> - slugs for chart would help here, and should be easy to implement - we
>> need that regardless
>> - side note: it'd be great to userison here for more readable URL encoding
>> - more generally - should an embeddable snippet point to a chart or a state
>> of an explore session? I think both use-cases make sense
>>
>> On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org> wrote:
>>
>>> Sorry for broken formatting. The URL contains an “slice_id”, see edited
>>> mail below for better readability.
>>>
>>>
>>>>
>>>> Hi,
>>>>
>>>> In the charts there’s an explicit embedding functionality generating
>>> Iframe code:
>>>>
>>>> <iframe
>>>> width="600"
>>>> height="400"
>>>> seamless
>>>> frameBorder="0"
>>>> scrolling="no"
>>>
>>> src="…slice_id=306…”
>>>
>>>
>>>> </iframe>
>>>>
>>>> It seems like there’s an ID hardcoded (see above) - which is very
>>> unfortunate when exporting and importing again and the embedding is in
>>> another application. In this case ids are different after import.
>>>>
>>>> I found a ticket about adding Slug to a chart, but it didn’t get too
>>> much attention.
>>>> https://github.com/apache/incubator-superset/issues/6537 <
>>> https://github.com/apache/incubator-superset/issues/6537>
>>>>
>>>> Is there currently a way to address this better if I wanted to use
>>> embedding code in dev, staging and prod environments?
>>>> Cloning the complete Superset database seems overkill for this. Also it
>>> won’t really work if you let production add charts by users and create
>> some
>>> major charts in dev you want to move to prod after some thorough testing.
>>>>
>>>> I’m glad for any suggestion.
>>>>
>>>> Thanks,
>>>> Dennis
>>>>
>>>>
>>>
>>>
>>
Re: Addressing Charts directly / how to handle embedding in dev,
stage, prod cycle?
Posted by Erik Ritter <er...@gmail.com>.
Hi Dennis,
You mention that cloning the complete Superset database seems overkill
here, but for our use case, we've found that copying almost the entire
Superset db into our dev and staging environments solves these types of
issues for us. To ensure the process doesn't take too long, we exclude the
`logs` and the `queries` tables from our DB dump, but everything else we
copy straight from prod to dev and staging for testing purposes.
I know this doesn't address a case where you build a chart in dev and want
to move it to prod, but that's not a pattern we've really seen or needed in
our deployment.
Hope this helps!
Erik
On Wed, Jun 17, 2020 at 7:46 AM Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:
> Few thoughts:
> - did you try removing the id? this should work without the id since it's
> totally possible to have an explore session without a saved chart
> - we spoke in the past about introducing *uuids* for exportable/importable
> objects that would be consistent across superset instances
> - slugs for chart would help here, and should be easy to implement - we
> need that regardless
> - side note: it'd be great to use rison here for more readable URL encoding
> - more generally - should an embeddable snippet point to a chart or a state
> of an explore session? I think both use-cases make sense
>
> On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org> wrote:
>
> > Sorry for broken formatting. The URL contains an “slice_id”, see edited
> > mail below for better readability.
> >
> >
> > >
> > > Hi,
> > >
> > > In the charts there’s an explicit embedding functionality generating
> > Iframe code:
> > >
> > > <iframe
> > > width="600"
> > > height="400"
> > > seamless
> > > frameBorder="0"
> > > scrolling="no"
> >
> > src="…slice_id=306…”
> >
> >
> > > </iframe>
> > >
> > > It seems like there’s an ID hardcoded (see above) - which is very
> > unfortunate when exporting and importing again and the embedding is in
> > another application. In this case ids are different after import.
> > >
> > > I found a ticket about adding Slug to a chart, but it didn’t get too
> > much attention.
> > > https://github.com/apache/incubator-superset/issues/6537 <
> > https://github.com/apache/incubator-superset/issues/6537>
> > >
> > > Is there currently a way to address this better if I wanted to use
> > embedding code in dev, staging and prod environments?
> > > Cloning the complete Superset database seems overkill for this. Also it
> > won’t really work if you let production add charts by users and create
> some
> > major charts in dev you want to move to prod after some thorough testing.
> > >
> > > I’m glad for any suggestion.
> > >
> > > Thanks,
> > > Dennis
> > >
> > >
> >
> >
>
Re: Addressing Charts directly / how to handle embedding in dev,
stage, prod cycle?
Posted by Maxime Beauchemin <ma...@gmail.com>.
Few thoughts:
- did you try removing the id? this should work without the id since it's
totally possible to have an explore session without a saved chart
- we spoke in the past about introducing *uuids* for exportable/importable
objects that would be consistent across superset instances
- slugs for chart would help here, and should be easy to implement - we
need that regardless
- side note: it'd be great to use rison here for more readable URL encoding
- more generally - should an embeddable snippet point to a chart or a state
of an explore session? I think both use-cases make sense
On Tue, Jun 16, 2020 at 12:39 PM Dennis Meyer <de...@jdeluxe.org> wrote:
> Sorry for broken formatting. The URL contains an “slice_id”, see edited
> mail below for better readability.
>
>
> >
> > Hi,
> >
> > In the charts there’s an explicit embedding functionality generating
> Iframe code:
> >
> > <iframe
> > width="600"
> > height="400"
> > seamless
> > frameBorder="0"
> > scrolling="no"
>
> src="…slice_id=306…”
>
>
> > </iframe>
> >
> > It seems like there’s an ID hardcoded (see above) - which is very
> unfortunate when exporting and importing again and the embedding is in
> another application. In this case ids are different after import.
> >
> > I found a ticket about adding Slug to a chart, but it didn’t get too
> much attention.
> > https://github.com/apache/incubator-superset/issues/6537 <
> https://github.com/apache/incubator-superset/issues/6537>
> >
> > Is there currently a way to address this better if I wanted to use
> embedding code in dev, staging and prod environments?
> > Cloning the complete Superset database seems overkill for this. Also it
> won’t really work if you let production add charts by users and create some
> major charts in dev you want to move to prod after some thorough testing.
> >
> > I’m glad for any suggestion.
> >
> > Thanks,
> > Dennis
> >
> >
>
>
Re: Addressing Charts directly / how to handle embedding in dev,
stage, prod cycle?
Posted by Dennis Meyer <de...@jdeluxe.org>.
Sorry for broken formatting. The URL contains an “slice_id”, see edited mail below for better readability.
>
> Hi,
>
> In the charts there’s an explicit embedding functionality generating Iframe code:
>
> <iframe
> width="600"
> height="400"
> seamless
> frameBorder="0"
> scrolling="no"
src="…slice_id=306…”
> </iframe>
>
> It seems like there’s an ID hardcoded (see above) - which is very unfortunate when exporting and importing again and the embedding is in another application. In this case ids are different after import.
>
> I found a ticket about adding Slug to a chart, but it didn’t get too much attention.
> https://github.com/apache/incubator-superset/issues/6537 <https://github.com/apache/incubator-superset/issues/6537>
>
> Is there currently a way to address this better if I wanted to use embedding code in dev, staging and prod environments?
> Cloning the complete Superset database seems overkill for this. Also it won’t really work if you let production add charts by users and create some major charts in dev you want to move to prod after some thorough testing.
>
> I’m glad for any suggestion.
>
> Thanks,
> Dennis
>
>