You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Jojy Varghese <jo...@mesosphere.io> on 2016/01/06 02:50:34 UTC
Review Request 41959: Statically initializing fetcher plugins.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/
-----------------------------------------------------------
Review request for mesos and Jie Yu.
Repository: mesos
Description
-------
Since fetcher plugins are not dynamically pluggable today, simplified the
factory method to use the statically intitalized plugins. This would be
useful when Fetchers are created multiple times.
Diffs
-----
src/uri/fetcher.cpp ac13fbdc7399045d183cbdcc48dc5cf9969e8ad5
Diff: https://reviews.apache.org/r/41959/diff/
Testing
-------
make check.
Thanks,
Jojy Varghese
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Mesos ReviewBot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/#review113036
-----------------------------------------------------------
Patch looks great!
Reviews applied: [41958, 41959]
Passed command: export OS=ubuntu:14.04;export CONFIGURATION="--verbose";export COMPILER=gcc; ./support/docker_build.sh
- Mesos ReviewBot
On Jan. 6, 2016, 2:14 a.m., Jojy Varghese wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41959/
> -----------------------------------------------------------
>
> (Updated Jan. 6, 2016, 2:14 a.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Since fetcher plugins are not dynamically pluggable today, simplified the
> factory method to use the statically intitalized plugins. This would be
> useful when Fetchers are created multiple times.
>
>
> Diffs
> -----
>
> src/uri/fetcher.cpp ac13fbdc7399045d183cbdcc48dc5cf9969e8ad5
>
> Diff: https://reviews.apache.org/r/41959/diff/
>
>
> Testing
> -------
>
> make check.
>
>
> Thanks,
>
> Jojy Varghese
>
>
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jojy Varghese <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/
-----------------------------------------------------------
(Updated Jan. 25, 2016, 9:19 p.m.)
Review request for mesos and Jie Yu.
Changes
-------
removed optimization of plugin creation.
Repository: mesos
Description
-------
Since fetcher plugins are not dynamically pluggable today, simplified the
factory method to use the statically intitalized plugins. This would be
useful when Fetchers are created multiple times.
Diffs (updated)
-----
src/uri/fetcher.cpp dfda732348fec3b686cf82b55ad94fda4829469b
Diff: https://reviews.apache.org/r/41959/diff/
Testing
-------
make check.
Thanks,
Jojy Varghese
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jie Yu <yu...@gmail.com>.
> On Jan. 25, 2016, 5:12 p.m., Jie Yu wrote:
> > I am wondering if you do see performance issue regarding this? We typically don't optimize the code too much until it becomes a problem. Also, we try to avoid global variable dependencies (i.e., static bool isCreateError in this case).
>
> Jojy Varghese wrote:
> Jie, thanks for looking at this.
>
> This change is two parts - one is simply replacing the initialization of `creators` hashmap. Initialization using universal `{` initialization is the preferred way in c++11 that calling `put`.
>
> The second change is to optimize the creation. I thought this was necessary because we will be calling `create` for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that `create` call, I thought it would be better to reuse the initialized structures.
>
> We can remove the 2nd part (optimization) if you like.
>
> Jojy Varghese wrote:
> I could make the static flag `isCreateError` as atomic_flag if you prefer.
> The second change is to optimize the creation. I thought this was necessary because we will be calling create for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that create call, I thought it would be better to reuse the initialized structures.
Why we will be calling create for every URI fetch in a image pull? Can we save the uri::Fetcher instance? Eventually, there'll be only one uri::Fetcher at the top level (similar to the Fetcher right now).
- Jie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/#review116115
-----------------------------------------------------------
On Jan. 25, 2016, 9:19 p.m., Jojy Varghese wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41959/
> -----------------------------------------------------------
>
> (Updated Jan. 25, 2016, 9:19 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Since fetcher plugins are not dynamically pluggable today, simplified the
> factory method to use the statically intitalized plugins. This would be
> useful when Fetchers are created multiple times.
>
>
> Diffs
> -----
>
> src/uri/fetcher.cpp dfda732348fec3b686cf82b55ad94fda4829469b
>
> Diff: https://reviews.apache.org/r/41959/diff/
>
>
> Testing
> -------
>
> make check.
>
>
> Thanks,
>
> Jojy Varghese
>
>
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jojy Varghese <jo...@mesosphere.io>.
> On Jan. 25, 2016, 5:12 p.m., Jie Yu wrote:
> > I am wondering if you do see performance issue regarding this? We typically don't optimize the code too much until it becomes a problem. Also, we try to avoid global variable dependencies (i.e., static bool isCreateError in this case).
>
> Jojy Varghese wrote:
> Jie, thanks for looking at this.
>
> This change is two parts - one is simply replacing the initialization of `creators` hashmap. Initialization using universal `{` initialization is the preferred way in c++11 that calling `put`.
>
> The second change is to optimize the creation. I thought this was necessary because we will be calling `create` for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that `create` call, I thought it would be better to reuse the initialized structures.
>
> We can remove the 2nd part (optimization) if you like.
>
> Jojy Varghese wrote:
> I could make the static flag `isCreateError` as atomic_flag if you prefer.
>
> Jie Yu wrote:
> > The second change is to optimize the creation. I thought this was necessary because we will be calling create for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that create call, I thought it would be better to reuse the initialized structures.
>
> Why we will be calling create for every URI fetch in a image pull? Can we save the uri::Fetcher instance? Eventually, there'll be only one uri::Fetcher at the top level (similar to the Fetcher right now).
One reason I can think of is to allow operaters to install missing plugings (say hdfs) without having to restart. Let me know if that use case is invalid.
- Jojy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/#review116115
-----------------------------------------------------------
On Jan. 25, 2016, 9:19 p.m., Jojy Varghese wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41959/
> -----------------------------------------------------------
>
> (Updated Jan. 25, 2016, 9:19 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Since fetcher plugins are not dynamically pluggable today, simplified the
> factory method to use the statically intitalized plugins. This would be
> useful when Fetchers are created multiple times.
>
>
> Diffs
> -----
>
> src/uri/fetcher.cpp dfda732348fec3b686cf82b55ad94fda4829469b
>
> Diff: https://reviews.apache.org/r/41959/diff/
>
>
> Testing
> -------
>
> make check.
>
>
> Thanks,
>
> Jojy Varghese
>
>
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jojy Varghese <jo...@mesosphere.io>.
> On Jan. 25, 2016, 5:12 p.m., Jie Yu wrote:
> > I am wondering if you do see performance issue regarding this? We typically don't optimize the code too much until it becomes a problem. Also, we try to avoid global variable dependencies (i.e., static bool isCreateError in this case).
>
> Jojy Varghese wrote:
> Jie, thanks for looking at this.
>
> This change is two parts - one is simply replacing the initialization of `creators` hashmap. Initialization using universal `{` initialization is the preferred way in c++11 that calling `put`.
>
> The second change is to optimize the creation. I thought this was necessary because we will be calling `create` for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that `create` call, I thought it would be better to reuse the initialized structures.
>
> We can remove the 2nd part (optimization) if you like.
>
> Jojy Varghese wrote:
> I could make the static flag `isCreateError` as atomic_flag if you prefer.
>
> Jie Yu wrote:
> > The second change is to optimize the creation. I thought this was necessary because we will be calling create for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that create call, I thought it would be better to reuse the initialized structures.
>
> Why we will be calling create for every URI fetch in a image pull? Can we save the uri::Fetcher instance? Eventually, there'll be only one uri::Fetcher at the top level (similar to the Fetcher right now).
>
> Jojy Varghese wrote:
> One reason I can think of is to allow operaters to install missing plugings (say hdfs) without having to restart. Let me know if that use case is invalid.
>
> Jie Yu wrote:
> Even though, i would rather add interfaces to allow us to update an existing Fetcher, instead of creating a Fetcher each time we pull an image.
I have updated the patch to remove the optimization part based on discussion above. thanks!
- Jojy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/#review116115
-----------------------------------------------------------
On Jan. 25, 2016, 9:19 p.m., Jojy Varghese wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41959/
> -----------------------------------------------------------
>
> (Updated Jan. 25, 2016, 9:19 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Since fetcher plugins are not dynamically pluggable today, simplified the
> factory method to use the statically intitalized plugins. This would be
> useful when Fetchers are created multiple times.
>
>
> Diffs
> -----
>
> src/uri/fetcher.cpp dfda732348fec3b686cf82b55ad94fda4829469b
>
> Diff: https://reviews.apache.org/r/41959/diff/
>
>
> Testing
> -------
>
> make check.
>
>
> Thanks,
>
> Jojy Varghese
>
>
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jojy Varghese <jo...@mesosphere.io>.
> On Jan. 25, 2016, 5:12 p.m., Jie Yu wrote:
> > I am wondering if you do see performance issue regarding this? We typically don't optimize the code too much until it becomes a problem. Also, we try to avoid global variable dependencies (i.e., static bool isCreateError in this case).
>
> Jojy Varghese wrote:
> Jie, thanks for looking at this.
>
> This change is two parts - one is simply replacing the initialization of `creators` hashmap. Initialization using universal `{` initialization is the preferred way in c++11 that calling `put`.
>
> The second change is to optimize the creation. I thought this was necessary because we will be calling `create` for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that `create` call, I thought it would be better to reuse the initialized structures.
>
> We can remove the 2nd part (optimization) if you like.
I could make the static flag `isCreateError` as atomic_flag if you prefer.
- Jojy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/#review116115
-----------------------------------------------------------
On Jan. 13, 2016, 11:08 p.m., Jojy Varghese wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41959/
> -----------------------------------------------------------
>
> (Updated Jan. 13, 2016, 11:08 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Since fetcher plugins are not dynamically pluggable today, simplified the
> factory method to use the statically intitalized plugins. This would be
> useful when Fetchers are created multiple times.
>
>
> Diffs
> -----
>
> src/uri/fetcher.cpp dfda732348fec3b686cf82b55ad94fda4829469b
>
> Diff: https://reviews.apache.org/r/41959/diff/
>
>
> Testing
> -------
>
> make check.
>
>
> Thanks,
>
> Jojy Varghese
>
>
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jie Yu <yu...@gmail.com>.
> On Jan. 25, 2016, 5:12 p.m., Jie Yu wrote:
> > I am wondering if you do see performance issue regarding this? We typically don't optimize the code too much until it becomes a problem. Also, we try to avoid global variable dependencies (i.e., static bool isCreateError in this case).
>
> Jojy Varghese wrote:
> Jie, thanks for looking at this.
>
> This change is two parts - one is simply replacing the initialization of `creators` hashmap. Initialization using universal `{` initialization is the preferred way in c++11 that calling `put`.
>
> The second change is to optimize the creation. I thought this was necessary because we will be calling `create` for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that `create` call, I thought it would be better to reuse the initialized structures.
>
> We can remove the 2nd part (optimization) if you like.
>
> Jojy Varghese wrote:
> I could make the static flag `isCreateError` as atomic_flag if you prefer.
>
> Jie Yu wrote:
> > The second change is to optimize the creation. I thought this was necessary because we will be calling create for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that create call, I thought it would be better to reuse the initialized structures.
>
> Why we will be calling create for every URI fetch in a image pull? Can we save the uri::Fetcher instance? Eventually, there'll be only one uri::Fetcher at the top level (similar to the Fetcher right now).
>
> Jojy Varghese wrote:
> One reason I can think of is to allow operaters to install missing plugings (say hdfs) without having to restart. Let me know if that use case is invalid.
Even though, i would rather add interfaces to allow us to update an existing Fetcher, instead of creating a Fetcher each time we pull an image.
- Jie
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/#review116115
-----------------------------------------------------------
On Jan. 25, 2016, 9:19 p.m., Jojy Varghese wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41959/
> -----------------------------------------------------------
>
> (Updated Jan. 25, 2016, 9:19 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Since fetcher plugins are not dynamically pluggable today, simplified the
> factory method to use the statically intitalized plugins. This would be
> useful when Fetchers are created multiple times.
>
>
> Diffs
> -----
>
> src/uri/fetcher.cpp dfda732348fec3b686cf82b55ad94fda4829469b
>
> Diff: https://reviews.apache.org/r/41959/diff/
>
>
> Testing
> -------
>
> make check.
>
>
> Thanks,
>
> Jojy Varghese
>
>
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jojy Varghese <jo...@mesosphere.io>.
> On Jan. 25, 2016, 5:12 p.m., Jie Yu wrote:
> > I am wondering if you do see performance issue regarding this? We typically don't optimize the code too much until it becomes a problem. Also, we try to avoid global variable dependencies (i.e., static bool isCreateError in this case).
Jie, thanks for looking at this.
This change is two parts - one is simply replacing the initialization of `creators` hashmap. Initialization using universal `{` initialization is the preferred way in c++11 that calling `put`.
The second change is to optimize the creation. I thought this was necessary because we will be calling `create` for every URI fetch in a image pull and its dependencies(or layers). As the information to create the fetcher does not change for each of that `create` call, I thought it would be better to reuse the initialized structures.
We can remove the 2nd part (optimization) if you like.
- Jojy
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/#review116115
-----------------------------------------------------------
On Jan. 13, 2016, 11:08 p.m., Jojy Varghese wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41959/
> -----------------------------------------------------------
>
> (Updated Jan. 13, 2016, 11:08 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Since fetcher plugins are not dynamically pluggable today, simplified the
> factory method to use the statically intitalized plugins. This would be
> useful when Fetchers are created multiple times.
>
>
> Diffs
> -----
>
> src/uri/fetcher.cpp dfda732348fec3b686cf82b55ad94fda4829469b
>
> Diff: https://reviews.apache.org/r/41959/diff/
>
>
> Testing
> -------
>
> make check.
>
>
> Thanks,
>
> Jojy Varghese
>
>
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/#review116115
-----------------------------------------------------------
I am wondering if you do see performance issue regarding this? We typically don't optimize the code too much until it becomes a problem. Also, we try to avoid global variable dependencies (i.e., static bool isCreateError in this case).
- Jie Yu
On Jan. 13, 2016, 11:08 p.m., Jojy Varghese wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41959/
> -----------------------------------------------------------
>
> (Updated Jan. 13, 2016, 11:08 p.m.)
>
>
> Review request for mesos and Jie Yu.
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Since fetcher plugins are not dynamically pluggable today, simplified the
> factory method to use the statically intitalized plugins. This would be
> useful when Fetchers are created multiple times.
>
>
> Diffs
> -----
>
> src/uri/fetcher.cpp dfda732348fec3b686cf82b55ad94fda4829469b
>
> Diff: https://reviews.apache.org/r/41959/diff/
>
>
> Testing
> -------
>
> make check.
>
>
> Thanks,
>
> Jojy Varghese
>
>
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jojy Varghese <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/
-----------------------------------------------------------
(Updated Jan. 13, 2016, 11:08 p.m.)
Review request for mesos and Jie Yu.
Changes
-------
rebased with master.
Repository: mesos
Description
-------
Since fetcher plugins are not dynamically pluggable today, simplified the
factory method to use the statically intitalized plugins. This would be
useful when Fetchers are created multiple times.
Diffs (updated)
-----
src/uri/fetcher.cpp dfda732348fec3b686cf82b55ad94fda4829469b
Diff: https://reviews.apache.org/r/41959/diff/
Testing
-------
make check.
Thanks,
Jojy Varghese
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jojy Varghese <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/
-----------------------------------------------------------
(Updated Jan. 12, 2016, 12:42 a.m.)
Review request for mesos and Jie Yu.
Changes
-------
rebased with master
Repository: mesos
Description
-------
Since fetcher plugins are not dynamically pluggable today, simplified the
factory method to use the statically intitalized plugins. This would be
useful when Fetchers are created multiple times.
Diffs (updated)
-----
src/uri/fetcher.cpp ac13fbdc7399045d183cbdcc48dc5cf9969e8ad5
Diff: https://reviews.apache.org/r/41959/diff/
Testing
-------
make check.
Thanks,
Jojy Varghese
Re: Review Request 41959: Statically initializing fetcher plugins.
Posted by Jojy Varghese <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41959/
-----------------------------------------------------------
(Updated Jan. 6, 2016, 2:14 a.m.)
Review request for mesos and Jie Yu.
Changes
-------
added flag to retry on error
Repository: mesos
Description
-------
Since fetcher plugins are not dynamically pluggable today, simplified the
factory method to use the statically intitalized plugins. This would be
useful when Fetchers are created multiple times.
Diffs (updated)
-----
src/uri/fetcher.cpp ac13fbdc7399045d183cbdcc48dc5cf9969e8ad5
Diff: https://reviews.apache.org/r/41959/diff/
Testing
-------
make check.
Thanks,
Jojy Varghese