You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Brian Wickman <wi...@apache.org> on 2015/07/14 00:53:28 UTC
Review Request 36459: Fix AuroraConfigLoader schema filtering when
loading from string.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36459/
-----------------------------------------------------------
Review request for Aurora and Bill Farner.
Repository: aurora
Description
-------
Fix AuroraConfigLoader schema filtering when loading from string.
We have an internal tool that needs to serialize configs into a string and retrieve them from stable storage. We had deployed a client containing shutdown_endpoint then reverted it in https://reviews.apache.org/r/35847/ but unfortunately deserialization broke the internal tool since it was not filtering out the unknown attributes.
Diffs
-----
src/main/python/apache/aurora/config/loader.py c8b045e9a668199feee0ac3cee9fac55fe09cf08
src/test/python/apache/aurora/config/test_loader.py 00b6eabadb8f4e40aba1191068e1ca04c78d1ff3
Diff: https://reviews.apache.org/r/36459/diff/
Testing
-------
Added test, it broke. Fixed code, tests passed.
Also updated tests to not create files when not necessary.
Thanks,
Brian Wickman
Re: Review Request 36459: Fix AuroraConfigLoader schema filtering when
loading from string.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36459/#review91563
-----------------------------------------------------------
Ship it!
Master (b7a02a5) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On July 13, 2015, 11:11 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36459/
> -----------------------------------------------------------
>
> (Updated July 13, 2015, 11:11 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Fix AuroraConfigLoader schema filtering when loading from string.
>
> We have an internal tool that needs to serialize configs into a string and retrieve them from stable storage. We had deployed a client containing shutdown_endpoint then reverted it in https://reviews.apache.org/r/35847/ but unfortunately deserialization broke the internal tool since it was not filtering out the unknown attributes.
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/config/loader.py c8b045e9a668199feee0ac3cee9fac55fe09cf08
> src/test/python/apache/aurora/config/test_loader.py 00b6eabadb8f4e40aba1191068e1ca04c78d1ff3
>
> Diff: https://reviews.apache.org/r/36459/diff/
>
>
> Testing
> -------
>
> Added test, it broke. Fixed code, tests passed.
>
> Also updated tests to not create files when not necessary.
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 36459: Fix AuroraConfigLoader schema filtering when
loading from string.
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36459/
-----------------------------------------------------------
(Updated July 13, 2015, 11:11 p.m.)
Review request for Aurora and Bill Farner.
Changes
-------
Remove unused import that the commit hook missed.
Repository: aurora
Description
-------
Fix AuroraConfigLoader schema filtering when loading from string.
We have an internal tool that needs to serialize configs into a string and retrieve them from stable storage. We had deployed a client containing shutdown_endpoint then reverted it in https://reviews.apache.org/r/35847/ but unfortunately deserialization broke the internal tool since it was not filtering out the unknown attributes.
Diffs (updated)
-----
src/main/python/apache/aurora/config/loader.py c8b045e9a668199feee0ac3cee9fac55fe09cf08
src/test/python/apache/aurora/config/test_loader.py 00b6eabadb8f4e40aba1191068e1ca04c78d1ff3
Diff: https://reviews.apache.org/r/36459/diff/
Testing
-------
Added test, it broke. Fixed code, tests passed.
Also updated tests to not create files when not necessary.
Thanks,
Brian Wickman
Re: Review Request 36459: Fix AuroraConfigLoader schema filtering when
loading from string.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36459/#review91553
-----------------------------------------------------------
Ship it!
For those not terribly familiar with pystachio, this fixes it because the `json_loads` function in `Struct` will filter, but the routine prior to this diff does not:
```
def json_loads(cls, json_string, strict=False):
return cls(json.loads(json_string) if strict
else cls._filter_against_schema(json.loads(json_string)))
```
https://github.com/wickman/pystachio/blob/6a9518fbb6b5732edf3112b76b9e70c0fa693877/pystachio/composite.py#L290-L293
- Bill Farner
On July 13, 2015, 10:53 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36459/
> -----------------------------------------------------------
>
> (Updated July 13, 2015, 10:53 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Fix AuroraConfigLoader schema filtering when loading from string.
>
> We have an internal tool that needs to serialize configs into a string and retrieve them from stable storage. We had deployed a client containing shutdown_endpoint then reverted it in https://reviews.apache.org/r/35847/ but unfortunately deserialization broke the internal tool since it was not filtering out the unknown attributes.
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/config/loader.py c8b045e9a668199feee0ac3cee9fac55fe09cf08
> src/test/python/apache/aurora/config/test_loader.py 00b6eabadb8f4e40aba1191068e1ca04c78d1ff3
>
> Diff: https://reviews.apache.org/r/36459/diff/
>
>
> Testing
> -------
>
> Added test, it broke. Fixed code, tests passed.
>
> Also updated tests to not create files when not necessary.
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 36459: Fix AuroraConfigLoader schema filtering when
loading from string.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36459/#review91555
-----------------------------------------------------------
Master (b7a02a5) is red with this patch.
./build-support/jenkins/build.sh
Using cached twitter.common.process-0.3.0.tar.gz
Collecting twitter.common.log==0.3.0 (from twitter.common.app==0.3.0->twitter.checkstyle==0.1.0)
Using cached twitter.common.log-0.3.0.tar.gz
Collecting twitter.common.util==0.3.0 (from twitter.common.app==0.3.0->twitter.checkstyle==0.1.0)
Using cached twitter.common.util-0.3.0.tar.gz
Collecting twitter.common.collections==0.3.0 (from twitter.common.app==0.3.0->twitter.checkstyle==0.1.0)
Using cached twitter.common.collections-0.3.0.tar.gz
Collecting smmap>=0.8.5 (from gitdb>=0.5.1->GitPython==0.3.2.RC1->twitter.checkstyle==0.1.0)
Using cached smmap-0.9.0.tar.gz
Collecting twitter.common.string==0.3.0 (from twitter.common.process==0.3.0->twitter.common.app==0.3.0->twitter.checkstyle==0.1.0)
Using cached twitter.common.string-0.3.0.tar.gz
Collecting twitter.common.options==0.3.0 (from twitter.common.log==0.3.0->twitter.common.app==0.3.0->twitter.checkstyle==0.1.0)
Using cached twitter.common.options-0.3.0.tar.gz
Collecting twitter.common.dirutil==0.3.0 (from twitter.common.log==0.3.0->twitter.common.app==0.3.0->twitter.checkstyle==0.1.0)
Using cached twitter.common.dirutil-0.3.0.tar.gz
Collecting twitter.common.contextutil==0.3.0 (from twitter.common.util==0.3.0->twitter.common.app==0.3.0->twitter.checkstyle==0.1.0)
Using cached twitter.common.contextutil-0.3.0.tar.gz
Collecting twitter.common.lang==0.3.0 (from twitter.common.collections==0.3.0->twitter.common.app==0.3.0->twitter.checkstyle==0.1.0)
Using cached twitter.common.lang-0.3.0.tar.gz
Installing collected packages: pyflakes, pep8, smmap, gitdb, GitPython, twitter.common.lang, twitter.common.string, twitter.common.process, twitter.common.options, twitter.common.dirutil, twitter.common.log, twitter.common.contextutil, twitter.common.util, twitter.common.collections, twitter.common.app, twitter.checkstyle
Running setup.py install for pyflakes
Running setup.py install for pep8
Running setup.py install for smmap
Running setup.py install for gitdb
Running setup.py install for GitPython
Running setup.py install for twitter.common.lang
Running setup.py install for twitter.common.string
Running setup.py install for twitter.common.process
Running setup.py install for twitter.common.options
Running setup.py install for twitter.common.dirutil
Running setup.py install for twitter.common.log
Running setup.py install for twitter.common.contextutil
Running setup.py install for twitter.common.util
Running setup.py install for twitter.common.collections
Running setup.py install for twitter.common.app
Running setup.py install for twitter.checkstyle
Successfully installed GitPython-0.3.2rc1 gitdb-0.6.4 pep8-1.4.5 pyflakes-0.7.2 smmap-0.9.0 twitter.checkstyle-0.1.0 twitter.common.app-0.3.0 twitter.common.collections-0.3.0 twitter.common.contextutil-0.3.0 twitter.common.dirutil-0.3.0 twitter.common.lang-0.3.0 twitter.common.log-0.3.0 twitter.common.options-0.3.0 twitter.common.process-0.3.0 twitter.common.string-0.3.0 twitter.common.util-0.3.0
F401:ERROR src/main/python/apache/aurora/config/loader.py:015 'json' imported but unused
|import json
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On July 13, 2015, 10:53 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36459/
> -----------------------------------------------------------
>
> (Updated July 13, 2015, 10:53 p.m.)
>
>
> Review request for Aurora and Bill Farner.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Fix AuroraConfigLoader schema filtering when loading from string.
>
> We have an internal tool that needs to serialize configs into a string and retrieve them from stable storage. We had deployed a client containing shutdown_endpoint then reverted it in https://reviews.apache.org/r/35847/ but unfortunately deserialization broke the internal tool since it was not filtering out the unknown attributes.
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/config/loader.py c8b045e9a668199feee0ac3cee9fac55fe09cf08
> src/test/python/apache/aurora/config/test_loader.py 00b6eabadb8f4e40aba1191068e1ca04c78d1ff3
>
> Diff: https://reviews.apache.org/r/36459/diff/
>
>
> Testing
> -------
>
> Added test, it broke. Fixed code, tests passed.
>
> Also updated tests to not create files when not necessary.
>
>
> Thanks,
>
> Brian Wickman
>
>