You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pirk.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/07/29 02:55:28 UTC
Jenkins build became unstable: pirk » Apache Pirk (incubating) Project #13
See <https://builds.apache.org/job/pirk/org.apache.pirk$pirk/13/changes>
Jenkins build is back to stable : pirk » Apache Pirk (incubating) Project #16
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/pirk/org.apache.pirk$pirk/16/changes>
Re: Jenkins build became unstable: pirk » Apache Pirk (incubating) Project #13
Posted by Ellison Anne Williams <ea...@gmail.com>.
Hi Tim,
The 'packaged' property files are first loaded in order:
(SystemConfiguration, line 65)
// First try to load the default properties file
loadPropsFromStream(DEFAULT_PROPERTY_FILE);
// Try to load props from the querier and responder property files, if
they exist
loadPropsFromStream(QUERIER_PROPERTIES_FILE);
loadPropsFromStream(RESPONDER_PROPERTIES_FILE);
And then any files ending in '.properties' are loaded from the specified
local directory:
// Try to load the local properties files, if they exists
loadPropsFromDir(LOCAL_PROPERTIES_DIR);
Thus, as the 'packaged' property files are loaded in order so there is no
chance of inconsistency. The files in the LOCAL_PROPERTIES_DIR are loaded
in arbitrary order and thus the properties placed in files within this
folder must be distinct (or, as you pointed out, they will be unpredictably
overwritten).
Ellison Anne
On Fri, Jul 29, 2016 at 5:23 AM, Tim Ellison <t....@gmail.com> wrote:
> On 29/07/16 03:55, Apache Jenkins Server wrote:
> > See <https://builds.apache.org/job/pirk/org.apache.pirk$pirk/13/changes>
>
> Travis liked this commit ok, but Jenkins failed with:
>
> java.io.IOException: No such file or directory
> at java.io.UnixFileSystem.createFileExclusively(Native Method)
> at java.io.File.createTempFile(File.java:1879)
> at java.io.File.createTempFile(File.java:1923)
> at
> org.apache.pirk.test.utils.TestUtils.writeToTmpFile(TestUtils.java:300)
> at
> org.apache.pirk.test.utils.Inputs.createPIRStopList(Inputs.java:411)
> at
> test.wideskies.standalone.StandaloneTest.<init>(StandaloneTest.java:55)
>
>
> == Observation 1 ==
>
> Looks like there was a change to
> org.apache.pirk.test.utils.Inputs#createPIRStopList(FileSystem,
> boolean):line 411
>
> that means the standalone test stop list file name property is now
> DistributedTestDriver.PIR_STOPLIST_FILE
> (a.k.a. "test.pir.stopListFile"), whereas before the commit it was
> "pir.stopListFile".
>
> Was that intentional? I'm suspicious because StandaloneTest:line 54 is
> still setting "pir.stopListFile"
>
> Also, is it correct that the hdfs=true case falls through and creates
> the local file stop list too?
>
> I've opened PR#30 to put the local property name back to
> "pir.stopListFile".
>
>
> == Observation 2 ==
>
> The new code in SystemConfiguration#loadPropsFromDir(String) to load all
> properties from a directory can cause inconsistent behavior.
>
> The final set of properties will depend upon the order of the files
> loaded, and that is not guaranteed, i.e. any property set in a loaded
> file will be overwritten by the same property set in a file that is
> loaded later.
>
> For example, in this run:
> INFO [main] 2016-07-29 02:49:19,345 (SystemConfiguration.java:202) -
> Loading file 'pirk.properties'
> INFO [main] 2016-07-29 02:49:19,362 (SystemConfiguration.java:202) -
> Loading file 'querier.properties'
> INFO [main] 2016-07-29 02:49:19,362 (SystemConfiguration.java:202) -
> Loading file 'responder.properties'
>
> any pirk.properties configuration would be overwritten by the same
> property given in querier.properties.
>
> While in this case the files were loaded in alphabetical order, there is
> no guarantee that the directory list, hence the properties had better be
> disjoint sets.
>
> Regards,
> Tim
>
Re: Jenkins build became unstable: pirk » Apache Pirk (incubating) Project #13
Posted by Tim Ellison <t....@gmail.com>.
On 29/07/16 03:55, Apache Jenkins Server wrote:
> See <https://builds.apache.org/job/pirk/org.apache.pirk$pirk/13/changes>
Travis liked this commit ok, but Jenkins failed with:
java.io.IOException: No such file or directory
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createTempFile(File.java:1879)
at java.io.File.createTempFile(File.java:1923)
at org.apache.pirk.test.utils.TestUtils.writeToTmpFile(TestUtils.java:300)
at org.apache.pirk.test.utils.Inputs.createPIRStopList(Inputs.java:411)
at test.wideskies.standalone.StandaloneTest.<init>(StandaloneTest.java:55)
== Observation 1 ==
Looks like there was a change to
org.apache.pirk.test.utils.Inputs#createPIRStopList(FileSystem,
boolean):line 411
that means the standalone test stop list file name property is now
DistributedTestDriver.PIR_STOPLIST_FILE
(a.k.a. "test.pir.stopListFile"), whereas before the commit it was
"pir.stopListFile".
Was that intentional? I'm suspicious because StandaloneTest:line 54 is
still setting "pir.stopListFile"
Also, is it correct that the hdfs=true case falls through and creates
the local file stop list too?
I've opened PR#30 to put the local property name back to "pir.stopListFile".
== Observation 2 ==
The new code in SystemConfiguration#loadPropsFromDir(String) to load all
properties from a directory can cause inconsistent behavior.
The final set of properties will depend upon the order of the files
loaded, and that is not guaranteed, i.e. any property set in a loaded
file will be overwritten by the same property set in a file that is
loaded later.
For example, in this run:
INFO [main] 2016-07-29 02:49:19,345 (SystemConfiguration.java:202) -
Loading file 'pirk.properties'
INFO [main] 2016-07-29 02:49:19,362 (SystemConfiguration.java:202) -
Loading file 'querier.properties'
INFO [main] 2016-07-29 02:49:19,362 (SystemConfiguration.java:202) -
Loading file 'responder.properties'
any pirk.properties configuration would be overwritten by the same
property given in querier.properties.
While in this case the files were loaded in alphabetical order, there is
no guarantee that the directory list, hence the properties had better be
disjoint sets.
Regards,
Tim