You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by Andy Seaborne <an...@apache.org> on 2015/01/20 11:52:49 UTC
Windows build issues
This is repeatedly going wrong and I'm not sure why.
On the surface, it looks like there is a rogue Jetty sitting on port
3535 which then blocks the tests.
But:
1/ Tracing back to the last job to run at all (#507), it's some TDB/JDBC
tests failing for weird low level reasons.
2/ Switch between windows1 and windows2 slaves today gets nasty file
errors (various flavours - some clearly jenkins problems)
3/ I can't even wipe the workspace at the moment.
so my guess is that OS resources are the problem and we can't fix them.
I've email builds@ to ask for help and disabled the job for now.
We might be contributing to the problems; the windows builds being
resource hungry for mmap files, but the job was running fine for quite
some time until #507 and I can't see any change that might be connected.
Andy
Re: Windows build issues
Posted by Andy Seaborne <an...@apache.org>.
Thanks - that is a very useful piece of information.
[[I don't have a windows box/VM; the barrier is the x-zillion updates
whenever I fired one up to run a build]]
Andy
On 20/01/15 12:17, Bruno P. Kinoshita wrote:
> Looks like an OS issues indeed Andy. Build works fine on my Windows box.
>
> c:\Users\kinow\Downloads\jena-master>mvn clean test -e -X
> Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T17:58:1
> 0-03:00)
> Maven home: C:\dev\apache-maven-3.2.3
> Java version: 1.7.0_71, vendor: Oracle Corporation
> Java home: C:\dev\jdk-7u71-windows-x64\jre
> Default locale: pt_BR, platform encoding: Cp1252
> OS name: "windows 7", version: "6.1", arch: "amd64", family: "windows"
>
>
> (...)
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Jena - Parent ............................... SUCCESS [ 11.715 s]
> [INFO] Apache Jena - IRI .................................. SUCCESS [ 13.274 s]
> [INFO] Apache Jena - Core ................................. SUCCESS [04:18 min]
> [INFO] Apache Jena - ARQ (SPARQL 1.1 Query Engine) ........ SUCCESS [01:59 min]
> [INFO] Apache Jena - TDB (Native Triple Store) ............ SUCCESS [01:00 min]
> [INFO] Apache Jena - Libraries POM ........................ SUCCESS [ 0.874 s]
> [INFO] Apache Jena - SPARQL Text Search ................... SUCCESS [ 39.652 s]
> [INFO] Apache Jena - SPARQL Spatial Search ................ SUCCESS [ 20.281 s]
> [INFO] Apache Jena - Data Tables for RDF and SPARQL ....... SUCCESS [ 14.909 s]
> [INFO] Apache Jena - SDB (SQL based triple store) ......... SUCCESS [ 50.005 s]
> [INFO] Apache Jena - Fuseki1 (SPARQL 1.1 Server) .......... SUCCESS [ 33.137 s]
> [INFO] Apache Jena - Fuseki ............................... SUCCESS [ 0.233 s]
> [INFO] Apache Jena - Fuseki Server Engine ................. SUCCESS [ 34.061 s]
> [INFO] Apache Jena - Fuseki WAR File ...................... SUCCESS [ 2.013 s]
> [INFO] Apache Jena - Fuseki Server Standalone Jar ......... SUCCESS [ 1.944 s]
> [INFO] Apache Jena - Fuseki Binary Distribution ........... SUCCESS [ 1.566 s]
> [INFO] Apache Jena - Security ............................. SUCCESS [ 20.831 s]
> [INFO] Apache Jena - JDBC Parent .......................... SUCCESS [ 3.368 s]
> [INFO] Apache Jena - JDBC Core API ........................ SUCCESS [ 8.339 s]
> [INFO] Apache Jena - JDBC Remote Endpoint Driver .......... SUCCESS [ 22.591 s]
> [INFO] Apache Jena - JDBC In-Memory Driver ................ SUCCESS [ 12.732 s]
> [INFO] Apache Jena - JDBC TDB Driver ...................... SUCCESS [02:17 min]
> [INFO] Apache Jena - JDBC Driver Bundle ................... SUCCESS [ 12.761 s]
> [INFO] Apache Jena - Maven Plugins, including schemagen ... SUCCESS [01:50 min]
> [INFO] Apache Jena - Distribution ......................... SUCCESS [ 1.056 s]
> [INFO] Apache Jena - Extras ............................... SUCCESS [ 0.227 s]
> [INFO] Apache Jena - Extras - Query Builder ............... SUCCESS [ 28.749 s]
> [INFO] Apache Jena ........................................ SUCCESS [ 15.301 s]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 17:17 min
> [INFO] Finished at: 2015-01-20T10:06:06-02:00
> [INFO] Final Memory: 71M/421M
> [INFO] ------------------------------------------------------------------------
>
> Bruno
>
>> ________________________________
>> From: Andy Seaborne <an...@apache.org>
>> To: dev@jena.apache.org
>> Sent: Tuesday, January 20, 2015 8:52 AM
>> Subject: Windows build issues
>>
>>
>> This is repeatedly going wrong and I'm not sure why.
>>
>> On the surface, it looks like there is a rogue Jetty sitting on port
>> 3535 which then blocks the tests.
>>
>> But:
>> 1/ Tracing back to the last job to run at all (#507), it's some TDB/JDBC
>> tests failing for weird low level reasons.
>>
>> 2/ Switch between windows1 and windows2 slaves today gets nasty file
>> errors (various flavours - some clearly jenkins problems)
>>
>> 3/ I can't even wipe the workspace at the moment.
>>
>> so my guess is that OS resources are the problem and we can't fix them.
>> I've email builds@ to ask for help and disabled the job for now.
>>
>> We might be contributing to the problems; the windows builds being
>> resource hungry for mmap files, but the job was running fine for quite
>> some time until #507 and I can't see any change that might be connected.
>>
>> Andy
>>
>>
>>
Re: Windows build issues
Posted by "Bruno P. Kinoshita" <ki...@apache.org>.
Looks like an OS issues indeed Andy. Build works fine on my Windows box.
c:\Users\kinow\Downloads\jena-master>mvn clean test -e -X
Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T17:58:1
0-03:00)
Maven home: C:\dev\apache-maven-3.2.3
Java version: 1.7.0_71, vendor: Oracle Corporation
Java home: C:\dev\jdk-7u71-windows-x64\jre
Default locale: pt_BR, platform encoding: Cp1252
OS name: "windows 7", version: "6.1", arch: "amd64", family: "windows"
(...)
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Jena - Parent ............................... SUCCESS [ 11.715 s]
[INFO] Apache Jena - IRI .................................. SUCCESS [ 13.274 s]
[INFO] Apache Jena - Core ................................. SUCCESS [04:18 min]
[INFO] Apache Jena - ARQ (SPARQL 1.1 Query Engine) ........ SUCCESS [01:59 min]
[INFO] Apache Jena - TDB (Native Triple Store) ............ SUCCESS [01:00 min]
[INFO] Apache Jena - Libraries POM ........................ SUCCESS [ 0.874 s]
[INFO] Apache Jena - SPARQL Text Search ................... SUCCESS [ 39.652 s]
[INFO] Apache Jena - SPARQL Spatial Search ................ SUCCESS [ 20.281 s]
[INFO] Apache Jena - Data Tables for RDF and SPARQL ....... SUCCESS [ 14.909 s]
[INFO] Apache Jena - SDB (SQL based triple store) ......... SUCCESS [ 50.005 s]
[INFO] Apache Jena - Fuseki1 (SPARQL 1.1 Server) .......... SUCCESS [ 33.137 s]
[INFO] Apache Jena - Fuseki ............................... SUCCESS [ 0.233 s]
[INFO] Apache Jena - Fuseki Server Engine ................. SUCCESS [ 34.061 s]
[INFO] Apache Jena - Fuseki WAR File ...................... SUCCESS [ 2.013 s]
[INFO] Apache Jena - Fuseki Server Standalone Jar ......... SUCCESS [ 1.944 s]
[INFO] Apache Jena - Fuseki Binary Distribution ........... SUCCESS [ 1.566 s]
[INFO] Apache Jena - Security ............................. SUCCESS [ 20.831 s]
[INFO] Apache Jena - JDBC Parent .......................... SUCCESS [ 3.368 s]
[INFO] Apache Jena - JDBC Core API ........................ SUCCESS [ 8.339 s]
[INFO] Apache Jena - JDBC Remote Endpoint Driver .......... SUCCESS [ 22.591 s]
[INFO] Apache Jena - JDBC In-Memory Driver ................ SUCCESS [ 12.732 s]
[INFO] Apache Jena - JDBC TDB Driver ...................... SUCCESS [02:17 min]
[INFO] Apache Jena - JDBC Driver Bundle ................... SUCCESS [ 12.761 s]
[INFO] Apache Jena - Maven Plugins, including schemagen ... SUCCESS [01:50 min]
[INFO] Apache Jena - Distribution ......................... SUCCESS [ 1.056 s]
[INFO] Apache Jena - Extras ............................... SUCCESS [ 0.227 s]
[INFO] Apache Jena - Extras - Query Builder ............... SUCCESS [ 28.749 s]
[INFO] Apache Jena ........................................ SUCCESS [ 15.301 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17:17 min
[INFO] Finished at: 2015-01-20T10:06:06-02:00
[INFO] Final Memory: 71M/421M
[INFO] ------------------------------------------------------------------------
Bruno
>________________________________
> From: Andy Seaborne <an...@apache.org>
>To: dev@jena.apache.org
>Sent: Tuesday, January 20, 2015 8:52 AM
>Subject: Windows build issues
>
>
>This is repeatedly going wrong and I'm not sure why.
>
>On the surface, it looks like there is a rogue Jetty sitting on port
>3535 which then blocks the tests.
>
>But:
>1/ Tracing back to the last job to run at all (#507), it's some TDB/JDBC
>tests failing for weird low level reasons.
>
>2/ Switch between windows1 and windows2 slaves today gets nasty file
>errors (various flavours - some clearly jenkins problems)
>
>3/ I can't even wipe the workspace at the moment.
>
>so my guess is that OS resources are the problem and we can't fix them.
> I've email builds@ to ask for help and disabled the job for now.
>
>We might be contributing to the problems; the windows builds being
>resource hungry for mmap files, but the job was running fine for quite
>some time until #507 and I can't see any change that might be connected.
>
> Andy
>
>
>
Re: Windows build issues
Posted by Andy Seaborne <an...@apache.org>.
On 20/01/15 14:18, Rob Hall wrote:
> The fix isn't in the mongo code, but rather the is os itself requires the
> hotfix. Check the mongodb windows installation instructions for ref.
Got it, thanks. It's about flushing files.
That's a hotfix to Windows7, not java. Maybe useful to apply but it
does not affect the fact that the JVM on windows simply does not have
the functionality to delete mmap files mid-process.
FWIW MMap files on Windows don't seem to be faster than TDB in direct
(old fashioned, in-JVM caching) or so I'm told. Nowadays it possible to
tune the internal TDB caches. Moving to Guava concurrent caches will
help (JENA-801, which principally about the node table but if in direct
mode on Windows, the same applies).
Andy
> On Jan 20, 2015 8:45 AM, "Andy Seaborne" <an...@apache.org> wrote:
>
>>
>> On 20/01/15 12:15, Rob Hall wrote:
>>
>>> I did randomly notice that mongo db had to have a hot fix for windows 7+
>>> users that solved a bug associated with Windows memory mapped files.
>>>
>>> If this is a Java+Windows environment problem, that might be a clue as to
>>> what's going on.
>>>
>>
>> The build problem looks like resources, not java.
>>
>> The reason why TDB has Windows specific testing is the (in)famous:
>>
>> http://bugs.java.com/view_bug.do?bug_id=4715154
>>
>> (and several others)
>>
>> Long time, won't fix, bug. (There are pragmatic, limited, JVM-specific
>> semi-workarounds.)
>>
>> If MongoDB have had a fix recently, I'd be interested to know more - my
>> Google foo wasn't strong enough to find it.
>>
>> TDB's test setup goes through hoops to workaround it and it leads to large
>> amounts of temporary file space that does not go away until end of JVM.
>> Plus users do sometimes want to delete a database awhile running.
>>
>> Andy
>>
>> Relevant hot fix: http://support.microsoft.com/kb/2731284
>>> On Jan 20, 2015 6:04 AM, "Andy Seaborne" <an...@apache.org> wrote:
>>>
>>> This is repeatedly going wrong and I'm not sure why.
>>>>
>>>> On the surface, it looks like there is a rogue Jetty sitting on port 3535
>>>> which then blocks the tests.
>>>>
>>>> But:
>>>> 1/ Tracing back to the last job to run at all (#507), it's some TDB/JDBC
>>>> tests failing for weird low level reasons.
>>>>
>>>> 2/ Switch between windows1 and windows2 slaves today gets nasty file
>>>> errors (various flavours - some clearly jenkins problems)
>>>>
>>>> 3/ I can't even wipe the workspace at the moment.
>>>>
>>>> so my guess is that OS resources are the problem and we can't fix them.
>>>> I've email builds@ to ask for help and disabled the job for now.
>>>>
>>>> We might be contributing to the problems; the windows builds being
>>>> resource hungry for mmap files, but the job was running fine for quite
>>>> some
>>>> time until #507 and I can't see any change that might be connected.
>>>>
>>>> Andy
>>>>
>>>>
>>>
>>
>
Re: Windows build issues
Posted by Rob Hall <kv...@gmail.com>.
The fix isn't in the mongo code, but rather the is os itself requires the
hotfix. Check the mongodb windows installation instructions for ref.
On Jan 20, 2015 8:45 AM, "Andy Seaborne" <an...@apache.org> wrote:
>
> On 20/01/15 12:15, Rob Hall wrote:
>
>> I did randomly notice that mongo db had to have a hot fix for windows 7+
>> users that solved a bug associated with Windows memory mapped files.
>>
>> If this is a Java+Windows environment problem, that might be a clue as to
>> what's going on.
>>
>
> The build problem looks like resources, not java.
>
> The reason why TDB has Windows specific testing is the (in)famous:
>
> http://bugs.java.com/view_bug.do?bug_id=4715154
>
> (and several others)
>
> Long time, won't fix, bug. (There are pragmatic, limited, JVM-specific
> semi-workarounds.)
>
> If MongoDB have had a fix recently, I'd be interested to know more - my
> Google foo wasn't strong enough to find it.
>
> TDB's test setup goes through hoops to workaround it and it leads to large
> amounts of temporary file space that does not go away until end of JVM.
> Plus users do sometimes want to delete a database awhile running.
>
> Andy
>
> Relevant hot fix: http://support.microsoft.com/kb/2731284
>> On Jan 20, 2015 6:04 AM, "Andy Seaborne" <an...@apache.org> wrote:
>>
>> This is repeatedly going wrong and I'm not sure why.
>>>
>>> On the surface, it looks like there is a rogue Jetty sitting on port 3535
>>> which then blocks the tests.
>>>
>>> But:
>>> 1/ Tracing back to the last job to run at all (#507), it's some TDB/JDBC
>>> tests failing for weird low level reasons.
>>>
>>> 2/ Switch between windows1 and windows2 slaves today gets nasty file
>>> errors (various flavours - some clearly jenkins problems)
>>>
>>> 3/ I can't even wipe the workspace at the moment.
>>>
>>> so my guess is that OS resources are the problem and we can't fix them.
>>> I've email builds@ to ask for help and disabled the job for now.
>>>
>>> We might be contributing to the problems; the windows builds being
>>> resource hungry for mmap files, but the job was running fine for quite
>>> some
>>> time until #507 and I can't see any change that might be connected.
>>>
>>> Andy
>>>
>>>
>>
>
Re: Windows build issues
Posted by Andy Seaborne <an...@apache.org>.
On 20/01/15 12:15, Rob Hall wrote:
> I did randomly notice that mongo db had to have a hot fix for windows 7+
> users that solved a bug associated with Windows memory mapped files.
>
> If this is a Java+Windows environment problem, that might be a clue as to
> what's going on.
The build problem looks like resources, not java.
The reason why TDB has Windows specific testing is the (in)famous:
http://bugs.java.com/view_bug.do?bug_id=4715154
(and several others)
Long time, won't fix, bug. (There are pragmatic, limited, JVM-specific
semi-workarounds.)
If MongoDB have had a fix recently, I'd be interested to know more - my
Google foo wasn't strong enough to find it.
TDB's test setup goes through hoops to workaround it and it leads to
large amounts of temporary file space that does not go away until end of
JVM. Plus users do sometimes want to delete a database awhile running.
Andy
> Relevant hot fix: http://support.microsoft.com/kb/2731284
> On Jan 20, 2015 6:04 AM, "Andy Seaborne" <an...@apache.org> wrote:
>
>> This is repeatedly going wrong and I'm not sure why.
>>
>> On the surface, it looks like there is a rogue Jetty sitting on port 3535
>> which then blocks the tests.
>>
>> But:
>> 1/ Tracing back to the last job to run at all (#507), it's some TDB/JDBC
>> tests failing for weird low level reasons.
>>
>> 2/ Switch between windows1 and windows2 slaves today gets nasty file
>> errors (various flavours - some clearly jenkins problems)
>>
>> 3/ I can't even wipe the workspace at the moment.
>>
>> so my guess is that OS resources are the problem and we can't fix them.
>> I've email builds@ to ask for help and disabled the job for now.
>>
>> We might be contributing to the problems; the windows builds being
>> resource hungry for mmap files, but the job was running fine for quite some
>> time until #507 and I can't see any change that might be connected.
>>
>> Andy
>>
>
Re: Windows build issues
Posted by Rob Hall <kv...@gmail.com>.
I did randomly notice that mongo db had to have a hot fix for windows 7+
users that solved a bug associated with Windows memory mapped files.
If this is a Java+Windows environment problem, that might be a clue as to
what's going on.
Relevant hot fix: http://support.microsoft.com/kb/2731284
On Jan 20, 2015 6:04 AM, "Andy Seaborne" <an...@apache.org> wrote:
> This is repeatedly going wrong and I'm not sure why.
>
> On the surface, it looks like there is a rogue Jetty sitting on port 3535
> which then blocks the tests.
>
> But:
> 1/ Tracing back to the last job to run at all (#507), it's some TDB/JDBC
> tests failing for weird low level reasons.
>
> 2/ Switch between windows1 and windows2 slaves today gets nasty file
> errors (various flavours - some clearly jenkins problems)
>
> 3/ I can't even wipe the workspace at the moment.
>
> so my guess is that OS resources are the problem and we can't fix them.
> I've email builds@ to ask for help and disabled the job for now.
>
> We might be contributing to the problems; the windows builds being
> resource hungry for mmap files, but the job was running fine for quite some
> time until #507 and I can't see any change that might be connected.
>
> Andy
>