You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@lucenenet.apache.org by Itamar Syn-Hershko <it...@code972.com> on 2015/04/08 19:07:11 UTC

Last failing tests remain to fix!

Hey all,

We are nearly there -- we are this close to a release!

We have only about 30 failing tests now in the core, and I'd like to push
hard during the next few days so we can get a beta out next week.

I'd appreciate your help with this!

I've narrowed down several failing tests that require some digging, if
folks can assign those to themselves and send us PRs fixing those tests,
that would help a lot!

1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
specifically it seems like Double and Float NaNs are treated differently
between C# and Java and therefore the failing tests. Or there is a bug in
the inner implementation handling those queries.

2. TestLucene40DocValuesFormat has a few failing tests, probably exposing
data corruption in one of the codec components. I believe this is contained
in the 40DocValues tests so the scope isn't huge but I could be wrong.

3. TestLucene42DocValuesFormat has two tests failing with output of lines
like "Fail: frozenHash=182658 vs h=5124052". Something is off in computing
the hashes of the nodes, or maybe the node construction is faulty (which is
a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll be a
good starting point. Also see
https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c

4. There seem to be some concurrency issues with file handling, probably on
the part that is in charge of acquiring and releasing locks. In particular
see TestDocValuesFormat,
.TestShardSearching.TestSimple(),
TestBufferedIndexInput.TestSetBufferSize(),
TestLockFactory.TestStressLocks(),
TestLockFactory.TestStressLocksNativeFSLockFactory()

5. RegexpQuery seems to be broken, I believe the underlying automaton
implementation doesn't emit the terms correctly. See the failures in
TestRegexpQuery and TestRegexpRandom.

6. Similar but probably unrelated failing test is
at TestSpanMultiTermQueryWrapper.TestFuzzy2()

7. An interesting failure to investigate is here:
TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
simple porting fail, a subtle Java <--> C# mismatch that we haven't noticed.

Additionally, any help with porting the subprojects would be great. Some
folks took some subprojects under their wings and never reported back. If
you could either complete the work, or donate whatever you have done so
far, that would be great.

As always, get the code from here: https://github.com/apache/lucenenet --
work on your fork and send us a PR when ready, with a good description of
what you've done.

To see the list of currently failing tests in real-time, go here (you can
login as guest):
http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv

Thanks, and happy searching,

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

Thanks! just pinged her on Twitter

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Thu, Apr 16, 2015 at 12:24 AM, Kevin Miller <sc...@gmail.com> wrote:

> Annie helps with the care and feeding of the CodeBetter TeamCity server.
> Reach out to her on Twitter and she will get you some help.
>
>
>
>
> "They can certainly contact me-I can either help or I can put them in
> contact with people who can help.”
>
>
>
>
>
>
> https://twitter.com/ajepst/status/588448988367433728
>
> On Tue, Apr 14, 2015 at 1:36 PM, Wyatt Barnett <wy...@gmail.com>
> wrote:
>
> > Sorry, my alter ego Wade and I just got back in town here.
> > I'm looking into the NuGet feed setup though I need to talk to someone
> over
> > at CodeBetter.com as some things need to be enabled and I need to figure
> > out where the artifacts. I know Martin has moved on from JetBrains, is
> > there another contact?
> > To answer your question, no we don't need to wait on passing builds -- we
> > can tell it to build the nuget regardless. We should probably talk about
> > what should trigger a nuget release -- probably every build is too much.
> > Maybe nightly builds for now?
> > On Mon, Apr 13, 2015 at 5:20 PM Itamar Syn-Hershko <it...@code972.com>
> > wrote:
> >> Ok, I just looked and the nuget feed is empty - probably because there
> was
> >> never a green build. Can we have nuget artifacts already, at least
> until we
> >> have green builds?
> >>
> >> --
> >>
> >> Itamar Syn-Hershko
> >> http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> Freelance Developer & Consultant
> >> Lucene.NET committer and PMC member
> >>
> >> On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <
> itamar@code972.com>
> >> wrote:
> >>
> >> > Errr.. I mean Wyatt :)
> >> >
> >> > --
> >> >
> >> > Itamar Syn-Hershko
> >> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> > Freelance Developer & Consultant
> >> > Lucene.NET committer and PMC member
> >> >
> >> > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <
> itamar@code972.com
> >> >
> >> > wrote:
> >> >
> >> >> Hi Wade, any luck?
> >> >>
> >> >> Is there a nuget feed on the build server we can start using to
> install
> >> >> the core / subproject pieces using nuget from?
> >> >>
> >> >> Thanks!
> >> >>
> >> >> --
> >> >>
> >> >> Itamar Syn-Hershko
> >> >> http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> >> Freelance Developer & Consultant
> >> >> Lucene.NET committer and PMC member
> >> >>
> >> >> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <
> wyatt.barnett@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Yup that makes sense -- somehow I conflated sub projects over to
> >> contrib
> >> >>> which I haven't seen any movement.
> >> >>>
> >> >>> I'm on the road at the moment but I'll see about getting the
> remaining
> >> >>> sub
> >> >>> projects setup and building, auditing the ignores and start banging
> on
> >> >>> nuget when I get back to the ranch.
> >> >>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com>
> >> wrote:
> >> >>>
> >> >>> > inline
> >> >>> >
> >> >>> > --
> >> >>> >
> >> >>> > Itamar Syn-Hershko
> >> >>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> >>> > Freelance Developer & Consultant
> >> >>> > Lucene.NET committer and PMC member
> >> >>> >
> >> >>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <
> >> wyatt.barnett@gmail.com
> >> >>> >
> >> >>> > wrote:
> >> >>> >
> >> >>> > > Thanks for the update Itamar -- this sounds exciting.
> >> >>> > >
> >> >>> > > A few notes / questions from the build side of things here:
> >> >>> > >
> >> >>> > > First, I'm not as hung up on persistently failing tests as the
> >> tests
> >> >>> we
> >> >>> > get
> >> >>> > > intermittent failures on. I suspect some of the below will
> address
> >> >>> this.
> >> >>> > >
> >> >>> >
> >> >>> > Yup
> >> >>> >
> >> >>> >
> >> >>> > >
> >> >>> > > Second, what do we want to do about the 112 or so ignored
> tests? I
> >> >>> think
> >> >>> > > some are marked as long-running via the [LongRunningTest]
> attribute
> >> >>> but
> >> >>> > > there are probably some ignored failures in that mix.
> >> >>> > >
> >> >>> >
> >> >>> > I guess that should be another task someone needs to take - can
> you
> >> >>> try and
> >> >>> > categorize them? some I know are JVM specfic tests that don't
> apply
> >> to
> >> >>> > Lucene.NET
> >> >>> >
> >> >>> >
> >> >>> > >
> >> >>> > > Third, regarding the sub-projects, what does the structure look
> >> like
> >> >>> so I
> >> >>> > > can get that integrated into the TeamCity setup? Or should I
> noodle
> >> >>> on it
> >> >>> > > and propose one?
> >> >>> > >
> >> >>> >
> >> >>> > What do you mean? See here
> >> >>> > https://github.com/apache/lucenenet/tree/master/src
> >> >>> >
> >> >>> > Every subproject has a test suite that needs to be run, and should
> >> >>> produce
> >> >>> > a class library that we could push to nuget when we release. Until
> >> >>> then we
> >> >>> > should be able to allow people to install via nuget from our build
> >> >>> server's
> >> >>> > feed.
> >> >>> >
> >> >>> > As far as dependencies and build triggers - that should be obvious
> >> and
> >> >>> self
> >> >>> > explanatory..
> >> >>> >
> >> >>> > Does that make sense?
> >> >>> >
> >> >>> >
> >> >>> > >
> >> >>> > > Thanks for all the help from everyone getting things this far.
> >> >>> > >
> >> >>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
> >> >>> itamar@code972.com>
> >> >>> > > wrote:
> >> >>> > >
> >> >>> > > > Hey all,
> >> >>> > > >
> >> >>> > > > We are nearly there -- we are this close to a release!
> >> >>> > > >
> >> >>> > > > We have only about 30 failing tests now in the core, and I'd
> like
> >> >>> to
> >> >>> > push
> >> >>> > > > hard during the next few days so we can get a beta out next
> week.
> >> >>> > > >
> >> >>> > > > I'd appreciate your help with this!
> >> >>> > > >
> >> >>> > > > I've narrowed down several failing tests that require some
> >> >>> digging, if
> >> >>> > > > folks can assign those to themselves and send us PRs fixing
> those
> >> >>> > tests,
> >> >>> > > > that would help a lot!
> >> >>> > > >
> >> >>> > > > 1.
> Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues
> >> ()
> >> >>> > > > and
> Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues
> >> ()
> >> >>> --
> >> >>> > > > specifically it seems like Double and Float NaNs are treated
> >> >>> > differently
> >> >>> > > > between C# and Java and therefore the failing tests. Or there
> is
> >> a
> >> >>> bug
> >> >>> > in
> >> >>> > > > the inner implementation handling those queries.
> >> >>> > > >
> >> >>> > > > 2. TestLucene40DocValuesFormat has a few failing tests,
> probably
> >> >>> > exposing
> >> >>> > > > data corruption in one of the codec components. I believe
> this is
> >> >>> > > contained
> >> >>> > > > in the 40DocValues tests so the scope isn't huge but I could
> be
> >> >>> wrong.
> >> >>> > > >
> >> >>> > > > 3. TestLucene42DocValuesFormat has two tests failing with
> output
> >> of
> >> >>> > lines
> >> >>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off
> in
> >> >>> > > computing
> >> >>> > > > the hashes of the nodes, or maybe the node construction is
> faulty
> >> >>> > (which
> >> >>> > > is
> >> >>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> >> >>> that'll
> >> >>> > > be a
> >> >>> > > > good starting point. Also see
> >> >>> > > >
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f910
> >> 90d995c729689fff9a0f54c
> >> >>> > > >
> >> >>> > > > 4. There seem to be some concurrency issues with file
> handling,
> >> >>> > probably
> >> >>> > > on
> >> >>> > > > the part that is in charge of acquiring and releasing locks.
> In
> >> >>> > > particular
> >> >>> > > > see TestDocValuesFormat,
> >> >>> > > > .TestShardSearching.TestSimple(),
> >> >>> > > > TestBufferedIndexInput.TestSetBufferSize(),
> >> >>> > > > TestLockFactory.TestStressLocks(),
> >> >>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> >> >>> > > >
> >> >>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
> >> >>> automaton
> >> >>> > > > implementation doesn't emit the terms correctly. See the
> failures
> >> >>> in
> >> >>> > > > TestRegexpQuery and TestRegexpRandom.
> >> >>> > > >
> >> >>> > > > 6. Similar but probably unrelated failing test is
> >> >>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> >> >>> > > >
> >> >>> > > > 7. An interesting failure to investigate is here:
> >> >>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
> >> >>> probably a
> >> >>> > > > simple porting fail, a subtle Java <--> C# mismatch that we
> >> haven't
> >> >>> > > > noticed.
> >> >>> > > >
> >> >>> > > > Additionally, any help with porting the subprojects would be
> >> great.
> >> >>> > Some
> >> >>> > > > folks took some subprojects under their wings and never
> reported
> >> >>> back.
> >> >>> > If
> >> >>> > > > you could either complete the work, or donate whatever you
> have
> >> >>> done so
> >> >>> > > > far, that would be great.
> >> >>> > > >
> >> >>> > > > As always, get the code from here:
> >> >>> https://github.com/apache/lucenenet
> >> >>> > > --
> >> >>> > > > work on your fork and send us a PR when ready, with a good
> >> >>> description
> >> >>> > of
> >> >>> > > > what you've done.
> >> >>> > > >
> >> >>> > > > To see the list of currently failing tests in real-time, go
> here
> >> >>> (you
> >> >>> > can
> >> >>> > > > login as guest):
> >> >>> > > >
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>> http://teamcity.codebetter.com/viewType.html?buildTypeId=Luc
> >> eneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> >> >>> > > >
> >> >>> > > > Thanks, and happy searching,
> >> >>> > > >
> >> >>> > > > --
> >> >>> > > >
> >> >>> > > > Itamar Syn-Hershko
> >> >>> > > > http://code972.com | @synhershko <
> https://twitter.com/synhershko
> >> >
> >> >>> > > > Freelance Developer & Consultant
> >> >>> > > > Lucene.NET committer and PMC member
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >
> >>
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

Thanks! just pinged her on Twitter

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Thu, Apr 16, 2015 at 12:24 AM, Kevin Miller <sc...@gmail.com> wrote:

> Annie helps with the care and feeding of the CodeBetter TeamCity server.
> Reach out to her on Twitter and she will get you some help.
>
>
>
>
> "They can certainly contact me-I can either help or I can put them in
> contact with people who can help.”
>
>
>
>
>
>
> https://twitter.com/ajepst/status/588448988367433728
>
> On Tue, Apr 14, 2015 at 1:36 PM, Wyatt Barnett <wy...@gmail.com>
> wrote:
>
> > Sorry, my alter ego Wade and I just got back in town here.
> > I'm looking into the NuGet feed setup though I need to talk to someone
> over
> > at CodeBetter.com as some things need to be enabled and I need to figure
> > out where the artifacts. I know Martin has moved on from JetBrains, is
> > there another contact?
> > To answer your question, no we don't need to wait on passing builds -- we
> > can tell it to build the nuget regardless. We should probably talk about
> > what should trigger a nuget release -- probably every build is too much.
> > Maybe nightly builds for now?
> > On Mon, Apr 13, 2015 at 5:20 PM Itamar Syn-Hershko <it...@code972.com>
> > wrote:
> >> Ok, I just looked and the nuget feed is empty - probably because there
> was
> >> never a green build. Can we have nuget artifacts already, at least
> until we
> >> have green builds?
> >>
> >> --
> >>
> >> Itamar Syn-Hershko
> >> http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> Freelance Developer & Consultant
> >> Lucene.NET committer and PMC member
> >>
> >> On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <
> itamar@code972.com>
> >> wrote:
> >>
> >> > Errr.. I mean Wyatt :)
> >> >
> >> > --
> >> >
> >> > Itamar Syn-Hershko
> >> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> > Freelance Developer & Consultant
> >> > Lucene.NET committer and PMC member
> >> >
> >> > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <
> itamar@code972.com
> >> >
> >> > wrote:
> >> >
> >> >> Hi Wade, any luck?
> >> >>
> >> >> Is there a nuget feed on the build server we can start using to
> install
> >> >> the core / subproject pieces using nuget from?
> >> >>
> >> >> Thanks!
> >> >>
> >> >> --
> >> >>
> >> >> Itamar Syn-Hershko
> >> >> http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> >> Freelance Developer & Consultant
> >> >> Lucene.NET committer and PMC member
> >> >>
> >> >> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <
> wyatt.barnett@gmail.com>
> >> >> wrote:
> >> >>
> >> >>> Yup that makes sense -- somehow I conflated sub projects over to
> >> contrib
> >> >>> which I haven't seen any movement.
> >> >>>
> >> >>> I'm on the road at the moment but I'll see about getting the
> remaining
> >> >>> sub
> >> >>> projects setup and building, auditing the ignores and start banging
> on
> >> >>> nuget when I get back to the ranch.
> >> >>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com>
> >> wrote:
> >> >>>
> >> >>> > inline
> >> >>> >
> >> >>> > --
> >> >>> >
> >> >>> > Itamar Syn-Hershko
> >> >>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> >>> > Freelance Developer & Consultant
> >> >>> > Lucene.NET committer and PMC member
> >> >>> >
> >> >>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <
> >> wyatt.barnett@gmail.com
> >> >>> >
> >> >>> > wrote:
> >> >>> >
> >> >>> > > Thanks for the update Itamar -- this sounds exciting.
> >> >>> > >
> >> >>> > > A few notes / questions from the build side of things here:
> >> >>> > >
> >> >>> > > First, I'm not as hung up on persistently failing tests as the
> >> tests
> >> >>> we
> >> >>> > get
> >> >>> > > intermittent failures on. I suspect some of the below will
> address
> >> >>> this.
> >> >>> > >
> >> >>> >
> >> >>> > Yup
> >> >>> >
> >> >>> >
> >> >>> > >
> >> >>> > > Second, what do we want to do about the 112 or so ignored
> tests? I
> >> >>> think
> >> >>> > > some are marked as long-running via the [LongRunningTest]
> attribute
> >> >>> but
> >> >>> > > there are probably some ignored failures in that mix.
> >> >>> > >
> >> >>> >
> >> >>> > I guess that should be another task someone needs to take - can
> you
> >> >>> try and
> >> >>> > categorize them? some I know are JVM specfic tests that don't
> apply
> >> to
> >> >>> > Lucene.NET
> >> >>> >
> >> >>> >
> >> >>> > >
> >> >>> > > Third, regarding the sub-projects, what does the structure look
> >> like
> >> >>> so I
> >> >>> > > can get that integrated into the TeamCity setup? Or should I
> noodle
> >> >>> on it
> >> >>> > > and propose one?
> >> >>> > >
> >> >>> >
> >> >>> > What do you mean? See here
> >> >>> > https://github.com/apache/lucenenet/tree/master/src
> >> >>> >
> >> >>> > Every subproject has a test suite that needs to be run, and should
> >> >>> produce
> >> >>> > a class library that we could push to nuget when we release. Until
> >> >>> then we
> >> >>> > should be able to allow people to install via nuget from our build
> >> >>> server's
> >> >>> > feed.
> >> >>> >
> >> >>> > As far as dependencies and build triggers - that should be obvious
> >> and
> >> >>> self
> >> >>> > explanatory..
> >> >>> >
> >> >>> > Does that make sense?
> >> >>> >
> >> >>> >
> >> >>> > >
> >> >>> > > Thanks for all the help from everyone getting things this far.
> >> >>> > >
> >> >>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
> >> >>> itamar@code972.com>
> >> >>> > > wrote:
> >> >>> > >
> >> >>> > > > Hey all,
> >> >>> > > >
> >> >>> > > > We are nearly there -- we are this close to a release!
> >> >>> > > >
> >> >>> > > > We have only about 30 failing tests now in the core, and I'd
> like
> >> >>> to
> >> >>> > push
> >> >>> > > > hard during the next few days so we can get a beta out next
> week.
> >> >>> > > >
> >> >>> > > > I'd appreciate your help with this!
> >> >>> > > >
> >> >>> > > > I've narrowed down several failing tests that require some
> >> >>> digging, if
> >> >>> > > > folks can assign those to themselves and send us PRs fixing
> those
> >> >>> > tests,
> >> >>> > > > that would help a lot!
> >> >>> > > >
> >> >>> > > > 1.
> Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues
> >> ()
> >> >>> > > > and
> Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues
> >> ()
> >> >>> --
> >> >>> > > > specifically it seems like Double and Float NaNs are treated
> >> >>> > differently
> >> >>> > > > between C# and Java and therefore the failing tests. Or there
> is
> >> a
> >> >>> bug
> >> >>> > in
> >> >>> > > > the inner implementation handling those queries.
> >> >>> > > >
> >> >>> > > > 2. TestLucene40DocValuesFormat has a few failing tests,
> probably
> >> >>> > exposing
> >> >>> > > > data corruption in one of the codec components. I believe
> this is
> >> >>> > > contained
> >> >>> > > > in the 40DocValues tests so the scope isn't huge but I could
> be
> >> >>> wrong.
> >> >>> > > >
> >> >>> > > > 3. TestLucene42DocValuesFormat has two tests failing with
> output
> >> of
> >> >>> > lines
> >> >>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off
> in
> >> >>> > > computing
> >> >>> > > > the hashes of the nodes, or maybe the node construction is
> faulty
> >> >>> > (which
> >> >>> > > is
> >> >>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> >> >>> that'll
> >> >>> > > be a
> >> >>> > > > good starting point. Also see
> >> >>> > > >
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f910
> >> 90d995c729689fff9a0f54c
> >> >>> > > >
> >> >>> > > > 4. There seem to be some concurrency issues with file
> handling,
> >> >>> > probably
> >> >>> > > on
> >> >>> > > > the part that is in charge of acquiring and releasing locks.
> In
> >> >>> > > particular
> >> >>> > > > see TestDocValuesFormat,
> >> >>> > > > .TestShardSearching.TestSimple(),
> >> >>> > > > TestBufferedIndexInput.TestSetBufferSize(),
> >> >>> > > > TestLockFactory.TestStressLocks(),
> >> >>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> >> >>> > > >
> >> >>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
> >> >>> automaton
> >> >>> > > > implementation doesn't emit the terms correctly. See the
> failures
> >> >>> in
> >> >>> > > > TestRegexpQuery and TestRegexpRandom.
> >> >>> > > >
> >> >>> > > > 6. Similar but probably unrelated failing test is
> >> >>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> >> >>> > > >
> >> >>> > > > 7. An interesting failure to investigate is here:
> >> >>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
> >> >>> probably a
> >> >>> > > > simple porting fail, a subtle Java <--> C# mismatch that we
> >> haven't
> >> >>> > > > noticed.
> >> >>> > > >
> >> >>> > > > Additionally, any help with porting the subprojects would be
> >> great.
> >> >>> > Some
> >> >>> > > > folks took some subprojects under their wings and never
> reported
> >> >>> back.
> >> >>> > If
> >> >>> > > > you could either complete the work, or donate whatever you
> have
> >> >>> done so
> >> >>> > > > far, that would be great.
> >> >>> > > >
> >> >>> > > > As always, get the code from here:
> >> >>> https://github.com/apache/lucenenet
> >> >>> > > --
> >> >>> > > > work on your fork and send us a PR when ready, with a good
> >> >>> description
> >> >>> > of
> >> >>> > > > what you've done.
> >> >>> > > >
> >> >>> > > > To see the list of currently failing tests in real-time, go
> here
> >> >>> (you
> >> >>> > can
> >> >>> > > > login as guest):
> >> >>> > > >
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>> http://teamcity.codebetter.com/viewType.html?buildTypeId=Luc
> >> eneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> >> >>> > > >
> >> >>> > > > Thanks, and happy searching,
> >> >>> > > >
> >> >>> > > > --
> >> >>> > > >
> >> >>> > > > Itamar Syn-Hershko
> >> >>> > > > http://code972.com | @synhershko <
> https://twitter.com/synhershko
> >> >
> >> >>> > > > Freelance Developer & Consultant
> >> >>> > > > Lucene.NET committer and PMC member
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>>
> >> >>
> >> >>
> >> >
> >>
>

Re: Last failing tests remain to fix!

Posted by Kevin Miller <sc...@gmail.com>.

Annie helps with the care and feeding of the CodeBetter TeamCity server. Reach out to her on Twitter and she will get you some help.




"They can certainly contact me-I can either help or I can put them in contact with people who can help.”






https://twitter.com/ajepst/status/588448988367433728

On Tue, Apr 14, 2015 at 1:36 PM, Wyatt Barnett <wy...@gmail.com>
wrote:

> Sorry, my alter ego Wade and I just got back in town here.
> I'm looking into the NuGet feed setup though I need to talk to someone over
> at CodeBetter.com as some things need to be enabled and I need to figure
> out where the artifacts. I know Martin has moved on from JetBrains, is
> there another contact?
> To answer your question, no we don't need to wait on passing builds -- we
> can tell it to build the nuget regardless. We should probably talk about
> what should trigger a nuget release -- probably every build is too much.
> Maybe nightly builds for now?
> On Mon, Apr 13, 2015 at 5:20 PM Itamar Syn-Hershko <it...@code972.com>
> wrote:
>> Ok, I just looked and the nuget feed is empty - probably because there was
>> never a green build. Can we have nuget artifacts already, at least until we
>> have green builds?
>>
>> --
>>
>> Itamar Syn-Hershko
>> http://code972.com | @synhershko <https://twitter.com/synhershko>
>> Freelance Developer & Consultant
>> Lucene.NET committer and PMC member
>>
>> On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <it...@code972.com>
>> wrote:
>>
>> > Errr.. I mean Wyatt :)
>> >
>> > --
>> >
>> > Itamar Syn-Hershko
>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
>> > Freelance Developer & Consultant
>> > Lucene.NET committer and PMC member
>> >
>> > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <itamar@code972.com
>> >
>> > wrote:
>> >
>> >> Hi Wade, any luck?
>> >>
>> >> Is there a nuget feed on the build server we can start using to install
>> >> the core / subproject pieces using nuget from?
>> >>
>> >> Thanks!
>> >>
>> >> --
>> >>
>> >> Itamar Syn-Hershko
>> >> http://code972.com | @synhershko <https://twitter.com/synhershko>
>> >> Freelance Developer & Consultant
>> >> Lucene.NET committer and PMC member
>> >>
>> >> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <wy...@gmail.com>
>> >> wrote:
>> >>
>> >>> Yup that makes sense -- somehow I conflated sub projects over to
>> contrib
>> >>> which I haven't seen any movement.
>> >>>
>> >>> I'm on the road at the moment but I'll see about getting the remaining
>> >>> sub
>> >>> projects setup and building, auditing the ignores and start banging on
>> >>> nuget when I get back to the ranch.
>> >>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com>
>> wrote:
>> >>>
>> >>> > inline
>> >>> >
>> >>> > --
>> >>> >
>> >>> > Itamar Syn-Hershko
>> >>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
>> >>> > Freelance Developer & Consultant
>> >>> > Lucene.NET committer and PMC member
>> >>> >
>> >>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <
>> wyatt.barnett@gmail.com
>> >>> >
>> >>> > wrote:
>> >>> >
>> >>> > > Thanks for the update Itamar -- this sounds exciting.
>> >>> > >
>> >>> > > A few notes / questions from the build side of things here:
>> >>> > >
>> >>> > > First, I'm not as hung up on persistently failing tests as the
>> tests
>> >>> we
>> >>> > get
>> >>> > > intermittent failures on. I suspect some of the below will address
>> >>> this.
>> >>> > >
>> >>> >
>> >>> > Yup
>> >>> >
>> >>> >
>> >>> > >
>> >>> > > Second, what do we want to do about the 112 or so ignored tests? I
>> >>> think
>> >>> > > some are marked as long-running via the [LongRunningTest] attribute
>> >>> but
>> >>> > > there are probably some ignored failures in that mix.
>> >>> > >
>> >>> >
>> >>> > I guess that should be another task someone needs to take - can you
>> >>> try and
>> >>> > categorize them? some I know are JVM specfic tests that don't apply
>> to
>> >>> > Lucene.NET
>> >>> >
>> >>> >
>> >>> > >
>> >>> > > Third, regarding the sub-projects, what does the structure look
>> like
>> >>> so I
>> >>> > > can get that integrated into the TeamCity setup? Or should I noodle
>> >>> on it
>> >>> > > and propose one?
>> >>> > >
>> >>> >
>> >>> > What do you mean? See here
>> >>> > https://github.com/apache/lucenenet/tree/master/src
>> >>> >
>> >>> > Every subproject has a test suite that needs to be run, and should
>> >>> produce
>> >>> > a class library that we could push to nuget when we release. Until
>> >>> then we
>> >>> > should be able to allow people to install via nuget from our build
>> >>> server's
>> >>> > feed.
>> >>> >
>> >>> > As far as dependencies and build triggers - that should be obvious
>> and
>> >>> self
>> >>> > explanatory..
>> >>> >
>> >>> > Does that make sense?
>> >>> >
>> >>> >
>> >>> > >
>> >>> > > Thanks for all the help from everyone getting things this far.
>> >>> > >
>> >>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
>> >>> itamar@code972.com>
>> >>> > > wrote:
>> >>> > >
>> >>> > > > Hey all,
>> >>> > > >
>> >>> > > > We are nearly there -- we are this close to a release!
>> >>> > > >
>> >>> > > > We have only about 30 failing tests now in the core, and I'd like
>> >>> to
>> >>> > push
>> >>> > > > hard during the next few days so we can get a beta out next week.
>> >>> > > >
>> >>> > > > I'd appreciate your help with this!
>> >>> > > >
>> >>> > > > I've narrowed down several failing tests that require some
>> >>> digging, if
>> >>> > > > folks can assign those to themselves and send us PRs fixing those
>> >>> > tests,
>> >>> > > > that would help a lot!
>> >>> > > >
>> >>> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues
>> ()
>> >>> > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues
>> ()
>> >>> --
>> >>> > > > specifically it seems like Double and Float NaNs are treated
>> >>> > differently
>> >>> > > > between C# and Java and therefore the failing tests. Or there is
>> a
>> >>> bug
>> >>> > in
>> >>> > > > the inner implementation handling those queries.
>> >>> > > >
>> >>> > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
>> >>> > exposing
>> >>> > > > data corruption in one of the codec components. I believe this is
>> >>> > > contained
>> >>> > > > in the 40DocValues tests so the scope isn't huge but I could be
>> >>> wrong.
>> >>> > > >
>> >>> > > > 3. TestLucene42DocValuesFormat has two tests failing with output
>> of
>> >>> > lines
>> >>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
>> >>> > > computing
>> >>> > > > the hashes of the nodes, or maybe the node construction is faulty
>> >>> > (which
>> >>> > > is
>> >>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
>> >>> that'll
>> >>> > > be a
>> >>> > > > good starting point. Also see
>> >>> > > >
>> >>> > > >
>> >>> > >
>> >>> >
>> >>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f910
>> 90d995c729689fff9a0f54c
>> >>> > > >
>> >>> > > > 4. There seem to be some concurrency issues with file handling,
>> >>> > probably
>> >>> > > on
>> >>> > > > the part that is in charge of acquiring and releasing locks. In
>> >>> > > particular
>> >>> > > > see TestDocValuesFormat,
>> >>> > > > .TestShardSearching.TestSimple(),
>> >>> > > > TestBufferedIndexInput.TestSetBufferSize(),
>> >>> > > > TestLockFactory.TestStressLocks(),
>> >>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
>> >>> > > >
>> >>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
>> >>> automaton
>> >>> > > > implementation doesn't emit the terms correctly. See the failures
>> >>> in
>> >>> > > > TestRegexpQuery and TestRegexpRandom.
>> >>> > > >
>> >>> > > > 6. Similar but probably unrelated failing test is
>> >>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
>> >>> > > >
>> >>> > > > 7. An interesting failure to investigate is here:
>> >>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
>> >>> probably a
>> >>> > > > simple porting fail, a subtle Java <--> C# mismatch that we
>> haven't
>> >>> > > > noticed.
>> >>> > > >
>> >>> > > > Additionally, any help with porting the subprojects would be
>> great.
>> >>> > Some
>> >>> > > > folks took some subprojects under their wings and never reported
>> >>> back.
>> >>> > If
>> >>> > > > you could either complete the work, or donate whatever you have
>> >>> done so
>> >>> > > > far, that would be great.
>> >>> > > >
>> >>> > > > As always, get the code from here:
>> >>> https://github.com/apache/lucenenet
>> >>> > > --
>> >>> > > > work on your fork and send us a PR when ready, with a good
>> >>> description
>> >>> > of
>> >>> > > > what you've done.
>> >>> > > >
>> >>> > > > To see the list of currently failing tests in real-time, go here
>> >>> (you
>> >>> > can
>> >>> > > > login as guest):
>> >>> > > >
>> >>> > > >
>> >>> > >
>> >>> >
>> >>> http://teamcity.codebetter.com/viewType.html?buildTypeId=Luc
>> eneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
>> >>> > > >
>> >>> > > > Thanks, and happy searching,
>> >>> > > >
>> >>> > > > --
>> >>> > > >
>> >>> > > > Itamar Syn-Hershko
>> >>> > > > http://code972.com | @synhershko <https://twitter.com/synhershko
>> >
>> >>> > > > Freelance Developer & Consultant
>> >>> > > > Lucene.NET committer and PMC member
>> >>> > > >
>> >>> > >
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>

Re: Last failing tests remain to fix!

Posted by Kevin Miller <sc...@gmail.com>.

Annie helps with the care and feeding of the CodeBetter TeamCity server. Reach out to her on Twitter and she will get you some help.




"They can certainly contact me-I can either help or I can put them in contact with people who can help.”






https://twitter.com/ajepst/status/588448988367433728

On Tue, Apr 14, 2015 at 1:36 PM, Wyatt Barnett <wy...@gmail.com>
wrote:

> Sorry, my alter ego Wade and I just got back in town here.
> I'm looking into the NuGet feed setup though I need to talk to someone over
> at CodeBetter.com as some things need to be enabled and I need to figure
> out where the artifacts. I know Martin has moved on from JetBrains, is
> there another contact?
> To answer your question, no we don't need to wait on passing builds -- we
> can tell it to build the nuget regardless. We should probably talk about
> what should trigger a nuget release -- probably every build is too much.
> Maybe nightly builds for now?
> On Mon, Apr 13, 2015 at 5:20 PM Itamar Syn-Hershko <it...@code972.com>
> wrote:
>> Ok, I just looked and the nuget feed is empty - probably because there was
>> never a green build. Can we have nuget artifacts already, at least until we
>> have green builds?
>>
>> --
>>
>> Itamar Syn-Hershko
>> http://code972.com | @synhershko <https://twitter.com/synhershko>
>> Freelance Developer & Consultant
>> Lucene.NET committer and PMC member
>>
>> On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <it...@code972.com>
>> wrote:
>>
>> > Errr.. I mean Wyatt :)
>> >
>> > --
>> >
>> > Itamar Syn-Hershko
>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
>> > Freelance Developer & Consultant
>> > Lucene.NET committer and PMC member
>> >
>> > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <itamar@code972.com
>> >
>> > wrote:
>> >
>> >> Hi Wade, any luck?
>> >>
>> >> Is there a nuget feed on the build server we can start using to install
>> >> the core / subproject pieces using nuget from?
>> >>
>> >> Thanks!
>> >>
>> >> --
>> >>
>> >> Itamar Syn-Hershko
>> >> http://code972.com | @synhershko <https://twitter.com/synhershko>
>> >> Freelance Developer & Consultant
>> >> Lucene.NET committer and PMC member
>> >>
>> >> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <wy...@gmail.com>
>> >> wrote:
>> >>
>> >>> Yup that makes sense -- somehow I conflated sub projects over to
>> contrib
>> >>> which I haven't seen any movement.
>> >>>
>> >>> I'm on the road at the moment but I'll see about getting the remaining
>> >>> sub
>> >>> projects setup and building, auditing the ignores and start banging on
>> >>> nuget when I get back to the ranch.
>> >>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com>
>> wrote:
>> >>>
>> >>> > inline
>> >>> >
>> >>> > --
>> >>> >
>> >>> > Itamar Syn-Hershko
>> >>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
>> >>> > Freelance Developer & Consultant
>> >>> > Lucene.NET committer and PMC member
>> >>> >
>> >>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <
>> wyatt.barnett@gmail.com
>> >>> >
>> >>> > wrote:
>> >>> >
>> >>> > > Thanks for the update Itamar -- this sounds exciting.
>> >>> > >
>> >>> > > A few notes / questions from the build side of things here:
>> >>> > >
>> >>> > > First, I'm not as hung up on persistently failing tests as the
>> tests
>> >>> we
>> >>> > get
>> >>> > > intermittent failures on. I suspect some of the below will address
>> >>> this.
>> >>> > >
>> >>> >
>> >>> > Yup
>> >>> >
>> >>> >
>> >>> > >
>> >>> > > Second, what do we want to do about the 112 or so ignored tests? I
>> >>> think
>> >>> > > some are marked as long-running via the [LongRunningTest] attribute
>> >>> but
>> >>> > > there are probably some ignored failures in that mix.
>> >>> > >
>> >>> >
>> >>> > I guess that should be another task someone needs to take - can you
>> >>> try and
>> >>> > categorize them? some I know are JVM specfic tests that don't apply
>> to
>> >>> > Lucene.NET
>> >>> >
>> >>> >
>> >>> > >
>> >>> > > Third, regarding the sub-projects, what does the structure look
>> like
>> >>> so I
>> >>> > > can get that integrated into the TeamCity setup? Or should I noodle
>> >>> on it
>> >>> > > and propose one?
>> >>> > >
>> >>> >
>> >>> > What do you mean? See here
>> >>> > https://github.com/apache/lucenenet/tree/master/src
>> >>> >
>> >>> > Every subproject has a test suite that needs to be run, and should
>> >>> produce
>> >>> > a class library that we could push to nuget when we release. Until
>> >>> then we
>> >>> > should be able to allow people to install via nuget from our build
>> >>> server's
>> >>> > feed.
>> >>> >
>> >>> > As far as dependencies and build triggers - that should be obvious
>> and
>> >>> self
>> >>> > explanatory..
>> >>> >
>> >>> > Does that make sense?
>> >>> >
>> >>> >
>> >>> > >
>> >>> > > Thanks for all the help from everyone getting things this far.
>> >>> > >
>> >>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
>> >>> itamar@code972.com>
>> >>> > > wrote:
>> >>> > >
>> >>> > > > Hey all,
>> >>> > > >
>> >>> > > > We are nearly there -- we are this close to a release!
>> >>> > > >
>> >>> > > > We have only about 30 failing tests now in the core, and I'd like
>> >>> to
>> >>> > push
>> >>> > > > hard during the next few days so we can get a beta out next week.
>> >>> > > >
>> >>> > > > I'd appreciate your help with this!
>> >>> > > >
>> >>> > > > I've narrowed down several failing tests that require some
>> >>> digging, if
>> >>> > > > folks can assign those to themselves and send us PRs fixing those
>> >>> > tests,
>> >>> > > > that would help a lot!
>> >>> > > >
>> >>> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues
>> ()
>> >>> > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues
>> ()
>> >>> --
>> >>> > > > specifically it seems like Double and Float NaNs are treated
>> >>> > differently
>> >>> > > > between C# and Java and therefore the failing tests. Or there is
>> a
>> >>> bug
>> >>> > in
>> >>> > > > the inner implementation handling those queries.
>> >>> > > >
>> >>> > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
>> >>> > exposing
>> >>> > > > data corruption in one of the codec components. I believe this is
>> >>> > > contained
>> >>> > > > in the 40DocValues tests so the scope isn't huge but I could be
>> >>> wrong.
>> >>> > > >
>> >>> > > > 3. TestLucene42DocValuesFormat has two tests failing with output
>> of
>> >>> > lines
>> >>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
>> >>> > > computing
>> >>> > > > the hashes of the nodes, or maybe the node construction is faulty
>> >>> > (which
>> >>> > > is
>> >>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
>> >>> that'll
>> >>> > > be a
>> >>> > > > good starting point. Also see
>> >>> > > >
>> >>> > > >
>> >>> > >
>> >>> >
>> >>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f910
>> 90d995c729689fff9a0f54c
>> >>> > > >
>> >>> > > > 4. There seem to be some concurrency issues with file handling,
>> >>> > probably
>> >>> > > on
>> >>> > > > the part that is in charge of acquiring and releasing locks. In
>> >>> > > particular
>> >>> > > > see TestDocValuesFormat,
>> >>> > > > .TestShardSearching.TestSimple(),
>> >>> > > > TestBufferedIndexInput.TestSetBufferSize(),
>> >>> > > > TestLockFactory.TestStressLocks(),
>> >>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
>> >>> > > >
>> >>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
>> >>> automaton
>> >>> > > > implementation doesn't emit the terms correctly. See the failures
>> >>> in
>> >>> > > > TestRegexpQuery and TestRegexpRandom.
>> >>> > > >
>> >>> > > > 6. Similar but probably unrelated failing test is
>> >>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
>> >>> > > >
>> >>> > > > 7. An interesting failure to investigate is here:
>> >>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
>> >>> probably a
>> >>> > > > simple porting fail, a subtle Java <--> C# mismatch that we
>> haven't
>> >>> > > > noticed.
>> >>> > > >
>> >>> > > > Additionally, any help with porting the subprojects would be
>> great.
>> >>> > Some
>> >>> > > > folks took some subprojects under their wings and never reported
>> >>> back.
>> >>> > If
>> >>> > > > you could either complete the work, or donate whatever you have
>> >>> done so
>> >>> > > > far, that would be great.
>> >>> > > >
>> >>> > > > As always, get the code from here:
>> >>> https://github.com/apache/lucenenet
>> >>> > > --
>> >>> > > > work on your fork and send us a PR when ready, with a good
>> >>> description
>> >>> > of
>> >>> > > > what you've done.
>> >>> > > >
>> >>> > > > To see the list of currently failing tests in real-time, go here
>> >>> (you
>> >>> > can
>> >>> > > > login as guest):
>> >>> > > >
>> >>> > > >
>> >>> > >
>> >>> >
>> >>> http://teamcity.codebetter.com/viewType.html?buildTypeId=Luc
>> eneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
>> >>> > > >
>> >>> > > > Thanks, and happy searching,
>> >>> > > >
>> >>> > > > --
>> >>> > > >
>> >>> > > > Itamar Syn-Hershko
>> >>> > > > http://code972.com | @synhershko <https://twitter.com/synhershko
>> >
>> >>> > > > Freelance Developer & Consultant
>> >>> > > > Lucene.NET committer and PMC member
>> >>> > > >
>> >>> > >
>> >>> >
>> >>>
>> >>
>> >>
>> >
>>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

Try contacting Maarten, he should still be able to help

Nightly builds sounds good

Thanks!

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Tue, Apr 14, 2015 at 9:34 PM, Wyatt Barnett <wy...@gmail.com>
wrote:

> Sorry, my alter ego Wade and I just got back in town here.
>
> I'm looking into the NuGet feed setup though I need to talk to someone over
> at CodeBetter.com as some things need to be enabled and I need to figure
> out where the artifacts. I know Martin has moved on from JetBrains, is
> there another contact?
>
> To answer your question, no we don't need to wait on passing builds -- we
> can tell it to build the nuget regardless. We should probably talk about
> what should trigger a nuget release -- probably every build is too much.
> Maybe nightly builds for now?
>
> On Mon, Apr 13, 2015 at 5:20 PM Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
> > Ok, I just looked and the nuget feed is empty - probably because there
> was
> > never a green build. Can we have nuget artifacts already, at least until
> we
> > have green builds?
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
> > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <itamar@code972.com
> >
> > wrote:
> >
> > > Errr.. I mean Wyatt :)
> > >
> > > --
> > >
> > > Itamar Syn-Hershko
> > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > Freelance Developer & Consultant
> > > Lucene.NET committer and PMC member
> > >
> > > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <
> itamar@code972.com
> > >
> > > wrote:
> > >
> > >> Hi Wade, any luck?
> > >>
> > >> Is there a nuget feed on the build server we can start using to
> install
> > >> the core / subproject pieces using nuget from?
> > >>
> > >> Thanks!
> > >>
> > >> --
> > >>
> > >> Itamar Syn-Hershko
> > >> http://code972.com | @synhershko <https://twitter.com/synhershko>
> > >> Freelance Developer & Consultant
> > >> Lucene.NET committer and PMC member
> > >>
> > >> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <
> wyatt.barnett@gmail.com>
> > >> wrote:
> > >>
> > >>> Yup that makes sense -- somehow I conflated sub projects over to
> > contrib
> > >>> which I haven't seen any movement.
> > >>>
> > >>> I'm on the road at the moment but I'll see about getting the
> remaining
> > >>> sub
> > >>> projects setup and building, auditing the ignores and start banging
> on
> > >>> nuget when I get back to the ranch.
> > >>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com>
> > wrote:
> > >>>
> > >>> > inline
> > >>> >
> > >>> > --
> > >>> >
> > >>> > Itamar Syn-Hershko
> > >>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > >>> > Freelance Developer & Consultant
> > >>> > Lucene.NET committer and PMC member
> > >>> >
> > >>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <
> > wyatt.barnett@gmail.com
> > >>> >
> > >>> > wrote:
> > >>> >
> > >>> > > Thanks for the update Itamar -- this sounds exciting.
> > >>> > >
> > >>> > > A few notes / questions from the build side of things here:
> > >>> > >
> > >>> > > First, I'm not as hung up on persistently failing tests as the
> > tests
> > >>> we
> > >>> > get
> > >>> > > intermittent failures on. I suspect some of the below will
> address
> > >>> this.
> > >>> > >
> > >>> >
> > >>> > Yup
> > >>> >
> > >>> >
> > >>> > >
> > >>> > > Second, what do we want to do about the 112 or so ignored tests?
> I
> > >>> think
> > >>> > > some are marked as long-running via the [LongRunningTest]
> attribute
> > >>> but
> > >>> > > there are probably some ignored failures in that mix.
> > >>> > >
> > >>> >
> > >>> > I guess that should be another task someone needs to take - can you
> > >>> try and
> > >>> > categorize them? some I know are JVM specfic tests that don't apply
> > to
> > >>> > Lucene.NET
> > >>> >
> > >>> >
> > >>> > >
> > >>> > > Third, regarding the sub-projects, what does the structure look
> > like
> > >>> so I
> > >>> > > can get that integrated into the TeamCity setup? Or should I
> noodle
> > >>> on it
> > >>> > > and propose one?
> > >>> > >
> > >>> >
> > >>> > What do you mean? See here
> > >>> > https://github.com/apache/lucenenet/tree/master/src
> > >>> >
> > >>> > Every subproject has a test suite that needs to be run, and should
> > >>> produce
> > >>> > a class library that we could push to nuget when we release. Until
> > >>> then we
> > >>> > should be able to allow people to install via nuget from our build
> > >>> server's
> > >>> > feed.
> > >>> >
> > >>> > As far as dependencies and build triggers - that should be obvious
> > and
> > >>> self
> > >>> > explanatory..
> > >>> >
> > >>> > Does that make sense?
> > >>> >
> > >>> >
> > >>> > >
> > >>> > > Thanks for all the help from everyone getting things this far.
> > >>> > >
> > >>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
> > >>> itamar@code972.com>
> > >>> > > wrote:
> > >>> > >
> > >>> > > > Hey all,
> > >>> > > >
> > >>> > > > We are nearly there -- we are this close to a release!
> > >>> > > >
> > >>> > > > We have only about 30 failing tests now in the core, and I'd
> like
> > >>> to
> > >>> > push
> > >>> > > > hard during the next few days so we can get a beta out next
> week.
> > >>> > > >
> > >>> > > > I'd appreciate your help with this!
> > >>> > > >
> > >>> > > > I've narrowed down several failing tests that require some
> > >>> digging, if
> > >>> > > > folks can assign those to themselves and send us PRs fixing
> those
> > >>> > tests,
> > >>> > > > that would help a lot!
> > >>> > > >
> > >>> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues
> > ()
> > >>> > > > and
> Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues
> > ()
> > >>> --
> > >>> > > > specifically it seems like Double and Float NaNs are treated
> > >>> > differently
> > >>> > > > between C# and Java and therefore the failing tests. Or there
> is
> > a
> > >>> bug
> > >>> > in
> > >>> > > > the inner implementation handling those queries.
> > >>> > > >
> > >>> > > > 2. TestLucene40DocValuesFormat has a few failing tests,
> probably
> > >>> > exposing
> > >>> > > > data corruption in one of the codec components. I believe this
> is
> > >>> > > contained
> > >>> > > > in the 40DocValues tests so the scope isn't huge but I could be
> > >>> wrong.
> > >>> > > >
> > >>> > > > 3. TestLucene42DocValuesFormat has two tests failing with
> output
> > of
> > >>> > lines
> > >>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off
> in
> > >>> > > computing
> > >>> > > > the hashes of the nodes, or maybe the node construction is
> faulty
> > >>> > (which
> > >>> > > is
> > >>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> > >>> that'll
> > >>> > > be a
> > >>> > > > good starting point. Also see
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f910
> > 90d995c729689fff9a0f54c
> > >>> > > >
> > >>> > > > 4. There seem to be some concurrency issues with file handling,
> > >>> > probably
> > >>> > > on
> > >>> > > > the part that is in charge of acquiring and releasing locks. In
> > >>> > > particular
> > >>> > > > see TestDocValuesFormat,
> > >>> > > > .TestShardSearching.TestSimple(),
> > >>> > > > TestBufferedIndexInput.TestSetBufferSize(),
> > >>> > > > TestLockFactory.TestStressLocks(),
> > >>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > >>> > > >
> > >>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
> > >>> automaton
> > >>> > > > implementation doesn't emit the terms correctly. See the
> failures
> > >>> in
> > >>> > > > TestRegexpQuery and TestRegexpRandom.
> > >>> > > >
> > >>> > > > 6. Similar but probably unrelated failing test is
> > >>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > >>> > > >
> > >>> > > > 7. An interesting failure to investigate is here:
> > >>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
> > >>> probably a
> > >>> > > > simple porting fail, a subtle Java <--> C# mismatch that we
> > haven't
> > >>> > > > noticed.
> > >>> > > >
> > >>> > > > Additionally, any help with porting the subprojects would be
> > great.
> > >>> > Some
> > >>> > > > folks took some subprojects under their wings and never
> reported
> > >>> back.
> > >>> > If
> > >>> > > > you could either complete the work, or donate whatever you have
> > >>> done so
> > >>> > > > far, that would be great.
> > >>> > > >
> > >>> > > > As always, get the code from here:
> > >>> https://github.com/apache/lucenenet
> > >>> > > --
> > >>> > > > work on your fork and send us a PR when ready, with a good
> > >>> description
> > >>> > of
> > >>> > > > what you've done.
> > >>> > > >
> > >>> > > > To see the list of currently failing tests in real-time, go
> here
> > >>> (you
> > >>> > can
> > >>> > > > login as guest):
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> http://teamcity.codebetter.com/viewType.html?buildTypeId=Luc
> > eneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > >>> > > >
> > >>> > > > Thanks, and happy searching,
> > >>> > > >
> > >>> > > > --
> > >>> > > >
> > >>> > > > Itamar Syn-Hershko
> > >>> > > > http://code972.com | @synhershko <
> https://twitter.com/synhershko
> > >
> > >>> > > > Freelance Developer & Consultant
> > >>> > > > Lucene.NET committer and PMC member
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

Try contacting Maarten, he should still be able to help

Nightly builds sounds good

Thanks!

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Tue, Apr 14, 2015 at 9:34 PM, Wyatt Barnett <wy...@gmail.com>
wrote:

> Sorry, my alter ego Wade and I just got back in town here.
>
> I'm looking into the NuGet feed setup though I need to talk to someone over
> at CodeBetter.com as some things need to be enabled and I need to figure
> out where the artifacts. I know Martin has moved on from JetBrains, is
> there another contact?
>
> To answer your question, no we don't need to wait on passing builds -- we
> can tell it to build the nuget regardless. We should probably talk about
> what should trigger a nuget release -- probably every build is too much.
> Maybe nightly builds for now?
>
> On Mon, Apr 13, 2015 at 5:20 PM Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
> > Ok, I just looked and the nuget feed is empty - probably because there
> was
> > never a green build. Can we have nuget artifacts already, at least until
> we
> > have green builds?
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
> > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <itamar@code972.com
> >
> > wrote:
> >
> > > Errr.. I mean Wyatt :)
> > >
> > > --
> > >
> > > Itamar Syn-Hershko
> > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > Freelance Developer & Consultant
> > > Lucene.NET committer and PMC member
> > >
> > > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <
> itamar@code972.com
> > >
> > > wrote:
> > >
> > >> Hi Wade, any luck?
> > >>
> > >> Is there a nuget feed on the build server we can start using to
> install
> > >> the core / subproject pieces using nuget from?
> > >>
> > >> Thanks!
> > >>
> > >> --
> > >>
> > >> Itamar Syn-Hershko
> > >> http://code972.com | @synhershko <https://twitter.com/synhershko>
> > >> Freelance Developer & Consultant
> > >> Lucene.NET committer and PMC member
> > >>
> > >> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <
> wyatt.barnett@gmail.com>
> > >> wrote:
> > >>
> > >>> Yup that makes sense -- somehow I conflated sub projects over to
> > contrib
> > >>> which I haven't seen any movement.
> > >>>
> > >>> I'm on the road at the moment but I'll see about getting the
> remaining
> > >>> sub
> > >>> projects setup and building, auditing the ignores and start banging
> on
> > >>> nuget when I get back to the ranch.
> > >>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com>
> > wrote:
> > >>>
> > >>> > inline
> > >>> >
> > >>> > --
> > >>> >
> > >>> > Itamar Syn-Hershko
> > >>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > >>> > Freelance Developer & Consultant
> > >>> > Lucene.NET committer and PMC member
> > >>> >
> > >>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <
> > wyatt.barnett@gmail.com
> > >>> >
> > >>> > wrote:
> > >>> >
> > >>> > > Thanks for the update Itamar -- this sounds exciting.
> > >>> > >
> > >>> > > A few notes / questions from the build side of things here:
> > >>> > >
> > >>> > > First, I'm not as hung up on persistently failing tests as the
> > tests
> > >>> we
> > >>> > get
> > >>> > > intermittent failures on. I suspect some of the below will
> address
> > >>> this.
> > >>> > >
> > >>> >
> > >>> > Yup
> > >>> >
> > >>> >
> > >>> > >
> > >>> > > Second, what do we want to do about the 112 or so ignored tests?
> I
> > >>> think
> > >>> > > some are marked as long-running via the [LongRunningTest]
> attribute
> > >>> but
> > >>> > > there are probably some ignored failures in that mix.
> > >>> > >
> > >>> >
> > >>> > I guess that should be another task someone needs to take - can you
> > >>> try and
> > >>> > categorize them? some I know are JVM specfic tests that don't apply
> > to
> > >>> > Lucene.NET
> > >>> >
> > >>> >
> > >>> > >
> > >>> > > Third, regarding the sub-projects, what does the structure look
> > like
> > >>> so I
> > >>> > > can get that integrated into the TeamCity setup? Or should I
> noodle
> > >>> on it
> > >>> > > and propose one?
> > >>> > >
> > >>> >
> > >>> > What do you mean? See here
> > >>> > https://github.com/apache/lucenenet/tree/master/src
> > >>> >
> > >>> > Every subproject has a test suite that needs to be run, and should
> > >>> produce
> > >>> > a class library that we could push to nuget when we release. Until
> > >>> then we
> > >>> > should be able to allow people to install via nuget from our build
> > >>> server's
> > >>> > feed.
> > >>> >
> > >>> > As far as dependencies and build triggers - that should be obvious
> > and
> > >>> self
> > >>> > explanatory..
> > >>> >
> > >>> > Does that make sense?
> > >>> >
> > >>> >
> > >>> > >
> > >>> > > Thanks for all the help from everyone getting things this far.
> > >>> > >
> > >>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
> > >>> itamar@code972.com>
> > >>> > > wrote:
> > >>> > >
> > >>> > > > Hey all,
> > >>> > > >
> > >>> > > > We are nearly there -- we are this close to a release!
> > >>> > > >
> > >>> > > > We have only about 30 failing tests now in the core, and I'd
> like
> > >>> to
> > >>> > push
> > >>> > > > hard during the next few days so we can get a beta out next
> week.
> > >>> > > >
> > >>> > > > I'd appreciate your help with this!
> > >>> > > >
> > >>> > > > I've narrowed down several failing tests that require some
> > >>> digging, if
> > >>> > > > folks can assign those to themselves and send us PRs fixing
> those
> > >>> > tests,
> > >>> > > > that would help a lot!
> > >>> > > >
> > >>> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues
> > ()
> > >>> > > > and
> Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues
> > ()
> > >>> --
> > >>> > > > specifically it seems like Double and Float NaNs are treated
> > >>> > differently
> > >>> > > > between C# and Java and therefore the failing tests. Or there
> is
> > a
> > >>> bug
> > >>> > in
> > >>> > > > the inner implementation handling those queries.
> > >>> > > >
> > >>> > > > 2. TestLucene40DocValuesFormat has a few failing tests,
> probably
> > >>> > exposing
> > >>> > > > data corruption in one of the codec components. I believe this
> is
> > >>> > > contained
> > >>> > > > in the 40DocValues tests so the scope isn't huge but I could be
> > >>> wrong.
> > >>> > > >
> > >>> > > > 3. TestLucene42DocValuesFormat has two tests failing with
> output
> > of
> > >>> > lines
> > >>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off
> in
> > >>> > > computing
> > >>> > > > the hashes of the nodes, or maybe the node construction is
> faulty
> > >>> > (which
> > >>> > > is
> > >>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> > >>> that'll
> > >>> > > be a
> > >>> > > > good starting point. Also see
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f910
> > 90d995c729689fff9a0f54c
> > >>> > > >
> > >>> > > > 4. There seem to be some concurrency issues with file handling,
> > >>> > probably
> > >>> > > on
> > >>> > > > the part that is in charge of acquiring and releasing locks. In
> > >>> > > particular
> > >>> > > > see TestDocValuesFormat,
> > >>> > > > .TestShardSearching.TestSimple(),
> > >>> > > > TestBufferedIndexInput.TestSetBufferSize(),
> > >>> > > > TestLockFactory.TestStressLocks(),
> > >>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > >>> > > >
> > >>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
> > >>> automaton
> > >>> > > > implementation doesn't emit the terms correctly. See the
> failures
> > >>> in
> > >>> > > > TestRegexpQuery and TestRegexpRandom.
> > >>> > > >
> > >>> > > > 6. Similar but probably unrelated failing test is
> > >>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > >>> > > >
> > >>> > > > 7. An interesting failure to investigate is here:
> > >>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
> > >>> probably a
> > >>> > > > simple porting fail, a subtle Java <--> C# mismatch that we
> > haven't
> > >>> > > > noticed.
> > >>> > > >
> > >>> > > > Additionally, any help with porting the subprojects would be
> > great.
> > >>> > Some
> > >>> > > > folks took some subprojects under their wings and never
> reported
> > >>> back.
> > >>> > If
> > >>> > > > you could either complete the work, or donate whatever you have
> > >>> done so
> > >>> > > > far, that would be great.
> > >>> > > >
> > >>> > > > As always, get the code from here:
> > >>> https://github.com/apache/lucenenet
> > >>> > > --
> > >>> > > > work on your fork and send us a PR when ready, with a good
> > >>> description
> > >>> > of
> > >>> > > > what you've done.
> > >>> > > >
> > >>> > > > To see the list of currently failing tests in real-time, go
> here
> > >>> (you
> > >>> > can
> > >>> > > > login as guest):
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>> http://teamcity.codebetter.com/viewType.html?buildTypeId=Luc
> > eneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > >>> > > >
> > >>> > > > Thanks, and happy searching,
> > >>> > > >
> > >>> > > > --
> > >>> > > >
> > >>> > > > Itamar Syn-Hershko
> > >>> > > > http://code972.com | @synhershko <
> https://twitter.com/synhershko
> > >
> > >>> > > > Freelance Developer & Consultant
> > >>> > > > Lucene.NET committer and PMC member
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> > >>
> > >>
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by Wyatt Barnett <wy...@gmail.com>.

Sorry, my alter ego Wade and I just got back in town here.

I'm looking into the NuGet feed setup though I need to talk to someone over
at CodeBetter.com as some things need to be enabled and I need to figure
out where the artifacts. I know Martin has moved on from JetBrains, is
there another contact?

To answer your question, no we don't need to wait on passing builds -- we
can tell it to build the nuget regardless. We should probably talk about
what should trigger a nuget release -- probably every build is too much.
Maybe nightly builds for now?

On Mon, Apr 13, 2015 at 5:20 PM Itamar Syn-Hershko <it...@code972.com>
wrote:

> Ok, I just looked and the nuget feed is empty - probably because there was
> never a green build. Can we have nuget artifacts already, at least until we
> have green builds?
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
> On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
> > Errr.. I mean Wyatt :)
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
> > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <itamar@code972.com
> >
> > wrote:
> >
> >> Hi Wade, any luck?
> >>
> >> Is there a nuget feed on the build server we can start using to install
> >> the core / subproject pieces using nuget from?
> >>
> >> Thanks!
> >>
> >> --
> >>
> >> Itamar Syn-Hershko
> >> http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> Freelance Developer & Consultant
> >> Lucene.NET committer and PMC member
> >>
> >> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <wy...@gmail.com>
> >> wrote:
> >>
> >>> Yup that makes sense -- somehow I conflated sub projects over to
> contrib
> >>> which I haven't seen any movement.
> >>>
> >>> I'm on the road at the moment but I'll see about getting the remaining
> >>> sub
> >>> projects setup and building, auditing the ignores and start banging on
> >>> nuget when I get back to the ranch.
> >>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com>
> wrote:
> >>>
> >>> > inline
> >>> >
> >>> > --
> >>> >
> >>> > Itamar Syn-Hershko
> >>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> >>> > Freelance Developer & Consultant
> >>> > Lucene.NET committer and PMC member
> >>> >
> >>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <
> wyatt.barnett@gmail.com
> >>> >
> >>> > wrote:
> >>> >
> >>> > > Thanks for the update Itamar -- this sounds exciting.
> >>> > >
> >>> > > A few notes / questions from the build side of things here:
> >>> > >
> >>> > > First, I'm not as hung up on persistently failing tests as the
> tests
> >>> we
> >>> > get
> >>> > > intermittent failures on. I suspect some of the below will address
> >>> this.
> >>> > >
> >>> >
> >>> > Yup
> >>> >
> >>> >
> >>> > >
> >>> > > Second, what do we want to do about the 112 or so ignored tests? I
> >>> think
> >>> > > some are marked as long-running via the [LongRunningTest] attribute
> >>> but
> >>> > > there are probably some ignored failures in that mix.
> >>> > >
> >>> >
> >>> > I guess that should be another task someone needs to take - can you
> >>> try and
> >>> > categorize them? some I know are JVM specfic tests that don't apply
> to
> >>> > Lucene.NET
> >>> >
> >>> >
> >>> > >
> >>> > > Third, regarding the sub-projects, what does the structure look
> like
> >>> so I
> >>> > > can get that integrated into the TeamCity setup? Or should I noodle
> >>> on it
> >>> > > and propose one?
> >>> > >
> >>> >
> >>> > What do you mean? See here
> >>> > https://github.com/apache/lucenenet/tree/master/src
> >>> >
> >>> > Every subproject has a test suite that needs to be run, and should
> >>> produce
> >>> > a class library that we could push to nuget when we release. Until
> >>> then we
> >>> > should be able to allow people to install via nuget from our build
> >>> server's
> >>> > feed.
> >>> >
> >>> > As far as dependencies and build triggers - that should be obvious
> and
> >>> self
> >>> > explanatory..
> >>> >
> >>> > Does that make sense?
> >>> >
> >>> >
> >>> > >
> >>> > > Thanks for all the help from everyone getting things this far.
> >>> > >
> >>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
> >>> itamar@code972.com>
> >>> > > wrote:
> >>> > >
> >>> > > > Hey all,
> >>> > > >
> >>> > > > We are nearly there -- we are this close to a release!
> >>> > > >
> >>> > > > We have only about 30 failing tests now in the core, and I'd like
> >>> to
> >>> > push
> >>> > > > hard during the next few days so we can get a beta out next week.
> >>> > > >
> >>> > > > I'd appreciate your help with this!
> >>> > > >
> >>> > > > I've narrowed down several failing tests that require some
> >>> digging, if
> >>> > > > folks can assign those to themselves and send us PRs fixing those
> >>> > tests,
> >>> > > > that would help a lot!
> >>> > > >
> >>> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues
> ()
> >>> > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues
> ()
> >>> --
> >>> > > > specifically it seems like Double and Float NaNs are treated
> >>> > differently
> >>> > > > between C# and Java and therefore the failing tests. Or there is
> a
> >>> bug
> >>> > in
> >>> > > > the inner implementation handling those queries.
> >>> > > >
> >>> > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> >>> > exposing
> >>> > > > data corruption in one of the codec components. I believe this is
> >>> > > contained
> >>> > > > in the 40DocValues tests so the scope isn't huge but I could be
> >>> wrong.
> >>> > > >
> >>> > > > 3. TestLucene42DocValuesFormat has two tests failing with output
> of
> >>> > lines
> >>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> >>> > > computing
> >>> > > > the hashes of the nodes, or maybe the node construction is faulty
> >>> > (which
> >>> > > is
> >>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> >>> that'll
> >>> > > be a
> >>> > > > good starting point. Also see
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f910
> 90d995c729689fff9a0f54c
> >>> > > >
> >>> > > > 4. There seem to be some concurrency issues with file handling,
> >>> > probably
> >>> > > on
> >>> > > > the part that is in charge of acquiring and releasing locks. In
> >>> > > particular
> >>> > > > see TestDocValuesFormat,
> >>> > > > .TestShardSearching.TestSimple(),
> >>> > > > TestBufferedIndexInput.TestSetBufferSize(),
> >>> > > > TestLockFactory.TestStressLocks(),
> >>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> >>> > > >
> >>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
> >>> automaton
> >>> > > > implementation doesn't emit the terms correctly. See the failures
> >>> in
> >>> > > > TestRegexpQuery and TestRegexpRandom.
> >>> > > >
> >>> > > > 6. Similar but probably unrelated failing test is
> >>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> >>> > > >
> >>> > > > 7. An interesting failure to investigate is here:
> >>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
> >>> probably a
> >>> > > > simple porting fail, a subtle Java <--> C# mismatch that we
> haven't
> >>> > > > noticed.
> >>> > > >
> >>> > > > Additionally, any help with porting the subprojects would be
> great.
> >>> > Some
> >>> > > > folks took some subprojects under their wings and never reported
> >>> back.
> >>> > If
> >>> > > > you could either complete the work, or donate whatever you have
> >>> done so
> >>> > > > far, that would be great.
> >>> > > >
> >>> > > > As always, get the code from here:
> >>> https://github.com/apache/lucenenet
> >>> > > --
> >>> > > > work on your fork and send us a PR when ready, with a good
> >>> description
> >>> > of
> >>> > > > what you've done.
> >>> > > >
> >>> > > > To see the list of currently failing tests in real-time, go here
> >>> (you
> >>> > can
> >>> > > > login as guest):
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>> http://teamcity.codebetter.com/viewType.html?buildTypeId=Luc
> eneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> >>> > > >
> >>> > > > Thanks, and happy searching,
> >>> > > >
> >>> > > > --
> >>> > > >
> >>> > > > Itamar Syn-Hershko
> >>> > > > http://code972.com | @synhershko <https://twitter.com/synhershko
> >
> >>> > > > Freelance Developer & Consultant
> >>> > > > Lucene.NET committer and PMC member
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>

Re: Last failing tests remain to fix!

Posted by Wyatt Barnett <wy...@gmail.com>.

Sorry, my alter ego Wade and I just got back in town here.

I'm looking into the NuGet feed setup though I need to talk to someone over
at CodeBetter.com as some things need to be enabled and I need to figure
out where the artifacts. I know Martin has moved on from JetBrains, is
there another contact?

To answer your question, no we don't need to wait on passing builds -- we
can tell it to build the nuget regardless. We should probably talk about
what should trigger a nuget release -- probably every build is too much.
Maybe nightly builds for now?

On Mon, Apr 13, 2015 at 5:20 PM Itamar Syn-Hershko <it...@code972.com>
wrote:

> Ok, I just looked and the nuget feed is empty - probably because there was
> never a green build. Can we have nuget artifacts already, at least until we
> have green builds?
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
> On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
> > Errr.. I mean Wyatt :)
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
> > On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <itamar@code972.com
> >
> > wrote:
> >
> >> Hi Wade, any luck?
> >>
> >> Is there a nuget feed on the build server we can start using to install
> >> the core / subproject pieces using nuget from?
> >>
> >> Thanks!
> >>
> >> --
> >>
> >> Itamar Syn-Hershko
> >> http://code972.com | @synhershko <https://twitter.com/synhershko>
> >> Freelance Developer & Consultant
> >> Lucene.NET committer and PMC member
> >>
> >> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <wy...@gmail.com>
> >> wrote:
> >>
> >>> Yup that makes sense -- somehow I conflated sub projects over to
> contrib
> >>> which I haven't seen any movement.
> >>>
> >>> I'm on the road at the moment but I'll see about getting the remaining
> >>> sub
> >>> projects setup and building, auditing the ignores and start banging on
> >>> nuget when I get back to the ranch.
> >>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com>
> wrote:
> >>>
> >>> > inline
> >>> >
> >>> > --
> >>> >
> >>> > Itamar Syn-Hershko
> >>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> >>> > Freelance Developer & Consultant
> >>> > Lucene.NET committer and PMC member
> >>> >
> >>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <
> wyatt.barnett@gmail.com
> >>> >
> >>> > wrote:
> >>> >
> >>> > > Thanks for the update Itamar -- this sounds exciting.
> >>> > >
> >>> > > A few notes / questions from the build side of things here:
> >>> > >
> >>> > > First, I'm not as hung up on persistently failing tests as the
> tests
> >>> we
> >>> > get
> >>> > > intermittent failures on. I suspect some of the below will address
> >>> this.
> >>> > >
> >>> >
> >>> > Yup
> >>> >
> >>> >
> >>> > >
> >>> > > Second, what do we want to do about the 112 or so ignored tests? I
> >>> think
> >>> > > some are marked as long-running via the [LongRunningTest] attribute
> >>> but
> >>> > > there are probably some ignored failures in that mix.
> >>> > >
> >>> >
> >>> > I guess that should be another task someone needs to take - can you
> >>> try and
> >>> > categorize them? some I know are JVM specfic tests that don't apply
> to
> >>> > Lucene.NET
> >>> >
> >>> >
> >>> > >
> >>> > > Third, regarding the sub-projects, what does the structure look
> like
> >>> so I
> >>> > > can get that integrated into the TeamCity setup? Or should I noodle
> >>> on it
> >>> > > and propose one?
> >>> > >
> >>> >
> >>> > What do you mean? See here
> >>> > https://github.com/apache/lucenenet/tree/master/src
> >>> >
> >>> > Every subproject has a test suite that needs to be run, and should
> >>> produce
> >>> > a class library that we could push to nuget when we release. Until
> >>> then we
> >>> > should be able to allow people to install via nuget from our build
> >>> server's
> >>> > feed.
> >>> >
> >>> > As far as dependencies and build triggers - that should be obvious
> and
> >>> self
> >>> > explanatory..
> >>> >
> >>> > Does that make sense?
> >>> >
> >>> >
> >>> > >
> >>> > > Thanks for all the help from everyone getting things this far.
> >>> > >
> >>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
> >>> itamar@code972.com>
> >>> > > wrote:
> >>> > >
> >>> > > > Hey all,
> >>> > > >
> >>> > > > We are nearly there -- we are this close to a release!
> >>> > > >
> >>> > > > We have only about 30 failing tests now in the core, and I'd like
> >>> to
> >>> > push
> >>> > > > hard during the next few days so we can get a beta out next week.
> >>> > > >
> >>> > > > I'd appreciate your help with this!
> >>> > > >
> >>> > > > I've narrowed down several failing tests that require some
> >>> digging, if
> >>> > > > folks can assign those to themselves and send us PRs fixing those
> >>> > tests,
> >>> > > > that would help a lot!
> >>> > > >
> >>> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues
> ()
> >>> > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues
> ()
> >>> --
> >>> > > > specifically it seems like Double and Float NaNs are treated
> >>> > differently
> >>> > > > between C# and Java and therefore the failing tests. Or there is
> a
> >>> bug
> >>> > in
> >>> > > > the inner implementation handling those queries.
> >>> > > >
> >>> > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> >>> > exposing
> >>> > > > data corruption in one of the codec components. I believe this is
> >>> > > contained
> >>> > > > in the 40DocValues tests so the scope isn't huge but I could be
> >>> wrong.
> >>> > > >
> >>> > > > 3. TestLucene42DocValuesFormat has two tests failing with output
> of
> >>> > lines
> >>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> >>> > > computing
> >>> > > > the hashes of the nodes, or maybe the node construction is faulty
> >>> > (which
> >>> > > is
> >>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> >>> that'll
> >>> > > be a
> >>> > > > good starting point. Also see
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f910
> 90d995c729689fff9a0f54c
> >>> > > >
> >>> > > > 4. There seem to be some concurrency issues with file handling,
> >>> > probably
> >>> > > on
> >>> > > > the part that is in charge of acquiring and releasing locks. In
> >>> > > particular
> >>> > > > see TestDocValuesFormat,
> >>> > > > .TestShardSearching.TestSimple(),
> >>> > > > TestBufferedIndexInput.TestSetBufferSize(),
> >>> > > > TestLockFactory.TestStressLocks(),
> >>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> >>> > > >
> >>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
> >>> automaton
> >>> > > > implementation doesn't emit the terms correctly. See the failures
> >>> in
> >>> > > > TestRegexpQuery and TestRegexpRandom.
> >>> > > >
> >>> > > > 6. Similar but probably unrelated failing test is
> >>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> >>> > > >
> >>> > > > 7. An interesting failure to investigate is here:
> >>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
> >>> probably a
> >>> > > > simple porting fail, a subtle Java <--> C# mismatch that we
> haven't
> >>> > > > noticed.
> >>> > > >
> >>> > > > Additionally, any help with porting the subprojects would be
> great.
> >>> > Some
> >>> > > > folks took some subprojects under their wings and never reported
> >>> back.
> >>> > If
> >>> > > > you could either complete the work, or donate whatever you have
> >>> done so
> >>> > > > far, that would be great.
> >>> > > >
> >>> > > > As always, get the code from here:
> >>> https://github.com/apache/lucenenet
> >>> > > --
> >>> > > > work on your fork and send us a PR when ready, with a good
> >>> description
> >>> > of
> >>> > > > what you've done.
> >>> > > >
> >>> > > > To see the list of currently failing tests in real-time, go here
> >>> (you
> >>> > can
> >>> > > > login as guest):
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>> http://teamcity.codebetter.com/viewType.html?buildTypeId=Luc
> eneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> >>> > > >
> >>> > > > Thanks, and happy searching,
> >>> > > >
> >>> > > > --
> >>> > > >
> >>> > > > Itamar Syn-Hershko
> >>> > > > http://code972.com | @synhershko <https://twitter.com/synhershko
> >
> >>> > > > Freelance Developer & Consultant
> >>> > > > Lucene.NET committer and PMC member
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

Ok, I just looked and the nuget feed is empty - probably because there was
never a green build. Can we have nuget artifacts already, at least until we
have green builds?

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <it...@code972.com>
wrote:

> Errr.. I mean Wyatt :)
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
> On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
>> Hi Wade, any luck?
>>
>> Is there a nuget feed on the build server we can start using to install
>> the core / subproject pieces using nuget from?
>>
>> Thanks!
>>
>> --
>>
>> Itamar Syn-Hershko
>> http://code972.com | @synhershko <https://twitter.com/synhershko>
>> Freelance Developer & Consultant
>> Lucene.NET committer and PMC member
>>
>> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <wy...@gmail.com>
>> wrote:
>>
>>> Yup that makes sense -- somehow I conflated sub projects over to contrib
>>> which I haven't seen any movement.
>>>
>>> I'm on the road at the moment but I'll see about getting the remaining
>>> sub
>>> projects setup and building, auditing the ignores and start banging on
>>> nuget when I get back to the ranch.
>>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com> wrote:
>>>
>>> > inline
>>> >
>>> > --
>>> >
>>> > Itamar Syn-Hershko
>>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
>>> > Freelance Developer & Consultant
>>> > Lucene.NET committer and PMC member
>>> >
>>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <wyatt.barnett@gmail.com
>>> >
>>> > wrote:
>>> >
>>> > > Thanks for the update Itamar -- this sounds exciting.
>>> > >
>>> > > A few notes / questions from the build side of things here:
>>> > >
>>> > > First, I'm not as hung up on persistently failing tests as the tests
>>> we
>>> > get
>>> > > intermittent failures on. I suspect some of the below will address
>>> this.
>>> > >
>>> >
>>> > Yup
>>> >
>>> >
>>> > >
>>> > > Second, what do we want to do about the 112 or so ignored tests? I
>>> think
>>> > > some are marked as long-running via the [LongRunningTest] attribute
>>> but
>>> > > there are probably some ignored failures in that mix.
>>> > >
>>> >
>>> > I guess that should be another task someone needs to take - can you
>>> try and
>>> > categorize them? some I know are JVM specfic tests that don't apply to
>>> > Lucene.NET
>>> >
>>> >
>>> > >
>>> > > Third, regarding the sub-projects, what does the structure look like
>>> so I
>>> > > can get that integrated into the TeamCity setup? Or should I noodle
>>> on it
>>> > > and propose one?
>>> > >
>>> >
>>> > What do you mean? See here
>>> > https://github.com/apache/lucenenet/tree/master/src
>>> >
>>> > Every subproject has a test suite that needs to be run, and should
>>> produce
>>> > a class library that we could push to nuget when we release. Until
>>> then we
>>> > should be able to allow people to install via nuget from our build
>>> server's
>>> > feed.
>>> >
>>> > As far as dependencies and build triggers - that should be obvious and
>>> self
>>> > explanatory..
>>> >
>>> > Does that make sense?
>>> >
>>> >
>>> > >
>>> > > Thanks for all the help from everyone getting things this far.
>>> > >
>>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
>>> itamar@code972.com>
>>> > > wrote:
>>> > >
>>> > > > Hey all,
>>> > > >
>>> > > > We are nearly there -- we are this close to a release!
>>> > > >
>>> > > > We have only about 30 failing tests now in the core, and I'd like
>>> to
>>> > push
>>> > > > hard during the next few days so we can get a beta out next week.
>>> > > >
>>> > > > I'd appreciate your help with this!
>>> > > >
>>> > > > I've narrowed down several failing tests that require some
>>> digging, if
>>> > > > folks can assign those to themselves and send us PRs fixing those
>>> > tests,
>>> > > > that would help a lot!
>>> > > >
>>> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
>>> > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues()
>>> --
>>> > > > specifically it seems like Double and Float NaNs are treated
>>> > differently
>>> > > > between C# and Java and therefore the failing tests. Or there is a
>>> bug
>>> > in
>>> > > > the inner implementation handling those queries.
>>> > > >
>>> > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
>>> > exposing
>>> > > > data corruption in one of the codec components. I believe this is
>>> > > contained
>>> > > > in the 40DocValues tests so the scope isn't huge but I could be
>>> wrong.
>>> > > >
>>> > > > 3. TestLucene42DocValuesFormat has two tests failing with output of
>>> > lines
>>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
>>> > > computing
>>> > > > the hashes of the nodes, or maybe the node construction is faulty
>>> > (which
>>> > > is
>>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
>>> that'll
>>> > > be a
>>> > > > good starting point. Also see
>>> > > >
>>> > > >
>>> > >
>>> >
>>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
>>> > > >
>>> > > > 4. There seem to be some concurrency issues with file handling,
>>> > probably
>>> > > on
>>> > > > the part that is in charge of acquiring and releasing locks. In
>>> > > particular
>>> > > > see TestDocValuesFormat,
>>> > > > .TestShardSearching.TestSimple(),
>>> > > > TestBufferedIndexInput.TestSetBufferSize(),
>>> > > > TestLockFactory.TestStressLocks(),
>>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
>>> > > >
>>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
>>> automaton
>>> > > > implementation doesn't emit the terms correctly. See the failures
>>> in
>>> > > > TestRegexpQuery and TestRegexpRandom.
>>> > > >
>>> > > > 6. Similar but probably unrelated failing test is
>>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
>>> > > >
>>> > > > 7. An interesting failure to investigate is here:
>>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
>>> probably a
>>> > > > simple porting fail, a subtle Java <--> C# mismatch that we haven't
>>> > > > noticed.
>>> > > >
>>> > > > Additionally, any help with porting the subprojects would be great.
>>> > Some
>>> > > > folks took some subprojects under their wings and never reported
>>> back.
>>> > If
>>> > > > you could either complete the work, or donate whatever you have
>>> done so
>>> > > > far, that would be great.
>>> > > >
>>> > > > As always, get the code from here:
>>> https://github.com/apache/lucenenet
>>> > > --
>>> > > > work on your fork and send us a PR when ready, with a good
>>> description
>>> > of
>>> > > > what you've done.
>>> > > >
>>> > > > To see the list of currently failing tests in real-time, go here
>>> (you
>>> > can
>>> > > > login as guest):
>>> > > >
>>> > > >
>>> > >
>>> >
>>> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
>>> > > >
>>> > > > Thanks, and happy searching,
>>> > > >
>>> > > > --
>>> > > >
>>> > > > Itamar Syn-Hershko
>>> > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
>>> > > > Freelance Developer & Consultant
>>> > > > Lucene.NET committer and PMC member
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

Errr.. I mean Wyatt :)

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Tue, Apr 14, 2015 at 12:15 AM, Itamar Syn-Hershko <it...@code972.com>
wrote:

> Hi Wade, any luck?
>
> Is there a nuget feed on the build server we can start using to install
> the core / subproject pieces using nuget from?
>
> Thanks!
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
> On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <wy...@gmail.com>
> wrote:
>
>> Yup that makes sense -- somehow I conflated sub projects over to contrib
>> which I haven't seen any movement.
>>
>> I'm on the road at the moment but I'll see about getting the remaining sub
>> projects setup and building, auditing the ignores and start banging on
>> nuget when I get back to the ranch.
>> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com> wrote:
>>
>> > inline
>> >
>> > --
>> >
>> > Itamar Syn-Hershko
>> > http://code972.com | @synhershko <https://twitter.com/synhershko>
>> > Freelance Developer & Consultant
>> > Lucene.NET committer and PMC member
>> >
>> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <wy...@gmail.com>
>> > wrote:
>> >
>> > > Thanks for the update Itamar -- this sounds exciting.
>> > >
>> > > A few notes / questions from the build side of things here:
>> > >
>> > > First, I'm not as hung up on persistently failing tests as the tests
>> we
>> > get
>> > > intermittent failures on. I suspect some of the below will address
>> this.
>> > >
>> >
>> > Yup
>> >
>> >
>> > >
>> > > Second, what do we want to do about the 112 or so ignored tests? I
>> think
>> > > some are marked as long-running via the [LongRunningTest] attribute
>> but
>> > > there are probably some ignored failures in that mix.
>> > >
>> >
>> > I guess that should be another task someone needs to take - can you try
>> and
>> > categorize them? some I know are JVM specfic tests that don't apply to
>> > Lucene.NET
>> >
>> >
>> > >
>> > > Third, regarding the sub-projects, what does the structure look like
>> so I
>> > > can get that integrated into the TeamCity setup? Or should I noodle
>> on it
>> > > and propose one?
>> > >
>> >
>> > What do you mean? See here
>> > https://github.com/apache/lucenenet/tree/master/src
>> >
>> > Every subproject has a test suite that needs to be run, and should
>> produce
>> > a class library that we could push to nuget when we release. Until then
>> we
>> > should be able to allow people to install via nuget from our build
>> server's
>> > feed.
>> >
>> > As far as dependencies and build triggers - that should be obvious and
>> self
>> > explanatory..
>> >
>> > Does that make sense?
>> >
>> >
>> > >
>> > > Thanks for all the help from everyone getting things this far.
>> > >
>> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <
>> itamar@code972.com>
>> > > wrote:
>> > >
>> > > > Hey all,
>> > > >
>> > > > We are nearly there -- we are this close to a release!
>> > > >
>> > > > We have only about 30 failing tests now in the core, and I'd like to
>> > push
>> > > > hard during the next few days so we can get a beta out next week.
>> > > >
>> > > > I'd appreciate your help with this!
>> > > >
>> > > > I've narrowed down several failing tests that require some digging,
>> if
>> > > > folks can assign those to themselves and send us PRs fixing those
>> > tests,
>> > > > that would help a lot!
>> > > >
>> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
>> > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues()
>> --
>> > > > specifically it seems like Double and Float NaNs are treated
>> > differently
>> > > > between C# and Java and therefore the failing tests. Or there is a
>> bug
>> > in
>> > > > the inner implementation handling those queries.
>> > > >
>> > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
>> > exposing
>> > > > data corruption in one of the codec components. I believe this is
>> > > contained
>> > > > in the 40DocValues tests so the scope isn't huge but I could be
>> wrong.
>> > > >
>> > > > 3. TestLucene42DocValuesFormat has two tests failing with output of
>> > lines
>> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
>> > > computing
>> > > > the hashes of the nodes, or maybe the node construction is faulty
>> > (which
>> > > is
>> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
>> that'll
>> > > be a
>> > > > good starting point. Also see
>> > > >
>> > > >
>> > >
>> >
>> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
>> > > >
>> > > > 4. There seem to be some concurrency issues with file handling,
>> > probably
>> > > on
>> > > > the part that is in charge of acquiring and releasing locks. In
>> > > particular
>> > > > see TestDocValuesFormat,
>> > > > .TestShardSearching.TestSimple(),
>> > > > TestBufferedIndexInput.TestSetBufferSize(),
>> > > > TestLockFactory.TestStressLocks(),
>> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
>> > > >
>> > > > 5. RegexpQuery seems to be broken, I believe the underlying
>> automaton
>> > > > implementation doesn't emit the terms correctly. See the failures in
>> > > > TestRegexpQuery and TestRegexpRandom.
>> > > >
>> > > > 6. Similar but probably unrelated failing test is
>> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
>> > > >
>> > > > 7. An interesting failure to investigate is here:
>> > > > TestIndexableBinaryStringTools.TestEncodedSortability() -- probably
>> a
>> > > > simple porting fail, a subtle Java <--> C# mismatch that we haven't
>> > > > noticed.
>> > > >
>> > > > Additionally, any help with porting the subprojects would be great.
>> > Some
>> > > > folks took some subprojects under their wings and never reported
>> back.
>> > If
>> > > > you could either complete the work, or donate whatever you have
>> done so
>> > > > far, that would be great.
>> > > >
>> > > > As always, get the code from here:
>> https://github.com/apache/lucenenet
>> > > --
>> > > > work on your fork and send us a PR when ready, with a good
>> description
>> > of
>> > > > what you've done.
>> > > >
>> > > > To see the list of currently failing tests in real-time, go here
>> (you
>> > can
>> > > > login as guest):
>> > > >
>> > > >
>> > >
>> >
>> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
>> > > >
>> > > > Thanks, and happy searching,
>> > > >
>> > > > --
>> > > >
>> > > > Itamar Syn-Hershko
>> > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
>> > > > Freelance Developer & Consultant
>> > > > Lucene.NET committer and PMC member
>> > > >
>> > >
>> >
>>
>
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

Hi Wade, any luck?

Is there a nuget feed on the build server we can start using to install the
core / subproject pieces using nuget from?

Thanks!

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Thu, Apr 9, 2015 at 1:53 AM, Wyatt Barnett <wy...@gmail.com>
wrote:

> Yup that makes sense -- somehow I conflated sub projects over to contrib
> which I haven't seen any movement.
>
> I'm on the road at the moment but I'll see about getting the remaining sub
> projects setup and building, auditing the ignores and start banging on
> nuget when I get back to the ranch.
> On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com> wrote:
>
> > inline
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
> > On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <wy...@gmail.com>
> > wrote:
> >
> > > Thanks for the update Itamar -- this sounds exciting.
> > >
> > > A few notes / questions from the build side of things here:
> > >
> > > First, I'm not as hung up on persistently failing tests as the tests we
> > get
> > > intermittent failures on. I suspect some of the below will address
> this.
> > >
> >
> > Yup
> >
> >
> > >
> > > Second, what do we want to do about the 112 or so ignored tests? I
> think
> > > some are marked as long-running via the [LongRunningTest] attribute but
> > > there are probably some ignored failures in that mix.
> > >
> >
> > I guess that should be another task someone needs to take - can you try
> and
> > categorize them? some I know are JVM specfic tests that don't apply to
> > Lucene.NET
> >
> >
> > >
> > > Third, regarding the sub-projects, what does the structure look like
> so I
> > > can get that integrated into the TeamCity setup? Or should I noodle on
> it
> > > and propose one?
> > >
> >
> > What do you mean? See here
> > https://github.com/apache/lucenenet/tree/master/src
> >
> > Every subproject has a test suite that needs to be run, and should
> produce
> > a class library that we could push to nuget when we release. Until then
> we
> > should be able to allow people to install via nuget from our build
> server's
> > feed.
> >
> > As far as dependencies and build triggers - that should be obvious and
> self
> > explanatory..
> >
> > Does that make sense?
> >
> >
> > >
> > > Thanks for all the help from everyone getting things this far.
> > >
> > > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <itamar@code972.com
> >
> > > wrote:
> > >
> > > > Hey all,
> > > >
> > > > We are nearly there -- we are this close to a release!
> > > >
> > > > We have only about 30 failing tests now in the core, and I'd like to
> > push
> > > > hard during the next few days so we can get a beta out next week.
> > > >
> > > > I'd appreciate your help with this!
> > > >
> > > > I've narrowed down several failing tests that require some digging,
> if
> > > > folks can assign those to themselves and send us PRs fixing those
> > tests,
> > > > that would help a lot!
> > > >
> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> > > > specifically it seems like Double and Float NaNs are treated
> > differently
> > > > between C# and Java and therefore the failing tests. Or there is a
> bug
> > in
> > > > the inner implementation handling those queries.
> > > >
> > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> > exposing
> > > > data corruption in one of the codec components. I believe this is
> > > contained
> > > > in the 40DocValues tests so the scope isn't huge but I could be
> wrong.
> > > >
> > > > 3. TestLucene42DocValuesFormat has two tests failing with output of
> > lines
> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> > > computing
> > > > the hashes of the nodes, or maybe the node construction is faulty
> > (which
> > > is
> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> that'll
> > > be a
> > > > good starting point. Also see
> > > >
> > > >
> > >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> > > >
> > > > 4. There seem to be some concurrency issues with file handling,
> > probably
> > > on
> > > > the part that is in charge of acquiring and releasing locks. In
> > > particular
> > > > see TestDocValuesFormat,
> > > > .TestShardSearching.TestSimple(),
> > > > TestBufferedIndexInput.TestSetBufferSize(),
> > > > TestLockFactory.TestStressLocks(),
> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > > >
> > > > 5. RegexpQuery seems to be broken, I believe the underlying automaton
> > > > implementation doesn't emit the terms correctly. See the failures in
> > > > TestRegexpQuery and TestRegexpRandom.
> > > >
> > > > 6. Similar but probably unrelated failing test is
> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > > >
> > > > 7. An interesting failure to investigate is here:
> > > > TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> > > > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> > > > noticed.
> > > >
> > > > Additionally, any help with porting the subprojects would be great.
> > Some
> > > > folks took some subprojects under their wings and never reported
> back.
> > If
> > > > you could either complete the work, or donate whatever you have done
> so
> > > > far, that would be great.
> > > >
> > > > As always, get the code from here:
> https://github.com/apache/lucenenet
> > > --
> > > > work on your fork and send us a PR when ready, with a good
> description
> > of
> > > > what you've done.
> > > >
> > > > To see the list of currently failing tests in real-time, go here (you
> > can
> > > > login as guest):
> > > >
> > > >
> > >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > > >
> > > > Thanks, and happy searching,
> > > >
> > > > --
> > > >
> > > > Itamar Syn-Hershko
> > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > > Freelance Developer & Consultant
> > > > Lucene.NET committer and PMC member
> > > >
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by Wyatt Barnett <wy...@gmail.com>.

Yup that makes sense -- somehow I conflated sub projects over to contrib
which I haven't seen any movement.

I'm on the road at the moment but I'll see about getting the remaining sub
projects setup and building, auditing the ignores and start banging on
nuget when I get back to the ranch.
On Apr 8, 2015 5:42 PM, "Itamar Syn-Hershko" <it...@code972.com> wrote:

> inline
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
> On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <wy...@gmail.com>
> wrote:
>
> > Thanks for the update Itamar -- this sounds exciting.
> >
> > A few notes / questions from the build side of things here:
> >
> > First, I'm not as hung up on persistently failing tests as the tests we
> get
> > intermittent failures on. I suspect some of the below will address this.
> >
>
> Yup
>
>
> >
> > Second, what do we want to do about the 112 or so ignored tests? I think
> > some are marked as long-running via the [LongRunningTest] attribute but
> > there are probably some ignored failures in that mix.
> >
>
> I guess that should be another task someone needs to take - can you try and
> categorize them? some I know are JVM specfic tests that don't apply to
> Lucene.NET
>
>
> >
> > Third, regarding the sub-projects, what does the structure look like so I
> > can get that integrated into the TeamCity setup? Or should I noodle on it
> > and propose one?
> >
>
> What do you mean? See here
> https://github.com/apache/lucenenet/tree/master/src
>
> Every subproject has a test suite that needs to be run, and should produce
> a class library that we could push to nuget when we release. Until then we
> should be able to allow people to install via nuget from our build server's
> feed.
>
> As far as dependencies and build triggers - that should be obvious and self
> explanatory..
>
> Does that make sense?
>
>
> >
> > Thanks for all the help from everyone getting things this far.
> >
> > On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <it...@code972.com>
> > wrote:
> >
> > > Hey all,
> > >
> > > We are nearly there -- we are this close to a release!
> > >
> > > We have only about 30 failing tests now in the core, and I'd like to
> push
> > > hard during the next few days so we can get a beta out next week.
> > >
> > > I'd appreciate your help with this!
> > >
> > > I've narrowed down several failing tests that require some digging, if
> > > folks can assign those to themselves and send us PRs fixing those
> tests,
> > > that would help a lot!
> > >
> > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> > > specifically it seems like Double and Float NaNs are treated
> differently
> > > between C# and Java and therefore the failing tests. Or there is a bug
> in
> > > the inner implementation handling those queries.
> > >
> > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> exposing
> > > data corruption in one of the codec components. I believe this is
> > contained
> > > in the 40DocValues tests so the scope isn't huge but I could be wrong.
> > >
> > > 3. TestLucene42DocValuesFormat has two tests failing with output of
> lines
> > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> > computing
> > > the hashes of the nodes, or maybe the node construction is faulty
> (which
> > is
> > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll
> > be a
> > > good starting point. Also see
> > >
> > >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> > >
> > > 4. There seem to be some concurrency issues with file handling,
> probably
> > on
> > > the part that is in charge of acquiring and releasing locks. In
> > particular
> > > see TestDocValuesFormat,
> > > .TestShardSearching.TestSimple(),
> > > TestBufferedIndexInput.TestSetBufferSize(),
> > > TestLockFactory.TestStressLocks(),
> > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > >
> > > 5. RegexpQuery seems to be broken, I believe the underlying automaton
> > > implementation doesn't emit the terms correctly. See the failures in
> > > TestRegexpQuery and TestRegexpRandom.
> > >
> > > 6. Similar but probably unrelated failing test is
> > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > >
> > > 7. An interesting failure to investigate is here:
> > > TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> > > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> > > noticed.
> > >
> > > Additionally, any help with porting the subprojects would be great.
> Some
> > > folks took some subprojects under their wings and never reported back.
> If
> > > you could either complete the work, or donate whatever you have done so
> > > far, that would be great.
> > >
> > > As always, get the code from here: https://github.com/apache/lucenenet
> > --
> > > work on your fork and send us a PR when ready, with a good description
> of
> > > what you've done.
> > >
> > > To see the list of currently failing tests in real-time, go here (you
> can
> > > login as guest):
> > >
> > >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > >
> > > Thanks, and happy searching,
> > >
> > > --
> > >
> > > Itamar Syn-Hershko
> > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > Freelance Developer & Consultant
> > > Lucene.NET committer and PMC member
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

inline

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <wy...@gmail.com>
wrote:

> Thanks for the update Itamar -- this sounds exciting.
>
> A few notes / questions from the build side of things here:
>
> First, I'm not as hung up on persistently failing tests as the tests we get
> intermittent failures on. I suspect some of the below will address this.
>

Yup


>
> Second, what do we want to do about the 112 or so ignored tests? I think
> some are marked as long-running via the [LongRunningTest] attribute but
> there are probably some ignored failures in that mix.
>

I guess that should be another task someone needs to take - can you try and
categorize them? some I know are JVM specfic tests that don't apply to
Lucene.NET


>
> Third, regarding the sub-projects, what does the structure look like so I
> can get that integrated into the TeamCity setup? Or should I noodle on it
> and propose one?
>

What do you mean? See here
https://github.com/apache/lucenenet/tree/master/src

Every subproject has a test suite that needs to be run, and should produce
a class library that we could push to nuget when we release. Until then we
should be able to allow people to install via nuget from our build server's
feed.

As far as dependencies and build triggers - that should be obvious and self
explanatory..

Does that make sense?


>
> Thanks for all the help from everyone getting things this far.
>
> On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
> > Hey all,
> >
> > We are nearly there -- we are this close to a release!
> >
> > We have only about 30 failing tests now in the core, and I'd like to push
> > hard during the next few days so we can get a beta out next week.
> >
> > I'd appreciate your help with this!
> >
> > I've narrowed down several failing tests that require some digging, if
> > folks can assign those to themselves and send us PRs fixing those tests,
> > that would help a lot!
> >
> > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> > specifically it seems like Double and Float NaNs are treated differently
> > between C# and Java and therefore the failing tests. Or there is a bug in
> > the inner implementation handling those queries.
> >
> > 2. TestLucene40DocValuesFormat has a few failing tests, probably exposing
> > data corruption in one of the codec components. I believe this is
> contained
> > in the 40DocValues tests so the scope isn't huge but I could be wrong.
> >
> > 3. TestLucene42DocValuesFormat has two tests failing with output of lines
> > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> computing
> > the hashes of the nodes, or maybe the node construction is faulty (which
> is
> > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll
> be a
> > good starting point. Also see
> >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> >
> > 4. There seem to be some concurrency issues with file handling, probably
> on
> > the part that is in charge of acquiring and releasing locks. In
> particular
> > see TestDocValuesFormat,
> > .TestShardSearching.TestSimple(),
> > TestBufferedIndexInput.TestSetBufferSize(),
> > TestLockFactory.TestStressLocks(),
> > TestLockFactory.TestStressLocksNativeFSLockFactory()
> >
> > 5. RegexpQuery seems to be broken, I believe the underlying automaton
> > implementation doesn't emit the terms correctly. See the failures in
> > TestRegexpQuery and TestRegexpRandom.
> >
> > 6. Similar but probably unrelated failing test is
> > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> >
> > 7. An interesting failure to investigate is here:
> > TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> > noticed.
> >
> > Additionally, any help with porting the subprojects would be great. Some
> > folks took some subprojects under their wings and never reported back. If
> > you could either complete the work, or donate whatever you have done so
> > far, that would be great.
> >
> > As always, get the code from here: https://github.com/apache/lucenenet
> --
> > work on your fork and send us a PR when ready, with a good description of
> > what you've done.
> >
> > To see the list of currently failing tests in real-time, go here (you can
> > login as guest):
> >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> >
> > Thanks, and happy searching,
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

inline

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Thu, Apr 9, 2015 at 1:09 AM, Wyatt Barnett <wy...@gmail.com>
wrote:

> Thanks for the update Itamar -- this sounds exciting.
>
> A few notes / questions from the build side of things here:
>
> First, I'm not as hung up on persistently failing tests as the tests we get
> intermittent failures on. I suspect some of the below will address this.
>

Yup


>
> Second, what do we want to do about the 112 or so ignored tests? I think
> some are marked as long-running via the [LongRunningTest] attribute but
> there are probably some ignored failures in that mix.
>

I guess that should be another task someone needs to take - can you try and
categorize them? some I know are JVM specfic tests that don't apply to
Lucene.NET


>
> Third, regarding the sub-projects, what does the structure look like so I
> can get that integrated into the TeamCity setup? Or should I noodle on it
> and propose one?
>

What do you mean? See here
https://github.com/apache/lucenenet/tree/master/src

Every subproject has a test suite that needs to be run, and should produce
a class library that we could push to nuget when we release. Until then we
should be able to allow people to install via nuget from our build server's
feed.

As far as dependencies and build triggers - that should be obvious and self
explanatory..

Does that make sense?


>
> Thanks for all the help from everyone getting things this far.
>
> On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
> > Hey all,
> >
> > We are nearly there -- we are this close to a release!
> >
> > We have only about 30 failing tests now in the core, and I'd like to push
> > hard during the next few days so we can get a beta out next week.
> >
> > I'd appreciate your help with this!
> >
> > I've narrowed down several failing tests that require some digging, if
> > folks can assign those to themselves and send us PRs fixing those tests,
> > that would help a lot!
> >
> > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> > specifically it seems like Double and Float NaNs are treated differently
> > between C# and Java and therefore the failing tests. Or there is a bug in
> > the inner implementation handling those queries.
> >
> > 2. TestLucene40DocValuesFormat has a few failing tests, probably exposing
> > data corruption in one of the codec components. I believe this is
> contained
> > in the 40DocValues tests so the scope isn't huge but I could be wrong.
> >
> > 3. TestLucene42DocValuesFormat has two tests failing with output of lines
> > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> computing
> > the hashes of the nodes, or maybe the node construction is faulty (which
> is
> > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll
> be a
> > good starting point. Also see
> >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> >
> > 4. There seem to be some concurrency issues with file handling, probably
> on
> > the part that is in charge of acquiring and releasing locks. In
> particular
> > see TestDocValuesFormat,
> > .TestShardSearching.TestSimple(),
> > TestBufferedIndexInput.TestSetBufferSize(),
> > TestLockFactory.TestStressLocks(),
> > TestLockFactory.TestStressLocksNativeFSLockFactory()
> >
> > 5. RegexpQuery seems to be broken, I believe the underlying automaton
> > implementation doesn't emit the terms correctly. See the failures in
> > TestRegexpQuery and TestRegexpRandom.
> >
> > 6. Similar but probably unrelated failing test is
> > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> >
> > 7. An interesting failure to investigate is here:
> > TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> > noticed.
> >
> > Additionally, any help with porting the subprojects would be great. Some
> > folks took some subprojects under their wings and never reported back. If
> > you could either complete the work, or donate whatever you have done so
> > far, that would be great.
> >
> > As always, get the code from here: https://github.com/apache/lucenenet
> --
> > work on your fork and send us a PR when ready, with a good description of
> > what you've done.
> >
> > To see the list of currently failing tests in real-time, go here (you can
> > login as guest):
> >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> >
> > Thanks, and happy searching,
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
>

Re: Last failing tests remain to fix!

Posted by Wyatt Barnett <wy...@gmail.com>.

Thanks for the update Itamar -- this sounds exciting.

A few notes / questions from the build side of things here:

First, I'm not as hung up on persistently failing tests as the tests we get
intermittent failures on. I suspect some of the below will address this.

Second, what do we want to do about the 112 or so ignored tests? I think
some are marked as long-running via the [LongRunningTest] attribute but
there are probably some ignored failures in that mix.

Third, regarding the sub-projects, what does the structure look like so I
can get that integrated into the TeamCity setup? Or should I noodle on it
and propose one?

Thanks for all the help from everyone getting things this far.

On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <it...@code972.com>
wrote:

> Hey all,
>
> We are nearly there -- we are this close to a release!
>
> We have only about 30 failing tests now in the core, and I'd like to push
> hard during the next few days so we can get a beta out next week.
>
> I'd appreciate your help with this!
>
> I've narrowed down several failing tests that require some digging, if
> folks can assign those to themselves and send us PRs fixing those tests,
> that would help a lot!
>
> 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> specifically it seems like Double and Float NaNs are treated differently
> between C# and Java and therefore the failing tests. Or there is a bug in
> the inner implementation handling those queries.
>
> 2. TestLucene40DocValuesFormat has a few failing tests, probably exposing
> data corruption in one of the codec components. I believe this is contained
> in the 40DocValues tests so the scope isn't huge but I could be wrong.
>
> 3. TestLucene42DocValuesFormat has two tests failing with output of lines
> like "Fail: frozenHash=182658 vs h=5124052". Something is off in computing
> the hashes of the nodes, or maybe the node construction is faulty (which is
> a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll be a
> good starting point. Also see
>
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
>
> 4. There seem to be some concurrency issues with file handling, probably on
> the part that is in charge of acquiring and releasing locks. In particular
> see TestDocValuesFormat,
> .TestShardSearching.TestSimple(),
> TestBufferedIndexInput.TestSetBufferSize(),
> TestLockFactory.TestStressLocks(),
> TestLockFactory.TestStressLocksNativeFSLockFactory()
>
> 5. RegexpQuery seems to be broken, I believe the underlying automaton
> implementation doesn't emit the terms correctly. See the failures in
> TestRegexpQuery and TestRegexpRandom.
>
> 6. Similar but probably unrelated failing test is
> at TestSpanMultiTermQueryWrapper.TestFuzzy2()
>
> 7. An interesting failure to investigate is here:
> TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> simple porting fail, a subtle Java <--> C# mismatch that we haven't
> noticed.
>
> Additionally, any help with porting the subprojects would be great. Some
> folks took some subprojects under their wings and never reported back. If
> you could either complete the work, or donate whatever you have done so
> far, that would be great.
>
> As always, get the code from here: https://github.com/apache/lucenenet --
> work on your fork and send us a PR when ready, with a good description of
> what you've done.
>
> To see the list of currently failing tests in real-time, go here (you can
> login as guest):
>
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
>
> Thanks, and happy searching,
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>

Re: Last failing tests remain to fix!

Posted by Laimonas Simutis <la...@gmail.com>.

James,

As far as the failing tests go, the failures highlighted in this thread
have been fixed and we are now at 8 consistently failing tests. We were
running into some very odd floating point rounding issues but just had a
breakthrough and now can explain why 5 of those tests fail. The fixes will
be coming up shortly. So close....

On Mon, Jun 1, 2015 at 9:32 PM, James Lewis <wi...@gmail.com>
wrote:

> I'm curious about how it's going.
>
> Cheers,
> James
>
>
>
> On Sun, May 10, 2015 at 2:27 PM, Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
> > We are now working on the last 5-10 failing tests in core. Still some
> code
> > left to port though.
> > On May 10, 2015 5:37 PM, "James Lewis" <wi...@gmail.com> wrote:
> >
> > > It seems it was on the verge a couple of of weeks ago. I'm wondering
> how
> > > it's going.
> > >
> > > Cheers,
> > > James
> > > On Apr 21, 2015 6:06 AM, "Itamar Syn-Hershko" <it...@code972.com>
> > wrote:
> > >
> > > > Status update:
> > > >
> > > > We are down to about 20 failing tests (thanks Laimis!)
> > > >
> > > > #1 + #2 are both fixed. Still more isolated tests left, and are up
> for
> > > > grabs.
> > > >
> > > > I'm currently working on the Analysis.Common sub-project, as it is a
> > > > prerequisite for a release (we can't release core without a proper
> > > analysis
> > > > package).
> > > >
> > > > Cheers,
> > > >
> > > > --
> > > >
> > > > Itamar Syn-Hershko
> > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > > Freelance Developer & Consultant
> > > > Lucene.NET committer and PMC member
> > > >
> > > > On Wed, Apr 8, 2015 at 10:07 AM, Itamar Syn-Hershko <
> > itamar@code972.com>
> > > > wrote:
> > > >
> > > > > Hey all,
> > > > >
> > > > > We are nearly there -- we are this close to a release!
> > > > >
> > > > > We have only about 30 failing tests now in the core, and I'd like
> to
> > > push
> > > > > hard during the next few days so we can get a beta out next week.
> > > > >
> > > > > I'd appreciate your help with this!
> > > > >
> > > > > I've narrowed down several failing tests that require some digging,
> > if
> > > > > folks can assign those to themselves and send us PRs fixing those
> > > tests,
> > > > > that would help a lot!
> > > > >
> > > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues()
> --
> > > > > specifically it seems like Double and Float NaNs are treated
> > > differently
> > > > > between C# and Java and therefore the failing tests. Or there is a
> > bug
> > > in
> > > > > the inner implementation handling those queries.
> > > > >
> > > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> > > exposing
> > > > > data corruption in one of the codec components. I believe this is
> > > > contained
> > > > > in the 40DocValues tests so the scope isn't huge but I could be
> > wrong.
> > > > >
> > > > > 3. TestLucene42DocValuesFormat has two tests failing with output of
> > > lines
> > > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> > > > computing
> > > > > the hashes of the nodes, or maybe the node construction is faulty
> > > (which
> > > > is
> > > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> > that'll
> > > > be a
> > > > > good starting point. Also see
> > > > >
> > > >
> > >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> > > > >
> > > > > 4. There seem to be some concurrency issues with file handling,
> > > probably
> > > > > on the part that is in charge of acquiring and releasing locks. In
> > > > > particular see TestDocValuesFormat,
> > > > > .TestShardSearching.TestSimple(),
> > > > TestBufferedIndexInput.TestSetBufferSize(),
> > > > TestLockFactory.TestStressLocks(),
> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > > > >
> > > > > 5. RegexpQuery seems to be broken, I believe the underlying
> automaton
> > > > > implementation doesn't emit the terms correctly. See the failures
> in
> > > > > TestRegexpQuery and TestRegexpRandom.
> > > > >
> > > > > 6. Similar but probably unrelated failing test is
> > > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > > > >
> > > > > 7. An interesting failure to investigate is here:
> > > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
> probably a
> > > > > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> > > > noticed.
> > > > >
> > > > > Additionally, any help with porting the subprojects would be great.
> > > Some
> > > > > folks took some subprojects under their wings and never reported
> > back.
> > > If
> > > > > you could either complete the work, or donate whatever you have
> done
> > so
> > > > > far, that would be great.
> > > > >
> > > > > As always, get the code from here:
> > https://github.com/apache/lucenenet
> > > > --
> > > > > work on your fork and send us a PR when ready, with a good
> > description
> > > of
> > > > > what you've done.
> > > > >
> > > > > To see the list of currently failing tests in real-time, go here
> (you
> > > can
> > > > > login as guest):
> > > > >
> > > >
> > >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > > > >
> > > > > Thanks, and happy searching,
> > > > >
> > > > > --
> > > > >
> > > > > Itamar Syn-Hershko
> > > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > > > Freelance Developer & Consultant
> > > > > Lucene.NET committer and PMC member
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by James Lewis <wi...@gmail.com>.

I'm curious about how it's going.

Cheers,
James



On Sun, May 10, 2015 at 2:27 PM, Itamar Syn-Hershko <it...@code972.com>
wrote:

> We are now working on the last 5-10 failing tests in core. Still some code
> left to port though.
> On May 10, 2015 5:37 PM, "James Lewis" <wi...@gmail.com> wrote:
>
> > It seems it was on the verge a couple of of weeks ago. I'm wondering how
> > it's going.
> >
> > Cheers,
> > James
> > On Apr 21, 2015 6:06 AM, "Itamar Syn-Hershko" <it...@code972.com>
> wrote:
> >
> > > Status update:
> > >
> > > We are down to about 20 failing tests (thanks Laimis!)
> > >
> > > #1 + #2 are both fixed. Still more isolated tests left, and are up for
> > > grabs.
> > >
> > > I'm currently working on the Analysis.Common sub-project, as it is a
> > > prerequisite for a release (we can't release core without a proper
> > analysis
> > > package).
> > >
> > > Cheers,
> > >
> > > --
> > >
> > > Itamar Syn-Hershko
> > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > Freelance Developer & Consultant
> > > Lucene.NET committer and PMC member
> > >
> > > On Wed, Apr 8, 2015 at 10:07 AM, Itamar Syn-Hershko <
> itamar@code972.com>
> > > wrote:
> > >
> > > > Hey all,
> > > >
> > > > We are nearly there -- we are this close to a release!
> > > >
> > > > We have only about 30 failing tests now in the core, and I'd like to
> > push
> > > > hard during the next few days so we can get a beta out next week.
> > > >
> > > > I'd appreciate your help with this!
> > > >
> > > > I've narrowed down several failing tests that require some digging,
> if
> > > > folks can assign those to themselves and send us PRs fixing those
> > tests,
> > > > that would help a lot!
> > > >
> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> > > > specifically it seems like Double and Float NaNs are treated
> > differently
> > > > between C# and Java and therefore the failing tests. Or there is a
> bug
> > in
> > > > the inner implementation handling those queries.
> > > >
> > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> > exposing
> > > > data corruption in one of the codec components. I believe this is
> > > contained
> > > > in the 40DocValues tests so the scope isn't huge but I could be
> wrong.
> > > >
> > > > 3. TestLucene42DocValuesFormat has two tests failing with output of
> > lines
> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> > > computing
> > > > the hashes of the nodes, or maybe the node construction is faulty
> > (which
> > > is
> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> that'll
> > > be a
> > > > good starting point. Also see
> > > >
> > >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> > > >
> > > > 4. There seem to be some concurrency issues with file handling,
> > probably
> > > > on the part that is in charge of acquiring and releasing locks. In
> > > > particular see TestDocValuesFormat,
> > > > .TestShardSearching.TestSimple(),
> > > TestBufferedIndexInput.TestSetBufferSize(),
> > > TestLockFactory.TestStressLocks(),
> > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > > >
> > > > 5. RegexpQuery seems to be broken, I believe the underlying automaton
> > > > implementation doesn't emit the terms correctly. See the failures in
> > > > TestRegexpQuery and TestRegexpRandom.
> > > >
> > > > 6. Similar but probably unrelated failing test is
> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > > >
> > > > 7. An interesting failure to investigate is here:
> > > > TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> > > > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> > > noticed.
> > > >
> > > > Additionally, any help with porting the subprojects would be great.
> > Some
> > > > folks took some subprojects under their wings and never reported
> back.
> > If
> > > > you could either complete the work, or donate whatever you have done
> so
> > > > far, that would be great.
> > > >
> > > > As always, get the code from here:
> https://github.com/apache/lucenenet
> > > --
> > > > work on your fork and send us a PR when ready, with a good
> description
> > of
> > > > what you've done.
> > > >
> > > > To see the list of currently failing tests in real-time, go here (you
> > can
> > > > login as guest):
> > > >
> > >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > > >
> > > > Thanks, and happy searching,
> > > >
> > > > --
> > > >
> > > > Itamar Syn-Hershko
> > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > > Freelance Developer & Consultant
> > > > Lucene.NET committer and PMC member
> > > >
> > > >
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by James Lewis <wi...@gmail.com>.

Thanks.  That's good to know. I'd offer to help if I wasn't busy myself.
Also, by the time I get up to speed with where you're at the thing will
probably be done.

Cheers,
James
On Jul 4, 2015 11:43 PM, "Itamar Syn-Hershko" <it...@code972.com> wrote:

> They will.
>
> It's going slower lately, mostly because me and apparently the other
> committers have become more busy with their other responsibilities. Like I
> said a few times before, we could use more help.
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
> On Sat, Jul 4, 2015 at 2:19 AM, James Lewis <wi...@gmail.com>
> wrote:
>
> > Once more, I'm wondering how it's going. Also, I was wondering the if the
> > KStem and other Lucene 4.0 filters will be available with this release.
> >
> > Cheers,
> > James
> >
> >
> >
> > On Sun, May 10, 2015 at 2:27 PM, Itamar Syn-Hershko <it...@code972.com>
> > wrote:
> >
> > > We are now working on the last 5-10 failing tests in core. Still some
> > code
> > > left to port though.
> > > On May 10, 2015 5:37 PM, "James Lewis" <wi...@gmail.com>
> wrote:
> > >
> > > > It seems it was on the verge a couple of of weeks ago. I'm wondering
> > how
> > > > it's going.
> > > >
> > > > Cheers,
> > > > James
> > > > On Apr 21, 2015 6:06 AM, "Itamar Syn-Hershko" <it...@code972.com>
> > > wrote:
> > > >
> > > > > Status update:
> > > > >
> > > > > We are down to about 20 failing tests (thanks Laimis!)
> > > > >
> > > > > #1 + #2 are both fixed. Still more isolated tests left, and are up
> > for
> > > > > grabs.
> > > > >
> > > > > I'm currently working on the Analysis.Common sub-project, as it is
> a
> > > > > prerequisite for a release (we can't release core without a proper
> > > > analysis
> > > > > package).
> > > > >
> > > > > Cheers,
> > > > >
> > > > > --
> > > > >
> > > > > Itamar Syn-Hershko
> > > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > > > Freelance Developer & Consultant
> > > > > Lucene.NET committer and PMC member
> > > > >
> > > > > On Wed, Apr 8, 2015 at 10:07 AM, Itamar Syn-Hershko <
> > > itamar@code972.com>
> > > > > wrote:
> > > > >
> > > > > > Hey all,
> > > > > >
> > > > > > We are nearly there -- we are this close to a release!
> > > > > >
> > > > > > We have only about 30 failing tests now in the core, and I'd like
> > to
> > > > push
> > > > > > hard during the next few days so we can get a beta out next week.
> > > > > >
> > > > > > I'd appreciate your help with this!
> > > > > >
> > > > > > I've narrowed down several failing tests that require some
> digging,
> > > if
> > > > > > folks can assign those to themselves and send us PRs fixing those
> > > > tests,
> > > > > > that would help a lot!
> > > > > >
> > > > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > > > > > and
> Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues()
> > --
> > > > > > specifically it seems like Double and Float NaNs are treated
> > > > differently
> > > > > > between C# and Java and therefore the failing tests. Or there is
> a
> > > bug
> > > > in
> > > > > > the inner implementation handling those queries.
> > > > > >
> > > > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> > > > exposing
> > > > > > data corruption in one of the codec components. I believe this is
> > > > > contained
> > > > > > in the 40DocValues tests so the scope isn't huge but I could be
> > > wrong.
> > > > > >
> > > > > > 3. TestLucene42DocValuesFormat has two tests failing with output
> of
> > > > lines
> > > > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> > > > > computing
> > > > > > the hashes of the nodes, or maybe the node construction is faulty
> > > > (which
> > > > > is
> > > > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> > > that'll
> > > > > be a
> > > > > > good starting point. Also see
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> > > > > >
> > > > > > 4. There seem to be some concurrency issues with file handling,
> > > > probably
> > > > > > on the part that is in charge of acquiring and releasing locks.
> In
> > > > > > particular see TestDocValuesFormat,
> > > > > > .TestShardSearching.TestSimple(),
> > > > > TestBufferedIndexInput.TestSetBufferSize(),
> > > > > TestLockFactory.TestStressLocks(),
> > > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > > > > >
> > > > > > 5. RegexpQuery seems to be broken, I believe the underlying
> > automaton
> > > > > > implementation doesn't emit the terms correctly. See the failures
> > in
> > > > > > TestRegexpQuery and TestRegexpRandom.
> > > > > >
> > > > > > 6. Similar but probably unrelated failing test is
> > > > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > > > > >
> > > > > > 7. An interesting failure to investigate is here:
> > > > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
> > probably a
> > > > > > simple porting fail, a subtle Java <--> C# mismatch that we
> haven't
> > > > > noticed.
> > > > > >
> > > > > > Additionally, any help with porting the subprojects would be
> great.
> > > > Some
> > > > > > folks took some subprojects under their wings and never reported
> > > back.
> > > > If
> > > > > > you could either complete the work, or donate whatever you have
> > done
> > > so
> > > > > > far, that would be great.
> > > > > >
> > > > > > As always, get the code from here:
> > > https://github.com/apache/lucenenet
> > > > > --
> > > > > > work on your fork and send us a PR when ready, with a good
> > > description
> > > > of
> > > > > > what you've done.
> > > > > >
> > > > > > To see the list of currently failing tests in real-time, go here
> > (you
> > > > can
> > > > > > login as guest):
> > > > > >
> > > > >
> > > >
> > >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > > > > >
> > > > > > Thanks, and happy searching,
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Itamar Syn-Hershko
> > > > > > http://code972.com | @synhershko <https://twitter.com/synhershko
> >
> > > > > > Freelance Developer & Consultant
> > > > > > Lucene.NET committer and PMC member
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

They will.

It's going slower lately, mostly because me and apparently the other
committers have become more busy with their other responsibilities. Like I
said a few times before, we could use more help.

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Sat, Jul 4, 2015 at 2:19 AM, James Lewis <wi...@gmail.com>
wrote:

> Once more, I'm wondering how it's going. Also, I was wondering the if the
> KStem and other Lucene 4.0 filters will be available with this release.
>
> Cheers,
> James
>
>
>
> On Sun, May 10, 2015 at 2:27 PM, Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
> > We are now working on the last 5-10 failing tests in core. Still some
> code
> > left to port though.
> > On May 10, 2015 5:37 PM, "James Lewis" <wi...@gmail.com> wrote:
> >
> > > It seems it was on the verge a couple of of weeks ago. I'm wondering
> how
> > > it's going.
> > >
> > > Cheers,
> > > James
> > > On Apr 21, 2015 6:06 AM, "Itamar Syn-Hershko" <it...@code972.com>
> > wrote:
> > >
> > > > Status update:
> > > >
> > > > We are down to about 20 failing tests (thanks Laimis!)
> > > >
> > > > #1 + #2 are both fixed. Still more isolated tests left, and are up
> for
> > > > grabs.
> > > >
> > > > I'm currently working on the Analysis.Common sub-project, as it is a
> > > > prerequisite for a release (we can't release core without a proper
> > > analysis
> > > > package).
> > > >
> > > > Cheers,
> > > >
> > > > --
> > > >
> > > > Itamar Syn-Hershko
> > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > > Freelance Developer & Consultant
> > > > Lucene.NET committer and PMC member
> > > >
> > > > On Wed, Apr 8, 2015 at 10:07 AM, Itamar Syn-Hershko <
> > itamar@code972.com>
> > > > wrote:
> > > >
> > > > > Hey all,
> > > > >
> > > > > We are nearly there -- we are this close to a release!
> > > > >
> > > > > We have only about 30 failing tests now in the core, and I'd like
> to
> > > push
> > > > > hard during the next few days so we can get a beta out next week.
> > > > >
> > > > > I'd appreciate your help with this!
> > > > >
> > > > > I've narrowed down several failing tests that require some digging,
> > if
> > > > > folks can assign those to themselves and send us PRs fixing those
> > > tests,
> > > > > that would help a lot!
> > > > >
> > > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues()
> --
> > > > > specifically it seems like Double and Float NaNs are treated
> > > differently
> > > > > between C# and Java and therefore the failing tests. Or there is a
> > bug
> > > in
> > > > > the inner implementation handling those queries.
> > > > >
> > > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> > > exposing
> > > > > data corruption in one of the codec components. I believe this is
> > > > contained
> > > > > in the 40DocValues tests so the scope isn't huge but I could be
> > wrong.
> > > > >
> > > > > 3. TestLucene42DocValuesFormat has two tests failing with output of
> > > lines
> > > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> > > > computing
> > > > > the hashes of the nodes, or maybe the node construction is faulty
> > > (which
> > > > is
> > > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> > that'll
> > > > be a
> > > > > good starting point. Also see
> > > > >
> > > >
> > >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> > > > >
> > > > > 4. There seem to be some concurrency issues with file handling,
> > > probably
> > > > > on the part that is in charge of acquiring and releasing locks. In
> > > > > particular see TestDocValuesFormat,
> > > > > .TestShardSearching.TestSimple(),
> > > > TestBufferedIndexInput.TestSetBufferSize(),
> > > > TestLockFactory.TestStressLocks(),
> > > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > > > >
> > > > > 5. RegexpQuery seems to be broken, I believe the underlying
> automaton
> > > > > implementation doesn't emit the terms correctly. See the failures
> in
> > > > > TestRegexpQuery and TestRegexpRandom.
> > > > >
> > > > > 6. Similar but probably unrelated failing test is
> > > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > > > >
> > > > > 7. An interesting failure to investigate is here:
> > > > > TestIndexableBinaryStringTools.TestEncodedSortability() --
> probably a
> > > > > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> > > > noticed.
> > > > >
> > > > > Additionally, any help with porting the subprojects would be great.
> > > Some
> > > > > folks took some subprojects under their wings and never reported
> > back.
> > > If
> > > > > you could either complete the work, or donate whatever you have
> done
> > so
> > > > > far, that would be great.
> > > > >
> > > > > As always, get the code from here:
> > https://github.com/apache/lucenenet
> > > > --
> > > > > work on your fork and send us a PR when ready, with a good
> > description
> > > of
> > > > > what you've done.
> > > > >
> > > > > To see the list of currently failing tests in real-time, go here
> (you
> > > can
> > > > > login as guest):
> > > > >
> > > >
> > >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > > > >
> > > > > Thanks, and happy searching,
> > > > >
> > > > > --
> > > > >
> > > > > Itamar Syn-Hershko
> > > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > > > Freelance Developer & Consultant
> > > > > Lucene.NET committer and PMC member
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by James Lewis <wi...@gmail.com>.

Once more, I'm wondering how it's going. Also, I was wondering the if the
KStem and other Lucene 4.0 filters will be available with this release.

Cheers,
James



On Sun, May 10, 2015 at 2:27 PM, Itamar Syn-Hershko <it...@code972.com>
wrote:

> We are now working on the last 5-10 failing tests in core. Still some code
> left to port though.
> On May 10, 2015 5:37 PM, "James Lewis" <wi...@gmail.com> wrote:
>
> > It seems it was on the verge a couple of of weeks ago. I'm wondering how
> > it's going.
> >
> > Cheers,
> > James
> > On Apr 21, 2015 6:06 AM, "Itamar Syn-Hershko" <it...@code972.com>
> wrote:
> >
> > > Status update:
> > >
> > > We are down to about 20 failing tests (thanks Laimis!)
> > >
> > > #1 + #2 are both fixed. Still more isolated tests left, and are up for
> > > grabs.
> > >
> > > I'm currently working on the Analysis.Common sub-project, as it is a
> > > prerequisite for a release (we can't release core without a proper
> > analysis
> > > package).
> > >
> > > Cheers,
> > >
> > > --
> > >
> > > Itamar Syn-Hershko
> > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > Freelance Developer & Consultant
> > > Lucene.NET committer and PMC member
> > >
> > > On Wed, Apr 8, 2015 at 10:07 AM, Itamar Syn-Hershko <
> itamar@code972.com>
> > > wrote:
> > >
> > > > Hey all,
> > > >
> > > > We are nearly there -- we are this close to a release!
> > > >
> > > > We have only about 30 failing tests now in the core, and I'd like to
> > push
> > > > hard during the next few days so we can get a beta out next week.
> > > >
> > > > I'd appreciate your help with this!
> > > >
> > > > I've narrowed down several failing tests that require some digging,
> if
> > > > folks can assign those to themselves and send us PRs fixing those
> > tests,
> > > > that would help a lot!
> > > >
> > > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> > > > specifically it seems like Double and Float NaNs are treated
> > differently
> > > > between C# and Java and therefore the failing tests. Or there is a
> bug
> > in
> > > > the inner implementation handling those queries.
> > > >
> > > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> > exposing
> > > > data corruption in one of the codec components. I believe this is
> > > contained
> > > > in the 40DocValues tests so the scope isn't huge but I could be
> wrong.
> > > >
> > > > 3. TestLucene42DocValuesFormat has two tests failing with output of
> > lines
> > > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> > > computing
> > > > the hashes of the nodes, or maybe the node construction is faulty
> > (which
> > > is
> > > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>,
> that'll
> > > be a
> > > > good starting point. Also see
> > > >
> > >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> > > >
> > > > 4. There seem to be some concurrency issues with file handling,
> > probably
> > > > on the part that is in charge of acquiring and releasing locks. In
> > > > particular see TestDocValuesFormat,
> > > > .TestShardSearching.TestSimple(),
> > > TestBufferedIndexInput.TestSetBufferSize(),
> > > TestLockFactory.TestStressLocks(),
> > > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > > >
> > > > 5. RegexpQuery seems to be broken, I believe the underlying automaton
> > > > implementation doesn't emit the terms correctly. See the failures in
> > > > TestRegexpQuery and TestRegexpRandom.
> > > >
> > > > 6. Similar but probably unrelated failing test is
> > > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > > >
> > > > 7. An interesting failure to investigate is here:
> > > > TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> > > > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> > > noticed.
> > > >
> > > > Additionally, any help with porting the subprojects would be great.
> > Some
> > > > folks took some subprojects under their wings and never reported
> back.
> > If
> > > > you could either complete the work, or donate whatever you have done
> so
> > > > far, that would be great.
> > > >
> > > > As always, get the code from here:
> https://github.com/apache/lucenenet
> > > --
> > > > work on your fork and send us a PR when ready, with a good
> description
> > of
> > > > what you've done.
> > > >
> > > > To see the list of currently failing tests in real-time, go here (you
> > can
> > > > login as guest):
> > > >
> > >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > > >
> > > > Thanks, and happy searching,
> > > >
> > > > --
> > > >
> > > > Itamar Syn-Hershko
> > > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > > Freelance Developer & Consultant
> > > > Lucene.NET committer and PMC member
> > > >
> > > >
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

We are now working on the last 5-10 failing tests in core. Still some code
left to port though.
On May 10, 2015 5:37 PM, "James Lewis" <wi...@gmail.com> wrote:

> It seems it was on the verge a couple of of weeks ago. I'm wondering how
> it's going.
>
> Cheers,
> James
> On Apr 21, 2015 6:06 AM, "Itamar Syn-Hershko" <it...@code972.com> wrote:
>
> > Status update:
> >
> > We are down to about 20 failing tests (thanks Laimis!)
> >
> > #1 + #2 are both fixed. Still more isolated tests left, and are up for
> > grabs.
> >
> > I'm currently working on the Analysis.Common sub-project, as it is a
> > prerequisite for a release (we can't release core without a proper
> analysis
> > package).
> >
> > Cheers,
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
> > On Wed, Apr 8, 2015 at 10:07 AM, Itamar Syn-Hershko <it...@code972.com>
> > wrote:
> >
> > > Hey all,
> > >
> > > We are nearly there -- we are this close to a release!
> > >
> > > We have only about 30 failing tests now in the core, and I'd like to
> push
> > > hard during the next few days so we can get a beta out next week.
> > >
> > > I'd appreciate your help with this!
> > >
> > > I've narrowed down several failing tests that require some digging, if
> > > folks can assign those to themselves and send us PRs fixing those
> tests,
> > > that would help a lot!
> > >
> > > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> > > specifically it seems like Double and Float NaNs are treated
> differently
> > > between C# and Java and therefore the failing tests. Or there is a bug
> in
> > > the inner implementation handling those queries.
> > >
> > > 2. TestLucene40DocValuesFormat has a few failing tests, probably
> exposing
> > > data corruption in one of the codec components. I believe this is
> > contained
> > > in the 40DocValues tests so the scope isn't huge but I could be wrong.
> > >
> > > 3. TestLucene42DocValuesFormat has two tests failing with output of
> lines
> > > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> > computing
> > > the hashes of the nodes, or maybe the node construction is faulty
> (which
> > is
> > > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll
> > be a
> > > good starting point. Also see
> > >
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> > >
> > > 4. There seem to be some concurrency issues with file handling,
> probably
> > > on the part that is in charge of acquiring and releasing locks. In
> > > particular see TestDocValuesFormat,
> > > .TestShardSearching.TestSimple(),
> > TestBufferedIndexInput.TestSetBufferSize(),
> > TestLockFactory.TestStressLocks(),
> > TestLockFactory.TestStressLocksNativeFSLockFactory()
> > >
> > > 5. RegexpQuery seems to be broken, I believe the underlying automaton
> > > implementation doesn't emit the terms correctly. See the failures in
> > > TestRegexpQuery and TestRegexpRandom.
> > >
> > > 6. Similar but probably unrelated failing test is
> > > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> > >
> > > 7. An interesting failure to investigate is here:
> > > TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> > > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> > noticed.
> > >
> > > Additionally, any help with porting the subprojects would be great.
> Some
> > > folks took some subprojects under their wings and never reported back.
> If
> > > you could either complete the work, or donate whatever you have done so
> > > far, that would be great.
> > >
> > > As always, get the code from here: https://github.com/apache/lucenenet
> > --
> > > work on your fork and send us a PR when ready, with a good description
> of
> > > what you've done.
> > >
> > > To see the list of currently failing tests in real-time, go here (you
> can
> > > login as guest):
> > >
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> > >
> > > Thanks, and happy searching,
> > >
> > > --
> > >
> > > Itamar Syn-Hershko
> > > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > > Freelance Developer & Consultant
> > > Lucene.NET committer and PMC member
> > >
> > >
> >
>

Re: Last failing tests remain to fix!

Posted by James Lewis <wi...@gmail.com>.

It seems it was on the verge a couple of of weeks ago. I'm wondering how
it's going.

Cheers,
James
On Apr 21, 2015 6:06 AM, "Itamar Syn-Hershko" <it...@code972.com> wrote:

> Status update:
>
> We are down to about 20 failing tests (thanks Laimis!)
>
> #1 + #2 are both fixed. Still more isolated tests left, and are up for
> grabs.
>
> I'm currently working on the Analysis.Common sub-project, as it is a
> prerequisite for a release (we can't release core without a proper analysis
> package).
>
> Cheers,
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
> On Wed, Apr 8, 2015 at 10:07 AM, Itamar Syn-Hershko <it...@code972.com>
> wrote:
>
> > Hey all,
> >
> > We are nearly there -- we are this close to a release!
> >
> > We have only about 30 failing tests now in the core, and I'd like to push
> > hard during the next few days so we can get a beta out next week.
> >
> > I'd appreciate your help with this!
> >
> > I've narrowed down several failing tests that require some digging, if
> > folks can assign those to themselves and send us PRs fixing those tests,
> > that would help a lot!
> >
> > 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> > and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> > specifically it seems like Double and Float NaNs are treated differently
> > between C# and Java and therefore the failing tests. Or there is a bug in
> > the inner implementation handling those queries.
> >
> > 2. TestLucene40DocValuesFormat has a few failing tests, probably exposing
> > data corruption in one of the codec components. I believe this is
> contained
> > in the 40DocValues tests so the scope isn't huge but I could be wrong.
> >
> > 3. TestLucene42DocValuesFormat has two tests failing with output of lines
> > like "Fail: frozenHash=182658 vs h=5124052". Something is off in
> computing
> > the hashes of the nodes, or maybe the node construction is faulty (which
> is
> > a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll
> be a
> > good starting point. Also see
> >
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
> >
> > 4. There seem to be some concurrency issues with file handling, probably
> > on the part that is in charge of acquiring and releasing locks. In
> > particular see TestDocValuesFormat,
> > .TestShardSearching.TestSimple(),
> TestBufferedIndexInput.TestSetBufferSize(),
> TestLockFactory.TestStressLocks(),
> TestLockFactory.TestStressLocksNativeFSLockFactory()
> >
> > 5. RegexpQuery seems to be broken, I believe the underlying automaton
> > implementation doesn't emit the terms correctly. See the failures in
> > TestRegexpQuery and TestRegexpRandom.
> >
> > 6. Similar but probably unrelated failing test is
> > at TestSpanMultiTermQueryWrapper.TestFuzzy2()
> >
> > 7. An interesting failure to investigate is here:
> > TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> > simple porting fail, a subtle Java <--> C# mismatch that we haven't
> noticed.
> >
> > Additionally, any help with porting the subprojects would be great. Some
> > folks took some subprojects under their wings and never reported back. If
> > you could either complete the work, or donate whatever you have done so
> > far, that would be great.
> >
> > As always, get the code from here: https://github.com/apache/lucenenet
> --
> > work on your fork and send us a PR when ready, with a good description of
> > what you've done.
> >
> > To see the list of currently failing tests in real-time, go here (you can
> > login as guest):
> >
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
> >
> > Thanks, and happy searching,
> >
> > --
> >
> > Itamar Syn-Hershko
> > http://code972.com | @synhershko <https://twitter.com/synhershko>
> > Freelance Developer & Consultant
> > Lucene.NET committer and PMC member
> >
> >
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

Status update:

We are down to about 20 failing tests (thanks Laimis!)

#1 + #2 are both fixed. Still more isolated tests left, and are up for
grabs.

I'm currently working on the Analysis.Common sub-project, as it is a
prerequisite for a release (we can't release core without a proper analysis
package).

Cheers,

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Wed, Apr 8, 2015 at 10:07 AM, Itamar Syn-Hershko <it...@code972.com>
wrote:

> Hey all,
>
> We are nearly there -- we are this close to a release!
>
> We have only about 30 failing tests now in the core, and I'd like to push
> hard during the next few days so we can get a beta out next week.
>
> I'd appreciate your help with this!
>
> I've narrowed down several failing tests that require some digging, if
> folks can assign those to themselves and send us PRs fixing those tests,
> that would help a lot!
>
> 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> specifically it seems like Double and Float NaNs are treated differently
> between C# and Java and therefore the failing tests. Or there is a bug in
> the inner implementation handling those queries.
>
> 2. TestLucene40DocValuesFormat has a few failing tests, probably exposing
> data corruption in one of the codec components. I believe this is contained
> in the 40DocValues tests so the scope isn't huge but I could be wrong.
>
> 3. TestLucene42DocValuesFormat has two tests failing with output of lines
> like "Fail: frozenHash=182658 vs h=5124052". Something is off in computing
> the hashes of the nodes, or maybe the node construction is faulty (which is
> a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll be a
> good starting point. Also see
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
>
> 4. There seem to be some concurrency issues with file handling, probably
> on the part that is in charge of acquiring and releasing locks. In
> particular see TestDocValuesFormat,
> .TestShardSearching.TestSimple(), TestBufferedIndexInput.TestSetBufferSize(), TestLockFactory.TestStressLocks(), TestLockFactory.TestStressLocksNativeFSLockFactory()
>
> 5. RegexpQuery seems to be broken, I believe the underlying automaton
> implementation doesn't emit the terms correctly. See the failures in
> TestRegexpQuery and TestRegexpRandom.
>
> 6. Similar but probably unrelated failing test is
> at TestSpanMultiTermQueryWrapper.TestFuzzy2()
>
> 7. An interesting failure to investigate is here:
> TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> simple porting fail, a subtle Java <--> C# mismatch that we haven't noticed.
>
> Additionally, any help with porting the subprojects would be great. Some
> folks took some subprojects under their wings and never reported back. If
> you could either complete the work, or donate whatever you have done so
> far, that would be great.
>
> As always, get the code from here: https://github.com/apache/lucenenet --
> work on your fork and send us a PR when ready, with a good description of
> what you've done.
>
> To see the list of currently failing tests in real-time, go here (you can
> login as guest):
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
>
> Thanks, and happy searching,
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
>

Re: Last failing tests remain to fix!

Posted by Itamar Syn-Hershko <it...@code972.com>.

Status update:

We are down to about 20 failing tests (thanks Laimis!)

#1 + #2 are both fixed. Still more isolated tests left, and are up for
grabs.

I'm currently working on the Analysis.Common sub-project, as it is a
prerequisite for a release (we can't release core without a proper analysis
package).

Cheers,

--

Itamar Syn-Hershko
http://code972.com | @synhershko <https://twitter.com/synhershko>
Freelance Developer & Consultant
Lucene.NET committer and PMC member

On Wed, Apr 8, 2015 at 10:07 AM, Itamar Syn-Hershko <it...@code972.com>
wrote:

> Hey all,
>
> We are nearly there -- we are this close to a release!
>
> We have only about 30 failing tests now in the core, and I'd like to push
> hard during the next few days so we can get a beta out next week.
>
> I'd appreciate your help with this!
>
> I've narrowed down several failing tests that require some digging, if
> folks can assign those to themselves and send us PRs fixing those tests,
> that would help a lot!
>
> 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> specifically it seems like Double and Float NaNs are treated differently
> between C# and Java and therefore the failing tests. Or there is a bug in
> the inner implementation handling those queries.
>
> 2. TestLucene40DocValuesFormat has a few failing tests, probably exposing
> data corruption in one of the codec components. I believe this is contained
> in the 40DocValues tests so the scope isn't huge but I could be wrong.
>
> 3. TestLucene42DocValuesFormat has two tests failing with output of lines
> like "Fail: frozenHash=182658 vs h=5124052". Something is off in computing
> the hashes of the nodes, or maybe the node construction is faulty (which is
> a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll be a
> good starting point. Also see
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
>
> 4. There seem to be some concurrency issues with file handling, probably
> on the part that is in charge of acquiring and releasing locks. In
> particular see TestDocValuesFormat,
> .TestShardSearching.TestSimple(), TestBufferedIndexInput.TestSetBufferSize(), TestLockFactory.TestStressLocks(), TestLockFactory.TestStressLocksNativeFSLockFactory()
>
> 5. RegexpQuery seems to be broken, I believe the underlying automaton
> implementation doesn't emit the terms correctly. See the failures in
> TestRegexpQuery and TestRegexpRandom.
>
> 6. Similar but probably unrelated failing test is
> at TestSpanMultiTermQueryWrapper.TestFuzzy2()
>
> 7. An interesting failure to investigate is here:
> TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> simple porting fail, a subtle Java <--> C# mismatch that we haven't noticed.
>
> Additionally, any help with porting the subprojects would be great. Some
> folks took some subprojects under their wings and never reported back. If
> you could either complete the work, or donate whatever you have done so
> far, that would be great.
>
> As always, get the code from here: https://github.com/apache/lucenenet --
> work on your fork and send us a PR when ready, with a good description of
> what you've done.
>
> To see the list of currently failing tests in real-time, go here (you can
> login as guest):
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
>
> Thanks, and happy searching,
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>
>

Re: Last failing tests remain to fix!

Posted by Wyatt Barnett <wy...@gmail.com>.

Thanks for the update Itamar -- this sounds exciting.

A few notes / questions from the build side of things here:

First, I'm not as hung up on persistently failing tests as the tests we get
intermittent failures on. I suspect some of the below will address this.

Second, what do we want to do about the 112 or so ignored tests? I think
some are marked as long-running via the [LongRunningTest] attribute but
there are probably some ignored failures in that mix.

Third, regarding the sub-projects, what does the structure look like so I
can get that integrated into the TeamCity setup? Or should I noodle on it
and propose one?

Thanks for all the help from everyone getting things this far.

On Wed, Apr 8, 2015 at 1:07 PM, Itamar Syn-Hershko <it...@code972.com>
wrote:

> Hey all,
>
> We are nearly there -- we are this close to a release!
>
> We have only about 30 failing tests now in the core, and I'd like to push
> hard during the next few days so we can get a beta out next week.
>
> I'd appreciate your help with this!
>
> I've narrowed down several failing tests that require some digging, if
> folks can assign those to themselves and send us PRs fixing those tests,
> that would help a lot!
>
> 1. Lucene.Net.Search.TestNumericRangeQuery32.TestInfiniteValues()
> and Lucene.Net.Search.TestNumericRangeQuery64.TestInfiniteValues() --
> specifically it seems like Double and Float NaNs are treated differently
> between C# and Java and therefore the failing tests. Or there is a bug in
> the inner implementation handling those queries.
>
> 2. TestLucene40DocValuesFormat has a few failing tests, probably exposing
> data corruption in one of the codec components. I believe this is contained
> in the 40DocValues tests so the scope isn't huge but I could be wrong.
>
> 3. TestLucene42DocValuesFormat has two tests failing with output of lines
> like "Fail: frozenHash=182658 vs h=5124052". Something is off in computing
> the hashes of the nodes, or maybe the node construction is faulty (which is
> a serious bug). Hashing is done in Lucene.Net.Fst.NodeHash<T>, that'll be a
> good starting point. Also see
>
> https://github.com/apache/lucenenet/commit/ea7cfe6972972f91090d995c729689fff9a0f54c
>
> 4. There seem to be some concurrency issues with file handling, probably on
> the part that is in charge of acquiring and releasing locks. In particular
> see TestDocValuesFormat,
> .TestShardSearching.TestSimple(),
> TestBufferedIndexInput.TestSetBufferSize(),
> TestLockFactory.TestStressLocks(),
> TestLockFactory.TestStressLocksNativeFSLockFactory()
>
> 5. RegexpQuery seems to be broken, I believe the underlying automaton
> implementation doesn't emit the terms correctly. See the failures in
> TestRegexpQuery and TestRegexpRandom.
>
> 6. Similar but probably unrelated failing test is
> at TestSpanMultiTermQueryWrapper.TestFuzzy2()
>
> 7. An interesting failure to investigate is here:
> TestIndexableBinaryStringTools.TestEncodedSortability() -- probably a
> simple porting fail, a subtle Java <--> C# mismatch that we haven't
> noticed.
>
> Additionally, any help with porting the subprojects would be great. Some
> folks took some subprojects under their wings and never reported back. If
> you could either complete the work, or donate whatever you have done so
> far, that would be great.
>
> As always, get the code from here: https://github.com/apache/lucenenet --
> work on your fork and send us a PR when ready, with a good description of
> what you've done.
>
> To see the list of currently failing tests in real-time, go here (you can
> login as guest):
>
> http://teamcity.codebetter.com/viewType.html?buildTypeId=LuceneNet_Core&branch_LuceneNet=%3Cdefault%3E&tab=buildTypeStatusDiv
>
> Thanks, and happy searching,
>
> --
>
> Itamar Syn-Hershko
> http://code972.com | @synhershko <https://twitter.com/synhershko>
> Freelance Developer & Consultant
> Lucene.NET committer and PMC member
>