You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Rishabh Patel <ri...@gmail.com> on 2015/07/14 16:33:53 UTC

Coverity scan results of Lucene

Hello!

I scanned the Lucene project with Coverity scanner. 444 defects have been
detected.
Please check the attached report on the breakup of the issues. Some of the
issues are false positives.

I would like to volunteer for fixing these defects.

Before I start, could you please tell me whether I should I create a single
JIRA for a kind of issue (e.g. "Concurrent data access" or "Null pointer
exception") or should multiple issues be created according to the module of
the files to be modified?

-- 
Sincerely,
*Rishabh Patel*

Re: Coverity scan results of Lucene

Posted by Mike Drob <md...@apache.org>.
When I click through on "View Defects" in the report, I get a "You are not
authorized to access this page." response.

On Tue, Jul 14, 2015 at 9:44 AM, Dawid Weiss <da...@gmail.com> wrote:

> The 444 defects is an overwhelming number. Most of those automated
> tools detect things that turn to be valid code (upon closer
> inspection). Could you start by listing, say, the first 5 defects that
> actually make sense and are indeed flawed code that should be fixed?
>
> Dawid
>
> On Tue, Jul 14, 2015 at 4:33 PM, Rishabh Patel
> <ri...@gmail.com> wrote:
> > Hello!
> >
> > I scanned the Lucene project with Coverity scanner. 444 defects have been
> > detected.
> > Please check the attached report on the breakup of the issues. Some of
> the
> > issues are false positives.
> >
> > I would like to volunteer for fixing these defects.
> >
> > Before I start, could you please tell me whether I should I create a
> single
> > JIRA for a kind of issue (e.g. "Concurrent data access" or "Null pointer
> > exception") or should multiple issues be created according to the module
> of
> > the files to be modified?
> >
> > --
> > Sincerely,
> > Rishabh Patel
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>

RE: Coverity scan results of Lucene

Posted by "Reitzel, Charles" <Ch...@tiaa-cref.org>.
What about the "High Impact" defects?    If I'm reading the report right, there appear to be only 12 of these.  

Rishabh, could you copy just the High Impact defects to the list?

-----Original Message-----
From: Dawid Weiss [mailto:dawid.weiss@gmail.com] 
Sent: Tuesday, July 14, 2015 11:59 AM
To: dev@lucene.apache.org
Subject: Re: Coverity scan results of Lucene

Yeah, that's exactly what I though. If you look at the code you'll see
that these are false positives. For example:

5) There is a loop, the following comments gives you a clue:

      // Loop until we succeed in calling doBody() without
      // hitting an IOException.

and inside the loop the exc variable is assigned to IOException,
should there be any.

2) There is a refcount check before this assignment (and this is using
atomic variables); it looks like it's harmless. To make it not
complain is not an easy fix; you can't add synchronization there.

Didn't look at other places, but it's definitely not an automated
pick-and-fix task to address those 444 issues....

Dawid


On Tue, Jul 14, 2015 at 5:48 PM, Rishabh Patel
<ri...@gmail.com> wrote:
> org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyWriter.java
>
> Line 498: Volatile not atomically updated. Updating nextID based on a stale
> value. Any intervening update in another thread is overwritten
>
> org/apache/lucene/index/IndexReader.java
>
> Line 249: Unguarded write. missing_lock: Accessing closed without holding
> lock IndexReader.this. Elsewhere,
> "org.apache.lucene.index.IndexReader.closed" is accessed
> withIndexReader.this held 2 out of 3 times.
>
> org/apache/lucene/index/SnapshotDeletionPolicy.java
>
> Line 116: Unguarded read. missing_lock: Accessing indexCommits without
> holding lock SnapshotDeletionPolicy.this. Elsewhere,
> "org.apache.lucene.index.SnapshotDeletionPolicy.indexCommits" is accessed
> withSnapshotDeletionPolicy.this held 4 out of 5 times.
> 'lastCommit' accessed in both synchronized and unsynchronized contexts.
>
> org/apache/lucene/queries/function/valuesource/QueryValueSource.java
>
> Line 76 and 116: Passing null pointer fcontext to createWeight, which
> dereferences it.
>
> org/apache/lucene/index/SegmentInfos.java
>
> Line 687: Throwing null exception exc.
>
>
> On Tue, Jul 14, 2015 at 10:59 AM, Rishabh Patel
> <ri...@gmail.com> wrote:
>>
>> org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyWriter.java:
>>
>>
>> On Tue, Jul 14, 2015 at 10:44 AM, Dawid Weiss <da...@gmail.com>
>> wrote:
>>>
>>> The 444 defects is an overwhelming number. Most of those automated
>>> tools detect things that turn to be valid code (upon closer
>>> inspection). Could you start by listing, say, the first 5 defects that
>>> actually make sense and are indeed flawed code that should be fixed?
>>>
>>> Dawid
>>>
>>> On Tue, Jul 14, 2015 at 4:33 PM, Rishabh Patel
>>> <ri...@gmail.com> wrote:
>>> > Hello!
>>> >
>>> > I scanned the Lucene project with Coverity scanner. 444 defects have
>>> > been
>>> > detected.
>>> > Please check the attached report on the breakup of the issues. Some of
>>> > the
>>> > issues are false positives.
>>> >
>>> > I would like to volunteer for fixing these defects.
>>> >
>>> > Before I start, could you please tell me whether I should I create a
>>> > single
>>> > JIRA for a kind of issue (e.g. "Concurrent data access" or "Null
>>> > pointer
>>> > exception") or should multiple issues be created according to the
>>> > module of
>>> > the files to be modified?
>>> >
>>> > --
>>> > Sincerely,
>>> > Rishabh Patel
>>> >
>>> >
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>>
>>
>> --
>> Sincerely,
>> Rishabh Patel
>>
>>
>
>
>
> --
> Sincerely,
> Rishabh Patel
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


*************************************************************************
This e-mail may contain confidential or privileged information.
If you are not the intended recipient, please notify the sender immediately and then delete it.

TIAA-CREF
*************************************************************************

Re: Coverity scan results of Lucene

Posted by Dawid Weiss <da...@gmail.com>.
Yeah, that's exactly what I though. If you look at the code you'll see
that these are false positives. For example:

5) There is a loop, the following comments gives you a clue:

      // Loop until we succeed in calling doBody() without
      // hitting an IOException.

and inside the loop the exc variable is assigned to IOException,
should there be any.

2) There is a refcount check before this assignment (and this is using
atomic variables); it looks like it's harmless. To make it not
complain is not an easy fix; you can't add synchronization there.

Didn't look at other places, but it's definitely not an automated
pick-and-fix task to address those 444 issues....

Dawid


On Tue, Jul 14, 2015 at 5:48 PM, Rishabh Patel
<ri...@gmail.com> wrote:
> org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyWriter.java
>
> Line 498: Volatile not atomically updated. Updating nextID based on a stale
> value. Any intervening update in another thread is overwritten
>
> org/apache/lucene/index/IndexReader.java
>
> Line 249: Unguarded write. missing_lock: Accessing closed without holding
> lock IndexReader.this. Elsewhere,
> "org.apache.lucene.index.IndexReader.closed" is accessed
> withIndexReader.this held 2 out of 3 times.
>
> org/apache/lucene/index/SnapshotDeletionPolicy.java
>
> Line 116: Unguarded read. missing_lock: Accessing indexCommits without
> holding lock SnapshotDeletionPolicy.this. Elsewhere,
> "org.apache.lucene.index.SnapshotDeletionPolicy.indexCommits" is accessed
> withSnapshotDeletionPolicy.this held 4 out of 5 times.
> 'lastCommit' accessed in both synchronized and unsynchronized contexts.
>
> org/apache/lucene/queries/function/valuesource/QueryValueSource.java
>
> Line 76 and 116: Passing null pointer fcontext to createWeight, which
> dereferences it.
>
> org/apache/lucene/index/SegmentInfos.java
>
> Line 687: Throwing null exception exc.
>
>
> On Tue, Jul 14, 2015 at 10:59 AM, Rishabh Patel
> <ri...@gmail.com> wrote:
>>
>> org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyWriter.java:
>>
>>
>> On Tue, Jul 14, 2015 at 10:44 AM, Dawid Weiss <da...@gmail.com>
>> wrote:
>>>
>>> The 444 defects is an overwhelming number. Most of those automated
>>> tools detect things that turn to be valid code (upon closer
>>> inspection). Could you start by listing, say, the first 5 defects that
>>> actually make sense and are indeed flawed code that should be fixed?
>>>
>>> Dawid
>>>
>>> On Tue, Jul 14, 2015 at 4:33 PM, Rishabh Patel
>>> <ri...@gmail.com> wrote:
>>> > Hello!
>>> >
>>> > I scanned the Lucene project with Coverity scanner. 444 defects have
>>> > been
>>> > detected.
>>> > Please check the attached report on the breakup of the issues. Some of
>>> > the
>>> > issues are false positives.
>>> >
>>> > I would like to volunteer for fixing these defects.
>>> >
>>> > Before I start, could you please tell me whether I should I create a
>>> > single
>>> > JIRA for a kind of issue (e.g. "Concurrent data access" or "Null
>>> > pointer
>>> > exception") or should multiple issues be created according to the
>>> > module of
>>> > the files to be modified?
>>> >
>>> > --
>>> > Sincerely,
>>> > Rishabh Patel
>>> >
>>> >
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>>
>>
>> --
>> Sincerely,
>> Rishabh Patel
>>
>>
>
>
>
> --
> Sincerely,
> Rishabh Patel
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Coverity scan results of Lucene

Posted by Rishabh Patel <ri...@gmail.com>.
   1.
   org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyWriter.java
      - Line 498: *Volatile not atomically updated*. Updating *nextID* based
      on a stale value. Any intervening update in another thread is overwritten
   2. org/apache/lucene/index/IndexReader.java
   - Line 249: *Unguarded write*. missing_lock: Accessing *closed* without
      holding lock *IndexReader.this*. Elsewhere,
      "org.apache.lucene.index.IndexReader.closed" is accessed with
      *IndexReader.this* held 2 out of 3 times.
   3. org/apache/lucene/index/SnapshotDeletionPolicy.java
   - Line 116: *Unguarded read*. missing_lock: Accessing *indexCommits* without
      holding lock *SnapshotDeletionPolicy.this*. Elsewhere,
      "org.apache.lucene.index.SnapshotDeletionPolicy.indexCommits" is accessed
      with*SnapshotDeletionPolicy.this* held 4 out of 5 times.
      - '*lastCommit*' accessed in both synchronized and unsynchronized
      contexts.
   4. org/apache/lucene/queries/function/valuesource/QueryValueSource.java
   - Line 76 and 116: Passing *null pointer* *fcontext* to *createWeight*,
      which dereferences it.
   5. org/apache/lucene/index/SegmentInfos.java
   - Line 687: Throwing *null exception* *exc*.


On Tue, Jul 14, 2015 at 10:59 AM, Rishabh Patel <
rishabh.mahendra.patel@gmail.com> wrote:

>
>    1.
>    org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyWriter.java:
>    2.
>
>
> On Tue, Jul 14, 2015 at 10:44 AM, Dawid Weiss <da...@gmail.com>
> wrote:
>
>> The 444 defects is an overwhelming number. Most of those automated
>> tools detect things that turn to be valid code (upon closer
>> inspection). Could you start by listing, say, the first 5 defects that
>> actually make sense and are indeed flawed code that should be fixed?
>>
>> Dawid
>>
>> On Tue, Jul 14, 2015 at 4:33 PM, Rishabh Patel
>> <ri...@gmail.com> wrote:
>> > Hello!
>> >
>> > I scanned the Lucene project with Coverity scanner. 444 defects have
>> been
>> > detected.
>> > Please check the attached report on the breakup of the issues. Some of
>> the
>> > issues are false positives.
>> >
>> > I would like to volunteer for fixing these defects.
>> >
>> > Before I start, could you please tell me whether I should I create a
>> single
>> > JIRA for a kind of issue (e.g. "Concurrent data access" or "Null pointer
>> > exception") or should multiple issues be created according to the
>> module of
>> > the files to be modified?
>> >
>> > --
>> > Sincerely,
>> > Rishabh Patel
>> >
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
>
> --
> Sincerely,
> *Rishabh Patel*
>
>
>


-- 
Sincerely,
*Rishabh Patel*

Re: Coverity scan results of Lucene

Posted by Benson Margulies <bi...@gmail.com>.
On Tue, Jul 14, 2015 at 10:59 AM, Rishabh Patel
<ri...@gmail.com> wrote:
> org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyWriter.java:
>

What does coverity complain of? Why do you, personally, think that the
complaint is legitimate?


>
> On Tue, Jul 14, 2015 at 10:44 AM, Dawid Weiss <da...@gmail.com> wrote:
>>
>> The 444 defects is an overwhelming number. Most of those automated
>> tools detect things that turn to be valid code (upon closer
>> inspection). Could you start by listing, say, the first 5 defects that
>> actually make sense and are indeed flawed code that should be fixed?
>>
>> Dawid
>>
>> On Tue, Jul 14, 2015 at 4:33 PM, Rishabh Patel
>> <ri...@gmail.com> wrote:
>> > Hello!
>> >
>> > I scanned the Lucene project with Coverity scanner. 444 defects have
>> > been
>> > detected.
>> > Please check the attached report on the breakup of the issues. Some of
>> > the
>> > issues are false positives.
>> >
>> > I would like to volunteer for fixing these defects.
>> >
>> > Before I start, could you please tell me whether I should I create a
>> > single
>> > JIRA for a kind of issue (e.g. "Concurrent data access" or "Null pointer
>> > exception") or should multiple issues be created according to the module
>> > of
>> > the files to be modified?
>> >
>> > --
>> > Sincerely,
>> > Rishabh Patel
>> >
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> > For additional commands, e-mail: dev-help@lucene.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
>
>
> --
> Sincerely,
> Rishabh Patel
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org

Re: Coverity scan results of Lucene

Posted by Rishabh Patel <ri...@gmail.com>.
   1.
   org/apache/lucene/facet/taxonomy/directory/DirectoryTaxonomyWriter.java:
   2.


On Tue, Jul 14, 2015 at 10:44 AM, Dawid Weiss <da...@gmail.com> wrote:

> The 444 defects is an overwhelming number. Most of those automated
> tools detect things that turn to be valid code (upon closer
> inspection). Could you start by listing, say, the first 5 defects that
> actually make sense and are indeed flawed code that should be fixed?
>
> Dawid
>
> On Tue, Jul 14, 2015 at 4:33 PM, Rishabh Patel
> <ri...@gmail.com> wrote:
> > Hello!
> >
> > I scanned the Lucene project with Coverity scanner. 444 defects have been
> > detected.
> > Please check the attached report on the breakup of the issues. Some of
> the
> > issues are false positives.
> >
> > I would like to volunteer for fixing these defects.
> >
> > Before I start, could you please tell me whether I should I create a
> single
> > JIRA for a kind of issue (e.g. "Concurrent data access" or "Null pointer
> > exception") or should multiple issues be created according to the module
> of
> > the files to be modified?
> >
> > --
> > Sincerely,
> > Rishabh Patel
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> > For additional commands, e-mail: dev-help@lucene.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>


-- 
Sincerely,
*Rishabh Patel*

Re: Coverity scan results of Lucene

Posted by Dawid Weiss <da...@gmail.com>.
The 444 defects is an overwhelming number. Most of those automated
tools detect things that turn to be valid code (upon closer
inspection). Could you start by listing, say, the first 5 defects that
actually make sense and are indeed flawed code that should be fixed?

Dawid

On Tue, Jul 14, 2015 at 4:33 PM, Rishabh Patel
<ri...@gmail.com> wrote:
> Hello!
>
> I scanned the Lucene project with Coverity scanner. 444 defects have been
> detected.
> Please check the attached report on the breakup of the issues. Some of the
> issues are false positives.
>
> I would like to volunteer for fixing these defects.
>
> Before I start, could you please tell me whether I should I create a single
> JIRA for a kind of issue (e.g. "Concurrent data access" or "Null pointer
> exception") or should multiple issues be created according to the module of
> the files to be modified?
>
> --
> Sincerely,
> Rishabh Patel
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org