[jira] [Created] (CASSANDRA-10393) LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref)

["Christian Winther (JIRA)" <ji...@apache.org>]

Christian Winther created CASSANDRA-10393:
---------------------------------------------

             Summary: LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref)
                 Key: CASSANDRA-10393
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10393
             Project: Cassandra
          Issue Type: Bug
         Environment: v 2.2.1 (from apt)

-> lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 7.8 (wheezy)
Release:	7.8
Codename:	wheezy

-> java -version
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

            Reporter: Christian Winther


When trying to repair full on a table with the following schema my nodes stall 
and end up with spamming this 

I've recently changed the table from SizeTieredCompactionStrategy to LeveledCompactionStrategy.

Coming from 2.1.9 -> 2.2.0 -> 2.2.1 i ran upgradesstable without issue as well

When trying to full repair post compaction change, I got "out of order" errors. A few google searches later, I was told to "scrub" the keyspace - did that during the night (no problems logged, and no data lost)

Now a repair just stalls and output memory leaks all over the place 

{code}
CREATE KEYSPACE sessions WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '3'}  AND durable_writes = true;

CREATE TABLE sessions.sessions (
    id text PRIMARY KEY,
    client_ip text,
    controller text,
    controller_action text,
    created timestamp,
    data text,
    expires timestamp,
    http_host text,
    modified timestamp,
    request_agent text,
    request_agent_bot boolean,
    request_path text,
    site_id int,
    user_id int
) WITH bloom_filter_fp_chance = 0.01
    AND caching = '{"keys":"NONE", "rows_per_partition":"NONE"}'
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy'}
    AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99.0PERCENTILE';
{code}


ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@4428a373) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1811114765:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104037-big was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@368dd97) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1811114765:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104037-big was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@66fb78be) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1811114765:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104037-big was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@9fdd2e6) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1460906269:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104788-big was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-09-24 10:25:28,475 Ref.java:187 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@84fcb91) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1460906269:/data/1/cassandra/sessions/sessions-77dd22f0ab9711e49cbc410c6b6f53a6/la-104788-big was not released before the reference was garbage collected




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)