You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Abdelkrim Fitouri <ab...@gmail.com> on 2017/11/15 20:33:05 UTC
Executing a check before replication / manual replication
Hi,
I know that cassandra handel properly data replication between cluster
nodes, but for some security reasons I am wonderning how to avoid data
replication after a server node have been compromised and someone is
executing modification via cqlsh ?
is there a posibility on Cassandra to execute a custom check / Hook before
replication ?
is there a posibilty to execute a manual replication between node ?
>
--
Best Regards.
*Abdelkarim FITOURI*
System And Security Engineer
Re: Executing a check before replication / manual replication
Posted by Subroto Barua <sb...@yahoo.com.INVALID>.
turn on audit on tables in question, scan the audit logs (using tools like Splunk) and send alerts based on the activity...
On Wednesday, November 15, 2017, 12:33:30 PM PST, Abdelkrim Fitouri <ab...@gmail.com> wrote:
Hi,
I know that cassandra handel properly data replication between cluster nodes, but for some security reasons I am wonderning how to avoid data replication after a server node have been compromised and someone is executing modification via cqlsh ?
is there a posibility on Cassandra to execute a custom check / Hook before replication ?
is there a posibilty to execute a manual replication between node ?
--
Best Regards.
Abdelkarim FITOURI
SystemAnd Security Engineer
Re: Executing a check before replication / manual replication
Posted by Jeff Jirsa <jj...@gmail.com>.
Yea there’s a whole lot of stuff here that doesn’t make sense
I’m not sure what the threat model really is, but there’s a lot of moving pieces here, and the place you’re thinking about adding validation isn’t the first place I’d be concerned with (internode tends to be a bigger problem).
Why do you think a compromised server will try to send data elsewhere?
Where is it going to send it?
--
Jeff Jirsa
> On Nov 16, 2017, at 12:40 PM, kurt greaves <ku...@instaclustr.com> wrote:
>
> What's the purpose here? If they have access to cqlsh, they have access to every nodes data, not just the one they are on. An attacker modifying RF would be the least of your worries. If you manage to detect that some node is compromise you should isolate it immediately.
>
> On 16 Nov. 2017 07:33, "Abdelkrim Fitouri" <ab...@gmail.com> wrote:
> Hi,
>
> I know that cassandra handel properly data replication between cluster nodes, but for some security reasons I am wonderning how to avoid data replication after a server node have been compromised and someone is executing modification via cqlsh ?
>
> is there a posibility on Cassandra to execute a custom check / Hook before replication ?
>
> is there a posibilty to execute a manual replication between node ?
>
>
>
> --
> Best Regards.
>
> Abdelkarim FITOURI
>
> System And Security Engineer
>
>
>
Re: Executing a check before replication / manual replication
Posted by Abdelkrim Fitouri <ab...@gmail.com>.
Hi, and thanks for all your replies,
in some ways, i see that my idea is so complicated to implement and is not
very "clean" ...
I will try to use trigger to do that with one account per application with
the good privileges, but in some ways i am trusting all the external
components like OS or other application layers, because like you mentioned
if some one have a root access to one node he can put down all the cluster
integrity.
As i know replication process is called after each write to the DB, i think
it is not a bad idea to plug a public interface at this level, because for
some project and due to some business constraints, a not reachable database
is better than a "corrupted" one ..
Kind regards.
2017-11-17 9:58 GMT+01:00 Jacques-Henri Berthemet <
jacques-henri.berthemet@genesys.com>:
> In the trigger API I mentioned you’ll get the data that is about to get
> inserted, you can decode that data and check that it is compliant to your
> security. If you want to kill the node, just call System.exit() or
> CassandraDaemon.stop(); The thing is that if you have RF=4 with 4 nodes,
> they will all receive the same update and this will kill your whole
> cluster. If instead you throw an exception, you’ll prevent the rogue
> write and your client will get an error.
>
>
>
> As far as I know there is no public interface to plug your code at the
> replication level, and even if there was one, it would only work when you
> have different DCs. One DC would stay with rogue data, and the other one
> would shutdown.
>
>
>
> *--*
>
> *Jacques-Henri Berthemet*
>
>
>
> *From:* Abdelkrim Fitouri [mailto:abdou.abf@gmail.com]
> *Sent:* jeudi 16 novembre 2017 22:33
> *To:* user@cassandra.apache.org
> *Subject:* Re: Executing a check before replication / manual replication
>
>
>
> ok please find bellow an example:
>
> Lets suppose that i have a cassandra cluster of 4 nodes / one DC /
> replication factor = 4, So in this architecture i have on full copy of the
> data on each node.
>
>
>
> Imagine now that one node have been hacked and in some way with full
> access to cqlsh session, if data is changed on that node, data will be
> changed on the three other, am i right ?
>
> imagine now that i am able to know (using cryptographic bases) if one
> column was modified by my API ( => normal way) or not ( => suspicious way),
> and i want to execute this check function just before any replication of a
> keyspace to avoid that all the replica will be affected by that and so a
> rollback will be not easy and the integrity of all the system will be down,
> the check will for example kill the local cassandra service ...
>
> Hope that my question is more clear now.
>
> Many thanks for any help.
>
>
>
>
>
> 2017-11-16 22:01 GMT+01:00 Oliver Ruebenacker <cu...@gmail.com>:
>
>
>
> Hello,
>
> If I understand the OP right, he wants an automated response one node
> displays suspicious activity.
>
> I suppose in that case, one would want the node to be removed from the
> cluster or shut down or both.
>
> Best, Oliver
>
>
>
> On Thu, Nov 16, 2017 at 3:40 PM, kurt greaves <ku...@instaclustr.com>
> wrote:
>
> What's the purpose here? If they have access to cqlsh, they have access to
> every nodes data, not just the one they are on. An attacker modifying RF
> would be the least of your worries. If you manage to detect that some node
> is compromise you should isolate it immediately.
>
>
>
>
>
> On 16 Nov. 2017 07:33, "Abdelkrim Fitouri" <ab...@gmail.com> wrote:
>
> Hi,
>
>
> I know that cassandra handel properly data replication between cluster
> nodes, but for some security reasons I am wonderning how to avoid data
> replication after a server node have been compromised and someone is
> executing modification via cqlsh ?
>
>
> is there a posibility on Cassandra to execute a custom check / Hook
> before replication ?
>
>
> is there a posibilty to execute a manual replication between node ?
>
>
>
>
> --
>
> Best Regards.
>
> *Abdelkarim FITOURI*
>
> System And Security Engineer
>
>
>
>
>
>
>
> --
>
> Oliver Ruebenacker
>
> Senior Software Engineer, Diabetes Portal
> <http://www.type2diabetesgenetics.org/>, Broad Institute
> <http://www.broadinstitute.org/>
>
>
>
>
>
>
>
>
RE: Executing a check before replication / manual replication
Posted by Jacques-Henri Berthemet <ja...@genesys.com>.
In the trigger API I mentioned you’ll get the data that is about to get inserted, you can decode that data and check that it is compliant to your security. If you want to kill the node, just call System.exit() or CassandraDaemon.stop(); The thing is that if you have RF=4 with 4 nodes, they will all receive the same update and this will kill your whole cluster. If instead you throw an exception, you’ll prevent the rogue write and your client will get an error.
As far as I know there is no public interface to plug your code at the replication level, and even if there was one, it would only work when you have different DCs. One DC would stay with rogue data, and the other one would shutdown.
--
Jacques-Henri Berthemet
From: Abdelkrim Fitouri [mailto:abdou.abf@gmail.com]
Sent: jeudi 16 novembre 2017 22:33
To: user@cassandra.apache.org
Subject: Re: Executing a check before replication / manual replication
ok please find bellow an example:
Lets suppose that i have a cassandra cluster of 4 nodes / one DC / replication factor = 4, So in this architecture i have on full copy of the data on each node.
Imagine now that one node have been hacked and in some way with full access to cqlsh session, if data is changed on that node, data will be changed on the three other, am i right ?
imagine now that i am able to know (using cryptographic bases) if one column was modified by my API ( => normal way) or not ( => suspicious way), and i want to execute this check function just before any replication of a keyspace to avoid that all the replica will be affected by that and so a rollback will be not easy and the integrity of all the system will be down, the check will for example kill the local cassandra service ...
Hope that my question is more clear now.
Many thanks for any help.
2017-11-16 22:01 GMT+01:00 Oliver Ruebenacker <cu...@gmail.com>>:
Hello,
If I understand the OP right, he wants an automated response one node displays suspicious activity.
I suppose in that case, one would want the node to be removed from the cluster or shut down or both.
Best, Oliver
On Thu, Nov 16, 2017 at 3:40 PM, kurt greaves <ku...@instaclustr.com>> wrote:
What's the purpose here? If they have access to cqlsh, they have access to every nodes data, not just the one they are on. An attacker modifying RF would be the least of your worries. If you manage to detect that some node is compromise you should isolate it immediately.
On 16 Nov. 2017 07:33, "Abdelkrim Fitouri" <ab...@gmail.com>> wrote:
Hi,
I know that cassandra handel properly data replication between cluster nodes, but for some security reasons I am wonderning how to avoid data replication after a server node have been compromised and someone is executing modification via cqlsh ?
is there a posibility on Cassandra to execute a custom check / Hook before replication ?
is there a posibilty to execute a manual replication between node ?
--
Best Regards.
Abdelkarim FITOURI
System And Security Engineer
--
Oliver Ruebenacker
Senior Software Engineer, Diabetes Portal<http://www.type2diabetesgenetics.org/>, Broad Institute<http://www.broadinstitute.org/>
Re: Executing a check before replication / manual replication
Posted by kurt greaves <ku...@instaclustr.com>.
That sounds like a great way to DoS yourself. While I'm sure it could be
achieved, probably in a pretty messy way, I don't think it's a good idea
and seems to me like way over the top security. Especially because sure,
you might be able to protect against CQL "attacks" via triggers (ugh) - but
if they have CQL access they could just drop the trigger (yes it requires
superuser, but I don't think that's sufficient protection). Anyway, even if
you do manage to achieve some kind of obscure protection in CQL you won't
be able to protect against internode attacks, or better yet, an attacker
just dumping a bunch of crafted SSTables in your data directories. Better
yet, using sstableloader to achieve the same on all nodes. You should focus
on locking down your client machines and nodes, as if they get access to
either I assure you they'll find a way around whatever security solution
you hack into C*.
Re: Executing a check before replication / manual replication
Posted by Abdelkrim Fitouri <ab...@gmail.com>.
ok please find bellow an example:
Lets suppose that i have a cassandra cluster of 4 nodes / one DC /
replication factor = 4, So in this architecture i have on full copy of the
data on each node.
Imagine now that one node have been hacked and in some way with full access
to cqlsh session, if data is changed on that node, data will be changed on
the three other, am i right ?
imagine now that i am able to know (using cryptographic bases) if one
column was modified by my API ( => normal way) or not ( => suspicious way),
and i want to execute this check function just before any replication of a
keyspace to avoid that all the replica will be affected by that and so a
rollback will be not easy and the integrity of all the system will be down,
the check will for example kill the local cassandra service ...
Hope that my question is more clear now.
Many thanks for any help.
2017-11-16 22:01 GMT+01:00 Oliver Ruebenacker <cu...@gmail.com>:
>
> Hello,
>
> If I understand the OP right, he wants an automated response one node
> displays suspicious activity.
>
> I suppose in that case, one would want the node to be removed from the
> cluster or shut down or both.
>
> Best, Oliver
>
> On Thu, Nov 16, 2017 at 3:40 PM, kurt greaves <ku...@instaclustr.com>
> wrote:
>
>> What's the purpose here? If they have access to cqlsh, they have access
>> to every nodes data, not just the one they are on. An attacker modifying RF
>> would be the least of your worries. If you manage to detect that some node
>> is compromise you should isolate it immediately.
>>
>>
>> On 16 Nov. 2017 07:33, "Abdelkrim Fitouri" <ab...@gmail.com> wrote:
>>
>> Hi,
>>
>> I know that cassandra handel properly data replication between cluster
>> nodes, but for some security reasons I am wonderning how to avoid data
>> replication after a server node have been compromised and someone is
>> executing modification via cqlsh ?
>>
>> is there a posibility on Cassandra to execute a custom check / Hook
>> before replication ?
>>
>> is there a posibilty to execute a manual replication between node ?
>>
>>>
>>
>>
>> --
>>
>> Best Regards.
>>
>> *Abdelkarim FITOURI*
>>
>> System And Security Engineer
>>
>>
>>
>>
>
>
> --
> Oliver Ruebenacker
> Senior Software Engineer, Diabetes Portal
> <http://www.type2diabetesgenetics.org/>, Broad Institute
> <http://www.broadinstitute.org/>
>
>
Re: Executing a check before replication / manual replication
Posted by Oliver Ruebenacker <cu...@gmail.com>.
Hello,
If I understand the OP right, he wants an automated response one node
displays suspicious activity.
I suppose in that case, one would want the node to be removed from the
cluster or shut down or both.
Best, Oliver
On Thu, Nov 16, 2017 at 3:40 PM, kurt greaves <ku...@instaclustr.com> wrote:
> What's the purpose here? If they have access to cqlsh, they have access to
> every nodes data, not just the one they are on. An attacker modifying RF
> would be the least of your worries. If you manage to detect that some node
> is compromise you should isolate it immediately.
>
>
> On 16 Nov. 2017 07:33, "Abdelkrim Fitouri" <ab...@gmail.com> wrote:
>
> Hi,
>
> I know that cassandra handel properly data replication between cluster
> nodes, but for some security reasons I am wonderning how to avoid data
> replication after a server node have been compromised and someone is
> executing modification via cqlsh ?
>
> is there a posibility on Cassandra to execute a custom check / Hook
> before replication ?
>
> is there a posibilty to execute a manual replication between node ?
>
>>
>
>
> --
>
> Best Regards.
>
> *Abdelkarim FITOURI*
>
> System And Security Engineer
>
>
>
>
--
Oliver Ruebenacker
Senior Software Engineer, Diabetes Portal
<http://www.type2diabetesgenetics.org/>, Broad Institute
<http://www.broadinstitute.org/>
Re: Executing a check before replication / manual replication
Posted by kurt greaves <ku...@instaclustr.com>.
What's the purpose here? If they have access to cqlsh, they have access to
every nodes data, not just the one they are on. An attacker modifying RF
would be the least of your worries. If you manage to detect that some node
is compromise you should isolate it immediately.
On 16 Nov. 2017 07:33, "Abdelkrim Fitouri" <ab...@gmail.com> wrote:
Hi,
I know that cassandra handel properly data replication between cluster
nodes, but for some security reasons I am wonderning how to avoid data
replication after a server node have been compromised and someone is
executing modification via cqlsh ?
is there a posibility on Cassandra to execute a custom check / Hook before
replication ?
is there a posibilty to execute a manual replication between node ?
>
--
Best Regards.
*Abdelkarim FITOURI*
System And Security Engineer