You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Sylvain Lebresne (JIRA)" <ji...@apache.org> on 2012/09/19 10:31:07 UTC
[jira] [Created] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Sylvain Lebresne created CASSANDRA-4684:
-------------------------------------------
Summary: Binary protocol: inform clients of schema changes
Key: CASSANDRA-4684
URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
Project: Cassandra
Issue Type: Improvement
Affects Versions: 1.2.0 beta 1
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 1.2.0 beta 2
It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
# return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
# add events notification on schema change.
Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470072#comment-13470072 ]
Sylvain Lebresne commented on CASSANDRA-4684:
---------------------------------------------
All I'm saying is that there is a lot of case where a client needs to access the schema. And for some tools, like an eclipse plugin for instance, they would need to access the schema all the time (to offer say completion or validation).
Now you don't need this to access the schema, you query the system table. However, if you do need to access the schema often, how do you do that? Well, either you query the database every damn time, and your plugin/tool/code will be super slow. Or, more likely, you cache the schema client side and implement some regular polling to refresh that cache. Which works, mostly, but has the defaults of polling: should you poll often or not? If you poll too often it's inefficient, if you poll not often enough you'll provide a bad user experience. Don't get me wrong, I'm not pretending that this is the worst problem database face today, but that is not far fetched either and all this ticket does is to provide a better solution to that problem. So why wouldn't we give people a better solution if we can? And we can very easily. It's not like this small patch touch any sensible part of the code, or somehow makes parts of the code unreadable (at least to me it seems like a very minor addition).
I also note that the patch don't even impose anything on clients implementors since events are optional (and optional by type of events).
As for libpq, as I said, this patch is an optimization so the fact that libpq don't support it is not a proof that it's useless either. And as a side note, and while I'm not expert in libpq, it has an aynchronous notification mechanism and I wouldn't be surprised that along with some simple trigger you can very easily have schema change notifications (does the eclipse plugin uses that if that does work? I don't know and frankly I don't care).
> Binary protocol: inform clients of schema changes
> -------------------------------------------------
>
> Key: CASSANDRA-4684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.2.0 beta 1
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-Return-schema-change-infos.txt, 0002-Add-migration-events.txt
>
>
> It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
> # return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
> # add events notification on schema change.
> Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467847#comment-13467847 ]
Jonathan Ellis commented on CASSANDRA-4684:
-------------------------------------------
TBH I'd rather hold off on this until we have a concrete use case for it. The one place this would be useful historically is ASSUME which we don't need anymore in CQL3.
> Binary protocol: inform clients of schema changes
> -------------------------------------------------
>
> Key: CASSANDRA-4684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.2.0 beta 1
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-Return-schema-change-infos.txt, 0002-Add-migration-events.txt
>
>
> It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
> # return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
> # add events notification on schema change.
> Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470487#comment-13470487 ]
Jonathan Ellis commented on CASSANDRA-4684:
-------------------------------------------
All right, +1
> Binary protocol: inform clients of schema changes
> -------------------------------------------------
>
> Key: CASSANDRA-4684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.2.0 beta 1
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-Return-schema-change-infos.txt, 0002-Add-migration-events.txt
>
>
> It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
> # return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
> # add events notification on schema change.
> Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-4684:
----------------------------------------
Attachment: 0002-Add-migration-events.txt
0001-Return-schema-change-infos.txt
Attaching 2 patches that adds the 2 "features" described above. Note that this only affect the binary protocol, and it's relatively minor changes, so I think it's probably fine to push in beta2.
> Binary protocol: inform clients of schema changes
> -------------------------------------------------
>
> Key: CASSANDRA-4684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.2.0 beta 1
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-Return-schema-change-infos.txt, 0002-Add-migration-events.txt
>
>
> It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
> # return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
> # add events notification on schema change.
> Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467872#comment-13467872 ]
Sylvain Lebresne commented on CASSANDRA-4684:
---------------------------------------------
I'll add that it's not a very disruptive email, so unless we're convince this is useless (as said above, I'm convinced of the contrary), I don't think we're taking much risk at supporting it.
> Binary protocol: inform clients of schema changes
> -------------------------------------------------
>
> Key: CASSANDRA-4684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.2.0 beta 1
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-Return-schema-change-infos.txt, 0002-Add-migration-events.txt
>
>
> It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
> # return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
> # add events notification on schema change.
> Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469710#comment-13469710 ]
Jonathan Ellis commented on CASSANDRA-4684:
-------------------------------------------
I'm 99% sure that libpq doesn't "push" schema changes back to clients so I'm skeptical that eclipse tools etc are proof that we need it.
> Binary protocol: inform clients of schema changes
> -------------------------------------------------
>
> Key: CASSANDRA-4684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.2.0 beta 1
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-Return-schema-change-infos.txt, 0002-Add-migration-events.txt
>
>
> It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
> # return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
> # add events notification on schema change.
> Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470399#comment-13470399 ]
Sylvain Lebresne commented on CASSANDRA-4684:
---------------------------------------------
But it's very very little complexification. I mean, it's about zero added complexity client side unless they want to care about optimizing that use case (and they are willing to, why limit them?). And server side, it adds a few lines of code, sure, but I hardly see how it "complexify" things.
bq. so let's put "push" in the YAGNI column until demonstrated otherwise
My problem with that is that this is that since there is an inferior workaround (polling or taking the hit of query the schema each time), it's unclear we'll hear from client that would benefit from that. Or we may only hear about that when lots of people have implemented that less then inferior workaround. I agree that we shouldn't add feature just because we can, but I really think that's not a case of that. Having to access the schema regularly client-side feels hardly like a crazy thing people would need. Why not optimize it if it cost us nothing (and again, I do think it cost us about nothing).
> Binary protocol: inform clients of schema changes
> -------------------------------------------------
>
> Key: CASSANDRA-4684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.2.0 beta 1
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-Return-schema-change-infos.txt, 0002-Add-migration-events.txt
>
>
> It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
> # return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
> # add events notification on schema change.
> Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467866#comment-13467866 ]
Sylvain Lebresne commented on CASSANDRA-4684:
---------------------------------------------
The use case is for client that want to maintain a view of the schema up to date. After all, with CQL3 we're going into a more typed world, so it make sense that high level client would need to know about those types. For instance, you cannot form a query properly unless you know the schema. And in 1.2, creating tables dynamically will be a fully supported use case, so in that case you will likely need to access the current schema programatically. But even without that, I'm pretty sure that any advanced tooling like say a good eclipse plugin will need to keep an up to date view of the schema.
Without this patch, said client will have to either query the schema each and every time they need it, or implement some form of ad-hoc polling. But events are here exactly to avoid having to do polling (and clients will already have support for events, so it will much easier for them to have this).
> Binary protocol: inform clients of schema changes
> -------------------------------------------------
>
> Key: CASSANDRA-4684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.2.0 beta 1
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-Return-schema-change-infos.txt, 0002-Add-migration-events.txt
>
>
> It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
> # return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
> # add events notification on schema change.
> Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4684) Binary protocol: inform clients
of schema changes
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470374#comment-13470374 ]
Jonathan Ellis commented on CASSANDRA-4684:
-------------------------------------------
I still think this is premature complexification. As mentioned, our only real prior art for schema caching is cli because it needs to for ASSUME; otherwise query-schema-every-time is completely fast enough. Of course any such tool will have to implement query-full-schema anyway to get initial state, so that's the simplest possible solution. My gut is that this is probably adequate so let's put "push" in the YAGNI column until demonstrated otherwise.
> Binary protocol: inform clients of schema changes
> -------------------------------------------------
>
> Key: CASSANDRA-4684
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4684
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.2.0 beta 1
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-Return-schema-change-infos.txt, 0002-Add-migration-events.txt
>
>
> It would be nice to inform clients when a schema change occurs as this would allow said client to maintain the current state of the schema, which might be useful/desirable. To allow that, we can:
> # return that a query has changed the schema (instead of simply a 'void' return), in the same spirit than CASSANDRA-3707.
> # add events notification on schema change.
> Just to be clear, the goal is only to inform that a change has occured, the client would still have to query the system table to know the exact content of the change, but at least it'll know when to do such query.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira