You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "T Jake Luciani (JIRA)" <ji...@apache.org> on 2009/11/08 22:21:32 UTC
[jira] Created: (CASSANDRA-535) Support remote clients via Java API
Support remote clients via Java API
-----------------------------------
Key: CASSANDRA-535
URL: https://issues.apache.org/jira/browse/CASSANDRA-535
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: T Jake Luciani
Cassandra should support Java based clients without requiring the overhead of thrift serialization.
To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-535) Support remote clients via Java API
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-535:
-------------------------------------
Fix Version/s: 0.9
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Fix For: 0.9
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-535) Support remote clients via Java API
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Dusbabek updated CASSANDRA-535:
------------------------------------
Attachment: 535-v1.diff
Here's a first stab. I tested using both random and ordered partitioning. commit log and data directories still get created, although nothing ever gets put there.
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 535-v1.diff
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-535) Support remote clients via Java
API
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784619#action_12784619 ]
Jonathan Ellis commented on CASSANDRA-535:
------------------------------------------
04
update bmt_example to use new client mode.
03
main method for testing only. optional.
02
improve testing of StorageService init, and fix shutdown of messagingservice.
patch by gdusbabek; reviewed by jbellis for CASSANDRA-535
01
add StorageService.initClient, which starts up Gossiper without setting a token or other server-specific tasks
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 0001-CASSANDRA-535.txt, 0002-improve-testing-of-StorageService-init-and-fix-shutdow.txt, 0003-main-method-for-testing-only.-do-not-commit.txt, 0004-update-bmt_example-to-use-new-client-mode.txt, 535-v1.diff, 535-v2-reentrant-fixes-and-tests.patch, 535-v2.patch
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-535) Support remote clients via Java
API
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783817#action_12783817 ]
Jonathan Ellis commented on CASSANDRA-535:
------------------------------------------
(it is fine to make the contrib example a separate ticket imo)
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 535-v1.diff, 535-v2-reentrant-fixes-and-tests.patch, 535-v2.patch
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-535) Support remote clients via Java
API
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783815#action_12783815 ]
Jonathan Ellis commented on CASSANDRA-535:
------------------------------------------
first patch: looking much better. i think we can simplify still further by not calling initMetadata at all for client mode. (the only reason would be to get a Generation, and we can always just use system.currentTimeMillis for that; generation is for determining what app state takes precedence and client mode won't use app state.)
adding an assert to gossip app state creation making sure we are not in client mode would be good.
the main() method should be moved to contrib as an example, and commented-out code cleaned out.
second patch: why create a new Timer on gossip stop? there is a Timer.stop method, so i must be missing something here.
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 535-v1.diff, 535-v2-reentrant-fixes-and-tests.patch, 535-v2.patch
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-535) Support remote clients via Java
API
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786383#action_12786383 ]
Hudson commented on CASSANDRA-535:
----------------------------------
Integrated in Cassandra #278 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/278/])
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 0001-CASSANDRA-535.txt, 0002-improve-testing-of-StorageService-init-and-fix-shutdow.txt, 0003-main-method-for-testing-only.-do-not-commit.txt, 0004-update-bmt_example-to-use-new-client-mode.txt, 535-v1.diff, 535-v2-reentrant-fixes-and-tests.patch, 535-v2.patch
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-535) Support remote clients via Java
API
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784947#action_12784947 ]
Jonathan Ellis commented on CASSANDRA-535:
------------------------------------------
ticket for moving 03 into contrib is CASSANDRA-596
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 0001-CASSANDRA-535.txt, 0002-improve-testing-of-StorageService-init-and-fix-shutdow.txt, 0003-main-method-for-testing-only.-do-not-commit.txt, 0004-update-bmt_example-to-use-new-client-mode.txt, 535-v1.diff, 535-v2-reentrant-fixes-and-tests.patch, 535-v2.patch
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-535) Support remote clients via Java API
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-535:
-------------------------------------
Attachment: 0004-update-bmt_example-to-use-new-client-mode.txt
0003-main-method-for-testing-only.-do-not-commit.txt
0002-improve-testing-of-StorageService-init-and-fix-shutdow.txt
0001-CASSANDRA-535.txt
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 0001-CASSANDRA-535.txt, 0002-improve-testing-of-StorageService-init-and-fix-shutdow.txt, 0003-main-method-for-testing-only.-do-not-commit.txt, 0004-update-bmt_example-to-use-new-client-mode.txt, 535-v1.diff, 535-v2-reentrant-fixes-and-tests.patch, 535-v2.patch
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CASSANDRA-535) Support remote clients via Java
API
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-535:
----------------------------------------
Assignee: Gary Dusbabek
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-535) Support remote clients via Java
API
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12784960#action_12784960 ]
Jonathan Ellis commented on CASSANDRA-535:
------------------------------------------
>From IRC:
> gdusbabek: Looks good
committed 01, 02, 04.
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 0001-CASSANDRA-535.txt, 0002-improve-testing-of-StorageService-init-and-fix-shutdow.txt, 0003-main-method-for-testing-only.-do-not-commit.txt, 0004-update-bmt_example-to-use-new-client-mode.txt, 535-v1.diff, 535-v2-reentrant-fixes-and-tests.patch, 535-v2.patch
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-535) Support remote clients via Java
API
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782604#action_12782604 ]
Jonathan Ellis commented on CASSANDRA-535:
------------------------------------------
This feels like a hack to me:
void apply(RowMutation mutation, DataOutputBuffer serializedMutation) throws IOException
{
+ // bail if we're in client-only mode. No need to write to anything, including the system table.
+ if (clientOnlyMode)
+ return;
we need to refactor things better so that we're not trying to apply mutations in client mode. having a storageservice.initClient for instance.
feels like we should not need a mode flag in the config file either -- if we need a flag in SS, initClient can set it. then move directory creation stuff out of DD, e.g. to SS.initServer.
clients should not have tokens associated with them, and should not participate in the ring at all at that level. (that is, we should start gossiper so we get updates, but never send any applicationstate of our own.)
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 535-v1.diff
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-535) Support remote clients via Java API
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Dusbabek updated CASSANDRA-535:
------------------------------------
Attachment: 535-v2-reentrant-fixes-and-tests.patch
535-v2.patch
creates SS.initClient, SS.stopClient. Push storage directory create down into Table and ensure that SS doesn't mess with tables in client-only mode.
ignore the testing main() in StorageService.
The second patch includes a method of testing aspects of the first (directory creation). This got a bit invasive as it required making some of the code more reentrant. Regardless, I think the changes to MessagingService are valid even if the tests aren't. MS.shutdown() was leaving the UDP connections open.
> Support remote clients via Java API
> -----------------------------------
>
> Key: CASSANDRA-535
> URL: https://issues.apache.org/jira/browse/CASSANDRA-535
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: T Jake Luciani
> Assignee: Gary Dusbabek
> Fix For: 0.9
>
> Attachments: 535-v1.diff, 535-v2-reentrant-fixes-and-tests.patch, 535-v2.patch
>
>
> Cassandra should support Java based clients without requiring the overhead of thrift serialization.
> To support this, a java client should be able to join the gossip pool but provide no tokens of its own (so it accepts no writes).
> The way this can be done is add a new startup routine to StorageService called startGossip() that would only initialize itself enough to get access to gossip.
> I don't think its quite this simple though since cassandra code assumes there is at least one local token in places.
> Also, all StorageProxy methods used for thrift service should be made public StorageProxy.getKeyRange() is not public.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.