You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Eric Evans (JIRA)" <ji...@apache.org> on 2010/04/05 01:09:27 UTC
[jira] Created: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
ArrayIndexOutOfBoundsException with nested maps
-----------------------------------------------
Key: AVRO-503
URL: https://issues.apache.org/jira/browse/AVRO-503
Project: Avro
Issue Type: Bug
Components: java, python
Affects Versions: 1.3.2
Reporter: Eric Evans
Attachments: cass.tar.gz
I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
Running the java example produces:
{noformat}
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
at $Proxy0.batch_mutate(Unknown Source)
at BatchMutate.main(BatchMutate.java:43)
{noformat}
Running the python examples produces:
{noformat}
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
self.test(*self.arg)
File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
self.client.request('batch_mutate', params)
File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
return self.request(message_name, request_datum)
File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
call_response_exists = self.read_handshake_response(buffer_decoder)
File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
return self.read_data(self.writers_schema, self.readers_schema, decoder)
File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
return self.read_record(writers_schema, readers_schema, decoder)
File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
field_val = self.read_data(field.type, readers_field.type, decoder)
File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
return self.read_union(writers_schema, readers_schema, decoder)
File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
selected_writers_schema = writers_schema.schemas[index_of_schema]
IndexError: list index out of range
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
Posted by "Jeff Hodges (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853307#action_12853307 ]
Jeff Hodges commented on AVRO-503:
----------------------------------
With a small change to the python client (a la AVRO-496), this is reproducible with trunk.
> ArrayIndexOutOfBoundsException with nested maps
> -----------------------------------------------
>
> Key: AVRO-503
> URL: https://issues.apache.org/jira/browse/AVRO-503
> Project: Avro
> Issue Type: Bug
> Components: java, python
> Affects Versions: 1.3.2
> Reporter: Eric Evans
> Attachments: cass.tar.gz
>
>
> I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
> Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
> Running the java example produces:
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
> at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
> at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
> at $Proxy0.batch_mutate(Unknown Source)
> at BatchMutate.main(BatchMutate.java:43)
> {noformat}
> Running the python examples produces:
> {noformat}
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
> self.test(*self.arg)
> File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
> self.client.request('batch_mutate', params)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
> return self.request(message_name, request_datum)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
> call_response_exists = self.read_handshake_response(buffer_decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
> handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
> return self.read_union(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
> selected_writers_schema = writers_schema.schemas[index_of_schema]
> IndexError: list index out of range
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
Posted by "Jeff Hodges (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853313#action_12853313 ]
Jeff Hodges commented on AVRO-503:
----------------------------------
Oh, and to get this up and running I did this:
{quote}
cd $AVRO_PROJECT
wget https://issues.apache.org/jira/secure/attachment/12440724/cass.tar.gz --no-check-certificate
tar xzvf cass.tar.gz
cd cass
mkdir -p build/classes
. clspth
find . -name "*.java" | xargs javac -d build/classes
./start
{quote}
I'm sure there's a quicker way but that worked. Note the sourcing of clspth.
> ArrayIndexOutOfBoundsException with nested maps
> -----------------------------------------------
>
> Key: AVRO-503
> URL: https://issues.apache.org/jira/browse/AVRO-503
> Project: Avro
> Issue Type: Bug
> Components: java, python
> Affects Versions: 1.3.2
> Reporter: Eric Evans
> Attachments: cass.tar.gz
>
>
> I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
> Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
> Running the java example produces:
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
> at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
> at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
> at $Proxy0.batch_mutate(Unknown Source)
> at BatchMutate.main(BatchMutate.java:43)
> {noformat}
> Running the python examples produces:
> {noformat}
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
> self.test(*self.arg)
> File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
> self.client.request('batch_mutate', params)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
> return self.request(message_name, request_datum)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
> call_response_exists = self.read_handshake_response(buffer_decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
> handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
> return self.read_union(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
> selected_writers_schema = writers_schema.schemas[index_of_schema]
> IndexError: list index out of range
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
Posted by "Jeff Hodges (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853318#action_12853318 ]
Jeff Hodges commented on AVRO-503:
----------------------------------
Digging deeper and using the avro jar in trunk, I find this curious value for the IO object in the decoder passed to HANDSHAKE_REQUESTOR_READER.read() in the python side.
{quote}
? org.apache.avro.AvroRuntimeException: Not in union ["string",{"type":"error","name":"InvalidRequestException","namespace":"org.apache.cassandra.avro","fields":[{"name":"why","type":["string","null"]}]},{"type":"error","name":"UnavailableException","fields":[{"name":"why","type":["string","null"]}]},{"type":"error","name":"TimedOutException","fields":[{"name":"why","type":["string","null"]}]}]: org.apache.avro.AvroRuntimeException: java.lang.NoSuchMethodException: org.apache.cassandra.avro.CassandraServer.batch_mutate(org.apache.avro.util.Utf8, java.util.Map, org.apache.cassandra.avro.ConsistencyLevel)
{quote}
Strange, strange.
> ArrayIndexOutOfBoundsException with nested maps
> -----------------------------------------------
>
> Key: AVRO-503
> URL: https://issues.apache.org/jira/browse/AVRO-503
> Project: Avro
> Issue Type: Bug
> Components: java, python
> Affects Versions: 1.3.2
> Reporter: Eric Evans
> Attachments: cass.tar.gz
>
>
> I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
> Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
> Running the java example produces:
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
> at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
> at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
> at $Proxy0.batch_mutate(Unknown Source)
> at BatchMutate.main(BatchMutate.java:43)
> {noformat}
> Running the python examples produces:
> {noformat}
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
> self.test(*self.arg)
> File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
> self.client.request('batch_mutate', params)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
> return self.request(message_name, request_datum)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
> call_response_exists = self.read_handshake_response(buffer_decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
> handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
> return self.read_union(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
> selected_writers_schema = writers_schema.schemas[index_of_schema]
> IndexError: list index out of range
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Evans updated AVRO-503:
----------------------------
Attachment: sample_bugfix.patch
Ok, the attached patch (sample_bugfix.patch) fixes the HTTPTransceiver initialization for the Python example, and uses the correct instance when constructing a SpecificResponder in the Java example.
With this applied the Java test-case (src/BatchMutate.java) now works against the Java test server (CassandraServer), but it still fails against the full implementation in Cassandra (same protocol schema). Here is the exception thrown on the RPC server (not sure why I didn't include that before):
{noformat}
org.apache.avro.AvroRuntimeException: Not in union ["string",{"type":"error","name":"InvalidRequestException","namespace":"org.apache.cassandra.avro","fields":[{"name":"why","type":["string","null"]}]},{"type":"error","name":"UnavailableException","fields":[{"name":"why","type":["string","null"]}]},{"type":"error","name":"TimedOutException","fields":[{"name":"why","type":["string","null"]}]}]: java.lang.NullPointerException
at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:340)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
at org.apache.avro.specific.SpecificResponder.writeError(SpecificResponder.java:81)
at org.apache.avro.ipc.Responder.respond(Responder.java:137)
at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:536)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:930)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:405)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
{noformat}
Jeff, since you've been known to get your Cassandra on, I went ahead and committed the code I had to trunk/ there, in case that helps. :)
> ArrayIndexOutOfBoundsException with nested maps
> -----------------------------------------------
>
> Key: AVRO-503
> URL: https://issues.apache.org/jira/browse/AVRO-503
> Project: Avro
> Issue Type: Bug
> Components: java, python
> Affects Versions: 1.3.2
> Reporter: Eric Evans
> Attachments: cass.tar.gz, sample_bugfix.patch
>
>
> I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
> Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
> Running the java example produces:
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
> at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
> at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
> at $Proxy0.batch_mutate(Unknown Source)
> at BatchMutate.main(BatchMutate.java:43)
> {noformat}
> Running the python examples produces:
> {noformat}
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
> self.test(*self.arg)
> File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
> self.client.request('batch_mutate', params)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
> return self.request(message_name, request_datum)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
> call_response_exists = self.read_handshake_response(buffer_decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
> handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
> return self.read_union(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
> selected_writers_schema = writers_schema.schemas[index_of_schema]
> IndexError: list index out of range
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
Posted by "Jeff Hodges (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853322#action_12853322 ]
Jeff Hodges commented on AVRO-503:
----------------------------------
Bonus for the Java hackers: Is there some generics magic (or even just runtime checks) we can do to prevent this accidental misuse of the SpecificResponder constructor?
> ArrayIndexOutOfBoundsException with nested maps
> -----------------------------------------------
>
> Key: AVRO-503
> URL: https://issues.apache.org/jira/browse/AVRO-503
> Project: Avro
> Issue Type: Bug
> Components: java, python
> Affects Versions: 1.3.2
> Reporter: Eric Evans
> Attachments: cass.tar.gz
>
>
> I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
> Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
> Running the java example produces:
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
> at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
> at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
> at $Proxy0.batch_mutate(Unknown Source)
> at BatchMutate.main(BatchMutate.java:43)
> {noformat}
> Running the python examples produces:
> {noformat}
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
> self.test(*self.arg)
> File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
> self.client.request('batch_mutate', params)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
> return self.request(message_name, request_datum)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
> call_response_exists = self.read_handshake_response(buffer_decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
> handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
> return self.read_union(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
> selected_writers_schema = writers_schema.schemas[index_of_schema]
> IndexError: list index out of range
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
Posted by "Jeff Hodges (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853320#action_12853320 ]
Jeff Hodges commented on AVRO-503:
----------------------------------
Okay! So! Eric!
It looks like your SpecificResponder instantiation was wrong! You're calling it as:
{quote}
SpecificResponder responder = new SpecificResponder(Cassandra.class, new CassandraServer());
{quote}
but CassandraServer doesn't implement the methods you need! You implemented them in CassandraImpl. The call should be:
{quote}
SpecificResponder responder = new SpecificResponder(Cassandra.class, new CassandraImpl());
{quote}
Now, that fixes this problem, but the real problem is that java side isn't handling AvroRuntimeExceptions correctly! In fact, the java side seems to just spitting out the error text for any error raised back on the wire instead of doing something "safe". The strange error you were seeing is because the error text (mentioned in the last comment) is (obviously) an invalid avro response!
So, we need to do a little more work around how we handle errors in the RPC layer. And you have an easy bug to fix.
> ArrayIndexOutOfBoundsException with nested maps
> -----------------------------------------------
>
> Key: AVRO-503
> URL: https://issues.apache.org/jira/browse/AVRO-503
> Project: Avro
> Issue Type: Bug
> Components: java, python
> Affects Versions: 1.3.2
> Reporter: Eric Evans
> Attachments: cass.tar.gz
>
>
> I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
> Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
> Running the java example produces:
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
> at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
> at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
> at $Proxy0.batch_mutate(Unknown Source)
> at BatchMutate.main(BatchMutate.java:43)
> {noformat}
> Running the python examples produces:
> {noformat}
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
> self.test(*self.arg)
> File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
> self.client.request('batch_mutate', params)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
> return self.request(message_name, request_datum)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
> call_response_exists = self.read_handshake_response(buffer_decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
> handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
> return self.read_union(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
> selected_writers_schema = writers_schema.schemas[index_of_schema]
> IndexError: list index out of range
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Evans updated AVRO-503:
----------------------------
Attachment: cass.tar.gz
> ArrayIndexOutOfBoundsException with nested maps
> -----------------------------------------------
>
> Key: AVRO-503
> URL: https://issues.apache.org/jira/browse/AVRO-503
> Project: Avro
> Issue Type: Bug
> Components: java, python
> Affects Versions: 1.3.2
> Reporter: Eric Evans
> Attachments: cass.tar.gz
>
>
> I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
> Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
> Running the java example produces:
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
> at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
> at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
> at $Proxy0.batch_mutate(Unknown Source)
> at BatchMutate.main(BatchMutate.java:43)
> {noformat}
> Running the python examples produces:
> {noformat}
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
> self.test(*self.arg)
> File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
> self.client.request('batch_mutate', params)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
> return self.request(message_name, request_datum)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
> call_response_exists = self.read_handshake_response(buffer_decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
> handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
> return self.read_union(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
> selected_writers_schema = writers_schema.schemas[index_of_schema]
> IndexError: list index out of range
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
Posted by "Jeff Hodges (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853462#action_12853462 ]
Jeff Hodges commented on AVRO-503:
----------------------------------
Totally does. Thanks, Eric. I'll take a look.
> ArrayIndexOutOfBoundsException with nested maps
> -----------------------------------------------
>
> Key: AVRO-503
> URL: https://issues.apache.org/jira/browse/AVRO-503
> Project: Avro
> Issue Type: Bug
> Components: java, python
> Affects Versions: 1.3.2
> Reporter: Eric Evans
> Attachments: cass.tar.gz, sample_bugfix.patch
>
>
> I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
> Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
> Running the java example produces:
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
> at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
> at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
> at $Proxy0.batch_mutate(Unknown Source)
> at BatchMutate.main(BatchMutate.java:43)
> {noformat}
> Running the python examples produces:
> {noformat}
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
> self.test(*self.arg)
> File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
> self.client.request('batch_mutate', params)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
> return self.request(message_name, request_datum)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
> call_response_exists = self.read_handshake_response(buffer_decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
> handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
> return self.read_union(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
> selected_writers_schema = writers_schema.schemas[index_of_schema]
> IndexError: list index out of range
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AVRO-503) ArrayIndexOutOfBoundsException with
nested maps
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853394#action_12853394 ]
Eric Evans commented on AVRO-503:
---------------------------------
bq. but CassandraServer doesn't implement the methods you need! You implemented them in CassandraImpl.
Gah. This was introduced in the process of creating the stand-alone example (that's where "CassandraServer" came from in fact). :/ I also just noticed that the Mutations array was empty (nothing is ever added to it).
> ArrayIndexOutOfBoundsException with nested maps
> -----------------------------------------------
>
> Key: AVRO-503
> URL: https://issues.apache.org/jira/browse/AVRO-503
> Project: Avro
> Issue Type: Bug
> Components: java, python
> Affects Versions: 1.3.2
> Reporter: Eric Evans
> Attachments: cass.tar.gz
>
>
> I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.
> Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.
> Running the java example produces:
> {noformat}
> Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
> at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
> at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
> at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
> at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
> at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
> at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
> at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
> at $Proxy0.batch_mutate(Unknown Source)
> at BatchMutate.main(BatchMutate.java:43)
> {noformat}
> Running the python examples produces:
> {noformat}
> Traceback (most recent call last):
> File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
> self.test(*self.arg)
> File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
> self.client.request('batch_mutate', params)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
> return self.request(message_name, request_datum)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
> call_response_exists = self.read_handshake_response(buffer_decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
> handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
> return self.read_data(self.writers_schema, self.readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
> return self.read_record(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
> field_val = self.read_data(field.type, readers_field.type, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
> return self.read_union(writers_schema, readers_schema, decoder)
> File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
> selected_writers_schema = writers_schema.schemas[index_of_schema]
> IndexError: list index out of range
> {noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.