Nutch 2 - mysql backend error

["hugo.ma" <hu...@gmail.com>]

Hi again.

So now i have a mysterious exception, at least for me.
I have 2 machines, one with windows 7 for testing, in this machine nutch
works just fine.
The other is a small server with windows xp, this is the problematic
machine, both have mysql for gora backend.

Since yesterday nutch started to throw exceptions at injector phase, the
exception is  related to mysql connector, ate least is what i understand
from hadoop log:


2012-09-17 10:56:07,657 INFO  regex.RegexURLNormalizer - can't find rules
for scope 'inject', using default
2012-09-17 10:56:07,688 WARN  mapred.LocalJobRunner - job_local_0001
java.io.IOException: java.sql.BatchUpdateException: Error reading from
InputStream java.io.EOFException
	at org.apache.gora.sql.store.SqlStore.flush(SqlStore.java:340)
	at org.apache.gora.sql.store.SqlStore.close(SqlStore.java:185)
	at
org.apache.gora.mapreduce.GoraRecordWriter.close(GoraRecordWriter.java:55)
	at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(MapTask.java:507)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:623)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
Caused by: java.sql.BatchUpdateException: Error reading from InputStream
java.io.EOFException
	at
com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2054)
	at
com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1467)
	at org.apache.gora.sql.store.SqlStore.flush(SqlStore.java:328)
	... 6 more
Caused by: java.sql.SQLException: Error reading from InputStream
java.io.EOFException
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1074)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:988)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:974)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:919)
	at com.mysql.jdbc.PreparedStatement.readblock(PreparedStatement.java:3097)
	at
com.mysql.jdbc.PreparedStatement.streamToBytes(PreparedStatement.java:5162)
	at
com.mysql.jdbc.PreparedStatement.fillSendPacket(PreparedStatement.java:2592)
	at
com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2432)
	at
com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2006)
	... 8 more
Caused by: java.io.EOFException
	at
org.apache.avro.ipc.ByteBufferInputStream.getBuffer(ByteBufferInputStream.java:84)
	at
org.apache.avro.ipc.ByteBufferInputStream.read(ByteBufferInputStream.java:46)
	at java.io.InputStream.read(InputStream.java:82)
	at com.mysql.jdbc.PreparedStatement.readblock(PreparedStatement.java:3095)
	... 12 more
2012-09-17 10:56:07,907 ERROR crawl.InjectorJob - InjectorJob:
java.lang.RuntimeException: job failed: name=inject
D:/NUTCH_BASE/job_id_5/seeds, jobid=null
	at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54)
	at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:219)
	at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:228)
	at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:248)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
	at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:258)

Anyone had a error like this?



--
View this message in context: http://lucene.472066.n3.nabble.com/Nutch-2-mysql-backend-error-tp4008255.html
Sent from the Nutch - User mailing list archive at Nabble.com.

Re: Nutch 2 - mysql backend error

["hugo.ma" <hu...@gmail.com>]

I've found the solution.

The problem is in the gora-sqlmapping.xml.

My original file was like this:
http://pastebin.com/ZZmQ3Rr9

And i changed to this:
http://pastebin.com/yaME13XZ

Looks like i cant change the outlinks,metadata and markers to LONGBLOB data
type.



--
View this message in context: http://lucene.472066.n3.nabble.com/Nutch-2-mysql-backend-error-tp4008255p4008339.html
Sent from the Nutch - User mailing list archive at Nabble.com.