You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ant.apache.org by Karl Stenerud <ka...@webartjapan.com> on 2004/04/13 06:31:29 UTC
sql task encoding problem using sjis
I'm getting errors when I try to import a mysql dump back into the database using the ant sql task (using ant 1.6.1).
I created the dump with the following command:
karl@polinki:~/projects/post> mysqldump -u root post --add-drop-table >test/src/sql/initdb.sql
The created file itself is indeed an sjis file.
The ant task looks like this:
---------------------------------------------------------------------
<target name="initdb-test" description="initialize the database">
<sql
autocommit="true"
driver="${test.database.driver}"
encoding="SJIS"
onerror="continue"
password="${test.database.password}"
src="${test.sql}/initdb.sql"
url="${test.database.url}"
userid="${test.database.user}">
<classpath refid="test.classpath"/>
</sql>
</target>
---------------------------------------------------------------------
This task worked fine until I added the postal information to the database.
When I run the task now it does the following:
karl@polinki:~/projects/post> ant initdb-test
Buildfile: build.xml
initdb-test:
[sql] Executing file: /home/karl/projects/post/test/src/sql/initdb.sql
[sql] Failed to execute: INSERT INTO PostalInformation VALUES (1393,4801,480135,'(IN/62D^3(B','(I=/B8^]8[OBE2A.3(B','(I68<^-3(B','$BKL3$F;(B','$B<wET749u>>FbD.(B','$B3Q==(B',0,0,0,0,0,0)
[sql] java.sql.SQLException: Syntax error or access violation, message from server: "You have an error in your SQL syntax near ''??',0,0,0,0,0,0)' at line 1"
[sql] Failed to execute: INSERT INTO PostalInformation VALUES (1438,7822,782212,'(IN/62D^3(B','(I3X-38^]GO@A.3(B','(ID3V3(B','$BKL3$F;(B','$B1+N574>BEDD.(B','$BElM=(B',0,0,0,0,0,0)
[sql] java.sql.SQLException: Syntax error or access violation, message from server: "You have an error in your SQL syntax near ''??\',0,0,0,0,0,0)' at line 1"
[sql] Failed to execute: INSERT INTO PostalInformation VALUES (1642,8927,892775,'(IN/62D^3(B','(IK[58^]K[5A.3(B','(I@F2?(B','$BKL3$F;(B','$B9-Hx749-HxD.(B','$B%?%K%$%=(B',0,0,0,0,0,0)
[sql] java.sql.SQLException: Syntax error or access violation, message from server: "You have an error in your SQL syntax near ''????\',0,0,0,0,0,0)' at line 1"
[sql] Failed to execute: INSERT INTO PostalInformation VALUES (3308,2876,287611,'(I2\C9](B','(I2\C8^]1<[A.3(B','(I57@5SC(B','$B4d<j8)(B','$B4d<j740BBeD.(B','$B2-EDI=(B',0,0,0,0,0,0)
[sql] java.sql.SQLException: Syntax error or access violation, message from server: "You have an error in your SQL syntax near ''???\',0,0,0,0,0,0)' at line 1"
[sql] Failed to execute: INSERT INTO PostalInformation VALUES (3384,2943,294454,'(I2\C9](B','(I2;\8^]:[S6^\QW(B','(I5SC(B','$B4d<j8)(B','$BC@Bt740a@nB<(B','$BI=(B',0,0,0,0,0,0)
[sql] java.sql.SQLException: Syntax error or access violation, message from server: "You have an error in your SQL syntax near ''?\',0,0,0,0,0,0)' at line 1"
[sql] Failed to execute: INSERT INTO PostalInformation VALUES (4204,98961,9896152,'(IPT7^9](B','(ILY6\<(B','(IFI6O4(B','$B5\>k8)(B','$B8E@n;T(B','$BFs%N9=(B',0,0,0,0,0,0)
[sql] java.sql.SQLException: Syntax error or access violation, message from server: "You have an error in your SQL syntax near ''???\',0,0,0,0,0,0)' at line 1"
(I hit ^C at this point)
Not all of the entries are failing (only about 300 out of 120000). I had similar problems with command line importing before when mysql defaulted to EUC encoding, but using "mysql --default-character-set=sjis post < initdb.sql" solved the problem, so it appears that ant is ignoring the encoding field...
P.S. using encoding="sjis" (lowercase) doesn't fix it either.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: sql task encoding problem using sjis
Posted by Ivan Ivanov <ra...@yahoo.com>.
> > You can also try with escapeprocessing and/or
> > keepformat attribuites of <sql> task.
>
> What's that?
They are described in Ant Manual[1].
Your problem astonished me since you have some rows
NOT inserted in the database and I had all my values
inserted but with wrong encoding (I am using mainly
the cyrillic encodings windows-1251/CP1251). The last
two things I can recommend you is to
1) remove or replace sjis letters with english letters
and see what happens
2) I often found useful to specify the jdbc connecting
string in this format:
jdbc:mysql://localhost:3306/somedb?useUnicode=true&characterEncoding=windows-1251
or in your case try:
jdbc:mysql://localhost:3306/somedb?useUnicode=true&characterEncoding=sjis
For more information see [2].
Sorry I am not able to give more clear advice.
Ivan
[1]http://ant.apache.org/manual/CoreTasks/sql.html
[2]http://dev.mysql.com/doc/connector/j/en/index.html#id2801179
__________________________________
Do you Yahoo!?
Yahoo! Tax Center - File online by April 15th
http://taxes.yahoo.com/filing.html
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: sql task encoding problem using sjis
Posted by Karl Stenerud <ka...@webartjapan.com>.
2004 4月 14 水曜日 04:06、Ivan Ivanov さんは書きました:
> How does this postal information looks like? I cannot
> read it in my browser. One reason for failing is that
> SJIS might be unknown encoding for JVM. (By the way
> why it is upper case?)
The table looks like this:
CREATE TABLE PostalInformation (
organizationCode int(11) default NULL,
oldPostalCode int(11) default NULL,
postalCode int(11) default NULL,
prefectureYomi varchar(250) default NULL,
cityYomi varchar(250) default NULL,
choYomi varchar(250) default NULL,
prefecture varchar(250) default NULL,
city varchar(250) default NULL,
cho mediumtext,
hasMultiplePostalCodes int(11) default NULL,
unknown int(11) default NULL,
hasChoNumber int(11) default NULL,
hasMultipleAddresses int(11) default NULL,
addressIsUpdated int(11) default NULL,
changeReason int(11) default NULL
) TYPE=MyISAM;
The JVM understands sjis, and we use this encoding for all of our webapps.
SJIS and sjis both work the same for specifying encoding (and I tried them
both just in case =).
>
> You can also try with escapeprocessing and/or
> keepformat attribuites of <sql> task.
What's that?
> It may also fail if you have add a comment with this
> sign: #. MySQL accepts it as a comment, but Ant not
> (this was my case and I simply stripped off lines
> starting with #).
It's a dump from the mysql database, so it uses "--" for comments.
e.g.
--
-- Table structure for table 'PostalInformation'
--
>
> HTH Ivan
> P.S. Just in every case: did you ensure yourself your
> test/src/sql/initdb.sql runs correctly from mysql
> console:
> mysql> source test/src/sql/initdb.sql
Yes, it works fine when I use mysql directly.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org
Re: sql task encoding problem using sjis
Posted by Ivan Ivanov <ra...@yahoo.com>.
Hi Karl, I had some troubles with MySQL and <sql> task
a mounth ago and might help you.
--- Karl Stenerud <ka...@webartjapan.com> wrote:
> I'm getting errors when I try to import a mysql dump
> back into the database using the ant sql task (using
> ant 1.6.1).
>
>
> I created the dump with the following command:
>
> karl@polinki:~/projects/post> mysqldump -u root post
> --add-drop-table >test/src/sql/initdb.sql
>
> The created file itself is indeed an sjis file.
>
>
> The ant task looks like this:
>
---------------------------------------------------------------------
> <target name="initdb-test" description="initialize
> the database">
> <sql
> autocommit="true"
> driver="${test.database.driver}"
> encoding="SJIS"
> onerror="continue"
> password="${test.database.password}"
> src="${test.sql}/initdb.sql"
> url="${test.database.url}"
> userid="${test.database.user}">
> <classpath refid="test.classpath"/>
> </sql>
> </target>
>
---------------------------------------------------------------------
>
> This task worked fine until I added the postal
> information to the database.
>
How does this postal information looks like? I cannot
read it in my browser. One reason for failing is that
SJIS might be unknown encoding for JVM. (By the way
why it is upper case?)
You can also try with escapeprocessing and/or
keepformat attribuites of <sql> task.
It may also fail if you have add a comment with this
sign: #. MySQL accepts it as a comment, but Ant not
(this was my case and I simply stripped off lines
starting with #).
HTH Ivan
P.S. Just in every case: did you ensure yourself your
test/src/sql/initdb.sql runs correctly from mysql
console:
mysql> source test/src/sql/initdb.sql
__________________________________
Do you Yahoo!?
Yahoo! Tax Center - File online by April 15th
http://taxes.yahoo.com/filing.html
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org