You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "anton (JIRA)" <ji...@apache.org> on 2011/06/23 14:14:47 UTC
[jira] [Created] (LANG-708) StringEscapeUtils.escapeEcmaScript from
lang3 cuts off long unicode string
StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
--------------------------------------------------------------------------
Key: LANG-708
URL: https://issues.apache.org/jira/browse/LANG-708
Project: Commons Lang
Issue Type: Bug
Reporter: anton
Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
the original string ends with:
"geonameId":6544329,"valueCode":""}]
and the produced string ends with:
\"geonameId\":6544329,\"value
So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
CharSequenceTranslator.translate(...){
...
int sz = Character.codePointCount(input, 0, input.length());
for (int i = 0; i < sz; i++) {
// consumed is the number of codepoints consumed
int consumed = translate(input, i, out);
if(consumed == 0) {
out.write( Character.toChars( Character.codePointAt(input, i) ) );
}
...
}
If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
So this is the original string (sorry for big text in message body - I can't see where files could be attached to this bugzilla):
"[{"geonameFeatureClass":"L","values":{"eu":"Mundua","ro":"Pamânt","it":"Globo","ca":"el món","tr":"Yeryüzü","no":"Jorden","hu":"Föld","lv":"Zeme","de":"Welt","el":"Υδρόγειος","fi":"Maa","la":"Terra","fr":"Monde","eo":"Mondo","en":"World","ru":"Земля","es":"el planeta","nl":"Aarde"},"geonameFeatureCode":"AREA","_id":32,"name":"","auto":true,"type":"GEO","geonameId":6295630,"valueCode":""},{"geonameFeatureClass":"L","values":{"ro":"Europa","zh":"欧洲","ca":"Europa","vi":"Châu Âu","tr":"Avrupa","no":"Europa","hu":"Európa","lv":"Eiropa","hi":"यूरोप","lt":"Europa","bs":"Evropa","ga":"an Eoraip","th":"ยุโรป","id":"Eropa","de":"Europa","fi":"Eurooppa","fr":"Europe","sv":"Europa","bg":"Европа","da":"Europa","eu":"Europa","is":"Evrópa","it":"Europa","cy":"Ewrop","ar":"أوروبا","se":"Eurohpá","he":"אירופה","cs":"Evropa","el":"Ευρώπη","nb":"Europa","pl":"Europa","la":"Europa","pt":"Europa","eo":"Eŭropo","en":"Europe","ru":"Европа","es":"Europa","ja":"ヨーロッパ","nl":"Europa"},"geonameFeatureCode":"CONT","_id":33,"name":"","auto":true,"type":"GEO","geonameId":6255148,"valueCode":""},{"geonameFeatureClass":"A","values":{"no":"Spania","nn":"Spania","fy":"Spanje","gd":"An Spàinn","ga":"An Spáinn","oc":"Espanha","arc":"ܐܣܦܢܝܐ","fi":"Espanja","fr":"Espagne","fo":"Spania","udm":"Испания","os":"Испани","he":"ספרד","gn":"Epaña","gl":"España","gv":"Yn Spaainey","pl":"Hiszpania","gu":"સ્પેઇન","lo":"ສະເປນ","ln":"Espania","vi":"Tây Ban Nha","dz":"Spain","pms":"Spagna","lv":"Spānija","lt":"Ispanija","vo":"Spanyän","de":"Spanien","mg":"Espaina","fur":"Spagne","mk":"Шпанија","ml":"സ്പെയ്ന്\u200D","ceb":"Espanya","mi":"Pāniora","uk":"Іспанія","eu":"Espainia","mr":"स्पेन","ug":"ئىسپانىيە","mt":"Spanja","ms":"Sepanyol","ur":"سپین","fa":"اسپانیا","ty":"Paniora","new":"स्पेन","na":"Pain","el":"Ισπανία","nb":"Spania","ne":"स्पेन","vls":"Spanje","eo":"Hispanio","en":"Kingdom of Spain","et":"Hispaania","es":"la Madre Patria","nl":"Spanje","vec":"Spagna","to":"Sepeni","ca":"Espanya","tl":"Espanya","tr":"İspanya","tg":"Испониё","haw":"Sepania","bs":"Španija","br":"Spagn","th":"ประเทศสเปน","bn":"স্পেন","bo":"སི་པན།","ta":"ஸ்பெயின்","sv":"Spanien","bg":"Испания","ka":"ესპანეთი","st":"Spain","sw":"Hispania","be":"Іспанія","kw":"Spayn","sl":"Španija","sk":"Španielsko","da":"Spanien","ang":"Spēonland","nds":"Spanien","ks":"Spēna","so":"Isbeyn","ku":"Spanya","sr":"Шпанија","sq":"Spanja","ko":"에스파냐","sc":"Ispagna","cy":"Sbaen","se":"Espánjja","sh":"Španija","cv":"Испани","km":"អេស្ប៉ាញ","cs":"Španělsko","li":"Spanje","co":"Spagna","default":"Spain","jbo":"sangu'e","la":"Hesperia","ru":"Испания","lb":"Spuenien","sco":"Spain","tet":"España","scn":"Spagna","hr":"Španjolska","zh":"西班牙","ro":"Spania","rm":"Spagna","ht":"Espay","hu":"Spanyolország","ast":"España","hi":"स्पेन","hsb":"Španiska","nah":"Caxtillān","war":"Espanya","lad":"Espanya","id":"Spanyol","ia":"Espania","nrm":"Espangne","hy":"Իսպանիա","qu":"Ispaña","ilo":"Espania","az":"İspaniya","is":"Spánn","it":"Spagna","tpi":"Spen","ar":"أسبانيا","io":"Hispania","pam":"Espanya","frp":"Èspagne","am":"እስፓንያ","an":"España","csb":"Szpańskô","pt":"Espanha","ja":"スペイン","ps":"اسپانيا","yi":"שפאניע","af":"Spanje"},"geonameFeatureCode":"PCLI","_id":260,"name":"","auto":true,"type":"GEO","geonameId":2510769,"valueCode":""},{"geonameFeatureClass":"A","values":{"ca":"Andalusia","tr":"Endülüs","krc":"Андалусия","no":"Andalucía","fy":"Andalûsje","bs":"Andaluzija","br":"Andalouzia","ext":"Andaluzia","ga":"An Andalúis","th":"แคว้นอันดาลูซีอา","bn":"আন্দালুসিয়া","oc":"Andalosia","ka":"ანდალუსია","sv":"Andalusien","fr":"Andalousie","bg":"Андалусия","glk":"آندالوسیا","be":"Андалусія","kw":"Andalousi","sk":"Andalúzia","os":"Андалуси","da":"Andalusien","sr":"Андалузија","ku":"Endulus","ko":"안달루시아 지방","he":"אנדלוסיה","sh":"Andaluzija","arz":"اندلوسيا","cs":"Andalusie","default":"Andalusia","stq":"Andalusien","la":"Vandalitia","pl":"Andaluzja","ru":"Андалусия","lb":"Andalusien","tet":"Andaluzia","got":"𐍅𐌰𐌽𐌳𐌰𐌻𐌹𐍄𐌾𐌰","hr":"Andaluzija","zh":"安達魯西亞","ro":"Andaluzia","hu":"Andalúzia","pms":"Andalusìa","lv":"Andalūzija","lt":"Andalūzija","nah":"Andalucia","lad":"Andaluziya","de":"Andalusien","als":"Andalusien","qu":"Andalusiya","hy":"Անդալուզիա","eu":"Andaluzia","is":"Andalúsía","uk":"Андалусія","az":"Andalusiya","mr":"आंदालुसिया","ug":"Andalusiye","fa":"اندلس","ar":"أندلوسيا","rmy":"Andalusiya","io":"Andaluzia","el":"Ανδαλουσία","frp":"Andalosie","pt":"Andaluzia","eo":"Andaluzio","en":"Andalusia","et":"Andaluusia","es":"Andalucía","ja":"アンダルシア州","nl":"Andalusië","af":"Andalusië","vec":"Andalusìa"},"geonameFeatureCode":"ADM1","_id":261,"name":"","auto":true,"type":"GEO","geonameId":2593109,"valueCode":""},{"geonameFeatureClass":"A","values":{"de":"Granada","default":"Province of Granada","fr":"Grenade","en":"Province of Granada","es":"Provincia de Granada","ja":"グラナダ"},"geonameFeatureCode":"ADM2","_id":262,"name":"","auto":true,"type":"GEO","geonameId":2517115,"valueCode":""},{"geonameFeatureClass":"A","values":{"default":"Monachil"},"geonameFeatureCode":"ADM3","_id":263,"name":"","auto":true,"type":"GEO","geonameId":6357744,"valueCode":""},{"geonameFeatureClass":"P","values":{"default":"Sierra Nevada"},"geonameFeatureCode":"PPL","_id":264,"name":"","auto":true,"type":"GEO","geonameId":6544329,"valueCode":""}]"
and this is string, produced by StringEscapeUtils.escapeEcmaScript(value):
"[{\"geonameFeatureClass\":\"L\",\"values\":{\"eu\":\"Mundua\",\"ro\":\"Pam\u00E2nt\",\"it\":\"Globo\",\"ca\":\"el m\u00F3n\",\"tr\":\"Yery\u00FCz\u00FC\",\"no\":\"Jorden\",\"hu\":\"F\u00F6ld\",\"lv\":\"Zeme\",\"de\":\"Welt\",\"el\":\"\u03A5\u03B4\u03C1\u03CC\u03B3\u03B5\u03B9\u03BF\u03C2\",\"fi\":\"Maa\",\"la\":\"Terra\",\"fr\":\"Monde\",\"eo\":\"Mondo\",\"en\":\"World\",\"ru\":\"\u0417\u0435\u043C\u043B\u044F\",\"es\":\"el planeta\",\"nl\":\"Aarde\"},\"geonameFeatureCode\":\"AREA\",\"_id\":32,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6295630,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"L\",\"values\":{\"ro\":\"Europa\",\"zh\":\"\u6B27\u6D32\",\"ca\":\"Europa\",\"vi\":\"Ch\u00E2u \u00C2u\",\"tr\":\"Avrupa\",\"no\":\"Europa\",\"hu\":\"Eur\u00F3pa\",\"lv\":\"Eiropa\",\"hi\":\"\u092F\u0942\u0930\u094B\u092A\",\"lt\":\"Europa\",\"bs\":\"Evropa\",\"ga\":\"an Eoraip\",\"th\":\"\u0E22\u0E38\u0E42\u0E23\u0E1B\",\"id\":\"Eropa\",\"de\":\"Europa\",\"fi\":\"Eurooppa\",\"fr\":\"Europe\",\"sv\":\"Europa\",\"bg\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"da\":\"Europa\",\"eu\":\"Europa\",\"is\":\"Evr\u00F3pa\",\"it\":\"Europa\",\"cy\":\"Ewrop\",\"ar\":\"\u0623\u0648\u0631\u0648\u0628\u0627\",\"se\":\"Eurohp\u00E1\",\"he\":\"\u05D0\u05D9\u05E8\u05D5\u05E4\u05D4\",\"cs\":\"Evropa\",\"el\":\"\u0395\u03C5\u03C1\u03CE\u03C0\u03B7\",\"nb\":\"Europa\",\"pl\":\"Europa\",\"la\":\"Europa\",\"pt\":\"Europa\",\"eo\":\"E\u016Dropo\",\"en\":\"Europe\",\"ru\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"es\":\"Europa\",\"ja\":\"\u30E8\u30FC\u30ED\u30C3\u30D1\",\"nl\":\"Europa\"},\"geonameFeatureCode\":\"CONT\",\"_id\":33,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6255148,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"no\":\"Spania\",\"nn\":\"Spania\",\"fy\":\"Spanje\",\"gd\":\"An Sp\u00E0inn\",\"ga\":\"An Sp\u00E1inn\",\"oc\":\"Espanha\",\"arc\":\"\u0710\u0723\u0726\u0722\u071D\u0710\",\"fi\":\"Espanja\",\"fr\":\"Espagne\",\"fo\":\"Spania\",\"udm\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"os\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"he\":\"\u05E1\u05E4\u05E8\u05D3\",\"gn\":\"Epa\u00F1a\",\"gl\":\"Espa\u00F1a\",\"gv\":\"Yn Spaainey\",\"pl\":\"Hiszpania\",\"gu\":\"\u0AB8\u0ACD\u0AAA\u0AC7\u0A87\u0AA8\",\"lo\":\"\u0EAA\u0EB0\u0EC0\u0E9B\u0E99\",\"ln\":\"Espania\",\"vi\":\"T\u00E2y Ban Nha\",\"dz\":\"Spain\",\"pms\":\"Spagna\",\"lv\":\"Sp\u0101nija\",\"lt\":\"Ispanija\",\"vo\":\"Spany\u00E4n\",\"de\":\"Spanien\",\"mg\":\"Espaina\",\"fur\":\"Spagne\",\"mk\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"ml\":\"\u0D38\u0D4D\u0D2A\u0D46\u0D2F\u0D4D\u0D28\u0D4D\\u200D\",\"ceb\":\"Espanya\",\"mi\":\"P\u0101niora\",\"uk\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"eu\":\"Espainia\",\"mr\":\"\u0938\u094D\u092A\u0947\u0928\",\"ug\":\"\u0626\u0649\u0633\u067E\u0627\u0646\u0649\u064A\u06D5\",\"mt\":\"Spanja\",\"ms\":\"Sepanyol\",\"ur\":\"\u0633\u067E\u06CC\u0646\",\"fa\":\"\u0627\u0633\u067E\u0627\u0646\u06CC\u0627\",\"ty\":\"Paniora\",\"new\":\"\u0938\u094D\u092A\u0947\u0928\",\"na\":\"Pain\",\"el\":\"\u0399\u03C3\u03C0\u03B1\u03BD\u03AF\u03B1\",\"nb\":\"Spania\",\"ne\":\"\u0938\u094D\u092A\u0947\u0928\",\"vls\":\"Spanje\",\"eo\":\"Hispanio\",\"en\":\"Kingdom of Spain\",\"et\":\"Hispaania\",\"es\":\"la Madre Patria\",\"nl\":\"Spanje\",\"vec\":\"Spagna\",\"to\":\"Sepeni\",\"ca\":\"Espanya\",\"tl\":\"Espanya\",\"tr\":\"\u0130spanya\",\"tg\":\"\u0418\u0441\u043F\u043E\u043D\u0438\u0451\",\"haw\":\"Sepania\",\"bs\":\"\u0160panija\",\"br\":\"Spagn\",\"th\":\"\u0E1B\u0E23\u0E30\u0E40\u0E17\u0E28\u0E2A\u0E40\u0E1B\u0E19\",\"bn\":\"\u09B8\u09CD\u09AA\u09C7\u09A8\",\"bo\":\"\u0F66\u0F72\u0F0B\u0F54\u0F53\u0F0D\",\"ta\":\"\u0BB8\u0BCD\u0BAA\u0BC6\u0BAF\u0BBF\u0BA9\u0BCD\",\"sv\":\"Spanien\",\"bg\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"ka\":\"\u10D4\u10E1\u10DE\u10D0\u10DC\u10D4\u10D7\u10D8\",\"st\":\"Spain\",\"sw\":\"Hispania\",\"be\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"kw\":\"Spayn\",\"sl\":\"\u0160panija\",\"sk\":\"\u0160panielsko\",\"da\":\"Spanien\",\"ang\":\"Sp\u0113onland\",\"nds\":\"Spanien\",\"ks\":\"Sp\u0113na\",\"so\":\"Isbeyn\",\"ku\":\"Spanya\",\"sr\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"sq\":\"Spanja\",\"ko\":\"\uC5D0\uC2A4\uD30C\uB0D0\",\"sc\":\"Ispagna\",\"cy\":\"Sbaen\",\"se\":\"Esp\u00E1njja\",\"sh\":\"\u0160panija\",\"cv\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"km\":\"\u17A2\u17C1\u179F\u17D2\u1794\u17C9\u17B6\u1789\",\"cs\":\"\u0160pan\u011Blsko\",\"li\":\"Spanje\",\"co\":\"Spagna\",\"default\":\"Spain\",\"jbo\":\"sangu\'e\",\"la\":\"Hesperia\",\"ru\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"lb\":\"Spuenien\",\"sco\":\"Spain\",\"tet\":\"Espa\u00F1a\",\"scn\":\"Spagna\",\"hr\":\"\u0160panjolska\",\"zh\":\"\u897F\u73ED\u7259\",\"ro\":\"Spania\",\"rm\":\"Spagna\",\"ht\":\"Espay\",\"hu\":\"Spanyolorsz\u00E1g\",\"ast\":\"Espa\u00F1a\",\"hi\":\"\u0938\u094D\u092A\u0947\u0928\",\"hsb\":\"\u0160paniska\",\"nah\":\"Caxtill\u0101n\",\"war\":\"Espanya\",\"lad\":\"Espanya\",\"id\":\"Spanyol\",\"ia\":\"Espania\",\"nrm\":\"Espangne\",\"hy\":\"\u053B\u057D\u057A\u0561\u0576\u056B\u0561\",\"qu\":\"Ispa\u00F1a\",\"ilo\":\"Espania\",\"az\":\"\u0130spaniya\",\"is\":\"Sp\u00E1nn\",\"it\":\"Spagna\",\"tpi\":\"Spen\",\"ar\":\"\u0623\u0633\u0628\u0627\u0646\u064A\u0627\",\"io\":\"Hispania\",\"pam\":\"Espanya\",\"frp\":\"\u00C8spagne\",\"am\":\"\u12A5\u1235\u1353\u1295\u12EB\",\"an\":\"Espa\u00F1a\",\"csb\":\"Szpa\u0144sk\u00F4\",\"pt\":\"Espanha\",\"ja\":\"\u30B9\u30DA\u30A4\u30F3\",\"ps\":\"\u0627\u0633\u067E\u0627\u0646\u064A\u0627\",\"yi\":\"\u05E9\u05E4\u05D0\u05E0\u05D9\u05E2\",\"af\":\"Spanje\"},\"geonameFeatureCode\":\"PCLI\",\"_id\":260,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2510769,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"ca\":\"Andalusia\",\"tr\":\"End\u00FCl\u00FCs\",\"krc\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"no\":\"Andaluc\u00EDa\",\"fy\":\"Andal\u00FBsje\",\"bs\":\"Andaluzija\",\"br\":\"Andalouzia\",\"ext\":\"Andaluzia\",\"ga\":\"An Andal\u00FAis\",\"th\":\"\u0E41\u0E04\u0E27\u0E49\u0E19\u0E2D\u0E31\u0E19\u0E14\u0E32\u0E25\u0E39\u0E0B\u0E35\u0E2D\u0E32\",\"bn\":\"\u0986\u09A8\u09CD\u09A6\u09BE\u09B2\u09C1\u09B8\u09BF\u09AF\u09BC\u09BE\",\"oc\":\"Andalosia\",\"ka\":\"\u10D0\u10DC\u10D3\u10D0\u10DA\u10E3\u10E1\u10D8\u10D0\",\"sv\":\"Andalusien\",\"fr\":\"Andalousie\",\"bg\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"glk\":\"\u0622\u0646\u062F\u0627\u0644\u0648\u0633\u06CC\u0627\",\"be\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"kw\":\"Andalousi\",\"sk\":\"Andal\u00FAzia\",\"os\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\",\"da\":\"Andalusien\",\"sr\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0437\u0438\u0458\u0430\",\"ku\":\"Endulus\",\"ko\":\"\uC548\uB2EC\uB8E8\uC2DC\uC544 \uC9C0\uBC29\",\"he\":\"\u05D0\u05E0\u05D3\u05DC\u05D5\u05E1\u05D9\u05D4\",\"sh\":\"Andaluzija\",\"arz\":\"\u0627\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"cs\":\"Andalusie\",\"default\":\"Andalusia\",\"stq\":\"Andalusien\",\"la\":\"Vandalitia\",\"pl\":\"Andaluzja\",\"ru\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"lb\":\"Andalusien\",\"tet\":\"Andaluzia\",\"got\":\"\u10345\u10330\u1033D\u10333\u10330\u1033B\u10339\u10344\u1033E\u10330\",\"hr\":\"Andaluzija\",\"zh\":\"\u5B89\u9054\u9B6F\u897F\u4E9E\",\"ro\":\"Andaluzia\",\"hu\":\"Andal\u00FAzia\",\"pms\":\"Andalus\u00ECa\",\"lv\":\"Andal\u016Bzija\",\"lt\":\"Andal\u016Bzija\",\"nah\":\"Andalucia\",\"lad\":\"Andaluziya\",\"de\":\"Andalusien\",\"als\":\"Andalusien\",\"qu\":\"Andalusiya\",\"hy\":\"\u0531\u0576\u0564\u0561\u056C\u0578\u0582\u0566\u056B\u0561\",\"eu\":\"Andaluzia\",\"is\":\"Andal\u00FAs\u00EDa\",\"uk\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"az\":\"Andalusiya\",\"mr\":\"\u0906\u0902\u0926\u093E\u0932\u0941\u0938\u093F\u092F\u093E\",\"ug\":\"Andalusiye\",\"fa\":\"\u0627\u0646\u062F\u0644\u0633\",\"ar\":\"\u0623\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"rmy\":\"Andalusiya\",\"io\":\"Andaluzia\",\"el\":\"\u0391\u03BD\u03B4\u03B1\u03BB\u03BF\u03C5\u03C3\u03AF\u03B1\",\"frp\":\"Andalosie\",\"pt\":\"Andaluzia\",\"eo\":\"Andaluzio\",\"en\":\"Andalusia\",\"et\":\"Andaluusia\",\"es\":\"Andaluc\u00EDa\",\"ja\":\"\u30A2\u30F3\u30C0\u30EB\u30B7\u30A2\u5DDE\",\"nl\":\"Andalusi\u00EB\",\"af\":\"Andalusi\u00EB\",\"vec\":\"Andalus\u00ECa\"},\"geonameFeatureCode\":\"ADM1\",\"_id\":261,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2593109,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"de\":\"Granada\",\"default\":\"Province of Granada\",\"fr\":\"Grenade\",\"en\":\"Province of Granada\",\"es\":\"Provincia de Granada\",\"ja\":\"\u30B0\u30E9\u30CA\u30C0\"},\"geonameFeatureCode\":\"ADM2\",\"_id\":262,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2517115,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"default\":\"Monachil\"},\"geonameFeatureCode\":\"ADM3\",\"_id\":263,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6357744,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"P\",\"values\":{\"default\":\"Sierra Nevada\"},\"geonameFeatureCode\":\"PPL\",\"_id\":264,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6544329,\"value"
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (LANG-708) StringEscapeUtils.escapeEcmaScript
from lang3 cuts off long unicode string
Posted by "anton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13053827#comment-13053827 ]
anton commented on LANG-708:
----------------------------
It seems that StringEscapeUtils.escapeJavaScript(value) from 2 commons.lang 2.1 does not have this problem with same string.
> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
> Key: LANG-708
> URL: https://issues.apache.org/jira/browse/LANG-708
> Project: Commons Lang
> Issue Type: Bug
> Reporter: anton
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
> the original string ends with:
> "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
> \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
> int sz = Character.codePointCount(input, 0, input.length());
> for (int i = 0; i < sz; i++) {
> // consumed is the number of codepoints consumed
> int consumed = translate(input, i, out);
> if(consumed == 0) {
> out.write( Character.toChars( Character.codePointAt(input, i) ) );
> }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
> So this is the original string (sorry for big text in message body - I can't see where files could be attached to this bugzilla):
> "[{"geonameFeatureClass":"L","values":{"eu":"Mundua","ro":"Pamânt","it":"Globo","ca":"el món","tr":"Yeryüzü","no":"Jorden","hu":"Föld","lv":"Zeme","de":"Welt","el":"Υδρόγειος","fi":"Maa","la":"Terra","fr":"Monde","eo":"Mondo","en":"World","ru":"Земля","es":"el planeta","nl":"Aarde"},"geonameFeatureCode":"AREA","_id":32,"name":"","auto":true,"type":"GEO","geonameId":6295630,"valueCode":""},{"geonameFeatureClass":"L","values":{"ro":"Europa","zh":"欧洲","ca":"Europa","vi":"Châu Âu","tr":"Avrupa","no":"Europa","hu":"Európa","lv":"Eiropa","hi":"यूरोप","lt":"Europa","bs":"Evropa","ga":"an Eoraip","th":"ยุโรป","id":"Eropa","de":"Europa","fi":"Eurooppa","fr":"Europe","sv":"Europa","bg":"Европа","da":"Europa","eu":"Europa","is":"Evrópa","it":"Europa","cy":"Ewrop","ar":"أوروبا","se":"Eurohpá","he":"אירופה","cs":"Evropa","el":"Ευρώπη","nb":"Europa","pl":"Europa","la":"Europa","pt":"Europa","eo":"Eŭropo","en":"Europe","ru":"Европа","es":"Europa","ja":"ヨーロッパ","nl":"Europa"},"geonameFeatureCode":"CONT","_id":33,"name":"","auto":true,"type":"GEO","geonameId":6255148,"valueCode":""},{"geonameFeatureClass":"A","values":{"no":"Spania","nn":"Spania","fy":"Spanje","gd":"An Spàinn","ga":"An Spáinn","oc":"Espanha","arc":"ܐܣܦܢܝܐ","fi":"Espanja","fr":"Espagne","fo":"Spania","udm":"Испания","os":"Испани","he":"ספרד","gn":"Epaña","gl":"España","gv":"Yn Spaainey","pl":"Hiszpania","gu":"સ્પેઇન","lo":"ສະເປນ","ln":"Espania","vi":"Tây Ban Nha","dz":"Spain","pms":"Spagna","lv":"Spānija","lt":"Ispanija","vo":"Spanyän","de":"Spanien","mg":"Espaina","fur":"Spagne","mk":"Шпанија","ml":"സ്പെയ്ന്\u200D","ceb":"Espanya","mi":"Pāniora","uk":"Іспанія","eu":"Espainia","mr":"स्पेन","ug":"ئىسپانىيە","mt":"Spanja","ms":"Sepanyol","ur":"سپین","fa":"اسپانیا","ty":"Paniora","new":"स्पेन","na":"Pain","el":"Ισπανία","nb":"Spania","ne":"स्पेन","vls":"Spanje","eo":"Hispanio","en":"Kingdom of Spain","et":"Hispaania","es":"la Madre Patria","nl":"Spanje","vec":"Spagna","to":"Sepeni","ca":"Espanya","tl":"Espanya","tr":"İspanya","tg":"Испониё","haw":"Sepania","bs":"Španija","br":"Spagn","th":"ประเทศสเปน","bn":"স্পেন","bo":"སི་པན།","ta":"ஸ்பெயின்","sv":"Spanien","bg":"Испания","ka":"ესპანეთი","st":"Spain","sw":"Hispania","be":"Іспанія","kw":"Spayn","sl":"Španija","sk":"Španielsko","da":"Spanien","ang":"Spēonland","nds":"Spanien","ks":"Spēna","so":"Isbeyn","ku":"Spanya","sr":"Шпанија","sq":"Spanja","ko":"에스파냐","sc":"Ispagna","cy":"Sbaen","se":"Espánjja","sh":"Španija","cv":"Испани","km":"អេស្ប៉ាញ","cs":"Španělsko","li":"Spanje","co":"Spagna","default":"Spain","jbo":"sangu'e","la":"Hesperia","ru":"Испания","lb":"Spuenien","sco":"Spain","tet":"España","scn":"Spagna","hr":"Španjolska","zh":"西班牙","ro":"Spania","rm":"Spagna","ht":"Espay","hu":"Spanyolország","ast":"España","hi":"स्पेन","hsb":"Španiska","nah":"Caxtillān","war":"Espanya","lad":"Espanya","id":"Spanyol","ia":"Espania","nrm":"Espangne","hy":"Իսպանիա","qu":"Ispaña","ilo":"Espania","az":"İspaniya","is":"Spánn","it":"Spagna","tpi":"Spen","ar":"أسبانيا","io":"Hispania","pam":"Espanya","frp":"Èspagne","am":"እስፓንያ","an":"España","csb":"Szpańskô","pt":"Espanha","ja":"スペイン","ps":"اسپانيا","yi":"שפאניע","af":"Spanje"},"geonameFeatureCode":"PCLI","_id":260,"name":"","auto":true,"type":"GEO","geonameId":2510769,"valueCode":""},{"geonameFeatureClass":"A","values":{"ca":"Andalusia","tr":"Endülüs","krc":"Андалусия","no":"Andalucía","fy":"Andalûsje","bs":"Andaluzija","br":"Andalouzia","ext":"Andaluzia","ga":"An Andalúis","th":"แคว้นอันดาลูซีอา","bn":"আন্দালুসিয়া","oc":"Andalosia","ka":"ანდალუსია","sv":"Andalusien","fr":"Andalousie","bg":"Андалусия","glk":"آندالوسیا","be":"Андалусія","kw":"Andalousi","sk":"Andalúzia","os":"Андалуси","da":"Andalusien","sr":"Андалузија","ku":"Endulus","ko":"안달루시아 지방","he":"אנדלוסיה","sh":"Andaluzija","arz":"اندلوسيا","cs":"Andalusie","default":"Andalusia","stq":"Andalusien","la":"Vandalitia","pl":"Andaluzja","ru":"Андалусия","lb":"Andalusien","tet":"Andaluzia","got":"𐍅𐌰𐌽𐌳𐌰𐌻𐌹𐍄𐌾𐌰","hr":"Andaluzija","zh":"安達魯西亞","ro":"Andaluzia","hu":"Andalúzia","pms":"Andalusìa","lv":"Andalūzija","lt":"Andalūzija","nah":"Andalucia","lad":"Andaluziya","de":"Andalusien","als":"Andalusien","qu":"Andalusiya","hy":"Անդալուզիա","eu":"Andaluzia","is":"Andalúsía","uk":"Андалусія","az":"Andalusiya","mr":"आंदालुसिया","ug":"Andalusiye","fa":"اندلس","ar":"أندلوسيا","rmy":"Andalusiya","io":"Andaluzia","el":"Ανδαλουσία","frp":"Andalosie","pt":"Andaluzia","eo":"Andaluzio","en":"Andalusia","et":"Andaluusia","es":"Andalucía","ja":"アンダルシア州","nl":"Andalusië","af":"Andalusië","vec":"Andalusìa"},"geonameFeatureCode":"ADM1","_id":261,"name":"","auto":true,"type":"GEO","geonameId":2593109,"valueCode":""},{"geonameFeatureClass":"A","values":{"de":"Granada","default":"Province of Granada","fr":"Grenade","en":"Province of Granada","es":"Provincia de Granada","ja":"グラナダ"},"geonameFeatureCode":"ADM2","_id":262,"name":"","auto":true,"type":"GEO","geonameId":2517115,"valueCode":""},{"geonameFeatureClass":"A","values":{"default":"Monachil"},"geonameFeatureCode":"ADM3","_id":263,"name":"","auto":true,"type":"GEO","geonameId":6357744,"valueCode":""},{"geonameFeatureClass":"P","values":{"default":"Sierra Nevada"},"geonameFeatureCode":"PPL","_id":264,"name":"","auto":true,"type":"GEO","geonameId":6544329,"valueCode":""}]"
> and this is string, produced by StringEscapeUtils.escapeEcmaScript(value):
> "[{\"geonameFeatureClass\":\"L\",\"values\":{\"eu\":\"Mundua\",\"ro\":\"Pam\u00E2nt\",\"it\":\"Globo\",\"ca\":\"el m\u00F3n\",\"tr\":\"Yery\u00FCz\u00FC\",\"no\":\"Jorden\",\"hu\":\"F\u00F6ld\",\"lv\":\"Zeme\",\"de\":\"Welt\",\"el\":\"\u03A5\u03B4\u03C1\u03CC\u03B3\u03B5\u03B9\u03BF\u03C2\",\"fi\":\"Maa\",\"la\":\"Terra\",\"fr\":\"Monde\",\"eo\":\"Mondo\",\"en\":\"World\",\"ru\":\"\u0417\u0435\u043C\u043B\u044F\",\"es\":\"el planeta\",\"nl\":\"Aarde\"},\"geonameFeatureCode\":\"AREA\",\"_id\":32,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6295630,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"L\",\"values\":{\"ro\":\"Europa\",\"zh\":\"\u6B27\u6D32\",\"ca\":\"Europa\",\"vi\":\"Ch\u00E2u \u00C2u\",\"tr\":\"Avrupa\",\"no\":\"Europa\",\"hu\":\"Eur\u00F3pa\",\"lv\":\"Eiropa\",\"hi\":\"\u092F\u0942\u0930\u094B\u092A\",\"lt\":\"Europa\",\"bs\":\"Evropa\",\"ga\":\"an Eoraip\",\"th\":\"\u0E22\u0E38\u0E42\u0E23\u0E1B\",\"id\":\"Eropa\",\"de\":\"Europa\",\"fi\":\"Eurooppa\",\"fr\":\"Europe\",\"sv\":\"Europa\",\"bg\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"da\":\"Europa\",\"eu\":\"Europa\",\"is\":\"Evr\u00F3pa\",\"it\":\"Europa\",\"cy\":\"Ewrop\",\"ar\":\"\u0623\u0648\u0631\u0648\u0628\u0627\",\"se\":\"Eurohp\u00E1\",\"he\":\"\u05D0\u05D9\u05E8\u05D5\u05E4\u05D4\",\"cs\":\"Evropa\",\"el\":\"\u0395\u03C5\u03C1\u03CE\u03C0\u03B7\",\"nb\":\"Europa\",\"pl\":\"Europa\",\"la\":\"Europa\",\"pt\":\"Europa\",\"eo\":\"E\u016Dropo\",\"en\":\"Europe\",\"ru\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"es\":\"Europa\",\"ja\":\"\u30E8\u30FC\u30ED\u30C3\u30D1\",\"nl\":\"Europa\"},\"geonameFeatureCode\":\"CONT\",\"_id\":33,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6255148,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"no\":\"Spania\",\"nn\":\"Spania\",\"fy\":\"Spanje\",\"gd\":\"An Sp\u00E0inn\",\"ga\":\"An Sp\u00E1inn\",\"oc\":\"Espanha\",\"arc\":\"\u0710\u0723\u0726\u0722\u071D\u0710\",\"fi\":\"Espanja\",\"fr\":\"Espagne\",\"fo\":\"Spania\",\"udm\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"os\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"he\":\"\u05E1\u05E4\u05E8\u05D3\",\"gn\":\"Epa\u00F1a\",\"gl\":\"Espa\u00F1a\",\"gv\":\"Yn Spaainey\",\"pl\":\"Hiszpania\",\"gu\":\"\u0AB8\u0ACD\u0AAA\u0AC7\u0A87\u0AA8\",\"lo\":\"\u0EAA\u0EB0\u0EC0\u0E9B\u0E99\",\"ln\":\"Espania\",\"vi\":\"T\u00E2y Ban Nha\",\"dz\":\"Spain\",\"pms\":\"Spagna\",\"lv\":\"Sp\u0101nija\",\"lt\":\"Ispanija\",\"vo\":\"Spany\u00E4n\",\"de\":\"Spanien\",\"mg\":\"Espaina\",\"fur\":\"Spagne\",\"mk\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"ml\":\"\u0D38\u0D4D\u0D2A\u0D46\u0D2F\u0D4D\u0D28\u0D4D\\u200D\",\"ceb\":\"Espanya\",\"mi\":\"P\u0101niora\",\"uk\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"eu\":\"Espainia\",\"mr\":\"\u0938\u094D\u092A\u0947\u0928\",\"ug\":\"\u0626\u0649\u0633\u067E\u0627\u0646\u0649\u064A\u06D5\",\"mt\":\"Spanja\",\"ms\":\"Sepanyol\",\"ur\":\"\u0633\u067E\u06CC\u0646\",\"fa\":\"\u0627\u0633\u067E\u0627\u0646\u06CC\u0627\",\"ty\":\"Paniora\",\"new\":\"\u0938\u094D\u092A\u0947\u0928\",\"na\":\"Pain\",\"el\":\"\u0399\u03C3\u03C0\u03B1\u03BD\u03AF\u03B1\",\"nb\":\"Spania\",\"ne\":\"\u0938\u094D\u092A\u0947\u0928\",\"vls\":\"Spanje\",\"eo\":\"Hispanio\",\"en\":\"Kingdom of Spain\",\"et\":\"Hispaania\",\"es\":\"la Madre Patria\",\"nl\":\"Spanje\",\"vec\":\"Spagna\",\"to\":\"Sepeni\",\"ca\":\"Espanya\",\"tl\":\"Espanya\",\"tr\":\"\u0130spanya\",\"tg\":\"\u0418\u0441\u043F\u043E\u043D\u0438\u0451\",\"haw\":\"Sepania\",\"bs\":\"\u0160panija\",\"br\":\"Spagn\",\"th\":\"\u0E1B\u0E23\u0E30\u0E40\u0E17\u0E28\u0E2A\u0E40\u0E1B\u0E19\",\"bn\":\"\u09B8\u09CD\u09AA\u09C7\u09A8\",\"bo\":\"\u0F66\u0F72\u0F0B\u0F54\u0F53\u0F0D\",\"ta\":\"\u0BB8\u0BCD\u0BAA\u0BC6\u0BAF\u0BBF\u0BA9\u0BCD\",\"sv\":\"Spanien\",\"bg\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"ka\":\"\u10D4\u10E1\u10DE\u10D0\u10DC\u10D4\u10D7\u10D8\",\"st\":\"Spain\",\"sw\":\"Hispania\",\"be\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"kw\":\"Spayn\",\"sl\":\"\u0160panija\",\"sk\":\"\u0160panielsko\",\"da\":\"Spanien\",\"ang\":\"Sp\u0113onland\",\"nds\":\"Spanien\",\"ks\":\"Sp\u0113na\",\"so\":\"Isbeyn\",\"ku\":\"Spanya\",\"sr\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"sq\":\"Spanja\",\"ko\":\"\uC5D0\uC2A4\uD30C\uB0D0\",\"sc\":\"Ispagna\",\"cy\":\"Sbaen\",\"se\":\"Esp\u00E1njja\",\"sh\":\"\u0160panija\",\"cv\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"km\":\"\u17A2\u17C1\u179F\u17D2\u1794\u17C9\u17B6\u1789\",\"cs\":\"\u0160pan\u011Blsko\",\"li\":\"Spanje\",\"co\":\"Spagna\",\"default\":\"Spain\",\"jbo\":\"sangu\'e\",\"la\":\"Hesperia\",\"ru\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"lb\":\"Spuenien\",\"sco\":\"Spain\",\"tet\":\"Espa\u00F1a\",\"scn\":\"Spagna\",\"hr\":\"\u0160panjolska\",\"zh\":\"\u897F\u73ED\u7259\",\"ro\":\"Spania\",\"rm\":\"Spagna\",\"ht\":\"Espay\",\"hu\":\"Spanyolorsz\u00E1g\",\"ast\":\"Espa\u00F1a\",\"hi\":\"\u0938\u094D\u092A\u0947\u0928\",\"hsb\":\"\u0160paniska\",\"nah\":\"Caxtill\u0101n\",\"war\":\"Espanya\",\"lad\":\"Espanya\",\"id\":\"Spanyol\",\"ia\":\"Espania\",\"nrm\":\"Espangne\",\"hy\":\"\u053B\u057D\u057A\u0561\u0576\u056B\u0561\",\"qu\":\"Ispa\u00F1a\",\"ilo\":\"Espania\",\"az\":\"\u0130spaniya\",\"is\":\"Sp\u00E1nn\",\"it\":\"Spagna\",\"tpi\":\"Spen\",\"ar\":\"\u0623\u0633\u0628\u0627\u0646\u064A\u0627\",\"io\":\"Hispania\",\"pam\":\"Espanya\",\"frp\":\"\u00C8spagne\",\"am\":\"\u12A5\u1235\u1353\u1295\u12EB\",\"an\":\"Espa\u00F1a\",\"csb\":\"Szpa\u0144sk\u00F4\",\"pt\":\"Espanha\",\"ja\":\"\u30B9\u30DA\u30A4\u30F3\",\"ps\":\"\u0627\u0633\u067E\u0627\u0646\u064A\u0627\",\"yi\":\"\u05E9\u05E4\u05D0\u05E0\u05D9\u05E2\",\"af\":\"Spanje\"},\"geonameFeatureCode\":\"PCLI\",\"_id\":260,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2510769,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"ca\":\"Andalusia\",\"tr\":\"End\u00FCl\u00FCs\",\"krc\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"no\":\"Andaluc\u00EDa\",\"fy\":\"Andal\u00FBsje\",\"bs\":\"Andaluzija\",\"br\":\"Andalouzia\",\"ext\":\"Andaluzia\",\"ga\":\"An Andal\u00FAis\",\"th\":\"\u0E41\u0E04\u0E27\u0E49\u0E19\u0E2D\u0E31\u0E19\u0E14\u0E32\u0E25\u0E39\u0E0B\u0E35\u0E2D\u0E32\",\"bn\":\"\u0986\u09A8\u09CD\u09A6\u09BE\u09B2\u09C1\u09B8\u09BF\u09AF\u09BC\u09BE\",\"oc\":\"Andalosia\",\"ka\":\"\u10D0\u10DC\u10D3\u10D0\u10DA\u10E3\u10E1\u10D8\u10D0\",\"sv\":\"Andalusien\",\"fr\":\"Andalousie\",\"bg\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"glk\":\"\u0622\u0646\u062F\u0627\u0644\u0648\u0633\u06CC\u0627\",\"be\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"kw\":\"Andalousi\",\"sk\":\"Andal\u00FAzia\",\"os\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\",\"da\":\"Andalusien\",\"sr\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0437\u0438\u0458\u0430\",\"ku\":\"Endulus\",\"ko\":\"\uC548\uB2EC\uB8E8\uC2DC\uC544 \uC9C0\uBC29\",\"he\":\"\u05D0\u05E0\u05D3\u05DC\u05D5\u05E1\u05D9\u05D4\",\"sh\":\"Andaluzija\",\"arz\":\"\u0627\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"cs\":\"Andalusie\",\"default\":\"Andalusia\",\"stq\":\"Andalusien\",\"la\":\"Vandalitia\",\"pl\":\"Andaluzja\",\"ru\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"lb\":\"Andalusien\",\"tet\":\"Andaluzia\",\"got\":\"\u10345\u10330\u1033D\u10333\u10330\u1033B\u10339\u10344\u1033E\u10330\",\"hr\":\"Andaluzija\",\"zh\":\"\u5B89\u9054\u9B6F\u897F\u4E9E\",\"ro\":\"Andaluzia\",\"hu\":\"Andal\u00FAzia\",\"pms\":\"Andalus\u00ECa\",\"lv\":\"Andal\u016Bzija\",\"lt\":\"Andal\u016Bzija\",\"nah\":\"Andalucia\",\"lad\":\"Andaluziya\",\"de\":\"Andalusien\",\"als\":\"Andalusien\",\"qu\":\"Andalusiya\",\"hy\":\"\u0531\u0576\u0564\u0561\u056C\u0578\u0582\u0566\u056B\u0561\",\"eu\":\"Andaluzia\",\"is\":\"Andal\u00FAs\u00EDa\",\"uk\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"az\":\"Andalusiya\",\"mr\":\"\u0906\u0902\u0926\u093E\u0932\u0941\u0938\u093F\u092F\u093E\",\"ug\":\"Andalusiye\",\"fa\":\"\u0627\u0646\u062F\u0644\u0633\",\"ar\":\"\u0623\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"rmy\":\"Andalusiya\",\"io\":\"Andaluzia\",\"el\":\"\u0391\u03BD\u03B4\u03B1\u03BB\u03BF\u03C5\u03C3\u03AF\u03B1\",\"frp\":\"Andalosie\",\"pt\":\"Andaluzia\",\"eo\":\"Andaluzio\",\"en\":\"Andalusia\",\"et\":\"Andaluusia\",\"es\":\"Andaluc\u00EDa\",\"ja\":\"\u30A2\u30F3\u30C0\u30EB\u30B7\u30A2\u5DDE\",\"nl\":\"Andalusi\u00EB\",\"af\":\"Andalusi\u00EB\",\"vec\":\"Andalus\u00ECa\"},\"geonameFeatureCode\":\"ADM1\",\"_id\":261,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2593109,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"de\":\"Granada\",\"default\":\"Province of Granada\",\"fr\":\"Grenade\",\"en\":\"Province of Granada\",\"es\":\"Provincia de Granada\",\"ja\":\"\u30B0\u30E9\u30CA\u30C0\"},\"geonameFeatureCode\":\"ADM2\",\"_id\":262,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2517115,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"default\":\"Monachil\"},\"geonameFeatureCode\":\"ADM3\",\"_id\":263,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6357744,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"P\",\"values\":{\"default\":\"Sierra Nevada\"},\"geonameFeatureCode\":\"PPL\",\"_id\":264,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6544329,\"value"
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (LANG-708) StringEscapeUtils.escapeEcmaScript
from lang3 cuts off long unicode string
Posted by "Matt Benson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Benson resolved LANG-708.
------------------------------
Resolution: Duplicate
With LANG-720 fixed, lang3 trunk no longer cuts off the end of the string.
> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
> Key: LANG-708
> URL: https://issues.apache.org/jira/browse/LANG-708
> Project: Commons Lang
> Issue Type: Bug
> Components: lang.text.translate.*
> Reporter: anton
> Fix For: 3.x
>
> Attachments: Test.java, input.txt
>
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
> the original string ends with:
> "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
> \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
> int sz = Character.codePointCount(input, 0, input.length());
> for (int i = 0; i < sz; i++) {
> // consumed is the number of codepoints consumed
> int consumed = translate(input, i, out);
> if(consumed == 0) {
> out.write( Character.toChars( Character.codePointAt(input, i) ) );
> }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (LANG-708) StringEscapeUtils.escapeEcmaScript from
lang3 cuts off long unicode string
Posted by "Henri Yandell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henri Yandell closed LANG-708.
------------------------------
Resolution: Fixed
Fix Version/s: (was: 3.x)
3.0.1
Setting fix version to 3.0.1.
Setting resolution to Fixed; I think we should treat this as a different bug entry to LANG-720, even if the same code change fixed it. It's a notably different symptom and worth calling out.
> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
> Key: LANG-708
> URL: https://issues.apache.org/jira/browse/LANG-708
> Project: Commons Lang
> Issue Type: Bug
> Components: lang.text.translate.*
> Reporter: anton
> Fix For: 3.0.1
>
> Attachments: Test.java, input.txt
>
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
> the original string ends with:
> "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
> \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
> int sz = Character.codePointCount(input, 0, input.length());
> for (int i = 0; i < sz; i++) {
> // consumed is the number of codepoints consumed
> int consumed = translate(input, i, out);
> if(consumed == 0) {
> out.write( Character.toChars( Character.codePointAt(input, i) ) );
> }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (LANG-708) StringEscapeUtils.escapeEcmaScript
from lang3 cuts off long unicode string
Posted by "Sebb (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13053840#comment-13053840 ]
Sebb commented on LANG-708:
---------------------------
Once you have created a JIRA issue, you can use More Actions / Attach Files to attach files.
Please could you do so with the test data? It's difficult to use as it is.
The issue text could then be editted to remove the inline data.
Thanks very much.
> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
> Key: LANG-708
> URL: https://issues.apache.org/jira/browse/LANG-708
> Project: Commons Lang
> Issue Type: Bug
> Reporter: anton
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
> the original string ends with:
> "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
> \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
> int sz = Character.codePointCount(input, 0, input.length());
> for (int i = 0; i < sz; i++) {
> // consumed is the number of codepoints consumed
> int consumed = translate(input, i, out);
> if(consumed == 0) {
> out.write( Character.toChars( Character.codePointAt(input, i) ) );
> }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
> So this is the original string (sorry for big text in message body - I can't see where files could be attached to this bugzilla):
> "[{"geonameFeatureClass":"L","values":{"eu":"Mundua","ro":"Pamânt","it":"Globo","ca":"el món","tr":"Yeryüzü","no":"Jorden","hu":"Föld","lv":"Zeme","de":"Welt","el":"Υδρόγειος","fi":"Maa","la":"Terra","fr":"Monde","eo":"Mondo","en":"World","ru":"Земля","es":"el planeta","nl":"Aarde"},"geonameFeatureCode":"AREA","_id":32,"name":"","auto":true,"type":"GEO","geonameId":6295630,"valueCode":""},{"geonameFeatureClass":"L","values":{"ro":"Europa","zh":"欧洲","ca":"Europa","vi":"Châu Âu","tr":"Avrupa","no":"Europa","hu":"Európa","lv":"Eiropa","hi":"यूरोप","lt":"Europa","bs":"Evropa","ga":"an Eoraip","th":"ยุโรป","id":"Eropa","de":"Europa","fi":"Eurooppa","fr":"Europe","sv":"Europa","bg":"Европа","da":"Europa","eu":"Europa","is":"Evrópa","it":"Europa","cy":"Ewrop","ar":"أوروبا","se":"Eurohpá","he":"אירופה","cs":"Evropa","el":"Ευρώπη","nb":"Europa","pl":"Europa","la":"Europa","pt":"Europa","eo":"Eŭropo","en":"Europe","ru":"Европа","es":"Europa","ja":"ヨーロッパ","nl":"Europa"},"geonameFeatureCode":"CONT","_id":33,"name":"","auto":true,"type":"GEO","geonameId":6255148,"valueCode":""},{"geonameFeatureClass":"A","values":{"no":"Spania","nn":"Spania","fy":"Spanje","gd":"An Spàinn","ga":"An Spáinn","oc":"Espanha","arc":"ܐܣܦܢܝܐ","fi":"Espanja","fr":"Espagne","fo":"Spania","udm":"Испания","os":"Испани","he":"ספרד","gn":"Epaña","gl":"España","gv":"Yn Spaainey","pl":"Hiszpania","gu":"સ્પેઇન","lo":"ສະເປນ","ln":"Espania","vi":"Tây Ban Nha","dz":"Spain","pms":"Spagna","lv":"Spānija","lt":"Ispanija","vo":"Spanyän","de":"Spanien","mg":"Espaina","fur":"Spagne","mk":"Шпанија","ml":"സ്പെയ്ന്\u200D","ceb":"Espanya","mi":"Pāniora","uk":"Іспанія","eu":"Espainia","mr":"स्पेन","ug":"ئىسپانىيە","mt":"Spanja","ms":"Sepanyol","ur":"سپین","fa":"اسپانیا","ty":"Paniora","new":"स्पेन","na":"Pain","el":"Ισπανία","nb":"Spania","ne":"स्पेन","vls":"Spanje","eo":"Hispanio","en":"Kingdom of Spain","et":"Hispaania","es":"la Madre Patria","nl":"Spanje","vec":"Spagna","to":"Sepeni","ca":"Espanya","tl":"Espanya","tr":"İspanya","tg":"Испониё","haw":"Sepania","bs":"Španija","br":"Spagn","th":"ประเทศสเปน","bn":"স্পেন","bo":"སི་པན།","ta":"ஸ்பெயின்","sv":"Spanien","bg":"Испания","ka":"ესპანეთი","st":"Spain","sw":"Hispania","be":"Іспанія","kw":"Spayn","sl":"Španija","sk":"Španielsko","da":"Spanien","ang":"Spēonland","nds":"Spanien","ks":"Spēna","so":"Isbeyn","ku":"Spanya","sr":"Шпанија","sq":"Spanja","ko":"에스파냐","sc":"Ispagna","cy":"Sbaen","se":"Espánjja","sh":"Španija","cv":"Испани","km":"អេស្ប៉ាញ","cs":"Španělsko","li":"Spanje","co":"Spagna","default":"Spain","jbo":"sangu'e","la":"Hesperia","ru":"Испания","lb":"Spuenien","sco":"Spain","tet":"España","scn":"Spagna","hr":"Španjolska","zh":"西班牙","ro":"Spania","rm":"Spagna","ht":"Espay","hu":"Spanyolország","ast":"España","hi":"स्पेन","hsb":"Španiska","nah":"Caxtillān","war":"Espanya","lad":"Espanya","id":"Spanyol","ia":"Espania","nrm":"Espangne","hy":"Իսպանիա","qu":"Ispaña","ilo":"Espania","az":"İspaniya","is":"Spánn","it":"Spagna","tpi":"Spen","ar":"أسبانيا","io":"Hispania","pam":"Espanya","frp":"Èspagne","am":"እስፓንያ","an":"España","csb":"Szpańskô","pt":"Espanha","ja":"スペイン","ps":"اسپانيا","yi":"שפאניע","af":"Spanje"},"geonameFeatureCode":"PCLI","_id":260,"name":"","auto":true,"type":"GEO","geonameId":2510769,"valueCode":""},{"geonameFeatureClass":"A","values":{"ca":"Andalusia","tr":"Endülüs","krc":"Андалусия","no":"Andalucía","fy":"Andalûsje","bs":"Andaluzija","br":"Andalouzia","ext":"Andaluzia","ga":"An Andalúis","th":"แคว้นอันดาลูซีอา","bn":"আন্দালুসিয়া","oc":"Andalosia","ka":"ანდალუსია","sv":"Andalusien","fr":"Andalousie","bg":"Андалусия","glk":"آندالوسیا","be":"Андалусія","kw":"Andalousi","sk":"Andalúzia","os":"Андалуси","da":"Andalusien","sr":"Андалузија","ku":"Endulus","ko":"안달루시아 지방","he":"אנדלוסיה","sh":"Andaluzija","arz":"اندلوسيا","cs":"Andalusie","default":"Andalusia","stq":"Andalusien","la":"Vandalitia","pl":"Andaluzja","ru":"Андалусия","lb":"Andalusien","tet":"Andaluzia","got":"𐍅𐌰𐌽𐌳𐌰𐌻𐌹𐍄𐌾𐌰","hr":"Andaluzija","zh":"安達魯西亞","ro":"Andaluzia","hu":"Andalúzia","pms":"Andalusìa","lv":"Andalūzija","lt":"Andalūzija","nah":"Andalucia","lad":"Andaluziya","de":"Andalusien","als":"Andalusien","qu":"Andalusiya","hy":"Անդալուզիա","eu":"Andaluzia","is":"Andalúsía","uk":"Андалусія","az":"Andalusiya","mr":"आंदालुसिया","ug":"Andalusiye","fa":"اندلس","ar":"أندلوسيا","rmy":"Andalusiya","io":"Andaluzia","el":"Ανδαλουσία","frp":"Andalosie","pt":"Andaluzia","eo":"Andaluzio","en":"Andalusia","et":"Andaluusia","es":"Andalucía","ja":"アンダルシア州","nl":"Andalusië","af":"Andalusië","vec":"Andalusìa"},"geonameFeatureCode":"ADM1","_id":261,"name":"","auto":true,"type":"GEO","geonameId":2593109,"valueCode":""},{"geonameFeatureClass":"A","values":{"de":"Granada","default":"Province of Granada","fr":"Grenade","en":"Province of Granada","es":"Provincia de Granada","ja":"グラナダ"},"geonameFeatureCode":"ADM2","_id":262,"name":"","auto":true,"type":"GEO","geonameId":2517115,"valueCode":""},{"geonameFeatureClass":"A","values":{"default":"Monachil"},"geonameFeatureCode":"ADM3","_id":263,"name":"","auto":true,"type":"GEO","geonameId":6357744,"valueCode":""},{"geonameFeatureClass":"P","values":{"default":"Sierra Nevada"},"geonameFeatureCode":"PPL","_id":264,"name":"","auto":true,"type":"GEO","geonameId":6544329,"valueCode":""}]"
> and this is string, produced by StringEscapeUtils.escapeEcmaScript(value):
> "[{\"geonameFeatureClass\":\"L\",\"values\":{\"eu\":\"Mundua\",\"ro\":\"Pam\u00E2nt\",\"it\":\"Globo\",\"ca\":\"el m\u00F3n\",\"tr\":\"Yery\u00FCz\u00FC\",\"no\":\"Jorden\",\"hu\":\"F\u00F6ld\",\"lv\":\"Zeme\",\"de\":\"Welt\",\"el\":\"\u03A5\u03B4\u03C1\u03CC\u03B3\u03B5\u03B9\u03BF\u03C2\",\"fi\":\"Maa\",\"la\":\"Terra\",\"fr\":\"Monde\",\"eo\":\"Mondo\",\"en\":\"World\",\"ru\":\"\u0417\u0435\u043C\u043B\u044F\",\"es\":\"el planeta\",\"nl\":\"Aarde\"},\"geonameFeatureCode\":\"AREA\",\"_id\":32,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6295630,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"L\",\"values\":{\"ro\":\"Europa\",\"zh\":\"\u6B27\u6D32\",\"ca\":\"Europa\",\"vi\":\"Ch\u00E2u \u00C2u\",\"tr\":\"Avrupa\",\"no\":\"Europa\",\"hu\":\"Eur\u00F3pa\",\"lv\":\"Eiropa\",\"hi\":\"\u092F\u0942\u0930\u094B\u092A\",\"lt\":\"Europa\",\"bs\":\"Evropa\",\"ga\":\"an Eoraip\",\"th\":\"\u0E22\u0E38\u0E42\u0E23\u0E1B\",\"id\":\"Eropa\",\"de\":\"Europa\",\"fi\":\"Eurooppa\",\"fr\":\"Europe\",\"sv\":\"Europa\",\"bg\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"da\":\"Europa\",\"eu\":\"Europa\",\"is\":\"Evr\u00F3pa\",\"it\":\"Europa\",\"cy\":\"Ewrop\",\"ar\":\"\u0623\u0648\u0631\u0648\u0628\u0627\",\"se\":\"Eurohp\u00E1\",\"he\":\"\u05D0\u05D9\u05E8\u05D5\u05E4\u05D4\",\"cs\":\"Evropa\",\"el\":\"\u0395\u03C5\u03C1\u03CE\u03C0\u03B7\",\"nb\":\"Europa\",\"pl\":\"Europa\",\"la\":\"Europa\",\"pt\":\"Europa\",\"eo\":\"E\u016Dropo\",\"en\":\"Europe\",\"ru\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"es\":\"Europa\",\"ja\":\"\u30E8\u30FC\u30ED\u30C3\u30D1\",\"nl\":\"Europa\"},\"geonameFeatureCode\":\"CONT\",\"_id\":33,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6255148,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"no\":\"Spania\",\"nn\":\"Spania\",\"fy\":\"Spanje\",\"gd\":\"An Sp\u00E0inn\",\"ga\":\"An Sp\u00E1inn\",\"oc\":\"Espanha\",\"arc\":\"\u0710\u0723\u0726\u0722\u071D\u0710\",\"fi\":\"Espanja\",\"fr\":\"Espagne\",\"fo\":\"Spania\",\"udm\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"os\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"he\":\"\u05E1\u05E4\u05E8\u05D3\",\"gn\":\"Epa\u00F1a\",\"gl\":\"Espa\u00F1a\",\"gv\":\"Yn Spaainey\",\"pl\":\"Hiszpania\",\"gu\":\"\u0AB8\u0ACD\u0AAA\u0AC7\u0A87\u0AA8\",\"lo\":\"\u0EAA\u0EB0\u0EC0\u0E9B\u0E99\",\"ln\":\"Espania\",\"vi\":\"T\u00E2y Ban Nha\",\"dz\":\"Spain\",\"pms\":\"Spagna\",\"lv\":\"Sp\u0101nija\",\"lt\":\"Ispanija\",\"vo\":\"Spany\u00E4n\",\"de\":\"Spanien\",\"mg\":\"Espaina\",\"fur\":\"Spagne\",\"mk\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"ml\":\"\u0D38\u0D4D\u0D2A\u0D46\u0D2F\u0D4D\u0D28\u0D4D\\u200D\",\"ceb\":\"Espanya\",\"mi\":\"P\u0101niora\",\"uk\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"eu\":\"Espainia\",\"mr\":\"\u0938\u094D\u092A\u0947\u0928\",\"ug\":\"\u0626\u0649\u0633\u067E\u0627\u0646\u0649\u064A\u06D5\",\"mt\":\"Spanja\",\"ms\":\"Sepanyol\",\"ur\":\"\u0633\u067E\u06CC\u0646\",\"fa\":\"\u0627\u0633\u067E\u0627\u0646\u06CC\u0627\",\"ty\":\"Paniora\",\"new\":\"\u0938\u094D\u092A\u0947\u0928\",\"na\":\"Pain\",\"el\":\"\u0399\u03C3\u03C0\u03B1\u03BD\u03AF\u03B1\",\"nb\":\"Spania\",\"ne\":\"\u0938\u094D\u092A\u0947\u0928\",\"vls\":\"Spanje\",\"eo\":\"Hispanio\",\"en\":\"Kingdom of Spain\",\"et\":\"Hispaania\",\"es\":\"la Madre Patria\",\"nl\":\"Spanje\",\"vec\":\"Spagna\",\"to\":\"Sepeni\",\"ca\":\"Espanya\",\"tl\":\"Espanya\",\"tr\":\"\u0130spanya\",\"tg\":\"\u0418\u0441\u043F\u043E\u043D\u0438\u0451\",\"haw\":\"Sepania\",\"bs\":\"\u0160panija\",\"br\":\"Spagn\",\"th\":\"\u0E1B\u0E23\u0E30\u0E40\u0E17\u0E28\u0E2A\u0E40\u0E1B\u0E19\",\"bn\":\"\u09B8\u09CD\u09AA\u09C7\u09A8\",\"bo\":\"\u0F66\u0F72\u0F0B\u0F54\u0F53\u0F0D\",\"ta\":\"\u0BB8\u0BCD\u0BAA\u0BC6\u0BAF\u0BBF\u0BA9\u0BCD\",\"sv\":\"Spanien\",\"bg\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"ka\":\"\u10D4\u10E1\u10DE\u10D0\u10DC\u10D4\u10D7\u10D8\",\"st\":\"Spain\",\"sw\":\"Hispania\",\"be\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"kw\":\"Spayn\",\"sl\":\"\u0160panija\",\"sk\":\"\u0160panielsko\",\"da\":\"Spanien\",\"ang\":\"Sp\u0113onland\",\"nds\":\"Spanien\",\"ks\":\"Sp\u0113na\",\"so\":\"Isbeyn\",\"ku\":\"Spanya\",\"sr\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"sq\":\"Spanja\",\"ko\":\"\uC5D0\uC2A4\uD30C\uB0D0\",\"sc\":\"Ispagna\",\"cy\":\"Sbaen\",\"se\":\"Esp\u00E1njja\",\"sh\":\"\u0160panija\",\"cv\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"km\":\"\u17A2\u17C1\u179F\u17D2\u1794\u17C9\u17B6\u1789\",\"cs\":\"\u0160pan\u011Blsko\",\"li\":\"Spanje\",\"co\":\"Spagna\",\"default\":\"Spain\",\"jbo\":\"sangu\'e\",\"la\":\"Hesperia\",\"ru\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"lb\":\"Spuenien\",\"sco\":\"Spain\",\"tet\":\"Espa\u00F1a\",\"scn\":\"Spagna\",\"hr\":\"\u0160panjolska\",\"zh\":\"\u897F\u73ED\u7259\",\"ro\":\"Spania\",\"rm\":\"Spagna\",\"ht\":\"Espay\",\"hu\":\"Spanyolorsz\u00E1g\",\"ast\":\"Espa\u00F1a\",\"hi\":\"\u0938\u094D\u092A\u0947\u0928\",\"hsb\":\"\u0160paniska\",\"nah\":\"Caxtill\u0101n\",\"war\":\"Espanya\",\"lad\":\"Espanya\",\"id\":\"Spanyol\",\"ia\":\"Espania\",\"nrm\":\"Espangne\",\"hy\":\"\u053B\u057D\u057A\u0561\u0576\u056B\u0561\",\"qu\":\"Ispa\u00F1a\",\"ilo\":\"Espania\",\"az\":\"\u0130spaniya\",\"is\":\"Sp\u00E1nn\",\"it\":\"Spagna\",\"tpi\":\"Spen\",\"ar\":\"\u0623\u0633\u0628\u0627\u0646\u064A\u0627\",\"io\":\"Hispania\",\"pam\":\"Espanya\",\"frp\":\"\u00C8spagne\",\"am\":\"\u12A5\u1235\u1353\u1295\u12EB\",\"an\":\"Espa\u00F1a\",\"csb\":\"Szpa\u0144sk\u00F4\",\"pt\":\"Espanha\",\"ja\":\"\u30B9\u30DA\u30A4\u30F3\",\"ps\":\"\u0627\u0633\u067E\u0627\u0646\u064A\u0627\",\"yi\":\"\u05E9\u05E4\u05D0\u05E0\u05D9\u05E2\",\"af\":\"Spanje\"},\"geonameFeatureCode\":\"PCLI\",\"_id\":260,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2510769,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"ca\":\"Andalusia\",\"tr\":\"End\u00FCl\u00FCs\",\"krc\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"no\":\"Andaluc\u00EDa\",\"fy\":\"Andal\u00FBsje\",\"bs\":\"Andaluzija\",\"br\":\"Andalouzia\",\"ext\":\"Andaluzia\",\"ga\":\"An Andal\u00FAis\",\"th\":\"\u0E41\u0E04\u0E27\u0E49\u0E19\u0E2D\u0E31\u0E19\u0E14\u0E32\u0E25\u0E39\u0E0B\u0E35\u0E2D\u0E32\",\"bn\":\"\u0986\u09A8\u09CD\u09A6\u09BE\u09B2\u09C1\u09B8\u09BF\u09AF\u09BC\u09BE\",\"oc\":\"Andalosia\",\"ka\":\"\u10D0\u10DC\u10D3\u10D0\u10DA\u10E3\u10E1\u10D8\u10D0\",\"sv\":\"Andalusien\",\"fr\":\"Andalousie\",\"bg\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"glk\":\"\u0622\u0646\u062F\u0627\u0644\u0648\u0633\u06CC\u0627\",\"be\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"kw\":\"Andalousi\",\"sk\":\"Andal\u00FAzia\",\"os\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\",\"da\":\"Andalusien\",\"sr\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0437\u0438\u0458\u0430\",\"ku\":\"Endulus\",\"ko\":\"\uC548\uB2EC\uB8E8\uC2DC\uC544 \uC9C0\uBC29\",\"he\":\"\u05D0\u05E0\u05D3\u05DC\u05D5\u05E1\u05D9\u05D4\",\"sh\":\"Andaluzija\",\"arz\":\"\u0627\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"cs\":\"Andalusie\",\"default\":\"Andalusia\",\"stq\":\"Andalusien\",\"la\":\"Vandalitia\",\"pl\":\"Andaluzja\",\"ru\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"lb\":\"Andalusien\",\"tet\":\"Andaluzia\",\"got\":\"\u10345\u10330\u1033D\u10333\u10330\u1033B\u10339\u10344\u1033E\u10330\",\"hr\":\"Andaluzija\",\"zh\":\"\u5B89\u9054\u9B6F\u897F\u4E9E\",\"ro\":\"Andaluzia\",\"hu\":\"Andal\u00FAzia\",\"pms\":\"Andalus\u00ECa\",\"lv\":\"Andal\u016Bzija\",\"lt\":\"Andal\u016Bzija\",\"nah\":\"Andalucia\",\"lad\":\"Andaluziya\",\"de\":\"Andalusien\",\"als\":\"Andalusien\",\"qu\":\"Andalusiya\",\"hy\":\"\u0531\u0576\u0564\u0561\u056C\u0578\u0582\u0566\u056B\u0561\",\"eu\":\"Andaluzia\",\"is\":\"Andal\u00FAs\u00EDa\",\"uk\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"az\":\"Andalusiya\",\"mr\":\"\u0906\u0902\u0926\u093E\u0932\u0941\u0938\u093F\u092F\u093E\",\"ug\":\"Andalusiye\",\"fa\":\"\u0627\u0646\u062F\u0644\u0633\",\"ar\":\"\u0623\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"rmy\":\"Andalusiya\",\"io\":\"Andaluzia\",\"el\":\"\u0391\u03BD\u03B4\u03B1\u03BB\u03BF\u03C5\u03C3\u03AF\u03B1\",\"frp\":\"Andalosie\",\"pt\":\"Andaluzia\",\"eo\":\"Andaluzio\",\"en\":\"Andalusia\",\"et\":\"Andaluusia\",\"es\":\"Andaluc\u00EDa\",\"ja\":\"\u30A2\u30F3\u30C0\u30EB\u30B7\u30A2\u5DDE\",\"nl\":\"Andalusi\u00EB\",\"af\":\"Andalusi\u00EB\",\"vec\":\"Andalus\u00ECa\"},\"geonameFeatureCode\":\"ADM1\",\"_id\":261,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2593109,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"de\":\"Granada\",\"default\":\"Province of Granada\",\"fr\":\"Grenade\",\"en\":\"Province of Granada\",\"es\":\"Provincia de Granada\",\"ja\":\"\u30B0\u30E9\u30CA\u30C0\"},\"geonameFeatureCode\":\"ADM2\",\"_id\":262,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2517115,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"default\":\"Monachil\"},\"geonameFeatureCode\":\"ADM3\",\"_id\":263,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6357744,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"P\",\"values\":{\"default\":\"Sierra Nevada\"},\"geonameFeatureCode\":\"PPL\",\"_id\":264,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6544329,\"value"
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (LANG-708) StringEscapeUtils.escapeEcmaScript from
lang3 cuts off long unicode string
Posted by "anton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
anton updated LANG-708:
-----------------------
Description:
Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
the original string ends with:
"geonameId":6544329,"valueCode":""}]
and the produced string ends with:
\"geonameId\":6544329,\"value
So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
CharSequenceTranslator.translate(...){
...
int sz = Character.codePointCount(input, 0, input.length());
for (int i = 0; i < sz; i++) {
// consumed is the number of codepoints consumed
int consumed = translate(input, i, out);
if(consumed == 0) {
out.write( Character.toChars( Character.codePointAt(input, i) ) );
}
...
}
If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
was:
Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
the original string ends with:
"geonameId":6544329,"valueCode":""}]
and the produced string ends with:
\"geonameId\":6544329,\"value
So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
CharSequenceTranslator.translate(...){
...
int sz = Character.codePointCount(input, 0, input.length());
for (int i = 0; i < sz; i++) {
// consumed is the number of codepoints consumed
int consumed = translate(input, i, out);
if(consumed == 0) {
out.write( Character.toChars( Character.codePointAt(input, i) ) );
}
...
}
If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
So this is the original string (sorry for big text in message body - I can't see where files could be attached to this bugzilla):
"[{"geonameFeatureClass":"L","values":{"eu":"Mundua","ro":"Pamânt","it":"Globo","ca":"el món","tr":"Yeryüzü","no":"Jorden","hu":"Föld","lv":"Zeme","de":"Welt","el":"Υδρόγειος","fi":"Maa","la":"Terra","fr":"Monde","eo":"Mondo","en":"World","ru":"Земля","es":"el planeta","nl":"Aarde"},"geonameFeatureCode":"AREA","_id":32,"name":"","auto":true,"type":"GEO","geonameId":6295630,"valueCode":""},{"geonameFeatureClass":"L","values":{"ro":"Europa","zh":"欧洲","ca":"Europa","vi":"Châu Âu","tr":"Avrupa","no":"Europa","hu":"Európa","lv":"Eiropa","hi":"यूरोप","lt":"Europa","bs":"Evropa","ga":"an Eoraip","th":"ยุโรป","id":"Eropa","de":"Europa","fi":"Eurooppa","fr":"Europe","sv":"Europa","bg":"Европа","da":"Europa","eu":"Europa","is":"Evrópa","it":"Europa","cy":"Ewrop","ar":"أوروبا","se":"Eurohpá","he":"אירופה","cs":"Evropa","el":"Ευρώπη","nb":"Europa","pl":"Europa","la":"Europa","pt":"Europa","eo":"Eŭropo","en":"Europe","ru":"Европа","es":"Europa","ja":"ヨーロッパ","nl":"Europa"},"geonameFeatureCode":"CONT","_id":33,"name":"","auto":true,"type":"GEO","geonameId":6255148,"valueCode":""},{"geonameFeatureClass":"A","values":{"no":"Spania","nn":"Spania","fy":"Spanje","gd":"An Spàinn","ga":"An Spáinn","oc":"Espanha","arc":"ܐܣܦܢܝܐ","fi":"Espanja","fr":"Espagne","fo":"Spania","udm":"Испания","os":"Испани","he":"ספרד","gn":"Epaña","gl":"España","gv":"Yn Spaainey","pl":"Hiszpania","gu":"સ્પેઇન","lo":"ສະເປນ","ln":"Espania","vi":"Tây Ban Nha","dz":"Spain","pms":"Spagna","lv":"Spānija","lt":"Ispanija","vo":"Spanyän","de":"Spanien","mg":"Espaina","fur":"Spagne","mk":"Шпанија","ml":"സ്പെയ്ന്\u200D","ceb":"Espanya","mi":"Pāniora","uk":"Іспанія","eu":"Espainia","mr":"स्पेन","ug":"ئىسپانىيە","mt":"Spanja","ms":"Sepanyol","ur":"سپین","fa":"اسپانیا","ty":"Paniora","new":"स्पेन","na":"Pain","el":"Ισπανία","nb":"Spania","ne":"स्पेन","vls":"Spanje","eo":"Hispanio","en":"Kingdom of Spain","et":"Hispaania","es":"la Madre Patria","nl":"Spanje","vec":"Spagna","to":"Sepeni","ca":"Espanya","tl":"Espanya","tr":"İspanya","tg":"Испониё","haw":"Sepania","bs":"Španija","br":"Spagn","th":"ประเทศสเปน","bn":"স্পেন","bo":"སི་པན།","ta":"ஸ்பெயின்","sv":"Spanien","bg":"Испания","ka":"ესპანეთი","st":"Spain","sw":"Hispania","be":"Іспанія","kw":"Spayn","sl":"Španija","sk":"Španielsko","da":"Spanien","ang":"Spēonland","nds":"Spanien","ks":"Spēna","so":"Isbeyn","ku":"Spanya","sr":"Шпанија","sq":"Spanja","ko":"에스파냐","sc":"Ispagna","cy":"Sbaen","se":"Espánjja","sh":"Španija","cv":"Испани","km":"អេស្ប៉ាញ","cs":"Španělsko","li":"Spanje","co":"Spagna","default":"Spain","jbo":"sangu'e","la":"Hesperia","ru":"Испания","lb":"Spuenien","sco":"Spain","tet":"España","scn":"Spagna","hr":"Španjolska","zh":"西班牙","ro":"Spania","rm":"Spagna","ht":"Espay","hu":"Spanyolország","ast":"España","hi":"स्पेन","hsb":"Španiska","nah":"Caxtillān","war":"Espanya","lad":"Espanya","id":"Spanyol","ia":"Espania","nrm":"Espangne","hy":"Իսպանիա","qu":"Ispaña","ilo":"Espania","az":"İspaniya","is":"Spánn","it":"Spagna","tpi":"Spen","ar":"أسبانيا","io":"Hispania","pam":"Espanya","frp":"Èspagne","am":"እስፓንያ","an":"España","csb":"Szpańskô","pt":"Espanha","ja":"スペイン","ps":"اسپانيا","yi":"שפאניע","af":"Spanje"},"geonameFeatureCode":"PCLI","_id":260,"name":"","auto":true,"type":"GEO","geonameId":2510769,"valueCode":""},{"geonameFeatureClass":"A","values":{"ca":"Andalusia","tr":"Endülüs","krc":"Андалусия","no":"Andalucía","fy":"Andalûsje","bs":"Andaluzija","br":"Andalouzia","ext":"Andaluzia","ga":"An Andalúis","th":"แคว้นอันดาลูซีอา","bn":"আন্দালুসিয়া","oc":"Andalosia","ka":"ანდალუსია","sv":"Andalusien","fr":"Andalousie","bg":"Андалусия","glk":"آندالوسیا","be":"Андалусія","kw":"Andalousi","sk":"Andalúzia","os":"Андалуси","da":"Andalusien","sr":"Андалузија","ku":"Endulus","ko":"안달루시아 지방","he":"אנדלוסיה","sh":"Andaluzija","arz":"اندلوسيا","cs":"Andalusie","default":"Andalusia","stq":"Andalusien","la":"Vandalitia","pl":"Andaluzja","ru":"Андалусия","lb":"Andalusien","tet":"Andaluzia","got":"𐍅𐌰𐌽𐌳𐌰𐌻𐌹𐍄𐌾𐌰","hr":"Andaluzija","zh":"安達魯西亞","ro":"Andaluzia","hu":"Andalúzia","pms":"Andalusìa","lv":"Andalūzija","lt":"Andalūzija","nah":"Andalucia","lad":"Andaluziya","de":"Andalusien","als":"Andalusien","qu":"Andalusiya","hy":"Անդալուզիա","eu":"Andaluzia","is":"Andalúsía","uk":"Андалусія","az":"Andalusiya","mr":"आंदालुसिया","ug":"Andalusiye","fa":"اندلس","ar":"أندلوسيا","rmy":"Andalusiya","io":"Andaluzia","el":"Ανδαλουσία","frp":"Andalosie","pt":"Andaluzia","eo":"Andaluzio","en":"Andalusia","et":"Andaluusia","es":"Andalucía","ja":"アンダルシア州","nl":"Andalusië","af":"Andalusië","vec":"Andalusìa"},"geonameFeatureCode":"ADM1","_id":261,"name":"","auto":true,"type":"GEO","geonameId":2593109,"valueCode":""},{"geonameFeatureClass":"A","values":{"de":"Granada","default":"Province of Granada","fr":"Grenade","en":"Province of Granada","es":"Provincia de Granada","ja":"グラナダ"},"geonameFeatureCode":"ADM2","_id":262,"name":"","auto":true,"type":"GEO","geonameId":2517115,"valueCode":""},{"geonameFeatureClass":"A","values":{"default":"Monachil"},"geonameFeatureCode":"ADM3","_id":263,"name":"","auto":true,"type":"GEO","geonameId":6357744,"valueCode":""},{"geonameFeatureClass":"P","values":{"default":"Sierra Nevada"},"geonameFeatureCode":"PPL","_id":264,"name":"","auto":true,"type":"GEO","geonameId":6544329,"valueCode":""}]"
and this is string, produced by StringEscapeUtils.escapeEcmaScript(value):
"[{\"geonameFeatureClass\":\"L\",\"values\":{\"eu\":\"Mundua\",\"ro\":\"Pam\u00E2nt\",\"it\":\"Globo\",\"ca\":\"el m\u00F3n\",\"tr\":\"Yery\u00FCz\u00FC\",\"no\":\"Jorden\",\"hu\":\"F\u00F6ld\",\"lv\":\"Zeme\",\"de\":\"Welt\",\"el\":\"\u03A5\u03B4\u03C1\u03CC\u03B3\u03B5\u03B9\u03BF\u03C2\",\"fi\":\"Maa\",\"la\":\"Terra\",\"fr\":\"Monde\",\"eo\":\"Mondo\",\"en\":\"World\",\"ru\":\"\u0417\u0435\u043C\u043B\u044F\",\"es\":\"el planeta\",\"nl\":\"Aarde\"},\"geonameFeatureCode\":\"AREA\",\"_id\":32,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6295630,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"L\",\"values\":{\"ro\":\"Europa\",\"zh\":\"\u6B27\u6D32\",\"ca\":\"Europa\",\"vi\":\"Ch\u00E2u \u00C2u\",\"tr\":\"Avrupa\",\"no\":\"Europa\",\"hu\":\"Eur\u00F3pa\",\"lv\":\"Eiropa\",\"hi\":\"\u092F\u0942\u0930\u094B\u092A\",\"lt\":\"Europa\",\"bs\":\"Evropa\",\"ga\":\"an Eoraip\",\"th\":\"\u0E22\u0E38\u0E42\u0E23\u0E1B\",\"id\":\"Eropa\",\"de\":\"Europa\",\"fi\":\"Eurooppa\",\"fr\":\"Europe\",\"sv\":\"Europa\",\"bg\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"da\":\"Europa\",\"eu\":\"Europa\",\"is\":\"Evr\u00F3pa\",\"it\":\"Europa\",\"cy\":\"Ewrop\",\"ar\":\"\u0623\u0648\u0631\u0648\u0628\u0627\",\"se\":\"Eurohp\u00E1\",\"he\":\"\u05D0\u05D9\u05E8\u05D5\u05E4\u05D4\",\"cs\":\"Evropa\",\"el\":\"\u0395\u03C5\u03C1\u03CE\u03C0\u03B7\",\"nb\":\"Europa\",\"pl\":\"Europa\",\"la\":\"Europa\",\"pt\":\"Europa\",\"eo\":\"E\u016Dropo\",\"en\":\"Europe\",\"ru\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"es\":\"Europa\",\"ja\":\"\u30E8\u30FC\u30ED\u30C3\u30D1\",\"nl\":\"Europa\"},\"geonameFeatureCode\":\"CONT\",\"_id\":33,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6255148,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"no\":\"Spania\",\"nn\":\"Spania\",\"fy\":\"Spanje\",\"gd\":\"An Sp\u00E0inn\",\"ga\":\"An Sp\u00E1inn\",\"oc\":\"Espanha\",\"arc\":\"\u0710\u0723\u0726\u0722\u071D\u0710\",\"fi\":\"Espanja\",\"fr\":\"Espagne\",\"fo\":\"Spania\",\"udm\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"os\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"he\":\"\u05E1\u05E4\u05E8\u05D3\",\"gn\":\"Epa\u00F1a\",\"gl\":\"Espa\u00F1a\",\"gv\":\"Yn Spaainey\",\"pl\":\"Hiszpania\",\"gu\":\"\u0AB8\u0ACD\u0AAA\u0AC7\u0A87\u0AA8\",\"lo\":\"\u0EAA\u0EB0\u0EC0\u0E9B\u0E99\",\"ln\":\"Espania\",\"vi\":\"T\u00E2y Ban Nha\",\"dz\":\"Spain\",\"pms\":\"Spagna\",\"lv\":\"Sp\u0101nija\",\"lt\":\"Ispanija\",\"vo\":\"Spany\u00E4n\",\"de\":\"Spanien\",\"mg\":\"Espaina\",\"fur\":\"Spagne\",\"mk\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"ml\":\"\u0D38\u0D4D\u0D2A\u0D46\u0D2F\u0D4D\u0D28\u0D4D\\u200D\",\"ceb\":\"Espanya\",\"mi\":\"P\u0101niora\",\"uk\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"eu\":\"Espainia\",\"mr\":\"\u0938\u094D\u092A\u0947\u0928\",\"ug\":\"\u0626\u0649\u0633\u067E\u0627\u0646\u0649\u064A\u06D5\",\"mt\":\"Spanja\",\"ms\":\"Sepanyol\",\"ur\":\"\u0633\u067E\u06CC\u0646\",\"fa\":\"\u0627\u0633\u067E\u0627\u0646\u06CC\u0627\",\"ty\":\"Paniora\",\"new\":\"\u0938\u094D\u092A\u0947\u0928\",\"na\":\"Pain\",\"el\":\"\u0399\u03C3\u03C0\u03B1\u03BD\u03AF\u03B1\",\"nb\":\"Spania\",\"ne\":\"\u0938\u094D\u092A\u0947\u0928\",\"vls\":\"Spanje\",\"eo\":\"Hispanio\",\"en\":\"Kingdom of Spain\",\"et\":\"Hispaania\",\"es\":\"la Madre Patria\",\"nl\":\"Spanje\",\"vec\":\"Spagna\",\"to\":\"Sepeni\",\"ca\":\"Espanya\",\"tl\":\"Espanya\",\"tr\":\"\u0130spanya\",\"tg\":\"\u0418\u0441\u043F\u043E\u043D\u0438\u0451\",\"haw\":\"Sepania\",\"bs\":\"\u0160panija\",\"br\":\"Spagn\",\"th\":\"\u0E1B\u0E23\u0E30\u0E40\u0E17\u0E28\u0E2A\u0E40\u0E1B\u0E19\",\"bn\":\"\u09B8\u09CD\u09AA\u09C7\u09A8\",\"bo\":\"\u0F66\u0F72\u0F0B\u0F54\u0F53\u0F0D\",\"ta\":\"\u0BB8\u0BCD\u0BAA\u0BC6\u0BAF\u0BBF\u0BA9\u0BCD\",\"sv\":\"Spanien\",\"bg\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"ka\":\"\u10D4\u10E1\u10DE\u10D0\u10DC\u10D4\u10D7\u10D8\",\"st\":\"Spain\",\"sw\":\"Hispania\",\"be\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"kw\":\"Spayn\",\"sl\":\"\u0160panija\",\"sk\":\"\u0160panielsko\",\"da\":\"Spanien\",\"ang\":\"Sp\u0113onland\",\"nds\":\"Spanien\",\"ks\":\"Sp\u0113na\",\"so\":\"Isbeyn\",\"ku\":\"Spanya\",\"sr\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"sq\":\"Spanja\",\"ko\":\"\uC5D0\uC2A4\uD30C\uB0D0\",\"sc\":\"Ispagna\",\"cy\":\"Sbaen\",\"se\":\"Esp\u00E1njja\",\"sh\":\"\u0160panija\",\"cv\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"km\":\"\u17A2\u17C1\u179F\u17D2\u1794\u17C9\u17B6\u1789\",\"cs\":\"\u0160pan\u011Blsko\",\"li\":\"Spanje\",\"co\":\"Spagna\",\"default\":\"Spain\",\"jbo\":\"sangu\'e\",\"la\":\"Hesperia\",\"ru\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"lb\":\"Spuenien\",\"sco\":\"Spain\",\"tet\":\"Espa\u00F1a\",\"scn\":\"Spagna\",\"hr\":\"\u0160panjolska\",\"zh\":\"\u897F\u73ED\u7259\",\"ro\":\"Spania\",\"rm\":\"Spagna\",\"ht\":\"Espay\",\"hu\":\"Spanyolorsz\u00E1g\",\"ast\":\"Espa\u00F1a\",\"hi\":\"\u0938\u094D\u092A\u0947\u0928\",\"hsb\":\"\u0160paniska\",\"nah\":\"Caxtill\u0101n\",\"war\":\"Espanya\",\"lad\":\"Espanya\",\"id\":\"Spanyol\",\"ia\":\"Espania\",\"nrm\":\"Espangne\",\"hy\":\"\u053B\u057D\u057A\u0561\u0576\u056B\u0561\",\"qu\":\"Ispa\u00F1a\",\"ilo\":\"Espania\",\"az\":\"\u0130spaniya\",\"is\":\"Sp\u00E1nn\",\"it\":\"Spagna\",\"tpi\":\"Spen\",\"ar\":\"\u0623\u0633\u0628\u0627\u0646\u064A\u0627\",\"io\":\"Hispania\",\"pam\":\"Espanya\",\"frp\":\"\u00C8spagne\",\"am\":\"\u12A5\u1235\u1353\u1295\u12EB\",\"an\":\"Espa\u00F1a\",\"csb\":\"Szpa\u0144sk\u00F4\",\"pt\":\"Espanha\",\"ja\":\"\u30B9\u30DA\u30A4\u30F3\",\"ps\":\"\u0627\u0633\u067E\u0627\u0646\u064A\u0627\",\"yi\":\"\u05E9\u05E4\u05D0\u05E0\u05D9\u05E2\",\"af\":\"Spanje\"},\"geonameFeatureCode\":\"PCLI\",\"_id\":260,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2510769,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"ca\":\"Andalusia\",\"tr\":\"End\u00FCl\u00FCs\",\"krc\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"no\":\"Andaluc\u00EDa\",\"fy\":\"Andal\u00FBsje\",\"bs\":\"Andaluzija\",\"br\":\"Andalouzia\",\"ext\":\"Andaluzia\",\"ga\":\"An Andal\u00FAis\",\"th\":\"\u0E41\u0E04\u0E27\u0E49\u0E19\u0E2D\u0E31\u0E19\u0E14\u0E32\u0E25\u0E39\u0E0B\u0E35\u0E2D\u0E32\",\"bn\":\"\u0986\u09A8\u09CD\u09A6\u09BE\u09B2\u09C1\u09B8\u09BF\u09AF\u09BC\u09BE\",\"oc\":\"Andalosia\",\"ka\":\"\u10D0\u10DC\u10D3\u10D0\u10DA\u10E3\u10E1\u10D8\u10D0\",\"sv\":\"Andalusien\",\"fr\":\"Andalousie\",\"bg\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"glk\":\"\u0622\u0646\u062F\u0627\u0644\u0648\u0633\u06CC\u0627\",\"be\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"kw\":\"Andalousi\",\"sk\":\"Andal\u00FAzia\",\"os\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\",\"da\":\"Andalusien\",\"sr\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0437\u0438\u0458\u0430\",\"ku\":\"Endulus\",\"ko\":\"\uC548\uB2EC\uB8E8\uC2DC\uC544 \uC9C0\uBC29\",\"he\":\"\u05D0\u05E0\u05D3\u05DC\u05D5\u05E1\u05D9\u05D4\",\"sh\":\"Andaluzija\",\"arz\":\"\u0627\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"cs\":\"Andalusie\",\"default\":\"Andalusia\",\"stq\":\"Andalusien\",\"la\":\"Vandalitia\",\"pl\":\"Andaluzja\",\"ru\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"lb\":\"Andalusien\",\"tet\":\"Andaluzia\",\"got\":\"\u10345\u10330\u1033D\u10333\u10330\u1033B\u10339\u10344\u1033E\u10330\",\"hr\":\"Andaluzija\",\"zh\":\"\u5B89\u9054\u9B6F\u897F\u4E9E\",\"ro\":\"Andaluzia\",\"hu\":\"Andal\u00FAzia\",\"pms\":\"Andalus\u00ECa\",\"lv\":\"Andal\u016Bzija\",\"lt\":\"Andal\u016Bzija\",\"nah\":\"Andalucia\",\"lad\":\"Andaluziya\",\"de\":\"Andalusien\",\"als\":\"Andalusien\",\"qu\":\"Andalusiya\",\"hy\":\"\u0531\u0576\u0564\u0561\u056C\u0578\u0582\u0566\u056B\u0561\",\"eu\":\"Andaluzia\",\"is\":\"Andal\u00FAs\u00EDa\",\"uk\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"az\":\"Andalusiya\",\"mr\":\"\u0906\u0902\u0926\u093E\u0932\u0941\u0938\u093F\u092F\u093E\",\"ug\":\"Andalusiye\",\"fa\":\"\u0627\u0646\u062F\u0644\u0633\",\"ar\":\"\u0623\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"rmy\":\"Andalusiya\",\"io\":\"Andaluzia\",\"el\":\"\u0391\u03BD\u03B4\u03B1\u03BB\u03BF\u03C5\u03C3\u03AF\u03B1\",\"frp\":\"Andalosie\",\"pt\":\"Andaluzia\",\"eo\":\"Andaluzio\",\"en\":\"Andalusia\",\"et\":\"Andaluusia\",\"es\":\"Andaluc\u00EDa\",\"ja\":\"\u30A2\u30F3\u30C0\u30EB\u30B7\u30A2\u5DDE\",\"nl\":\"Andalusi\u00EB\",\"af\":\"Andalusi\u00EB\",\"vec\":\"Andalus\u00ECa\"},\"geonameFeatureCode\":\"ADM1\",\"_id\":261,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2593109,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"de\":\"Granada\",\"default\":\"Province of Granada\",\"fr\":\"Grenade\",\"en\":\"Province of Granada\",\"es\":\"Provincia de Granada\",\"ja\":\"\u30B0\u30E9\u30CA\u30C0\"},\"geonameFeatureCode\":\"ADM2\",\"_id\":262,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2517115,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"default\":\"Monachil\"},\"geonameFeatureCode\":\"ADM3\",\"_id\":263,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6357744,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"P\",\"values\":{\"default\":\"Sierra Nevada\"},\"geonameFeatureCode\":\"PPL\",\"_id\":264,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6544329,\"value"
> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
> Key: LANG-708
> URL: https://issues.apache.org/jira/browse/LANG-708
> Project: Commons Lang
> Issue Type: Bug
> Reporter: anton
> Attachments: Test.java, input.txt
>
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
> the original string ends with:
> "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
> \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
> int sz = Character.codePointCount(input, 0, input.length());
> for (int i = 0; i < sz; i++) {
> // consumed is the number of codepoints consumed
> int consumed = translate(input, i, out);
> if(consumed == 0) {
> out.write( Character.toChars( Character.codePointAt(input, i) ) );
> }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (LANG-708) StringEscapeUtils.escapeEcmaScript
from lang3 cuts off long unicode string
Posted by "Henri Yandell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henri Yandell reopened LANG-708:
--------------------------------
> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
> Key: LANG-708
> URL: https://issues.apache.org/jira/browse/LANG-708
> Project: Commons Lang
> Issue Type: Bug
> Components: lang.text.translate.*
> Reporter: anton
> Fix For: 3.x
>
> Attachments: Test.java, input.txt
>
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
> the original string ends with:
> "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
> \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
> int sz = Character.codePointCount(input, 0, input.length());
> for (int i = 0; i < sz; i++) {
> // consumed is the number of codepoints consumed
> int consumed = translate(input, i, out);
> if(consumed == 0) {
> out.write( Character.toChars( Character.codePointAt(input, i) ) );
> }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (LANG-708) StringEscapeUtils.escapeEcmaScript from
lang3 cuts off long unicode string
Posted by "anton (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
anton updated LANG-708:
-----------------------
Attachment: input.txt
Test.java
drop both files to same dir and run Test.java as main class - it would print result of conversion with 3.0 and 2.1 to system out - 3.0 will have described problem, 2.1 will not (both libs should be added as project deps).
> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
> Key: LANG-708
> URL: https://issues.apache.org/jira/browse/LANG-708
> Project: Commons Lang
> Issue Type: Bug
> Reporter: anton
> Attachments: Test.java, input.txt
>
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
> the original string ends with:
> "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
> \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
> int sz = Character.codePointCount(input, 0, input.length());
> for (int i = 0; i < sz; i++) {
> // consumed is the number of codepoints consumed
> int consumed = translate(input, i, out);
> if(consumed == 0) {
> out.write( Character.toChars( Character.codePointAt(input, i) ) );
> }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
> So this is the original string (sorry for big text in message body - I can't see where files could be attached to this bugzilla):
> "[{"geonameFeatureClass":"L","values":{"eu":"Mundua","ro":"Pamânt","it":"Globo","ca":"el món","tr":"Yeryüzü","no":"Jorden","hu":"Föld","lv":"Zeme","de":"Welt","el":"Υδρόγειος","fi":"Maa","la":"Terra","fr":"Monde","eo":"Mondo","en":"World","ru":"Земля","es":"el planeta","nl":"Aarde"},"geonameFeatureCode":"AREA","_id":32,"name":"","auto":true,"type":"GEO","geonameId":6295630,"valueCode":""},{"geonameFeatureClass":"L","values":{"ro":"Europa","zh":"欧洲","ca":"Europa","vi":"Châu Âu","tr":"Avrupa","no":"Europa","hu":"Európa","lv":"Eiropa","hi":"यूरोप","lt":"Europa","bs":"Evropa","ga":"an Eoraip","th":"ยุโรป","id":"Eropa","de":"Europa","fi":"Eurooppa","fr":"Europe","sv":"Europa","bg":"Европа","da":"Europa","eu":"Europa","is":"Evrópa","it":"Europa","cy":"Ewrop","ar":"أوروبا","se":"Eurohpá","he":"אירופה","cs":"Evropa","el":"Ευρώπη","nb":"Europa","pl":"Europa","la":"Europa","pt":"Europa","eo":"Eŭropo","en":"Europe","ru":"Европа","es":"Europa","ja":"ヨーロッパ","nl":"Europa"},"geonameFeatureCode":"CONT","_id":33,"name":"","auto":true,"type":"GEO","geonameId":6255148,"valueCode":""},{"geonameFeatureClass":"A","values":{"no":"Spania","nn":"Spania","fy":"Spanje","gd":"An Spàinn","ga":"An Spáinn","oc":"Espanha","arc":"ܐܣܦܢܝܐ","fi":"Espanja","fr":"Espagne","fo":"Spania","udm":"Испания","os":"Испани","he":"ספרד","gn":"Epaña","gl":"España","gv":"Yn Spaainey","pl":"Hiszpania","gu":"સ્પેઇન","lo":"ສະເປນ","ln":"Espania","vi":"Tây Ban Nha","dz":"Spain","pms":"Spagna","lv":"Spānija","lt":"Ispanija","vo":"Spanyän","de":"Spanien","mg":"Espaina","fur":"Spagne","mk":"Шпанија","ml":"സ്പെയ്ന്\u200D","ceb":"Espanya","mi":"Pāniora","uk":"Іспанія","eu":"Espainia","mr":"स्पेन","ug":"ئىسپانىيە","mt":"Spanja","ms":"Sepanyol","ur":"سپین","fa":"اسپانیا","ty":"Paniora","new":"स्पेन","na":"Pain","el":"Ισπανία","nb":"Spania","ne":"स्पेन","vls":"Spanje","eo":"Hispanio","en":"Kingdom of Spain","et":"Hispaania","es":"la Madre Patria","nl":"Spanje","vec":"Spagna","to":"Sepeni","ca":"Espanya","tl":"Espanya","tr":"İspanya","tg":"Испониё","haw":"Sepania","bs":"Španija","br":"Spagn","th":"ประเทศสเปน","bn":"স্পেন","bo":"སི་པན།","ta":"ஸ்பெயின்","sv":"Spanien","bg":"Испания","ka":"ესპანეთი","st":"Spain","sw":"Hispania","be":"Іспанія","kw":"Spayn","sl":"Španija","sk":"Španielsko","da":"Spanien","ang":"Spēonland","nds":"Spanien","ks":"Spēna","so":"Isbeyn","ku":"Spanya","sr":"Шпанија","sq":"Spanja","ko":"에스파냐","sc":"Ispagna","cy":"Sbaen","se":"Espánjja","sh":"Španija","cv":"Испани","km":"អេស្ប៉ាញ","cs":"Španělsko","li":"Spanje","co":"Spagna","default":"Spain","jbo":"sangu'e","la":"Hesperia","ru":"Испания","lb":"Spuenien","sco":"Spain","tet":"España","scn":"Spagna","hr":"Španjolska","zh":"西班牙","ro":"Spania","rm":"Spagna","ht":"Espay","hu":"Spanyolország","ast":"España","hi":"स्पेन","hsb":"Španiska","nah":"Caxtillān","war":"Espanya","lad":"Espanya","id":"Spanyol","ia":"Espania","nrm":"Espangne","hy":"Իսպանիա","qu":"Ispaña","ilo":"Espania","az":"İspaniya","is":"Spánn","it":"Spagna","tpi":"Spen","ar":"أسبانيا","io":"Hispania","pam":"Espanya","frp":"Èspagne","am":"እስፓንያ","an":"España","csb":"Szpańskô","pt":"Espanha","ja":"スペイン","ps":"اسپانيا","yi":"שפאניע","af":"Spanje"},"geonameFeatureCode":"PCLI","_id":260,"name":"","auto":true,"type":"GEO","geonameId":2510769,"valueCode":""},{"geonameFeatureClass":"A","values":{"ca":"Andalusia","tr":"Endülüs","krc":"Андалусия","no":"Andalucía","fy":"Andalûsje","bs":"Andaluzija","br":"Andalouzia","ext":"Andaluzia","ga":"An Andalúis","th":"แคว้นอันดาลูซีอา","bn":"আন্দালুসিয়া","oc":"Andalosia","ka":"ანდალუსია","sv":"Andalusien","fr":"Andalousie","bg":"Андалусия","glk":"آندالوسیا","be":"Андалусія","kw":"Andalousi","sk":"Andalúzia","os":"Андалуси","da":"Andalusien","sr":"Андалузија","ku":"Endulus","ko":"안달루시아 지방","he":"אנדלוסיה","sh":"Andaluzija","arz":"اندلوسيا","cs":"Andalusie","default":"Andalusia","stq":"Andalusien","la":"Vandalitia","pl":"Andaluzja","ru":"Андалусия","lb":"Andalusien","tet":"Andaluzia","got":"𐍅𐌰𐌽𐌳𐌰𐌻𐌹𐍄𐌾𐌰","hr":"Andaluzija","zh":"安達魯西亞","ro":"Andaluzia","hu":"Andalúzia","pms":"Andalusìa","lv":"Andalūzija","lt":"Andalūzija","nah":"Andalucia","lad":"Andaluziya","de":"Andalusien","als":"Andalusien","qu":"Andalusiya","hy":"Անդալուզիա","eu":"Andaluzia","is":"Andalúsía","uk":"Андалусія","az":"Andalusiya","mr":"आंदालुसिया","ug":"Andalusiye","fa":"اندلس","ar":"أندلوسيا","rmy":"Andalusiya","io":"Andaluzia","el":"Ανδαλουσία","frp":"Andalosie","pt":"Andaluzia","eo":"Andaluzio","en":"Andalusia","et":"Andaluusia","es":"Andalucía","ja":"アンダルシア州","nl":"Andalusië","af":"Andalusië","vec":"Andalusìa"},"geonameFeatureCode":"ADM1","_id":261,"name":"","auto":true,"type":"GEO","geonameId":2593109,"valueCode":""},{"geonameFeatureClass":"A","values":{"de":"Granada","default":"Province of Granada","fr":"Grenade","en":"Province of Granada","es":"Provincia de Granada","ja":"グラナダ"},"geonameFeatureCode":"ADM2","_id":262,"name":"","auto":true,"type":"GEO","geonameId":2517115,"valueCode":""},{"geonameFeatureClass":"A","values":{"default":"Monachil"},"geonameFeatureCode":"ADM3","_id":263,"name":"","auto":true,"type":"GEO","geonameId":6357744,"valueCode":""},{"geonameFeatureClass":"P","values":{"default":"Sierra Nevada"},"geonameFeatureCode":"PPL","_id":264,"name":"","auto":true,"type":"GEO","geonameId":6544329,"valueCode":""}]"
> and this is string, produced by StringEscapeUtils.escapeEcmaScript(value):
> "[{\"geonameFeatureClass\":\"L\",\"values\":{\"eu\":\"Mundua\",\"ro\":\"Pam\u00E2nt\",\"it\":\"Globo\",\"ca\":\"el m\u00F3n\",\"tr\":\"Yery\u00FCz\u00FC\",\"no\":\"Jorden\",\"hu\":\"F\u00F6ld\",\"lv\":\"Zeme\",\"de\":\"Welt\",\"el\":\"\u03A5\u03B4\u03C1\u03CC\u03B3\u03B5\u03B9\u03BF\u03C2\",\"fi\":\"Maa\",\"la\":\"Terra\",\"fr\":\"Monde\",\"eo\":\"Mondo\",\"en\":\"World\",\"ru\":\"\u0417\u0435\u043C\u043B\u044F\",\"es\":\"el planeta\",\"nl\":\"Aarde\"},\"geonameFeatureCode\":\"AREA\",\"_id\":32,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6295630,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"L\",\"values\":{\"ro\":\"Europa\",\"zh\":\"\u6B27\u6D32\",\"ca\":\"Europa\",\"vi\":\"Ch\u00E2u \u00C2u\",\"tr\":\"Avrupa\",\"no\":\"Europa\",\"hu\":\"Eur\u00F3pa\",\"lv\":\"Eiropa\",\"hi\":\"\u092F\u0942\u0930\u094B\u092A\",\"lt\":\"Europa\",\"bs\":\"Evropa\",\"ga\":\"an Eoraip\",\"th\":\"\u0E22\u0E38\u0E42\u0E23\u0E1B\",\"id\":\"Eropa\",\"de\":\"Europa\",\"fi\":\"Eurooppa\",\"fr\":\"Europe\",\"sv\":\"Europa\",\"bg\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"da\":\"Europa\",\"eu\":\"Europa\",\"is\":\"Evr\u00F3pa\",\"it\":\"Europa\",\"cy\":\"Ewrop\",\"ar\":\"\u0623\u0648\u0631\u0648\u0628\u0627\",\"se\":\"Eurohp\u00E1\",\"he\":\"\u05D0\u05D9\u05E8\u05D5\u05E4\u05D4\",\"cs\":\"Evropa\",\"el\":\"\u0395\u03C5\u03C1\u03CE\u03C0\u03B7\",\"nb\":\"Europa\",\"pl\":\"Europa\",\"la\":\"Europa\",\"pt\":\"Europa\",\"eo\":\"E\u016Dropo\",\"en\":\"Europe\",\"ru\":\"\u0415\u0432\u0440\u043E\u043F\u0430\",\"es\":\"Europa\",\"ja\":\"\u30E8\u30FC\u30ED\u30C3\u30D1\",\"nl\":\"Europa\"},\"geonameFeatureCode\":\"CONT\",\"_id\":33,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6255148,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"no\":\"Spania\",\"nn\":\"Spania\",\"fy\":\"Spanje\",\"gd\":\"An Sp\u00E0inn\",\"ga\":\"An Sp\u00E1inn\",\"oc\":\"Espanha\",\"arc\":\"\u0710\u0723\u0726\u0722\u071D\u0710\",\"fi\":\"Espanja\",\"fr\":\"Espagne\",\"fo\":\"Spania\",\"udm\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"os\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"he\":\"\u05E1\u05E4\u05E8\u05D3\",\"gn\":\"Epa\u00F1a\",\"gl\":\"Espa\u00F1a\",\"gv\":\"Yn Spaainey\",\"pl\":\"Hiszpania\",\"gu\":\"\u0AB8\u0ACD\u0AAA\u0AC7\u0A87\u0AA8\",\"lo\":\"\u0EAA\u0EB0\u0EC0\u0E9B\u0E99\",\"ln\":\"Espania\",\"vi\":\"T\u00E2y Ban Nha\",\"dz\":\"Spain\",\"pms\":\"Spagna\",\"lv\":\"Sp\u0101nija\",\"lt\":\"Ispanija\",\"vo\":\"Spany\u00E4n\",\"de\":\"Spanien\",\"mg\":\"Espaina\",\"fur\":\"Spagne\",\"mk\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"ml\":\"\u0D38\u0D4D\u0D2A\u0D46\u0D2F\u0D4D\u0D28\u0D4D\\u200D\",\"ceb\":\"Espanya\",\"mi\":\"P\u0101niora\",\"uk\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"eu\":\"Espainia\",\"mr\":\"\u0938\u094D\u092A\u0947\u0928\",\"ug\":\"\u0626\u0649\u0633\u067E\u0627\u0646\u0649\u064A\u06D5\",\"mt\":\"Spanja\",\"ms\":\"Sepanyol\",\"ur\":\"\u0633\u067E\u06CC\u0646\",\"fa\":\"\u0627\u0633\u067E\u0627\u0646\u06CC\u0627\",\"ty\":\"Paniora\",\"new\":\"\u0938\u094D\u092A\u0947\u0928\",\"na\":\"Pain\",\"el\":\"\u0399\u03C3\u03C0\u03B1\u03BD\u03AF\u03B1\",\"nb\":\"Spania\",\"ne\":\"\u0938\u094D\u092A\u0947\u0928\",\"vls\":\"Spanje\",\"eo\":\"Hispanio\",\"en\":\"Kingdom of Spain\",\"et\":\"Hispaania\",\"es\":\"la Madre Patria\",\"nl\":\"Spanje\",\"vec\":\"Spagna\",\"to\":\"Sepeni\",\"ca\":\"Espanya\",\"tl\":\"Espanya\",\"tr\":\"\u0130spanya\",\"tg\":\"\u0418\u0441\u043F\u043E\u043D\u0438\u0451\",\"haw\":\"Sepania\",\"bs\":\"\u0160panija\",\"br\":\"Spagn\",\"th\":\"\u0E1B\u0E23\u0E30\u0E40\u0E17\u0E28\u0E2A\u0E40\u0E1B\u0E19\",\"bn\":\"\u09B8\u09CD\u09AA\u09C7\u09A8\",\"bo\":\"\u0F66\u0F72\u0F0B\u0F54\u0F53\u0F0D\",\"ta\":\"\u0BB8\u0BCD\u0BAA\u0BC6\u0BAF\u0BBF\u0BA9\u0BCD\",\"sv\":\"Spanien\",\"bg\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"ka\":\"\u10D4\u10E1\u10DE\u10D0\u10DC\u10D4\u10D7\u10D8\",\"st\":\"Spain\",\"sw\":\"Hispania\",\"be\":\"\u0406\u0441\u043F\u0430\u043D\u0456\u044F\",\"kw\":\"Spayn\",\"sl\":\"\u0160panija\",\"sk\":\"\u0160panielsko\",\"da\":\"Spanien\",\"ang\":\"Sp\u0113onland\",\"nds\":\"Spanien\",\"ks\":\"Sp\u0113na\",\"so\":\"Isbeyn\",\"ku\":\"Spanya\",\"sr\":\"\u0428\u043F\u0430\u043D\u0438\u0458\u0430\",\"sq\":\"Spanja\",\"ko\":\"\uC5D0\uC2A4\uD30C\uB0D0\",\"sc\":\"Ispagna\",\"cy\":\"Sbaen\",\"se\":\"Esp\u00E1njja\",\"sh\":\"\u0160panija\",\"cv\":\"\u0418\u0441\u043F\u0430\u043D\u0438\",\"km\":\"\u17A2\u17C1\u179F\u17D2\u1794\u17C9\u17B6\u1789\",\"cs\":\"\u0160pan\u011Blsko\",\"li\":\"Spanje\",\"co\":\"Spagna\",\"default\":\"Spain\",\"jbo\":\"sangu\'e\",\"la\":\"Hesperia\",\"ru\":\"\u0418\u0441\u043F\u0430\u043D\u0438\u044F\",\"lb\":\"Spuenien\",\"sco\":\"Spain\",\"tet\":\"Espa\u00F1a\",\"scn\":\"Spagna\",\"hr\":\"\u0160panjolska\",\"zh\":\"\u897F\u73ED\u7259\",\"ro\":\"Spania\",\"rm\":\"Spagna\",\"ht\":\"Espay\",\"hu\":\"Spanyolorsz\u00E1g\",\"ast\":\"Espa\u00F1a\",\"hi\":\"\u0938\u094D\u092A\u0947\u0928\",\"hsb\":\"\u0160paniska\",\"nah\":\"Caxtill\u0101n\",\"war\":\"Espanya\",\"lad\":\"Espanya\",\"id\":\"Spanyol\",\"ia\":\"Espania\",\"nrm\":\"Espangne\",\"hy\":\"\u053B\u057D\u057A\u0561\u0576\u056B\u0561\",\"qu\":\"Ispa\u00F1a\",\"ilo\":\"Espania\",\"az\":\"\u0130spaniya\",\"is\":\"Sp\u00E1nn\",\"it\":\"Spagna\",\"tpi\":\"Spen\",\"ar\":\"\u0623\u0633\u0628\u0627\u0646\u064A\u0627\",\"io\":\"Hispania\",\"pam\":\"Espanya\",\"frp\":\"\u00C8spagne\",\"am\":\"\u12A5\u1235\u1353\u1295\u12EB\",\"an\":\"Espa\u00F1a\",\"csb\":\"Szpa\u0144sk\u00F4\",\"pt\":\"Espanha\",\"ja\":\"\u30B9\u30DA\u30A4\u30F3\",\"ps\":\"\u0627\u0633\u067E\u0627\u0646\u064A\u0627\",\"yi\":\"\u05E9\u05E4\u05D0\u05E0\u05D9\u05E2\",\"af\":\"Spanje\"},\"geonameFeatureCode\":\"PCLI\",\"_id\":260,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2510769,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"ca\":\"Andalusia\",\"tr\":\"End\u00FCl\u00FCs\",\"krc\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"no\":\"Andaluc\u00EDa\",\"fy\":\"Andal\u00FBsje\",\"bs\":\"Andaluzija\",\"br\":\"Andalouzia\",\"ext\":\"Andaluzia\",\"ga\":\"An Andal\u00FAis\",\"th\":\"\u0E41\u0E04\u0E27\u0E49\u0E19\u0E2D\u0E31\u0E19\u0E14\u0E32\u0E25\u0E39\u0E0B\u0E35\u0E2D\u0E32\",\"bn\":\"\u0986\u09A8\u09CD\u09A6\u09BE\u09B2\u09C1\u09B8\u09BF\u09AF\u09BC\u09BE\",\"oc\":\"Andalosia\",\"ka\":\"\u10D0\u10DC\u10D3\u10D0\u10DA\u10E3\u10E1\u10D8\u10D0\",\"sv\":\"Andalusien\",\"fr\":\"Andalousie\",\"bg\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"glk\":\"\u0622\u0646\u062F\u0627\u0644\u0648\u0633\u06CC\u0627\",\"be\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"kw\":\"Andalousi\",\"sk\":\"Andal\u00FAzia\",\"os\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\",\"da\":\"Andalusien\",\"sr\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0437\u0438\u0458\u0430\",\"ku\":\"Endulus\",\"ko\":\"\uC548\uB2EC\uB8E8\uC2DC\uC544 \uC9C0\uBC29\",\"he\":\"\u05D0\u05E0\u05D3\u05DC\u05D5\u05E1\u05D9\u05D4\",\"sh\":\"Andaluzija\",\"arz\":\"\u0627\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"cs\":\"Andalusie\",\"default\":\"Andalusia\",\"stq\":\"Andalusien\",\"la\":\"Vandalitia\",\"pl\":\"Andaluzja\",\"ru\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0438\u044F\",\"lb\":\"Andalusien\",\"tet\":\"Andaluzia\",\"got\":\"\u10345\u10330\u1033D\u10333\u10330\u1033B\u10339\u10344\u1033E\u10330\",\"hr\":\"Andaluzija\",\"zh\":\"\u5B89\u9054\u9B6F\u897F\u4E9E\",\"ro\":\"Andaluzia\",\"hu\":\"Andal\u00FAzia\",\"pms\":\"Andalus\u00ECa\",\"lv\":\"Andal\u016Bzija\",\"lt\":\"Andal\u016Bzija\",\"nah\":\"Andalucia\",\"lad\":\"Andaluziya\",\"de\":\"Andalusien\",\"als\":\"Andalusien\",\"qu\":\"Andalusiya\",\"hy\":\"\u0531\u0576\u0564\u0561\u056C\u0578\u0582\u0566\u056B\u0561\",\"eu\":\"Andaluzia\",\"is\":\"Andal\u00FAs\u00EDa\",\"uk\":\"\u0410\u043D\u0434\u0430\u043B\u0443\u0441\u0456\u044F\",\"az\":\"Andalusiya\",\"mr\":\"\u0906\u0902\u0926\u093E\u0932\u0941\u0938\u093F\u092F\u093E\",\"ug\":\"Andalusiye\",\"fa\":\"\u0627\u0646\u062F\u0644\u0633\",\"ar\":\"\u0623\u0646\u062F\u0644\u0648\u0633\u064A\u0627\",\"rmy\":\"Andalusiya\",\"io\":\"Andaluzia\",\"el\":\"\u0391\u03BD\u03B4\u03B1\u03BB\u03BF\u03C5\u03C3\u03AF\u03B1\",\"frp\":\"Andalosie\",\"pt\":\"Andaluzia\",\"eo\":\"Andaluzio\",\"en\":\"Andalusia\",\"et\":\"Andaluusia\",\"es\":\"Andaluc\u00EDa\",\"ja\":\"\u30A2\u30F3\u30C0\u30EB\u30B7\u30A2\u5DDE\",\"nl\":\"Andalusi\u00EB\",\"af\":\"Andalusi\u00EB\",\"vec\":\"Andalus\u00ECa\"},\"geonameFeatureCode\":\"ADM1\",\"_id\":261,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2593109,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"de\":\"Granada\",\"default\":\"Province of Granada\",\"fr\":\"Grenade\",\"en\":\"Province of Granada\",\"es\":\"Provincia de Granada\",\"ja\":\"\u30B0\u30E9\u30CA\u30C0\"},\"geonameFeatureCode\":\"ADM2\",\"_id\":262,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":2517115,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"A\",\"values\":{\"default\":\"Monachil\"},\"geonameFeatureCode\":\"ADM3\",\"_id\":263,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6357744,\"valueCode\":\"\"},{\"geonameFeatureClass\":\"P\",\"values\":{\"default\":\"Sierra Nevada\"},\"geonameFeatureCode\":\"PPL\",\"_id\":264,\"name\":\"\",\"auto\":true,\"type\":\"GEO\",\"geonameId\":6544329,\"value"
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (LANG-708) StringEscapeUtils.escapeEcmaScript from
lang3 cuts off long unicode string
Posted by "Henri Yandell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henri Yandell updated LANG-708:
-------------------------------
Fix Version/s: 3.x
> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
> Key: LANG-708
> URL: https://issues.apache.org/jira/browse/LANG-708
> Project: Commons Lang
> Issue Type: Bug
> Reporter: anton
> Fix For: 3.x
>
> Attachments: Test.java, input.txt
>
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
> the original string ends with:
> "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
> \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
> int sz = Character.codePointCount(input, 0, input.length());
> for (int i = 0; i < sz; i++) {
> // consumed is the number of codepoints consumed
> int consumed = translate(input, i, out);
> if(consumed == 0) {
> out.write( Character.toChars( Character.codePointAt(input, i) ) );
> }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (LANG-708) StringEscapeUtils.escapeEcmaScript from
lang3 cuts off long unicode string
Posted by "Henri Yandell (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LANG-708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henri Yandell updated LANG-708:
-------------------------------
Component/s: lang.text.translate.*
> StringEscapeUtils.escapeEcmaScript from lang3 cuts off long unicode string
> --------------------------------------------------------------------------
>
> Key: LANG-708
> URL: https://issues.apache.org/jira/browse/LANG-708
> Project: Commons Lang
> Issue Type: Bug
> Components: lang.text.translate.*
> Reporter: anton
> Fix For: 3.x
>
> Attachments: Test.java, input.txt
>
>
> Hello, I have a really big JSON string (generated from db) with unicode chars and I need to pass it though StringEscapeUtils.escapeEcmaScript(value). This string is generated and in most cases it works ok, but I have met a specific string (attached below) which is not correctly converted - few symbols (about 10) at the end of the string are cut-off (and actually they are not already unicode chars).
> the original string ends with:
> "geonameId":6544329,"valueCode":""}]
> and the produced string ends with:
> \"geonameId\":6544329,\"value
> So Code":""}] part is missing and this does not allow to parse the result as JSON on the client side.
> I have tried to debug a bit with StringEscapeUtils.escapeEcmaScript source code and is seems that the problem is somewhere around here:
> CharSequenceTranslator.translate(...){
> ...
> int sz = Character.codePointCount(input, 0, input.length());
> for (int i = 0; i < sz; i++) {
> // consumed is the number of codepoints consumed
> int consumed = translate(input, i, out);
> if(consumed == 0) {
> out.write( Character.toChars( Character.codePointAt(input, i) ) );
> }
> ...
> }
> If I put breakpoint condition to stop in the loop when i==(sz-5), I can see that the last chars of "valueCode" literal are being added to the end of "out" stream, but the counter condition ends too early to reach the end of original input String.
> So, it seems that somehow with the provided string either the sz value is calculated incorrectly or the processing loop did wrong counter adjustmes at some point.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira