[GitHub] chauthai opened a new issue #284: readAsText function causes encoding error reading large files

[GitBox <gi...@apache.org>]

chauthai opened a new issue #284: readAsText function causes encoding error reading large files
URL: https://github.com/apache/cordova-plugin-file/issues/284
 
 
   # cordova-file-plugin encoding error bug report
   ## Abstract
   The `readAsText` function of the cordova-file-plugin causes an encoding error when reading an UTF-8 file larger than 2 megabytes on platforms iOS, Android and Windows.
   
   ## Description
   The error is caused by the native implementation of the [`readAsText`](https://github.com/apache/cordova-plugin-file/blob/d135cd0e006736ea60c9dec46590de0927d57b6b/src/ios/CDVFile.m#L866) function of the `cordova-file-plugin`. Files are cut into chunks at a predefined [size](https://github.com/apache/cordova-plugin-file/blob/d135cd0e006736ea60c9dec46590de0927d57b6b/www/FileReader.js#L52) and are immediately converted to UTF-8. UTF-8 encoded characters use 4 bytes and if the cut is exactly in one character with 4 bytes, an encoding error is thrown.
   You can see the UTF-8 leading byte `e2` at the end of [data-chunk.dump file](./data-chunk.dump). The missing bytes are in the next chunk and if you convert the dump file back to text a filler character is shown (e.g. a `?`).
   
   ## Workaround
   We implemented a workaround using the [`readAsArrayBuffer`](https://github.com/chauthai/cordova-file-plugin-bug-report/blob/628b8cd65b295042ee953d6241d8d84fe5e27565/index-array-buffer.html#L39) function of the `cordova-file-plugin` and converting the typed array to UTF-8 in JavaScript with [TextDecoder.decode()](https://developer.mozilla.org/en-US/docs/Web/API/TextDecoder/decode).
   
   ## How to reproduce the `encoding error`
   1. Check out [sampe-app](https://github.com/chauthai/cordova-file-plugin-bug-report)
   1. Install dependencies
   `npm i`
   1. Initialize Cordova
   `npm run init`
   1. Deploy on platform
   `npm run ios/android/windows`
   1. Press `Fire Parser` in Demo App
   1. Watch the logs
   1. Set a breakpoint at the throw of [encoding error](https://github.com/apache/cordova-plugin-file/blob/d135cd0e006736ea60c9dec46590de0927d57b6b/src/ios/CDVFile.m#L872) in iOS
   1. Look at the `data` variable
   
   Related to #238

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cordova.apache.org
For additional commands, e-mail: commits-help@cordova.apache.org