You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tajo.apache.org by jinossy <gi...@git.apache.org> on 2014/11/03 15:29:41 UTC
[GitHub] tajo pull request: TAJO-1152: RawFile ByteBuffer should be reuse
GitHub user jinossy opened a pull request:
https://github.com/apache/tajo/pull/224
TAJO-1152: RawFile ByteBuffer should be reuse
This patch depend on TAJO-1149.
I've refactor the EOF checking
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jinossy/tajo TAJO-1152
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/tajo/pull/224.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #224
----
commit aed406357e016e602b11dbaed3d20994dd651a17
Author: jhkim <jh...@apache.org>
Date: 2014-10-31T12:10:08Z
TAJO-1149: Implement direct read of TextFile scanner
commit f95d8dc0ef04dfb82290e23d5430eefc70f2552d
Author: jhkim <jh...@apache.org>
Date: 2014-10-31T13:42:47Z
remove the past buffer reading
commit f87e56f942b523f2bd2675d2c75484d3fea6023e
Author: jhkim <jh...@apache.org>
Date: 2014-11-01T14:47:56Z
refactor the compact buffer
commit bd15ea07a0187ff1d76bb0ba887bf899a281aa07
Author: jhkim <jh...@apache.org>
Date: 2014-11-03T08:03:30Z
rename LineDelimitedReader to DelimitedLineReader
commit 166e913be69d8ee7944d67c3fd82f015762d82b1
Author: jhkim <jh...@apache.org>
Date: 2014-11-03T09:36:38Z
rename csvfile.delimiter, csvfile.null to text.delimiter, text.null
commit 48962d005cc39d2016b864e07d0f985eb31ed69f
Author: jhkim <jh...@apache.org>
Date: 2014-11-03T14:22:01Z
TAJO-1152: RawFile ByteBuffer should be reuse
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1152: RawFile ByteBuffer should be reuse
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/tajo/pull/224
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1152: RawFile ByteBuffer should be reuse
Posted by hyunsik <gi...@git.apache.org>.
Github user hyunsik commented on the pull request:
https://github.com/apache/tajo/pull/224#issuecomment-62530492
+1
The patch looks good to me. This work will be helpful to efficiently manage direct memory in JVM. Could you add one comment to the source before committing the patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] tajo pull request: TAJO-1152: RawFile ByteBuffer should be reuse
Posted by hyunsik <gi...@git.apache.org>.
Github user hyunsik commented on a diff in the pull request:
https://github.com/apache/tajo/pull/224#discussion_r20142448
--- Diff: tajo-storage/src/main/java/org/apache/tajo/storage/RawFile.java ---
@@ -109,56 +106,50 @@ public void init() throws IOException {
// initial read
if (fragment.getStartKey() > 0) {
- channel.position(fragment.getStartKey());
+ channel.position(fragment.getStartKey());
}
- numBytesRead = channel.read(buffer);
- buffer.flip();
+ forceFillBuffer = true;
super.init();
}
@Override
public long getNextOffset() throws IOException {
- return channel.position() - buffer.remaining();
+ return filePosition - (forceFillBuffer ? 0 : buffer.remaining());
}
@Override
public void seek(long offset) throws IOException {
- long currentPos = channel.position();
- if(currentPos < offset && offset < currentPos + buffer.limit()){
- buffer.position((int)(offset - currentPos));
+ eos = false;
+ filePosition = channel.position();
+
--- End diff --
Could you add the comment in this line?
```
// do not fill the buffer if the offset is already included in the buffer.
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---