You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Kamesh B <ka...@gmail.com> on 2015/01/27 11:27:05 UTC
Review Request 30313: Fixing Mongo join issue when * is selected
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/
-----------------------------------------------------------
Review request for drill, Jacques Nadeau and Jinfeng Ni.
Repository: drill-git
Description
-------
Fixing Mongo join issue when * is selected.
Diffs
-----
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java 4b7360057e9d95fb0176b035469bca02128a0b34
Diff: https://reviews.apache.org/r/30313/diff/
Testing
-------
Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
Thanks,
Kamesh B
Re: Review Request 30313: Fixing Mongo join issue when * is selected
Posted by Kamesh B <ka...@gmail.com>.
> On Feb. 27, 2015, 12:19 a.m., Hanifi Gunes wrote:
> > contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java, line 200
> > <https://reviews.apache.org/r/30313/diff/1/?file=836234#file836234line200>
> >
> > This does not seem quite right to me. What if multiple writes fail? Also note that this will change the order of records being written into vectors.
The reason for doing this is, if there is any issue while storing documents into value vectors, and if the vectors don't have any buffers, if fails to write it. But by that time, Mongo iterator advances its iterator. When the flow again reaches to next method, first we need to process the previously read document, otherwise, we will miss that document.
But, I think, with DRILL-1960, we may not encounter this scenario.
Will update the patch.
- Kamesh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/#review74406
-----------------------------------------------------------
On Jan. 27, 2015, 10:27 a.m., Kamesh B wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30313/
> -----------------------------------------------------------
>
> (Updated Jan. 27, 2015, 10:27 a.m.)
>
>
> Review request for drill, Jacques Nadeau and Jinfeng Ni.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Fixing Mongo join issue when * is selected.
>
>
> Diffs
> -----
>
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java 4b7360057e9d95fb0176b035469bca02128a0b34
>
> Diff: https://reviews.apache.org/r/30313/diff/
>
>
> Testing
> -------
>
> Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
>
>
> Thanks,
>
> Kamesh B
>
>
Re: Review Request 30313: Fixing Mongo join issue when * is selected
Posted by Hanifi Gunes <hg...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/#review74406
-----------------------------------------------------------
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
<https://reviews.apache.org/r/30313/#comment120980>
This does not seem quite right to me. What if multiple writes fail? Also note that this will change the order of records being written into vectors.
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
<https://reviews.apache.org/r/30313/#comment120983>
DRILL-1960 handles auto-reallocation of buffers. I don't think we will ever hit to this check due to out of memory issues. We should remove this.
- Hanifi Gunes
On Jan. 27, 2015, 10:27 a.m., Kamesh B wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30313/
> -----------------------------------------------------------
>
> (Updated Jan. 27, 2015, 10:27 a.m.)
>
>
> Review request for drill, Jacques Nadeau and Jinfeng Ni.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Fixing Mongo join issue when * is selected.
>
>
> Diffs
> -----
>
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java 4b7360057e9d95fb0176b035469bca02128a0b34
>
> Diff: https://reviews.apache.org/r/30313/diff/
>
>
> Testing
> -------
>
> Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
>
>
> Thanks,
>
> Kamesh B
>
>
Re: Review Request 30313: Fixing Mongo join issue when * is selected
Posted by Mariano Ruiz <mr...@gmail.com>.
> On Marzo 3, 2015, 3:10 p.m., Mariano Ruiz wrote:
> > This patch solve the issue #DRILL-2352 reported by me. Thanks!
Ups, I made some extra tests, and found some errors. I not sure if they are related to this patch. Please check my comments in the bug reported [here](https://issues.apache.org/jira/browse/DRILL-2352?focusedCommentId=14345256&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14345256).
- Mariano
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/#review74955
-----------------------------------------------------------
On Marzo 3, 2015, 3:54 p.m., Kamesh B wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30313/
> -----------------------------------------------------------
>
> (Updated Marzo 3, 2015, 3:54 p.m.)
>
>
> Review request for drill, Jacques Nadeau and Jinfeng Ni.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Fixing Mongo join issue when * is selected.
>
>
> Diffs
> -----
>
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da9670154e38b6248e826b03dd71ec396ddef5f4
>
> Diff: https://reviews.apache.org/r/30313/diff/
>
>
> Testing
> -------
>
> Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
>
>
> Thanks,
>
> Kamesh B
>
>
Re: Review Request 30313: Fixing Mongo join issue when * is selected
Posted by Mariano Ruiz <mr...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/#review74955
-----------------------------------------------------------
This patch solve the issue #DRILL-2352 reported by me. Thanks!
- Mariano Ruiz
On Feb. 28, 2015, 3:15 p.m., Kamesh B wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30313/
> -----------------------------------------------------------
>
> (Updated Feb. 28, 2015, 3:15 p.m.)
>
>
> Review request for drill, Jacques Nadeau and Jinfeng Ni.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Fixing Mongo join issue when * is selected.
>
>
> Diffs
> -----
>
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da9670154e38b6248e826b03dd71ec396ddef5f4
>
> Diff: https://reviews.apache.org/r/30313/diff/
>
>
> Testing
> -------
>
> Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
>
>
> Thanks,
>
> Kamesh B
>
>
Re: Review Request 30313: Fixing Mongo join issue when * is selected
Posted by Hanifi Gunes <hg...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/#review75019
-----------------------------------------------------------
Ship it!
- Hanifi Gunes
On March 3, 2015, 3:54 p.m., Kamesh B wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30313/
> -----------------------------------------------------------
>
> (Updated March 3, 2015, 3:54 p.m.)
>
>
> Review request for drill, Jacques Nadeau and Jinfeng Ni.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Fixing Mongo join issue when * is selected.
>
>
> Diffs
> -----
>
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da9670154e38b6248e826b03dd71ec396ddef5f4
>
> Diff: https://reviews.apache.org/r/30313/diff/
>
>
> Testing
> -------
>
> Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
>
>
> Thanks,
>
> Kamesh B
>
>
Re: Review Request 30313: Fixing Mongo join issue when * is selected
Posted by Kamesh B <ka...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/
-----------------------------------------------------------
(Updated March 4, 2015, 2:11 a.m.)
Review request for drill, Jacques Nadeau and Jinfeng Ni.
Changes
-------
Attaching patch generated using git format-patch
Repository: drill-git
Description
-------
Fixing Mongo join issue when * is selected.
Diffs (updated)
-----
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da96701
Diff: https://reviews.apache.org/r/30313/diff/
Testing
-------
Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
Thanks,
Kamesh B
Re: Review Request 30313: Fixing Mongo join issue when * is selected
Posted by Kamesh B <ka...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/
-----------------------------------------------------------
(Updated March 3, 2015, 3:54 p.m.)
Review request for drill, Jacques Nadeau and Jinfeng Ni.
Changes
-------
Fixing review comments
Repository: drill-git
Description
-------
Fixing Mongo join issue when * is selected.
Diffs (updated)
-----
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da9670154e38b6248e826b03dd71ec396ddef5f4
Diff: https://reviews.apache.org/r/30313/diff/
Testing
-------
Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
Thanks,
Kamesh B
Re: Review Request 30313: Fixing Mongo join issue when * is selected
Posted by Hanifi Gunes <hg...@maprtech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/#review74825
-----------------------------------------------------------
Overall looks good. We can go ahead and merge this in once we address these issues.
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
<https://reviews.apache.org/r/30313/#comment121609>
this is unneeded. see #setColumns.
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
<https://reviews.apache.org/r/30313/#comment121611>
As a minor comment, I think the old way of externalizing the doc string was cleaner here.
- Hanifi Gunes
On Feb. 28, 2015, 3:15 p.m., Kamesh B wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30313/
> -----------------------------------------------------------
>
> (Updated Feb. 28, 2015, 3:15 p.m.)
>
>
> Review request for drill, Jacques Nadeau and Jinfeng Ni.
>
>
> Repository: drill-git
>
>
> Description
> -------
>
> Fixing Mongo join issue when * is selected.
>
>
> Diffs
> -----
>
> contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da9670154e38b6248e826b03dd71ec396ddef5f4
>
> Diff: https://reviews.apache.org/r/30313/diff/
>
>
> Testing
> -------
>
> Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
>
>
> Thanks,
>
> Kamesh B
>
>
Re: Review Request 30313: Fixing Mongo join issue when * is selected
Posted by Kamesh B <ka...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30313/
-----------------------------------------------------------
(Updated Feb. 28, 2015, 3:15 p.m.)
Review request for drill, Jacques Nadeau and Jinfeng Ni.
Repository: drill-git
Description
-------
Fixing Mongo join issue when * is selected.
Diffs (updated)
-----
contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java da9670154e38b6248e826b03dd71ec396ddef5f4
Diff: https://reviews.apache.org/r/30313/diff/
Testing
-------
Tested the patch by selecting * in join operation. It is working fine. This patch also fixes some of issues caused by earlier commits.
Thanks,
Kamesh B