You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Rahul Challapalli (JIRA)" <ji...@apache.org> on 2015/04/07 20:57:14 UTC
[jira] [Updated] (DRILL-2707) Projecting a required varchar column
after a Full Outer Join results in an IOOBException
[ https://issues.apache.org/jira/browse/DRILL-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Rahul Challapalli updated DRILL-2707:
-------------------------------------
Component/s: (was: Storage - Parquet)
Execution - Data Types
Description:
git.commit.id.abbrev=a53e123
I tried to project a required varchar column after a FOJ. Below is what I see
{code}
0: jdbc:drill:schema=dfs_eea> select
. . . . . . . . . . . . . . > p.varchar_col
. . . . . . . . . . . . . . > from dfs.`cross-sources`.`fewtypes.parquet` p
. . . . . . . . . . . . . . > full outer join dfs.`cross-sources`.`fewtypes_null.json` o
. . . . . . . . . . . . . . > on p.int_col=o.int_col;
+-------------+
| varchar_col |
+-------------+
java.lang.IndexOutOfBoundsException: index: 180, length: 10 (expected: range(0, 180))
at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1143)
at io.netty.buffer.PooledUnsafeDirectByteBuf.getBytes(PooledUnsafeDirectByteBuf.java:136)
at io.netty.buffer.WrappedByteBuf.getBytes(WrappedByteBuf.java:289)
at io.netty.buffer.UnsafeDirectLittleEndian.getBytes(UnsafeDirectLittleEndian.java:25)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at org.apache.drill.exec.vector.VarCharVector$Accessor.get(VarCharVector.java:387)
at org.apache.drill.exec.vector.VarCharVector$Accessor.getObject(VarCharVector.java:411)
at org.apache.drill.exec.vector.accessor.VarCharAccessor.getObject(VarCharAccessor.java:108)
at org.apache.drill.exec.vector.accessor.BoundCheckingAccessor.getObject(BoundCheckingAccessor.java:137)
at org.apache.drill.jdbc.AvaticaDrillSqlAccessor.getObject(AvaticaDrillSqlAccessor.java:165)
at net.hydromatic.avatica.AvaticaResultSet.getObject(AvaticaResultSet.java:351)
at sqlline.SqlLine$Rows$Row.<init>(SqlLine.java:2388)
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2504)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Not sure if this is a client-specific issue as there is no exception from the drillbit log files
However if I project a varchar column (nullable) from a json file after a FOJ, there seems to be no issues
{code}
0: jdbc:drill:schema=dfs_eea> select
. . . . . . . . . . . . . . > o.varchar_col
. . . . . . . . . . . . . . > from dfs.`cross-sources`.`fewtypes.parquet` p
. . . . . . . . . . . . . . > full outer join dfs.`cross-sources`.`fewtypes_null.json` o
. . . . . . . . . . . . . . > on p.int_col=o.int_col;
+-------------+
| varchar_col |
+-------------+
| jllkjsdhfg |
| null |
| gfdstweopiu |
| gjklhsdfgkjhkASDF |
| oieoiutriotureWERTgwgEWRg |
| gjkdfkjglfd |
| ioerutklsdfASDgerGWEr |
| lkjgfiurtoUYFHfahui |
| IOUfiuodsfIUfjkh |
| iweuoHUIhUwer |
| null |
| dfgoiuert |
| uitreo |
| uigoMnvjjkdf |
| NvvdfHVG |
| null |
| null |
| uiuikjk |
| null |
| hjiwgh |
| null |
| jhgduitweriuoert |
| KfijUIwre |
| Nhkhuivb |
| null |
| null |
+-------------+
26 rows selected (0.212 seconds)
{code}
I attached the parquet and json files used. Let me know if you need anything more.
was:
git.commit.id.abbrev=a53e123
I did a full outer join between a parquet file and a json file and projected a varchar column. Below is what I see
{code}
0: jdbc:drill:schema=dfs_eea> select
. . . . . . . . . . . . . . > p.varchar_col
. . . . . . . . . . . . . . > from dfs.`cross-sources`.`fewtypes.parquet` p
. . . . . . . . . . . . . . > full outer join dfs.`cross-sources`.`fewtypes_null.json` o
. . . . . . . . . . . . . . > on p.int_col=o.int_col;
+-------------+
| varchar_col |
+-------------+
java.lang.IndexOutOfBoundsException: index: 180, length: 10 (expected: range(0, 180))
at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1143)
at io.netty.buffer.PooledUnsafeDirectByteBuf.getBytes(PooledUnsafeDirectByteBuf.java:136)
at io.netty.buffer.WrappedByteBuf.getBytes(WrappedByteBuf.java:289)
at io.netty.buffer.UnsafeDirectLittleEndian.getBytes(UnsafeDirectLittleEndian.java:25)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at org.apache.drill.exec.vector.VarCharVector$Accessor.get(VarCharVector.java:387)
at org.apache.drill.exec.vector.VarCharVector$Accessor.getObject(VarCharVector.java:411)
at org.apache.drill.exec.vector.accessor.VarCharAccessor.getObject(VarCharAccessor.java:108)
at org.apache.drill.exec.vector.accessor.BoundCheckingAccessor.getObject(BoundCheckingAccessor.java:137)
at org.apache.drill.jdbc.AvaticaDrillSqlAccessor.getObject(AvaticaDrillSqlAccessor.java:165)
at net.hydromatic.avatica.AvaticaResultSet.getObject(AvaticaResultSet.java:351)
at sqlline.SqlLine$Rows$Row.<init>(SqlLine.java:2388)
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2504)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Not sure if this is a client-specific issue as there is no exception from the drillbit log files
However if I project a varchar column from a json file after a FOJ, there seems to be no issues
{code}
0: jdbc:drill:schema=dfs_eea> select
. . . . . . . . . . . . . . > o.varchar_col
. . . . . . . . . . . . . . > from dfs.`cross-sources`.`fewtypes.parquet` p
. . . . . . . . . . . . . . > full outer join dfs.`cross-sources`.`fewtypes_null.json` o
. . . . . . . . . . . . . . > on p.int_col=o.int_col;
+-------------+
| varchar_col |
+-------------+
| jllkjsdhfg |
| null |
| gfdstweopiu |
| gjklhsdfgkjhkASDF |
| oieoiutriotureWERTgwgEWRg |
| gjkdfkjglfd |
| ioerutklsdfASDgerGWEr |
| lkjgfiurtoUYFHfahui |
| IOUfiuodsfIUfjkh |
| iweuoHUIhUwer |
| null |
| dfgoiuert |
| uitreo |
| uigoMnvjjkdf |
| NvvdfHVG |
| null |
| null |
| uiuikjk |
| null |
| hjiwgh |
| null |
| jhgduitweriuoert |
| KfijUIwre |
| Nhkhuivb |
| null |
| null |
+-------------+
26 rows selected (0.212 seconds)
{code}
I attached the parquet and json files used. Let me know if you need anything more.
Priority: Major (was: Critical)
Summary: Projecting a required varchar column after a Full Outer Join results in an IOOBException (was: Projecting a varchar column from a parquet file after a full outer join with a json file results in IOOBException)
> Projecting a required varchar column after a Full Outer Join results in an IOOBException
> -----------------------------------------------------------------------------------------
>
> Key: DRILL-2707
> URL: https://issues.apache.org/jira/browse/DRILL-2707
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types, Execution - Relational Operators
> Reporter: Rahul Challapalli
> Assignee: Chris Westin
> Attachments: fewtypes.parquet, fewtypes_null.json
>
>
> git.commit.id.abbrev=a53e123
> I tried to project a required varchar column after a FOJ. Below is what I see
> {code}
> 0: jdbc:drill:schema=dfs_eea> select
> . . . . . . . . . . . . . . > p.varchar_col
> . . . . . . . . . . . . . . > from dfs.`cross-sources`.`fewtypes.parquet` p
> . . . . . . . . . . . . . . > full outer join dfs.`cross-sources`.`fewtypes_null.json` o
> . . . . . . . . . . . . . . > on p.int_col=o.int_col;
> +-------------+
> | varchar_col |
> +-------------+
> java.lang.IndexOutOfBoundsException: index: 180, length: 10 (expected: range(0, 180))
> at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1143)
> at io.netty.buffer.PooledUnsafeDirectByteBuf.getBytes(PooledUnsafeDirectByteBuf.java:136)
> at io.netty.buffer.WrappedByteBuf.getBytes(WrappedByteBuf.java:289)
> at io.netty.buffer.UnsafeDirectLittleEndian.getBytes(UnsafeDirectLittleEndian.java:25)
> at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
> at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
> at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
> at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
> at org.apache.drill.exec.vector.VarCharVector$Accessor.get(VarCharVector.java:387)
> at org.apache.drill.exec.vector.VarCharVector$Accessor.getObject(VarCharVector.java:411)
> at org.apache.drill.exec.vector.accessor.VarCharAccessor.getObject(VarCharAccessor.java:108)
> at org.apache.drill.exec.vector.accessor.BoundCheckingAccessor.getObject(BoundCheckingAccessor.java:137)
> at org.apache.drill.jdbc.AvaticaDrillSqlAccessor.getObject(AvaticaDrillSqlAccessor.java:165)
> at net.hydromatic.avatica.AvaticaResultSet.getObject(AvaticaResultSet.java:351)
> at sqlline.SqlLine$Rows$Row.<init>(SqlLine.java:2388)
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2504)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
> Not sure if this is a client-specific issue as there is no exception from the drillbit log files
> However if I project a varchar column (nullable) from a json file after a FOJ, there seems to be no issues
> {code}
> 0: jdbc:drill:schema=dfs_eea> select
> . . . . . . . . . . . . . . > o.varchar_col
> . . . . . . . . . . . . . . > from dfs.`cross-sources`.`fewtypes.parquet` p
> . . . . . . . . . . . . . . > full outer join dfs.`cross-sources`.`fewtypes_null.json` o
> . . . . . . . . . . . . . . > on p.int_col=o.int_col;
> +-------------+
> | varchar_col |
> +-------------+
> | jllkjsdhfg |
> | null |
> | gfdstweopiu |
> | gjklhsdfgkjhkASDF |
> | oieoiutriotureWERTgwgEWRg |
> | gjkdfkjglfd |
> | ioerutklsdfASDgerGWEr |
> | lkjgfiurtoUYFHfahui |
> | IOUfiuodsfIUfjkh |
> | iweuoHUIhUwer |
> | null |
> | dfgoiuert |
> | uitreo |
> | uigoMnvjjkdf |
> | NvvdfHVG |
> | null |
> | null |
> | uiuikjk |
> | null |
> | hjiwgh |
> | null |
> | jhgduitweriuoert |
> | KfijUIwre |
> | Nhkhuivb |
> | null |
> | null |
> +-------------+
> 26 rows selected (0.212 seconds)
> {code}
> I attached the parquet and json files used. Let me know if you need anything more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)