[jira] [Commented] (ARROW-5658) [JAVA]apache arrow-flight cannot send listvector

["Liya Fan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868294#comment-16868294 ] 

Liya Fan commented on ARROW-5658:
---------------------------------

Hi [~liaotian1005] thanks a lot for reporting the problem and the detailed information.

I am not familiar with arrow-flight, but will take a look at this issue. 

> [JAVA]apache arrow-flight cannot send listvector 
> -------------------------------------------------
>
>                 Key: ARROW-5658
>                 URL: https://issues.apache.org/jira/browse/ARROW-5658
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Java
>    Affects Versions: 0.13.0
>         Environment: java8 arrow-java 0.13.0
>            Reporter: luckily
>            Priority: Blocker
>
> I can't transfer using apache arrow-flihgt. Contains listvector data. The problem description is as follows:
> {quote} # I parse an xml file and convert it to an arrow format and finally convert it to a parquet data format. The address of the .xml file data is url [http://www.w3school.com.cn/example/xmle/cd_catalog.xml|http://www.w3school.com.cn/example/xmle/cd_catalog.xml)]
>  # I created a schema that uses listvector.
> code show as below:
> List<FiledVector> list = childrenBuilder.add(ListVector.empty(column.getId().toString(),allocator));
> VectorSchemaRoot root = VectorSchemaRoot.of(inVector)
>  # Parse the xml file to get the list data in "cd". Use api use listvector.
> `ListVector listVector = (ListVector) valueVectors;
>                 List<Column> columns = column.getColumns();
>                 Column column1 = columns.get(0);
>                 String name = column1.getId().toString();
>                 UnionListWriter writer = listVector.getWriter();
>                 Writer.allocate();
>                 For (int j = 0; j < column1.getColumns().size();j++) {
>                     writer.setPosition(j);
>                     writer.startList();
>                     Writer.list().startList();
>                     Column column2 = column1.getColumns().get(j);
>                     List<Map<String, String>> lst = (List<Map<String, String>>) ((Map) val).get(name);
>                     For (int k = 0; k < lst.size(); k++) {
>                         Map<String, String> stringStringMap = lst.get(k);
>                         String value = stringStringMap.get(column2.getId().toString());
>                         Switch (column2.getType()) {
>                             Case FLOAT:
>                                 Writer.list().float4().writeFloat4(stringConvertFloat(value));
>                                 Break;
>                             Case BOOLEAN:
>                                 Writer.list().bit().writeBit(stringConvertBoolean(value));
>                                 Break;
>                             Case DECIMAL:
>                                 Writer.list().decimal().writeDecimal(stringConvertDecimal(value,column2.getScale()));
>                                 Break;
>                             Case TIMESTAMP:
>                                 Writer.list().dateMilli().writeDateMilli(stringConvertTimestamp(value,column2.format.toString()));
>                                 Break;
>                             Case INTEGER:
>                             Case BIGINT:
>                                 Writer.list().bigInt().writeBigInt(stringConvertLong(value));
>                                 Break;
>                             Case VARCHAR:
>                                 VarCharHolder varBinaryHolder = new VarCharHolder();
>                                 varBinaryHolder.start = 0;
>                                 Byte[] bytes =value.getBytes();
>                                 ArrowBuf buffer = listVector.getAllocator().buffer(bytes.length);
>                                 varBinaryHolder.buffer = buffer;
>                                 buffer.writeBytes(bytes);
>                                 varBinaryHolder.end=bytes.length;
>                                 Writer.list().varChar().write(varBinaryHolder);
>                                 Break;
>                             Default:
>                                 Throw new IllegalArgumentException(" error no type !!");
>                         }
>                     }
>                     Writer.list().endList();
>                     writer.endList();
>                 }`
>  4. 
> After the write is complete, I will send to the arrow-flight server. server code :
> {quote}
> {quote}@Override
> public Callable<Flight.PutResult> acceptPut(FlightStream flightStream) {
>  return () -> {
>  try (VectorSchemaRoot root = flightStream.getRoot()) {
>  while (flightStream.next()) {
>  VectorSchemaRoot other = null;
>  try {
>  logger.info(" Receive message ...... size: " + root.getRowCount());
>  other = copyRoot(root);
>  ArrowMessage arrowMessage = new ArrowMessage(other, other.getSchema());
>  spmc.offer(arrowMessage);
>  } catch (Exception e) {
>  logger.error(e.getMessage(), e);
>  }
>  }
>  }
>  return Flight.PutResult.parseFrom("ok".getBytes());
>  };
> }{quote}
> {quote} But the server did not receive any information.!! it is error .{quote}
> {quote}client code :{quote}
> {quote}root = message.getRoot();
> //client.close();
> FlightClient.ClientStreamListener listener =
>  client.startPut(FlightDescriptor.path(message.getFilename()), root);
> listener.putNext();
> listener.completed();
> client.close();
> listener.putNext();
> listener.completed();
> Flight.PutResult result =
>  listener.getResult();
> String s = new String(result.toByteArray());
> System.out.println(s);{quote}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)