[jira] [Commented] (ARROW-8719) Stream data easily- function to convert record batch to streamable format

["Wes McKinney (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ARROW-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17101183#comment-17101183 ] 

Wes McKinney commented on ARROW-8719:
-------------------------------------

Just looking at the client code

{code}
	int sock;
	sock = socket(AF_INET, SOCK_STREAM, 0);

	//setup an address
  struct sockaddr_in server_address;
	server_address.sin_family = AF_INET;
	server_address.sin_port = htons(9002);
	server_address.sin_addr.s_addr = INADDR_ANY;

	connect(sock, (struct sockaddr *) &server_address, sizeof(server_address));

	//send(sock, request, sizeof(request), 0);

	GArrowResizableBuffer *data = NULL;
	recv(sock, &data, garrow_buffer_get_size(GARROW_BUFFER(data)), 0);

	gsize data_size = sizeof(data);
	display_arrow_buffer(GARROW_BUFFER(data), data_size);
{code}

These lines won't do what you want:

{code}
GArrowResizableBuffer *data = NULL;
recv(sock, &data, garrow_buffer_get_size(GARROW_BUFFER(data)), 0);
{code}

You need to allocate memory before you use {{recv}} to write data into it. Consider using {{garrow_resizable_buffer_new}} (you will need to send a buffer size or something from the server so the client knows how much memory to allocate)

> Stream data easily- function to convert record batch to streamable format
> -------------------------------------------------------------------------
>
>                 Key: ARROW-8719
>                 URL: https://issues.apache.org/jira/browse/ARROW-8719
>             Project: Apache Arrow
>          Issue Type: Wish
>          Components: C
>         Environment: Linux
>            Reporter: Steven
>            Priority: Minor
>              Labels: buffer, data-transfer, socket
>         Attachments: code.zip
>
>
> I am currently working on a project that  need to send the data in different processes. Instead of storing it in a file, I am using the IPC stream method, storing it into a buffer. When I try to send the data(bytes) through a socket, I loose the data in the client side. ( Which gives me the idea that when i leave the function(create buffer in the server side folder), I loose acces to that data. Can there be an example that explains this. 
> Also, can you implement a function that convert the record batch into the format that can be sent through the socket directly.
>  
> The code are in the attachment, one for the server side code and the other for the client side code.
> Thank you



--
This message was sent by Atlassian Jira
(v8.3.4#803005)