[GitHub] [tvm] apeskov opened a new pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov opened a new pull request #7876:
URL: https://github.com/apache/tvm/pull/7876


   This patch introduces tracker support for iOS roc application. That is mandatory functionality for auto tuning process for iOS platform. 
   
   This is a port of app/cpp_rpc sources.
   
   Also Fontaine's
   * Minor improvements for UX (proper keyboard events handling, cache text fields values)
   * Changes in build process 
     - Use tvm_runtime as external module (allow to avoid source inclusion)
     - Search TVM sources and modules via build variables TVM_ROOT and TVM_BUILD_DIR
     - Disable 32 bit build.
     - Add verification build step to check compatibility with TVM prebuilt artefacts


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-925259356


   and thanks @echuraev !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-919687971


   @areusch Could you please take a look on this PR one again? I rewrote README for ios_rpc and described all modes and what is the difference between PureRPC, Tracker and Proxy.
   
   > @apeskov sorry i think the thing i'm confused about is: what functionality does RPCServerPure add over the existing tracker (or: why was the existing tracker not able to be run over usbmux)? perhaps it would be helpful to see an e.g. README.md explaining how to use RPCServerPure. Does it e.g. allow you to start/stop the tracker or RPC server? sorry my obj-c/iOS is not super great so I keep trying to review this but going in circles.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r652185799



##########
File path: apps/ios_rpc/tvmrpc/RPCArgs.mm
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#import "RPCArgs.h"
+
+#import <Foundation/Foundation.h>
+
+#import "../../../src/support/socket.h"
+#import "../../../src/support/utils.h"
+
+#import <string>
+
+using std::string;
+
+const char* kUsage =
+    "\n"
+    "iOS tvmrpc application supported flags:\n"
+    "--host_url      The tracker/proxy address, Default=0.0.0.0\n"
+    "--host_port     The tracker/proxy port, Default=9190\n"
+    "--key           The key used to identify the device type in tracker. Default=\"\"\n"
+    "--custom_addr   Custom IP Address to Report to RPC Tracker. Default=\"\"\n"
+    "--immediate_connect   No UI interconnection, connect to tracker immediately. Default=False\n"
+    "--verbose       Allow to print status info to std out. Default=False\n"
+    "--server_mode   Server mode. Can be \"pure_server\", \"proxy\" or \"tracker\". "
+    "Default=pure_server \n"
+    "\n";
+
+struct RPCArgs_cpp {
+  string host_url = "0.0.0.0";
+  int host_port = 9190;
+
+  string key;
+  string custom_addr = "";
+
+  bool immediate_connect = false;
+  bool verbose = false;
+  char server_mode = 0;
+
+  operator RPCArgs() const {
+    return RPCArgs{.host_url = host_url.c_str(),
+                   .host_port = host_port,
+                   .key = key.c_str(),
+                   .custom_addr = custom_addr.c_str(),
+                   .verbose = verbose,
+                   .immediate_connect = immediate_connect,
+                   .server_mode = server_mode};
+  };
+
+  RPCArgs_cpp& operator=(const RPCArgs& args) {
+    host_url = args.host_url;
+    host_port = args.host_port;
+    key = args.key;
+    custom_addr = args.custom_addr;
+    verbose = args.verbose;
+    immediate_connect = args.immediate_connect;
+    server_mode = args.server_mode;
+    return *this;
+  }
+};
+
+struct RPCArgs_cpp g_rpc_args;
+
+static void restore_from_cache() {
+  NSUserDefaults* defaults = [NSUserDefaults standardUserDefaults];
+
+  auto get_string_from_cache = [defaults](const char* key) {
+    NSString* ns_key = [NSString stringWithUTF8String:key];
+    NSString* ns_val = [defaults stringForKey:ns_key];
+    return std::string(ns_val != nil ? [ns_val UTF8String] : "");
+  };
+
+  auto get_int_from_cache = [defaults](const char* key) {
+    NSString* ns_key = [NSString stringWithUTF8String:key];
+    return static_cast<int>([defaults integerForKey:ns_key]);
+  };
+
+  g_rpc_args.host_url = get_string_from_cache("tmvrpc_url");
+  g_rpc_args.host_port = get_int_from_cache("tmvrpc_port");
+  g_rpc_args.key = get_string_from_cache("tmvrpc_key");
+}
+
+static void update_in_cache() {
+  NSUserDefaults* defaults = [NSUserDefaults standardUserDefaults];
+
+  [defaults setObject:[NSString stringWithUTF8String:g_rpc_args.host_url.c_str()]
+               forKey:@"tmvrpc_url"];
+  [defaults setInteger:g_rpc_args.host_port forKey:@"tmvrpc_port"];
+  [defaults setObject:[NSString stringWithUTF8String:g_rpc_args.key.c_str()] forKey:@"tmvrpc_key"];

Review comment:
       nit: tvm, here and above

##########
File path: apps/ios_rpc/tvmrpc/RPCServer.mm
##########
@@ -0,0 +1,809 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file ViewController.mm
+ */
+
+#import "RPCServer.h"
+
+#include <tvm/runtime/packed_func.h>
+#include <tvm/runtime/registry.h>
+
+#include <random>
+#include <string>
+
+// To get device WiFi IP
+#include <arpa/inet.h>
+#include <ifaddrs.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+
+// TVM internal header to access Magic keys like kRPCMagic and others
+#include "../../../src/runtime/rpc/rpc_endpoint.h"
+
+namespace tvm {
+namespace runtime {
+
+/*!
+ * \brief Message handling function for event driven server.
+ *
+ * \param in_bytes The incoming bytes.
+ * \param event_flag  1: read_available, 2: write_avaiable.
+ * \return State flag.
+ *     1: continue running, no need to write,
+ *     2: need to write
+ *     0: shutdown
+ */
+using FEventHandler = PackedFunc;
+
+/*!
+ * \brief Create a server event handler.
+ *
+ * \param outputStream The output stream used to send outputs.
+ * \param name The name of the server.
+ * \param remote_key The remote key
+ * \return The event handler.
+ */
+FEventHandler CreateServerEventHandler(NSOutputStream* outputStream, std::string name,
+                                       std::string remote_key) {
+  const PackedFunc* event_handler_factor = Registry::Get("rpc.CreateEventDrivenServer");
+  ICHECK(event_handler_factor != nullptr)
+      << "You are using tvm_runtime module built without RPC support. "
+      << "Please rebuild it with USE_RPC flag.";
+
+  PackedFunc writer_func([outputStream](TVMArgs args, TVMRetValue* rv) {
+    TVMByteArray* data = args[0].ptr<TVMByteArray>();
+    int64_t nbytes = [outputStream write:reinterpret_cast<const uint8_t*>(data->data)
+                               maxLength:data->size];
+    if (nbytes < 0) {
+      NSLog(@"%@", [outputStream streamError].localizedDescription);
+      throw tvm::Error("Stream error");
+    }
+    *rv = nbytes;
+  });
+
+  return (*event_handler_factor)(writer_func, name, remote_key);
+}
+
+/*!
+ * \brief Helper function to query real IP of device in WiFi network
+ * \return string with IPv4 in format "192.168.0.1" or "unknown" if cannot detect
+ */
+static std::string getWiFiAddress() {
+  std::string address = "unknown";
+  ifaddrs* interfaces = nullptr;
+
+  int success = getifaddrs(&interfaces);
+  if (success == 0) {
+    ifaddrs* temp_addr = interfaces;
+    while (temp_addr != NULL) {
+      if (temp_addr->ifa_addr->sa_family == AF_INET) {
+        // Check if interface is en0 which is the wifi connection on the iPhone
+        if (std::string(temp_addr->ifa_name) == "en0") {
+          address = inet_ntoa(((sockaddr_in*)temp_addr->ifa_addr)->sin_addr);
+        }
+      }
+      temp_addr = temp_addr->ifa_next;
+    }
+  }
+
+  freeifaddrs(interfaces);
+  return address;
+}
+
+}  // namespace runtime
+}  // namespace tvm
+
+// Base class for any type of RPC servicing
+@interface RPCServerBase : RPCServer
+
+/*!
+ * Methods should be implemented in inherited classes
+ */
+- (bool)onReadHandler;   // return true - continue feeding, false - stop, try to drain output buffer
+- (bool)onWriteHandler;  // return true - continue draining, false - no data to write
+- (void)onEndEncountered;  // called on disconnect or session desided that it's shutdown time
+- (void)open;              // Initiate listening objects like i/o streams and other resources
+- (void)close;             // Deinitialize resources opend in "open" method
+@end
+
+@implementation RPCServerBase {
+  // Worker thread
+  NSThread* worker_thread_;
+  // Triger to continue RunLoop processing inside worker_thread_
+  BOOL shouldKeepRunning;
+  // Input socket stream
+ @protected
+  NSInputStream* inputStream_;
+  // Output socket stream
+  NSOutputStream* outputStream_;
+  // Temporal buffer with data to send
+  std::string sendBuffer_;
+  // Temporal receive buffer
+  std::string recvBuffer_;
+  // Requested data size to accumulate in recvBuffer_ before continue processing
+  int requiredToRecv_;
+}
+
+/*!
+ * Start internal worker thread with RunLoop and submit correspoding open handlers into it
+ * Not blocking
+ */
+- (void)start {
+  worker_thread_ = [[NSThread alloc] initWithBlock:^{
+    @autoreleasepool {
+      [self notifyState:RPCServerStatus_Launched];
+      [self open];
+      shouldKeepRunning = YES;
+      while (shouldKeepRunning && [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
+                                                           beforeDate:[NSDate distantFuture]])
+        ;
+      [self notifyState:RPCServerStatus_Stopped];
+    }
+  }];
+  [worker_thread_ start];
+}
+
+/*!
+ * Send message to workel thread runloop to finish processing
+ * Not blocking
+ */
+- (void)stop {
+  if (worker_thread_ == nil) return;
+
+  [self performSelector:@selector(stop_) onThread:worker_thread_ withObject:nil waitUntilDone:NO];
+  worker_thread_ = nil;  // TODO: is it valide? may be better to do that inside NSThread?
+}
+
+- (void)stop_ {
+  [self close];
+  shouldKeepRunning = NO;
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will connect to host and port specified in corresponding properties
+ */
+- (void)open {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)self.host, self.port, &readStream,
+                                     &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will assign i/o streams to provided socket connection.
+ */
+- (void)openWithSocket:(CFSocketNativeHandle)sock {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocket(NULL, sock, &readStream, &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Close i/o streams assosiated with connection
+ */
+- (void)close {
+  [inputStream_ close];
+  [outputStream_ close];
+  [inputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [inputStream_ setDelegate:nil];
+  [outputStream_ setDelegate:nil];
+  inputStream_ = nil;
+  outputStream_ = nil;
+}
+
+/// Unimplemented stubs
+- (bool)onReadHandler {
+  return false;
+}
+- (bool)onWriteHandler {
+  return false;
+}
+- (void)onEndEncountered {
+}
+
+/*!
+ * Try to read data from stream and call processing hadnler
+ */
+- (void)tryToRead {
+  const int kBufferSize = 4 << 10;  // 4kB buffer
+  const int prev_size = recvBuffer_.size();
+  recvBuffer_.resize(kBufferSize);
+  size_t nbytes = [inputStream_ read:(uint8_t*)recvBuffer_.data() + prev_size
+                           maxLength:recvBuffer_.size() - prev_size];
+  recvBuffer_.resize(nbytes + prev_size);
+
+  // feed while it accept or requested particulat buffer size
+  while (!recvBuffer_.empty() && requiredToRecv_ <= recvBuffer_.size() && [self onReadHandler])
+    ;
+}
+
+/*!
+ * Try to write remaining data to stream and call processing hadnler
+ */
+- (void)tryToWrite {
+  if (!sendBuffer_.empty()) {
+    size_t nbytes = [outputStream_ write:(uint8_t*)sendBuffer_.data() maxLength:sendBuffer_.size()];
+    sendBuffer_.erase(0, nbytes);
+  }
+  // call write handler while it want be called and space is available
+  while (sendBuffer_.empty() && [outputStream_ hasSpaceAvailable] && [self onWriteHandler])
+    ;
+}
+
+/*!
+ * Main event handler of socket stream events
+ */
+- (void)stream:(NSStream*)strm handleEvent:(NSStreamEvent)event {
+  std::string buffer;
+  switch (event) {
+    case NSStreamEventOpenCompleted: {
+      // Nothing
+      break;
+    }
+    case NSStreamEventHasBytesAvailable:
+      if (strm == inputStream_) {
+        [self tryToRead];
+        if ([outputStream_ hasSpaceAvailable]) [self tryToWrite];
+      }
+      break;
+    case NSStreamEventHasSpaceAvailable: {
+      if (strm == outputStream_) {
+        [self tryToWrite];
+        if ([inputStream_ hasBytesAvailable]) [self tryToRead];
+      }
+      break;
+    }
+    case NSStreamEventErrorOccurred: {
+      [self notifyError:[strm streamError].localizedDescription];
+      break;
+    }
+    case NSStreamEventEndEncountered: {
+      [self onEndEncountered];
+      break;
+    }
+    default: {
+      NSLog(@"Unknown event");
+    }
+  }
+}
+
+#pragma mark - Helpers
+
+/*!
+ * Set buffer to send into stream. Try to send immediatly or submit to lazy sending
+ * Non blocking operation
+ */
+- (void)toSend:(NSData*)data {
+  sendBuffer_.append(static_cast<const char*>(data.bytes), data.length);
+
+  // try to flush buffer
+  NSInteger sent_size = [outputStream_ write:(uint8_t*)sendBuffer_.data()
+                                   maxLength:sendBuffer_.size()];
+  sendBuffer_.erase(0, sent_size);
+}
+
+/*!
+ * Set buffer to send  in packet format [size, data]. Behaviour is same as for toSend.
+ */
+- (void)toSendPacked:(NSData*)data {
+  int packet_size = data.length;
+  [self toSend:[NSData dataWithBytes:&packet_size length:sizeof(packet_size)]];
+  [self toSend:data];
+}
+
+/*!
+ */
+- (NSData*)requestInputDataWithSize:(NSInteger)size {
+  if (recvBuffer_.size() < size) {
+    requiredToRecv_ = size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() length:size];
+  recvBuffer_.erase(0, size);
+  return res;
+}
+
+/*!
+ */
+- (NSData*)requestInputDataPacked {
+  int size;

Review comment:
       prefer to use the inttypes when taking sizeof() to compute payload lengths for wire data

##########
File path: apps/ios_rpc/tvmrpc/RPCArgs.mm
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#import "RPCArgs.h"
+
+#import <Foundation/Foundation.h>
+
+#import "../../../src/support/socket.h"
+#import "../../../src/support/utils.h"
+
+#import <string>
+
+using std::string;
+
+const char* kUsage =
+    "\n"
+    "iOS tvmrpc application supported flags:\n"
+    "--host_url      The tracker/proxy address, Default=0.0.0.0\n"
+    "--host_port     The tracker/proxy port, Default=9190\n"
+    "--key           The key used to identify the device type in tracker. Default=\"\"\n"
+    "--custom_addr   Custom IP Address to Report to RPC Tracker. Default=\"\"\n"
+    "--immediate_connect   No UI interconnection, connect to tracker immediately. Default=False\n"
+    "--verbose       Allow to print status info to std out. Default=False\n"
+    "--server_mode   Server mode. Can be \"pure_server\", \"proxy\" or \"tracker\". "
+    "Default=pure_server \n"
+    "\n";
+
+struct RPCArgs_cpp {
+  string host_url = "0.0.0.0";
+  int host_port = 9190;
+
+  string key;
+  string custom_addr = "";
+
+  bool immediate_connect = false;
+  bool verbose = false;
+  char server_mode = 0;
+
+  operator RPCArgs() const {
+    return RPCArgs{.host_url = host_url.c_str(),
+                   .host_port = host_port,
+                   .key = key.c_str(),
+                   .custom_addr = custom_addr.c_str(),
+                   .verbose = verbose,
+                   .immediate_connect = immediate_connect,
+                   .server_mode = server_mode};
+  };
+
+  RPCArgs_cpp& operator=(const RPCArgs& args) {
+    host_url = args.host_url;
+    host_port = args.host_port;
+    key = args.key;
+    custom_addr = args.custom_addr;
+    verbose = args.verbose;
+    immediate_connect = args.immediate_connect;
+    server_mode = args.server_mode;
+    return *this;
+  }
+};
+
+struct RPCArgs_cpp g_rpc_args;
+
+static void restore_from_cache() {
+  NSUserDefaults* defaults = [NSUserDefaults standardUserDefaults];
+
+  auto get_string_from_cache = [defaults](const char* key) {
+    NSString* ns_key = [NSString stringWithUTF8String:key];
+    NSString* ns_val = [defaults stringForKey:ns_key];
+    return std::string(ns_val != nil ? [ns_val UTF8String] : "");
+  };
+
+  auto get_int_from_cache = [defaults](const char* key) {
+    NSString* ns_key = [NSString stringWithUTF8String:key];
+    return static_cast<int>([defaults integerForKey:ns_key]);
+  };
+
+  g_rpc_args.host_url = get_string_from_cache("tmvrpc_url");
+  g_rpc_args.host_port = get_int_from_cache("tmvrpc_port");
+  g_rpc_args.key = get_string_from_cache("tmvrpc_key");
+}
+
+static void update_in_cache() {
+  NSUserDefaults* defaults = [NSUserDefaults standardUserDefaults];
+
+  [defaults setObject:[NSString stringWithUTF8String:g_rpc_args.host_url.c_str()]
+               forKey:@"tmvrpc_url"];
+  [defaults setInteger:g_rpc_args.host_port forKey:@"tmvrpc_port"];
+  [defaults setObject:[NSString stringWithUTF8String:g_rpc_args.key.c_str()] forKey:@"tmvrpc_key"];
+}
+
+string GetCmdOption(int argc, char* argv[], string option, bool key = false) {
+  string cmd;
+  for (int i = 1; i < argc; ++i) {
+    string arg = argv[i];
+    if (arg.find(option) == 0) {
+      if (key) {
+        cmd = argv[i];
+        return cmd;
+      }
+      // We assume "=" is the end of option.
+      ICHECK_EQ(*option.rbegin(), '=');
+      cmd = arg.substr(arg.find('=') + 1);
+      return cmd;
+    }
+  }
+  return cmd;
+}
+
+void update_rpc_args(int argc, char* argv[]) {
+  restore_from_cache();
+  RPCArgs_cpp& args = g_rpc_args;
+
+  using tvm::support::IsNumber;
+  using tvm::support::ValidateIP;
+  constexpr int MAX_PORT_NUM = 65535;
+
+  const string immediate_connect = GetCmdOption(argc, argv, "--immediate_connect", true);
+  args.immediate_connect = !immediate_connect.empty();
+
+  const string verbose = GetCmdOption(argc, argv, "--verbose", true);
+  args.verbose = !verbose.empty();
+
+  const string server_mode = GetCmdOption(argc, argv, "--server_mode=", false);
+  if (!server_mode.empty()) {
+    if (server_mode == "tracker") {
+      args.server_mode = 0;
+    } else if (server_mode == "proxy") {
+      args.server_mode = 1;
+    } else if (server_mode == "pure_server") {

Review comment:
       what's the use case for gRPC?

##########
File path: apps/ios_rpc/tvmrpc/RPCArgs.h
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#ifndef TVM_APPS_IOS_RPC_ARGS_H_
+#define TVM_APPS_IOS_RPC_ARGS_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*!
+ * \brief Struct representing arguments of iOS RPC app
+ */
+typedef struct RPCArgs_t {
+  /// Tracker or Proxy address (actually ip)
+  const char* host_url;
+
+  /// Tracker or Proxy port
+  int host_port;
+
+  /// device key to report
+  const char* key;
+
+  /// custom adress to report into Tracker. Ignored for other server modes.
+  const char* custom_addr;
+
+  /// Verbose mode. Will print status messages to std out.
+  /// 0 - no prints , 1 - print state to output
+  char verbose;
+
+  /// Immediate server launch. No UI interaction.
+  /// 0 - UI interaction, 1 - automatically connect on launch
+  char immediate_connect;

Review comment:
       why do you represent bool quantities with char? (instead of bool or uint8_t)

##########
File path: apps/ios_rpc/tvmrpc/RPCArgs.mm
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#import "RPCArgs.h"
+
+#import <Foundation/Foundation.h>
+
+#import "../../../src/support/socket.h"
+#import "../../../src/support/utils.h"
+
+#import <string>
+
+using std::string;
+
+const char* kUsage =
+    "\n"
+    "iOS tvmrpc application supported flags:\n"
+    "--host_url      The tracker/proxy address, Default=0.0.0.0\n"
+    "--host_port     The tracker/proxy port, Default=9190\n"
+    "--key           The key used to identify the device type in tracker. Default=\"\"\n"
+    "--custom_addr   Custom IP Address to Report to RPC Tracker. Default=\"\"\n"
+    "--immediate_connect   No UI interconnection, connect to tracker immediately. Default=False\n"
+    "--verbose       Allow to print status info to std out. Default=False\n"
+    "--server_mode   Server mode. Can be \"pure_server\", \"proxy\" or \"tracker\". "
+    "Default=pure_server \n"
+    "\n";
+
+struct RPCArgs_cpp {
+  string host_url = "0.0.0.0";
+  int host_port = 9190;
+
+  string key;
+  string custom_addr = "";
+
+  bool immediate_connect = false;
+  bool verbose = false;
+  char server_mode = 0;

Review comment:
       should that be an enum?

##########
File path: apps/ios_rpc/tvmrpc/RPCServer.mm
##########
@@ -0,0 +1,809 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file ViewController.mm
+ */
+
+#import "RPCServer.h"
+
+#include <tvm/runtime/packed_func.h>
+#include <tvm/runtime/registry.h>
+
+#include <random>
+#include <string>
+
+// To get device WiFi IP
+#include <arpa/inet.h>
+#include <ifaddrs.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+
+// TVM internal header to access Magic keys like kRPCMagic and others
+#include "../../../src/runtime/rpc/rpc_endpoint.h"
+
+namespace tvm {
+namespace runtime {
+
+/*!
+ * \brief Message handling function for event driven server.
+ *
+ * \param in_bytes The incoming bytes.
+ * \param event_flag  1: read_available, 2: write_avaiable.
+ * \return State flag.
+ *     1: continue running, no need to write,
+ *     2: need to write
+ *     0: shutdown
+ */
+using FEventHandler = PackedFunc;
+
+/*!
+ * \brief Create a server event handler.
+ *
+ * \param outputStream The output stream used to send outputs.
+ * \param name The name of the server.
+ * \param remote_key The remote key
+ * \return The event handler.
+ */
+FEventHandler CreateServerEventHandler(NSOutputStream* outputStream, std::string name,
+                                       std::string remote_key) {
+  const PackedFunc* event_handler_factor = Registry::Get("rpc.CreateEventDrivenServer");
+  ICHECK(event_handler_factor != nullptr)
+      << "You are using tvm_runtime module built without RPC support. "
+      << "Please rebuild it with USE_RPC flag.";
+
+  PackedFunc writer_func([outputStream](TVMArgs args, TVMRetValue* rv) {
+    TVMByteArray* data = args[0].ptr<TVMByteArray>();
+    int64_t nbytes = [outputStream write:reinterpret_cast<const uint8_t*>(data->data)
+                               maxLength:data->size];
+    if (nbytes < 0) {
+      NSLog(@"%@", [outputStream streamError].localizedDescription);
+      throw tvm::Error("Stream error");
+    }
+    *rv = nbytes;
+  });
+
+  return (*event_handler_factor)(writer_func, name, remote_key);
+}
+
+/*!
+ * \brief Helper function to query real IP of device in WiFi network
+ * \return string with IPv4 in format "192.168.0.1" or "unknown" if cannot detect
+ */
+static std::string getWiFiAddress() {
+  std::string address = "unknown";
+  ifaddrs* interfaces = nullptr;
+
+  int success = getifaddrs(&interfaces);
+  if (success == 0) {
+    ifaddrs* temp_addr = interfaces;
+    while (temp_addr != NULL) {
+      if (temp_addr->ifa_addr->sa_family == AF_INET) {
+        // Check if interface is en0 which is the wifi connection on the iPhone
+        if (std::string(temp_addr->ifa_name) == "en0") {
+          address = inet_ntoa(((sockaddr_in*)temp_addr->ifa_addr)->sin_addr);
+        }
+      }
+      temp_addr = temp_addr->ifa_next;
+    }
+  }
+
+  freeifaddrs(interfaces);
+  return address;
+}
+
+}  // namespace runtime
+}  // namespace tvm
+
+// Base class for any type of RPC servicing
+@interface RPCServerBase : RPCServer
+
+/*!
+ * Methods should be implemented in inherited classes
+ */
+- (bool)onReadHandler;   // return true - continue feeding, false - stop, try to drain output buffer
+- (bool)onWriteHandler;  // return true - continue draining, false - no data to write
+- (void)onEndEncountered;  // called on disconnect or session desided that it's shutdown time
+- (void)open;              // Initiate listening objects like i/o streams and other resources
+- (void)close;             // Deinitialize resources opend in "open" method
+@end
+
+@implementation RPCServerBase {
+  // Worker thread
+  NSThread* worker_thread_;
+  // Triger to continue RunLoop processing inside worker_thread_
+  BOOL shouldKeepRunning;
+  // Input socket stream
+ @protected
+  NSInputStream* inputStream_;
+  // Output socket stream
+  NSOutputStream* outputStream_;
+  // Temporal buffer with data to send
+  std::string sendBuffer_;
+  // Temporal receive buffer
+  std::string recvBuffer_;
+  // Requested data size to accumulate in recvBuffer_ before continue processing
+  int requiredToRecv_;
+}
+
+/*!
+ * Start internal worker thread with RunLoop and submit correspoding open handlers into it
+ * Not blocking
+ */
+- (void)start {
+  worker_thread_ = [[NSThread alloc] initWithBlock:^{
+    @autoreleasepool {
+      [self notifyState:RPCServerStatus_Launched];
+      [self open];
+      shouldKeepRunning = YES;
+      while (shouldKeepRunning && [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
+                                                           beforeDate:[NSDate distantFuture]])
+        ;
+      [self notifyState:RPCServerStatus_Stopped];
+    }
+  }];
+  [worker_thread_ start];
+}
+
+/*!
+ * Send message to workel thread runloop to finish processing
+ * Not blocking
+ */
+- (void)stop {
+  if (worker_thread_ == nil) return;
+
+  [self performSelector:@selector(stop_) onThread:worker_thread_ withObject:nil waitUntilDone:NO];
+  worker_thread_ = nil;  // TODO: is it valide? may be better to do that inside NSThread?
+}
+
+- (void)stop_ {
+  [self close];
+  shouldKeepRunning = NO;
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will connect to host and port specified in corresponding properties
+ */
+- (void)open {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)self.host, self.port, &readStream,
+                                     &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will assign i/o streams to provided socket connection.
+ */
+- (void)openWithSocket:(CFSocketNativeHandle)sock {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocket(NULL, sock, &readStream, &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Close i/o streams assosiated with connection
+ */
+- (void)close {
+  [inputStream_ close];
+  [outputStream_ close];
+  [inputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [inputStream_ setDelegate:nil];
+  [outputStream_ setDelegate:nil];
+  inputStream_ = nil;
+  outputStream_ = nil;
+}
+
+/// Unimplemented stubs
+- (bool)onReadHandler {
+  return false;
+}
+- (bool)onWriteHandler {
+  return false;
+}
+- (void)onEndEncountered {
+}
+
+/*!
+ * Try to read data from stream and call processing hadnler
+ */
+- (void)tryToRead {
+  const int kBufferSize = 4 << 10;  // 4kB buffer
+  const int prev_size = recvBuffer_.size();
+  recvBuffer_.resize(kBufferSize);
+  size_t nbytes = [inputStream_ read:(uint8_t*)recvBuffer_.data() + prev_size
+                           maxLength:recvBuffer_.size() - prev_size];
+  recvBuffer_.resize(nbytes + prev_size);
+
+  // feed while it accept or requested particulat buffer size
+  while (!recvBuffer_.empty() && requiredToRecv_ <= recvBuffer_.size() && [self onReadHandler])
+    ;
+}
+
+/*!
+ * Try to write remaining data to stream and call processing hadnler
+ */
+- (void)tryToWrite {
+  if (!sendBuffer_.empty()) {
+    size_t nbytes = [outputStream_ write:(uint8_t*)sendBuffer_.data() maxLength:sendBuffer_.size()];
+    sendBuffer_.erase(0, nbytes);
+  }
+  // call write handler while it want be called and space is available
+  while (sendBuffer_.empty() && [outputStream_ hasSpaceAvailable] && [self onWriteHandler])
+    ;
+}
+
+/*!
+ * Main event handler of socket stream events
+ */
+- (void)stream:(NSStream*)strm handleEvent:(NSStreamEvent)event {
+  std::string buffer;
+  switch (event) {
+    case NSStreamEventOpenCompleted: {
+      // Nothing
+      break;
+    }
+    case NSStreamEventHasBytesAvailable:
+      if (strm == inputStream_) {
+        [self tryToRead];
+        if ([outputStream_ hasSpaceAvailable]) [self tryToWrite];
+      }
+      break;
+    case NSStreamEventHasSpaceAvailable: {
+      if (strm == outputStream_) {
+        [self tryToWrite];
+        if ([inputStream_ hasBytesAvailable]) [self tryToRead];
+      }
+      break;
+    }
+    case NSStreamEventErrorOccurred: {
+      [self notifyError:[strm streamError].localizedDescription];
+      break;
+    }
+    case NSStreamEventEndEncountered: {
+      [self onEndEncountered];
+      break;
+    }
+    default: {
+      NSLog(@"Unknown event");
+    }
+  }
+}
+
+#pragma mark - Helpers
+
+/*!
+ * Set buffer to send into stream. Try to send immediatly or submit to lazy sending
+ * Non blocking operation
+ */
+- (void)toSend:(NSData*)data {
+  sendBuffer_.append(static_cast<const char*>(data.bytes), data.length);
+
+  // try to flush buffer
+  NSInteger sent_size = [outputStream_ write:(uint8_t*)sendBuffer_.data()
+                                   maxLength:sendBuffer_.size()];
+  sendBuffer_.erase(0, sent_size);
+}
+
+/*!
+ * Set buffer to send  in packet format [size, data]. Behaviour is same as for toSend.
+ */
+- (void)toSendPacked:(NSData*)data {
+  int packet_size = data.length;
+  [self toSend:[NSData dataWithBytes:&packet_size length:sizeof(packet_size)]];
+  [self toSend:data];
+}
+
+/*!
+ */
+- (NSData*)requestInputDataWithSize:(NSInteger)size {
+  if (recvBuffer_.size() < size) {
+    requiredToRecv_ = size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() length:size];
+  recvBuffer_.erase(0, size);
+  return res;
+}
+
+/*!
+ */
+- (NSData*)requestInputDataPacked {
+  int size;
+  if (recvBuffer_.size() < sizeof(size)) {
+    requiredToRecv_ = sizeof(size);
+    return nil;
+  }
+  size = *(int*)recvBuffer_.data();
+  if (recvBuffer_.size() < sizeof(size) + size) {
+    requiredToRecv_ = sizeof(size) + size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() + sizeof(size) length:size];
+  recvBuffer_.erase(0, sizeof(size) + size);
+  return res;
+};
+
+#pragma mark - Notifiers
+
+/*!
+ * Notify external listener about error.
+ * Also print error message to std out in case of Verbose mode
+ */
+- (void)notifyError:(NSString*)msg {
+  // Duplicate error message in std output. Host launcher script may listen it.
+  if (self.verbose) NSLog(@"[IOS-RPC] ERROR: %@", msg);
+  if (self.delegate) [self.delegate onError:msg];
+}
+
+/*!
+ * Notify external listener about server state changes.
+ * Also print information to std out in case of Verbose mode
+ */
+- (void)notifyState:(RPCServerStatus)state {
+  // Duplicate sattus changing in std output. Host launcher script may listen it.
+  if (self.verbose) NSLog(@"[IOS-RPC] STATE: %d", state);
+  if (self.delegate != nil) [self.delegate onStatusChanged:state];
+}
+
+@end
+
+@interface RPCServerProxy : RPCServerBase
+@end
+
+typedef enum {
+  RPCServerProxyState_Idle,
+  RPCServerProxyState_HandshakeToSend,
+  RPCServerProxyState_HandshakeToRecv,
+  RPCServerProxyState_Processing,
+} RPCServerProxyState;
+
+@implementation RPCServerProxy {
+  /// Original TVM RPC event handler
+  tvm::runtime::FEventHandler handler_;
+ @protected
+  /// Sate of Proxy client implementation
+  RPCServerProxyState state_;
+}
+
+- (instancetype)init {
+  if (self = [super init]) {
+    handler_ = nullptr;
+    state_ = RPCServerProxyState_Idle;
+  }
+  return self;
+}
+
+/*!
+ * Implement matching of internat state on state available for outside users
+ */
+- (void)setState:(RPCServerProxyState)new_state {
+  // Send Connected notification because Proxy doesn't responce until client connected.
+  if (new_state == RPCServerProxyState_HandshakeToRecv)
+    [self notifyState:RPCServerStatus_Connected];
+  if (new_state == RPCServerProxyState_Idle) [self notifyState:RPCServerStatus_Disconnected];
+  if (state_ == RPCServerProxyState_HandshakeToRecv && new_state == RPCServerProxyState_Processing)
+    [self notifyState:RPCServerStatus_RPCSessionStarted];
+  if (state_ == RPCServerProxyState_Processing && new_state == RPCServerProxyState_Idle)
+    [self notifyState:RPCServerStatus_RPCSessionStarted];
+
+  state_ = new_state;
+}
+
+- (bool)onWriteHandler {
+  switch (state_) {
+    case RPCServerProxyState_HandshakeToSend: {
+      // Send together kRPCMagic and server descriptor because of Proxy
+      int code = tvm::runtime::kRPCMagic;

Review comment:
       inttypes

##########
File path: apps/ios_rpc/tvmrpc/ViewController.mm
##########
@@ -22,168 +22,144 @@
  */
 
 #import "ViewController.h"
-#include <string>
+#import "RPCArgs.h"
 
-@implementation ViewController
-
-- (void)stream:(NSStream*)strm handleEvent:(NSStreamEvent)event {
-  std::string buffer;
-  switch (event) {
-    case NSStreamEventOpenCompleted: {
-      self.statusLabel.text = @"Connected";
-      break;
-    }
-    case NSStreamEventHasBytesAvailable:
-      if (strm == inputStream_) {
-        [self onReadAvailable];
-      }
-      break;
-    case NSStreamEventHasSpaceAvailable: {
-      if (strm == outputStream_) {
-        [self onWriteAvailable];
-      }
-      break;
-    }
-    case NSStreamEventErrorOccurred: {
-      NSLog(@"%@", [strm streamError].localizedDescription);
-      break;
-    }
-    case NSStreamEventEndEncountered: {
-      [self close];
-      // auto reconnect when normal end.
-      [self open];
-      break;
-    }
-    default: {
-      NSLog(@"Unknown event");
-    }
-  }
+@implementation ViewController {
+  // server implementation
+  RPCServer* server_;
+  // verbose flag to print status info
+  bool verbose_;
+  // Button state. True - push will start connection, false - push will disconnect
+  bool to_connect_;
 }
 
-- (void)onReadAvailable {
-  constexpr int kRPCMagic = 0xff271;
-  if (!initialized_) {
-    int code;
-    size_t nbytes = [inputStream_ read:reinterpret_cast<uint8_t*>(&code) maxLength:sizeof(code)];
-    if (nbytes != sizeof(code)) {
-      self.infoText.text = @"Fail to receive remote confirmation code.";
-      [self close];
-    } else if (code == kRPCMagic + 2) {
-      self.infoText.text = @"Proxy server cannot find client that matches the key";
-      [self close];
-    } else if (code == kRPCMagic + 1) {
-      self.infoText.text = @"Proxy server already have another server with same key";
-      [self close];
-    } else if (code != kRPCMagic) {
-      self.infoText.text = @"Given address is not a TVM RPC Proxy";
-      [self close];
-    } else {
-      initialized_ = true;
-      self.statusLabel.text = @"Proxy connected.";
-      ICHECK(handler_ != nullptr);
-    }
-  }
-  const int kBufferSize = 4 << 10;
-  if (initialized_) {
-    while ([inputStream_ hasBytesAvailable]) {
-      recvBuffer_.resize(kBufferSize);
-      uint8_t* bptr = reinterpret_cast<uint8_t*>(&recvBuffer_[0]);
-      size_t nbytes = [inputStream_ read:bptr maxLength:kBufferSize];
-      recvBuffer_.resize(nbytes);
-      int flag = 1;
-      if ([outputStream_ hasSpaceAvailable]) {
-        flag |= 2;
-      }
-      // always try to write
-      try {
-        flag = handler_(recvBuffer_, flag);
-        if (flag == 2) {
-          [self onShutdownReceived];
-        }
-      } catch (const tvm::Error& e) {
-        [self close];
-      }
-    }
+- (void)viewDidLoad {
+  // To handle end editing events
+  self.proxyURL.delegate = self;
+  self.proxyPort.delegate = self;
+  self.proxyKey.delegate = self;
+
+  RPCArgs args = get_current_rpc_args();
+  self.proxyURL.text = @(args.host_url);
+  self.proxyPort.text = @(args.host_port).stringValue;
+  self.proxyKey.text = @(args.key);
+
+  self.ModeSelector.selectedSegmentIndex = args.server_mode;
+  self->verbose_ = args.verbose;
+  self->to_connect_ = true;
+
+  // Add border to button
+  void (^addBorder)(UIButton* btn) = ^(UIButton* btn) {
+    btn.layer.borderWidth = 2.0f;
+    btn.layer.borderColor = self.ConnectButton.currentTitleColor.CGColor;
+    btn.layer.cornerRadius = 10;
+  };
+  addBorder(self.ConnectButton);
+
+  // Connect to tracker immediately
+  if (args.immediate_connect) {
+    [self disableUIInteraction];
+    [self open];
   }
 }
 
-- (void)onShutdownReceived {
-  [self close];
-}
+/*!
+ * \brief Disable all UI elements
+ */
+- (void)disableUIInteraction {
+  void (^disable)(UITextField* field) = ^(UITextField* field) {
+    field.enabled = NO;
+    field.backgroundColor = [UIColor lightGrayColor];
+  };
 
-- (void)onWriteAvailable {
-  if (initSendPtr_ < initBytes_.length()) {
-    initSendPtr_ += [outputStream_ write:reinterpret_cast<uint8_t*>(&initBytes_[initSendPtr_])
-                               maxLength:(initBytes_.length() - initSendPtr_)];
-  }
-  if (initialized_) {
-    try {
-      std::string dummy;
-      int flag = handler_(dummy, 2);
-      if (flag == 2) {
-        [self onShutdownReceived];
-      }
-    } catch (const tvm::Error& e) {
-      [self close];
-    }
-  }
+  void (^disableButton)(UIButton* btn) = ^(UIButton* btn) {
+    btn.enabled = NO;
+    btn.layer.borderColor = btn.currentTitleColor.CGColor;
+  };
+
+  disable(self.proxyURL);
+  disable(self.proxyPort);
+  disable(self.proxyKey);
+  disableButton(self.ConnectButton);
+  self.ModeSelector.enabled = NO;
 }
 
+/*!
+ * \brief Start RPC server
+ */
 - (void)open {
-  constexpr int kRPCMagic = 0xff271;
-  NSLog(@"Connecting to the proxy server..");
-  // Initialize the data states.
-  key_ = [self.proxyKey.text UTF8String];
-  key_ = "server:" + key_;
-  std::ostringstream os;
-  int rpc_magic = kRPCMagic;
-  os.write(reinterpret_cast<char*>(&rpc_magic), sizeof(rpc_magic));
-  int keylen = static_cast<int>(key_.length());
-  os.write(reinterpret_cast<char*>(&keylen), sizeof(keylen));
-  os.write(key_.c_str(), key_.length());
-  initialized_ = false;
-  initBytes_ = os.str();
-  initSendPtr_ = 0;
-  // Initialize the network.
-  CFReadStreamRef readStream;
-  CFWriteStreamRef writeStream;
-  CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)self.proxyURL.text,
-                                     [self.proxyPort.text intValue], &readStream, &writeStream);
-  inputStream_ = (NSInputStream*)readStream;
-  outputStream_ = (NSOutputStream*)writeStream;
-  [inputStream_ setDelegate:self];
-  [outputStream_ setDelegate:self];
-  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [outputStream_ open];
-  [inputStream_ open];
-  handler_ = tvm::runtime::CreateServerEventHandler(outputStream_, key_, "%toinit");
-  ICHECK(handler_ != nullptr);
+  RPCServerMode server_mode = static_cast<RPCServerMode>(self.ModeSelector.selectedSegmentIndex);
+
+  server_ = [RPCServer serverWithMode:server_mode];
+  server_.host = self.proxyURL.text;
+  server_.port = self.proxyPort.text.intValue;
+  server_.key = self.proxyKey.text;
+  server_.verbose = self->verbose_;
+  server_.delegate = self;
+
+  [server_ start];
+
   self.infoText.text = @"";
   self.statusLabel.text = @"Connecting...";
 }
 
+/*!
+ * \brief Stop RPC server
+ */
 - (void)close {
-  NSLog(@"Closing the streams.");
-  [inputStream_ close];
-  [outputStream_ close];
-  [inputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [outputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [inputStream_ setDelegate:nil];
-  [outputStream_ setDelegate:nil];
-  inputStream_ = nil;
-  outputStream_ = nil;
-  handler_ = nullptr;
-  self.statusLabel.text = @"Disconnected";
+  [server_ stop];
+  self.statusLabel.text = @"Disconnecting...";

Review comment:
       maybe swap this with previous line?

##########
File path: apps/ios_rpc/tvmrpc/RPCServer.mm
##########
@@ -0,0 +1,809 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file ViewController.mm
+ */
+
+#import "RPCServer.h"
+
+#include <tvm/runtime/packed_func.h>
+#include <tvm/runtime/registry.h>
+
+#include <random>
+#include <string>
+
+// To get device WiFi IP
+#include <arpa/inet.h>
+#include <ifaddrs.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+
+// TVM internal header to access Magic keys like kRPCMagic and others
+#include "../../../src/runtime/rpc/rpc_endpoint.h"
+
+namespace tvm {
+namespace runtime {
+
+/*!
+ * \brief Message handling function for event driven server.
+ *
+ * \param in_bytes The incoming bytes.
+ * \param event_flag  1: read_available, 2: write_avaiable.
+ * \return State flag.
+ *     1: continue running, no need to write,
+ *     2: need to write
+ *     0: shutdown
+ */
+using FEventHandler = PackedFunc;
+
+/*!
+ * \brief Create a server event handler.
+ *
+ * \param outputStream The output stream used to send outputs.
+ * \param name The name of the server.
+ * \param remote_key The remote key
+ * \return The event handler.
+ */
+FEventHandler CreateServerEventHandler(NSOutputStream* outputStream, std::string name,
+                                       std::string remote_key) {
+  const PackedFunc* event_handler_factor = Registry::Get("rpc.CreateEventDrivenServer");
+  ICHECK(event_handler_factor != nullptr)
+      << "You are using tvm_runtime module built without RPC support. "
+      << "Please rebuild it with USE_RPC flag.";
+
+  PackedFunc writer_func([outputStream](TVMArgs args, TVMRetValue* rv) {
+    TVMByteArray* data = args[0].ptr<TVMByteArray>();
+    int64_t nbytes = [outputStream write:reinterpret_cast<const uint8_t*>(data->data)
+                               maxLength:data->size];
+    if (nbytes < 0) {
+      NSLog(@"%@", [outputStream streamError].localizedDescription);
+      throw tvm::Error("Stream error");
+    }
+    *rv = nbytes;
+  });
+
+  return (*event_handler_factor)(writer_func, name, remote_key);
+}
+
+/*!
+ * \brief Helper function to query real IP of device in WiFi network
+ * \return string with IPv4 in format "192.168.0.1" or "unknown" if cannot detect
+ */
+static std::string getWiFiAddress() {
+  std::string address = "unknown";
+  ifaddrs* interfaces = nullptr;
+
+  int success = getifaddrs(&interfaces);
+  if (success == 0) {
+    ifaddrs* temp_addr = interfaces;
+    while (temp_addr != NULL) {
+      if (temp_addr->ifa_addr->sa_family == AF_INET) {
+        // Check if interface is en0 which is the wifi connection on the iPhone
+        if (std::string(temp_addr->ifa_name) == "en0") {
+          address = inet_ntoa(((sockaddr_in*)temp_addr->ifa_addr)->sin_addr);
+        }
+      }
+      temp_addr = temp_addr->ifa_next;
+    }
+  }
+
+  freeifaddrs(interfaces);
+  return address;
+}
+
+}  // namespace runtime
+}  // namespace tvm
+
+// Base class for any type of RPC servicing
+@interface RPCServerBase : RPCServer
+
+/*!
+ * Methods should be implemented in inherited classes
+ */
+- (bool)onReadHandler;   // return true - continue feeding, false - stop, try to drain output buffer
+- (bool)onWriteHandler;  // return true - continue draining, false - no data to write
+- (void)onEndEncountered;  // called on disconnect or session desided that it's shutdown time
+- (void)open;              // Initiate listening objects like i/o streams and other resources
+- (void)close;             // Deinitialize resources opend in "open" method
+@end
+
+@implementation RPCServerBase {
+  // Worker thread
+  NSThread* worker_thread_;
+  // Triger to continue RunLoop processing inside worker_thread_
+  BOOL shouldKeepRunning;
+  // Input socket stream
+ @protected
+  NSInputStream* inputStream_;
+  // Output socket stream
+  NSOutputStream* outputStream_;
+  // Temporal buffer with data to send
+  std::string sendBuffer_;
+  // Temporal receive buffer
+  std::string recvBuffer_;
+  // Requested data size to accumulate in recvBuffer_ before continue processing
+  int requiredToRecv_;
+}
+
+/*!
+ * Start internal worker thread with RunLoop and submit correspoding open handlers into it
+ * Not blocking
+ */
+- (void)start {
+  worker_thread_ = [[NSThread alloc] initWithBlock:^{
+    @autoreleasepool {
+      [self notifyState:RPCServerStatus_Launched];
+      [self open];
+      shouldKeepRunning = YES;
+      while (shouldKeepRunning && [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
+                                                           beforeDate:[NSDate distantFuture]])
+        ;
+      [self notifyState:RPCServerStatus_Stopped];
+    }
+  }];
+  [worker_thread_ start];
+}
+
+/*!
+ * Send message to workel thread runloop to finish processing
+ * Not blocking
+ */
+- (void)stop {
+  if (worker_thread_ == nil) return;
+
+  [self performSelector:@selector(stop_) onThread:worker_thread_ withObject:nil waitUntilDone:NO];
+  worker_thread_ = nil;  // TODO: is it valide? may be better to do that inside NSThread?
+}
+
+- (void)stop_ {
+  [self close];
+  shouldKeepRunning = NO;
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will connect to host and port specified in corresponding properties
+ */
+- (void)open {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)self.host, self.port, &readStream,
+                                     &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will assign i/o streams to provided socket connection.
+ */
+- (void)openWithSocket:(CFSocketNativeHandle)sock {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocket(NULL, sock, &readStream, &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Close i/o streams assosiated with connection
+ */
+- (void)close {
+  [inputStream_ close];
+  [outputStream_ close];
+  [inputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [inputStream_ setDelegate:nil];
+  [outputStream_ setDelegate:nil];
+  inputStream_ = nil;
+  outputStream_ = nil;
+}
+
+/// Unimplemented stubs
+- (bool)onReadHandler {
+  return false;
+}
+- (bool)onWriteHandler {
+  return false;
+}
+- (void)onEndEncountered {
+}
+
+/*!
+ * Try to read data from stream and call processing hadnler
+ */
+- (void)tryToRead {
+  const int kBufferSize = 4 << 10;  // 4kB buffer
+  const int prev_size = recvBuffer_.size();
+  recvBuffer_.resize(kBufferSize);
+  size_t nbytes = [inputStream_ read:(uint8_t*)recvBuffer_.data() + prev_size
+                           maxLength:recvBuffer_.size() - prev_size];
+  recvBuffer_.resize(nbytes + prev_size);
+
+  // feed while it accept or requested particulat buffer size
+  while (!recvBuffer_.empty() && requiredToRecv_ <= recvBuffer_.size() && [self onReadHandler])
+    ;
+}
+
+/*!
+ * Try to write remaining data to stream and call processing hadnler
+ */
+- (void)tryToWrite {
+  if (!sendBuffer_.empty()) {
+    size_t nbytes = [outputStream_ write:(uint8_t*)sendBuffer_.data() maxLength:sendBuffer_.size()];
+    sendBuffer_.erase(0, nbytes);
+  }
+  // call write handler while it want be called and space is available
+  while (sendBuffer_.empty() && [outputStream_ hasSpaceAvailable] && [self onWriteHandler])
+    ;
+}
+
+/*!
+ * Main event handler of socket stream events
+ */
+- (void)stream:(NSStream*)strm handleEvent:(NSStreamEvent)event {
+  std::string buffer;
+  switch (event) {
+    case NSStreamEventOpenCompleted: {
+      // Nothing
+      break;
+    }
+    case NSStreamEventHasBytesAvailable:
+      if (strm == inputStream_) {
+        [self tryToRead];
+        if ([outputStream_ hasSpaceAvailable]) [self tryToWrite];
+      }
+      break;
+    case NSStreamEventHasSpaceAvailable: {
+      if (strm == outputStream_) {
+        [self tryToWrite];
+        if ([inputStream_ hasBytesAvailable]) [self tryToRead];
+      }
+      break;
+    }
+    case NSStreamEventErrorOccurred: {
+      [self notifyError:[strm streamError].localizedDescription];
+      break;
+    }
+    case NSStreamEventEndEncountered: {
+      [self onEndEncountered];
+      break;
+    }
+    default: {
+      NSLog(@"Unknown event");
+    }
+  }
+}
+
+#pragma mark - Helpers
+
+/*!
+ * Set buffer to send into stream. Try to send immediatly or submit to lazy sending
+ * Non blocking operation
+ */
+- (void)toSend:(NSData*)data {
+  sendBuffer_.append(static_cast<const char*>(data.bytes), data.length);
+
+  // try to flush buffer
+  NSInteger sent_size = [outputStream_ write:(uint8_t*)sendBuffer_.data()
+                                   maxLength:sendBuffer_.size()];
+  sendBuffer_.erase(0, sent_size);
+}
+
+/*!
+ * Set buffer to send  in packet format [size, data]. Behaviour is same as for toSend.
+ */
+- (void)toSendPacked:(NSData*)data {
+  int packet_size = data.length;
+  [self toSend:[NSData dataWithBytes:&packet_size length:sizeof(packet_size)]];
+  [self toSend:data];
+}
+
+/*!
+ */
+- (NSData*)requestInputDataWithSize:(NSInteger)size {
+  if (recvBuffer_.size() < size) {
+    requiredToRecv_ = size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() length:size];
+  recvBuffer_.erase(0, size);
+  return res;
+}
+
+/*!
+ */
+- (NSData*)requestInputDataPacked {
+  int size;
+  if (recvBuffer_.size() < sizeof(size)) {
+    requiredToRecv_ = sizeof(size);
+    return nil;
+  }
+  size = *(int*)recvBuffer_.data();
+  if (recvBuffer_.size() < sizeof(size) + size) {
+    requiredToRecv_ = sizeof(size) + size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() + sizeof(size) length:size];
+  recvBuffer_.erase(0, sizeof(size) + size);
+  return res;
+};
+
+#pragma mark - Notifiers
+
+/*!
+ * Notify external listener about error.
+ * Also print error message to std out in case of Verbose mode
+ */
+- (void)notifyError:(NSString*)msg {
+  // Duplicate error message in std output. Host launcher script may listen it.
+  if (self.verbose) NSLog(@"[IOS-RPC] ERROR: %@", msg);
+  if (self.delegate) [self.delegate onError:msg];
+}
+
+/*!
+ * Notify external listener about server state changes.
+ * Also print information to std out in case of Verbose mode
+ */
+- (void)notifyState:(RPCServerStatus)state {
+  // Duplicate sattus changing in std output. Host launcher script may listen it.
+  if (self.verbose) NSLog(@"[IOS-RPC] STATE: %d", state);
+  if (self.delegate != nil) [self.delegate onStatusChanged:state];
+}
+
+@end
+
+@interface RPCServerProxy : RPCServerBase
+@end
+
+typedef enum {
+  RPCServerProxyState_Idle,
+  RPCServerProxyState_HandshakeToSend,
+  RPCServerProxyState_HandshakeToRecv,
+  RPCServerProxyState_Processing,
+} RPCServerProxyState;
+
+@implementation RPCServerProxy {
+  /// Original TVM RPC event handler
+  tvm::runtime::FEventHandler handler_;
+ @protected
+  /// Sate of Proxy client implementation
+  RPCServerProxyState state_;
+}
+
+- (instancetype)init {
+  if (self = [super init]) {
+    handler_ = nullptr;
+    state_ = RPCServerProxyState_Idle;
+  }
+  return self;
+}
+
+/*!
+ * Implement matching of internat state on state available for outside users
+ */
+- (void)setState:(RPCServerProxyState)new_state {
+  // Send Connected notification because Proxy doesn't responce until client connected.
+  if (new_state == RPCServerProxyState_HandshakeToRecv)
+    [self notifyState:RPCServerStatus_Connected];
+  if (new_state == RPCServerProxyState_Idle) [self notifyState:RPCServerStatus_Disconnected];
+  if (state_ == RPCServerProxyState_HandshakeToRecv && new_state == RPCServerProxyState_Processing)
+    [self notifyState:RPCServerStatus_RPCSessionStarted];
+  if (state_ == RPCServerProxyState_Processing && new_state == RPCServerProxyState_Idle)
+    [self notifyState:RPCServerStatus_RPCSessionStarted];
+
+  state_ = new_state;
+}
+
+- (bool)onWriteHandler {
+  switch (state_) {
+    case RPCServerProxyState_HandshakeToSend: {
+      // Send together kRPCMagic and server descriptor because of Proxy
+      int code = tvm::runtime::kRPCMagic;
+      [self toSend:[NSData dataWithBytes:&code length:sizeof(code)]];
+
+      std::string full_key = std::string("server:") + self.key.UTF8String;
+      [self toSendPacked:[NSData dataWithBytes:full_key.data() length:full_key.size()]];
+
+      self.state = RPCServerProxyState_HandshakeToRecv;
+      return TRUE;
+    }
+    case RPCServerProxyState_Processing: {
+      try {
+        TVMByteArray dummy{nullptr, 0};
+        int flag = handler_(dummy, 2);
+        if (flag == 0) {
+          [self onEndEncountered];
+        }
+        return flag == 2;
+      } catch (const tvm::Error& e) {
+        [self close];
+      }
+      break;
+    }
+    default:
+      // Nothing
+      break;
+  }
+  return FALSE;
+}
+
+- (bool)onReadHandler {
+  switch (state_) {
+    case RPCServerProxyState_HandshakeToRecv: {
+      NSData* data = [self requestInputDataWithSize:sizeof(int)];
+      if (data == nil) return FALSE;
+
+      if (*(int*)data.bytes != tvm::runtime::kRPCMagic) {
+        [self notifyError:@"Wrong responce, is not RPC client."];
+        [self close];
+        return FALSE;
+        break;
+      }
+
+      handler_ = tvm::runtime::CreateServerEventHandler(outputStream_, "iphone", "%toinit");
+
+      self.state = RPCServerProxyState_Processing;
+      return TRUE;
+      break;
+    }
+    case RPCServerProxyState_Processing: {
+      int flag = 1;
+      if ([outputStream_ hasSpaceAvailable]) {
+        flag |= 2;
+      }
+      // always try to write
+      try {
+        TVMByteArray arr{recvBuffer_.data(), recvBuffer_.size()};
+        flag = handler_(arr, flag);
+        recvBuffer_.clear();
+        if (flag == 0) {
+          [self onEndEncountered];
+        }
+        return flag == 1;
+      } catch (const tvm::Error& e) {
+        [self close];
+      }
+      break;
+    }
+    default:
+      // Nothing
+      break;
+  }
+  return FALSE;
+}
+
+- (void)onEndEncountered {
+  // Automatic reconnection when session is finished.

Review comment:
       worth logging?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r651295319



##########
File path: apps/ios_rpc/tvmrpc/rpc_server.h
##########
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file rpc_server.h
+ * \brief RPC Server implementation.
+ */
+#ifndef TVM_APPS_IOS_RPC_SERVER_H_
+#define TVM_APPS_IOS_RPC_SERVER_H_
+
+#include <string>
+#include <future>
+#include <chrono>
+#include <dirent.h>
+
+#include "tvm/runtime/c_runtime_api.h"
+#include "runtime/rpc/rpc_endpoint.h"
+#include "runtime/rpc/rpc_socket_impl.h"
+#include "support/socket.h"
+#include "rpc_tracker_client.h"
+
+namespace tvm {
+namespace runtime {
+
+std::vector<std::string> ListDir(const std::string& dirname) {
+  std::vector<std::string> vec;
+  DIR* dp = opendir(dirname.c_str());
+  if (dp == nullptr) {
+    int errsv = errno;
+    LOG(FATAL) << "ListDir " << dirname << " error: " << strerror(errsv);
+  }
+  dirent* d;
+  while ((d = readdir(dp)) != nullptr) {
+    std::string filename = d->d_name;
+    if (filename != "." && filename != "..") {
+      std::string f = dirname;
+      if (f[f.length() - 1] != '/') {
+        f += '/';
+      }
+      f += d->d_name;
+      vec.push_back(f);
+    }
+  }
+  closedir(dp);
+  return vec;
+}
+
+/*!
+ * \brief CleanDir Removes the files from the directory
+ * \param dirname The name of the directory
+ */
+void CleanDir(const std::string& dirname) {
+  auto files = ListDir(dirname);
+  for (const auto& filename : files) {
+    std::string file_path = dirname + "/";
+    file_path += filename;
+    const int ret = std::remove(filename.c_str());
+    if (ret != 0) {
+      LOG(WARNING) << "Remove file " << filename << " failed";
+    }
+  }
+}
+
+// Runtime environment
+struct RPCEnv {
+ public:
+  RPCEnv(const std::string &base):base_(base) {}
+  // Get Path.
+  std::string GetPath(const std::string& file_name) { return base_ + file_name; }
+
+  void CleanUp() const {
+    CleanDir(base_);
+  }
+ private:
+  std::string base_;
+};
+
+
+/*!
+ * \brief RPCServer RPC Server class.
+ * \param host The hostname of the server, Default=0.0.0.0
+ * \param port The port of the RPC, Default=9090
+ * \param port_end The end search port of the RPC, Default=9099
+ * \param tracker The address of RPC tracker in host:port format e.g. 10.77.1.234:9190 Default=""
+ * \param key The key used to identify the device type in tracker. Default=""
+ * \param custom_addr Custom IP Address to Report to RPC Tracker. Default=""
+ */
+class RPCServer {
+ public:
+  /*!
+   * \brief Constructor.
+   */
+  RPCServer(std::string host, int port, int port_end, std::string tracker_addr, std::string key,
+            std::string custom_addr, std::string work_dir)
+      : host_(std::move(host)),
+        port_(port),
+        my_port_(0),
+        port_end_(port_end),
+        tracker_addr_(std::move(tracker_addr)),
+        key_(std::move(key)),
+        custom_addr_(std::move(custom_addr)),
+        work_dir_(std::move(work_dir)),
+        tracker_(tracker_addr_, key_, custom_addr_) {}
+
+  /*!
+   * \brief Destructor.
+   */
+  ~RPCServer() {
+    try {
+      // Free the resources
+      listen_sock_.Close();
+      tracker_.Close();
+    } catch (...) {
+    }
+  }
+
+  /*!
+   * \brief Start Creates the RPC listen process and execution.
+   */
+  void Start() {
+    listen_sock_.Create();
+    my_port_ = listen_sock_.TryBindHost(host_, port_, port_end_);
+    LOG(INFO) << "bind to " << host_ << ":" << my_port_;
+    listen_sock_.Listen(1);
+    continue_processing = true;
+    proc_ = std::future<void>(std::async(std::launch::async, &RPCServer::ListenLoopProc, this));
+  }
+  
+  void Stop() {
+    continue_processing = false;
+    tracker_.Close();
+  }
+    
+  void setCompletionCallbacks(std::function<void()> callback_start, std::function<void()> callback_stop) {
+    completion_callback_start_ = callback_start;
+    completion_callback_stop_ = callback_stop;
+  }
+
+ private:
+  /*!
+   * \brief ListenLoopProc The listen process.
+   */
+  void ListenLoopProc() {
+    
+    while (continue_processing) {
+      support::TCPSocket conn;
+      support::SockAddr addr("0.0.0.0", 0);
+      std::string opts;
+      try {
+        // step 1: setup tracker and report to tracker
+        tracker_.TryConnect();
+        if (completion_callback_start_)
+          completion_callback_start_();
+        // step 2: wait for in-coming connections
+        AcceptConnection(&tracker_, &conn, &addr, &opts);
+      } catch (const char* msg) {
+        LOG(WARNING) << "Socket exception: " << msg;
+        // close tracker resource
+        tracker_.Close();
+        continue;
+      } catch (const std::exception& e) {
+        // close tracker resource
+        tracker_.Close();
+        LOG(WARNING) << "Exception standard: " << e.what();
+        continue;
+      }
+
+      auto start_time = std::chrono::high_resolution_clock::now();
+      ServerLoopProc(conn, addr, work_dir_);
+      auto dur = std::chrono::high_resolution_clock::now() - start_time;
+
+      LOG(INFO) << "Serve Time " << std::chrono::duration_cast<std::chrono::milliseconds>(dur).count() << "ms";
+
+      // close from our side.
+      LOG(INFO) << "Socket Connection Closed";
+      conn.Close();
+    }
+    if (completion_callback_stop_)
+      completion_callback_stop_();
+
+  }
+
+  /*!
+   * \brief AcceptConnection Accepts the RPC Server connection.
+   * \param tracker Tracker details.
+   * \param conn_sock New connection information.
+   * \param addr New connection address information.
+   * \param opts Parsed options for socket
+   * \param ping_period Timeout for select call waiting
+   */
+  void AcceptConnection(TrackerClient* tracker, support::TCPSocket* conn_sock,
+                        support::SockAddr* addr, std::string* opts, int ping_period = 2) {
+    std::set<std::string> old_keyset;
+    std::string matchkey;
+
+    // Report resource to tracker and get key
+    tracker->ReportResourceAndGetKey(my_port_, &matchkey);
+
+    while (continue_processing) {
+      tracker->WaitConnectionAndUpdateKey(listen_sock_, my_port_, ping_period, &matchkey);
+      support::TCPSocket conn = listen_sock_.Accept(addr);
+
+      int code = kRPCMagic;
+      ICHECK_EQ(conn.RecvAll(&code, sizeof(code)), sizeof(code));
+      if (code != kRPCMagic) {
+        conn.Close();
+        LOG(FATAL) << "Client connected is not TVM RPC server";
+        continue;
+      }
+
+      int keylen = 0;
+      ICHECK_EQ(conn.RecvAll(&keylen, sizeof(keylen)), sizeof(keylen));
+
+      const char* CLIENT_HEADER = "client:";
+      const char* SERVER_HEADER = "server:";
+      std::string expect_header = CLIENT_HEADER + matchkey;
+      std::string server_key = SERVER_HEADER + key_;
+      if (size_t(keylen) < expect_header.length()) {
+        conn.Close();
+        LOG(INFO) << "Wrong client header length";
+        continue;
+      }
+
+      ICHECK_NE(keylen, 0);
+      std::string remote_key;
+      remote_key.resize(keylen);
+      ICHECK_EQ(conn.RecvAll(&remote_key[0], keylen), keylen);
+
+      std::stringstream ssin(remote_key);
+      std::string arg0;
+      ssin >> arg0;
+
+      if (arg0 != expect_header) {
+        code = kRPCMismatch;
+        ICHECK_EQ(conn.SendAll(&code, sizeof(code)), sizeof(code));
+        conn.Close();
+        LOG(WARNING) << "Mismatch key from" << addr->AsString();
+        continue;
+      } else {
+        code = kRPCSuccess;
+        ICHECK_EQ(conn.SendAll(&code, sizeof(code)), sizeof(code));
+        keylen = int(server_key.length());
+        ICHECK_EQ(conn.SendAll(&keylen, sizeof(keylen)), sizeof(keylen));
+        ICHECK_EQ(conn.SendAll(server_key.c_str(), keylen), keylen);
+        LOG(INFO) << "Connection success " << addr->AsString();
+        ssin >> *opts;
+        *conn_sock = conn;
+        return;
+      }
+    }
+  }
+
+  /*!
+   * \brief ServerLoopProc The Server loop process.
+   * \param sock The socket information
+   * \param addr The socket address information
+   */
+  static void ServerLoopProc(support::TCPSocket sock, support::SockAddr addr,
+                             std::string work_dir) {
+    // Server loop
+    const auto env = RPCEnv(work_dir);
+    RPCServerLoop(int(sock.sockfd));
+    LOG(INFO) << "Finish serving " << addr.AsString();
+    env.CleanUp();
+  }
+
+  /*!
+   * \brief GetTimeOutFromOpts Parse and get the timeout option.
+   * \param opts The option string
+   */
+  int GetTimeOutFromOpts(const std::string& opts) const {
+    const std::string option = "-timeout=";

Review comment:
       Agree. And already removed.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r711967426



##########
File path: apps/ios_rpc/README.md
##########
@@ -103,7 +40,13 @@ This custom `dlopen` mechanic is integrated into TVM RPC as plugin and registere
 application.
 
 The custom implementation of `dlopen` and other functions from `dlfcn.h` header are placed in separate repository,
-and will be downloaded automatically during cmake build for iOS. To run cmake build you may use next flags:
+and will be downloaded automatically during cmake build for iOS.
+
+Also, it is necessary to build `tvm_runtime.dylib` for our iOS device. The iOS

Review comment:
       You are right. Thank you.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r651305686



##########
File path: apps/ios_rpc/tvmrpc/TVMRuntime.mm
##########
@@ -71,88 +73,19 @@ void LogMessageImpl(const std::string& file, int lineno, const std::string& mess
 namespace tvm {
 namespace runtime {
 
-class NSStreamChannel final : public RPCChannel {
- public:
-  explicit NSStreamChannel(NSOutputStream* stream) : stream_(stream) {}
-
-  size_t Send(const void* data, size_t size) final {
-    ssize_t nbytes = [stream_ write:reinterpret_cast<const uint8_t*>(data) maxLength:size];
-    if (nbytes < 0) {
-      NSLog(@"%@", [stream_ streamError].localizedDescription);
-      throw tvm::Error("Stream error");
-    }
-    return nbytes;
-  }
-
-  size_t Recv(void* data, size_t size) final {
-    LOG(FATAL) << "Do not allow explicit receive for";
-    return 0;
-  }
-
- private:
-  NSOutputStream* stream_;
-};
-
-FEventHandler CreateServerEventHandler(NSOutputStream* outputStream, std::string name,
-                                       std::string remote_key) {
-  std::unique_ptr<NSStreamChannel> ch(new NSStreamChannel(outputStream));
-  std::shared_ptr<RPCEndpoint> sess = RPCEndpoint::Create(std::move(ch), name, remote_key);
-  return [sess](const std::string& in_bytes, int flag) {
-    return sess->ServerAsyncIOEventHandler(in_bytes, flag);
-  };
-}
-
-// Runtime environment
-struct RPCEnv {
- public:
-  RPCEnv() {
-    NSString* path = NSTemporaryDirectory();
-    base_ = [path UTF8String];
-    if (base_[base_.length() - 1] != '/') {
-      base_ = base_ + '/';
-    }
-  }
-  // Get Path.
-  std::string GetPath(const std::string& file_name) { return base_ + file_name; }
-
- private:
-  std::string base_;
-};
-
-void LaunchSyncServer() {
-  // only load dylib from frameworks.
-  NSBundle* bundle = [NSBundle mainBundle];
-  NSString* base = [bundle privateFrameworksPath];
-  NSString* path = [base stringByAppendingPathComponent:@"tvm/rpc_config.txt"];
-  std::string name = [path UTF8String];
-  std::ifstream fs(name, std::ios::in);
-  std::string url, key;
-  int port;
-  ICHECK(fs >> url >> port >> key) << "Invalid RPC config file " << name;
-  RPCConnect(url, port, "server:" + key, TVMArgs(nullptr, nullptr, 0))->ServerLoop();
-}
-
 TVM_REGISTER_GLOBAL("tvm.rpc.server.workpath").set_body([](TVMArgs args, TVMRetValue* rv) {
-  static RPCEnv env;
-  *rv = env.GetPath(args[0]);
+  std::string name = args[0];
+  std::string base = [NSTemporaryDirectory() UTF8String];
+  *rv = base + "/" + name;
 });
 
 TVM_REGISTER_GLOBAL("tvm.rpc.server.load_module").set_body([](TVMArgs args, TVMRetValue* rv) {
   std::string name = args[0];
-  std::string fmt = GetFileFormat(name, "");

Review comment:
       Previous version of this patch was designed before merging of patch with custom DSO loader. But current PR state keeps this line unchanged.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-864056719






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r654011693



##########
File path: apps/ios_rpc/tvmrpc/ViewController.mm
##########
@@ -22,168 +22,144 @@
  */
 
 #import "ViewController.h"
-#include <string>
+#import "RPCArgs.h"
 
-@implementation ViewController
-
-- (void)stream:(NSStream*)strm handleEvent:(NSStreamEvent)event {
-  std::string buffer;
-  switch (event) {
-    case NSStreamEventOpenCompleted: {
-      self.statusLabel.text = @"Connected";
-      break;
-    }
-    case NSStreamEventHasBytesAvailable:
-      if (strm == inputStream_) {
-        [self onReadAvailable];
-      }
-      break;
-    case NSStreamEventHasSpaceAvailable: {
-      if (strm == outputStream_) {
-        [self onWriteAvailable];
-      }
-      break;
-    }
-    case NSStreamEventErrorOccurred: {
-      NSLog(@"%@", [strm streamError].localizedDescription);
-      break;
-    }
-    case NSStreamEventEndEncountered: {
-      [self close];
-      // auto reconnect when normal end.
-      [self open];
-      break;
-    }
-    default: {
-      NSLog(@"Unknown event");
-    }
-  }
+@implementation ViewController {
+  // server implementation
+  RPCServer* server_;
+  // verbose flag to print status info
+  bool verbose_;
+  // Button state. True - push will start connection, false - push will disconnect
+  bool to_connect_;
 }
 
-- (void)onReadAvailable {
-  constexpr int kRPCMagic = 0xff271;
-  if (!initialized_) {
-    int code;
-    size_t nbytes = [inputStream_ read:reinterpret_cast<uint8_t*>(&code) maxLength:sizeof(code)];
-    if (nbytes != sizeof(code)) {
-      self.infoText.text = @"Fail to receive remote confirmation code.";
-      [self close];
-    } else if (code == kRPCMagic + 2) {
-      self.infoText.text = @"Proxy server cannot find client that matches the key";
-      [self close];
-    } else if (code == kRPCMagic + 1) {
-      self.infoText.text = @"Proxy server already have another server with same key";
-      [self close];
-    } else if (code != kRPCMagic) {
-      self.infoText.text = @"Given address is not a TVM RPC Proxy";
-      [self close];
-    } else {
-      initialized_ = true;
-      self.statusLabel.text = @"Proxy connected.";
-      ICHECK(handler_ != nullptr);
-    }
-  }
-  const int kBufferSize = 4 << 10;
-  if (initialized_) {
-    while ([inputStream_ hasBytesAvailable]) {
-      recvBuffer_.resize(kBufferSize);
-      uint8_t* bptr = reinterpret_cast<uint8_t*>(&recvBuffer_[0]);
-      size_t nbytes = [inputStream_ read:bptr maxLength:kBufferSize];
-      recvBuffer_.resize(nbytes);
-      int flag = 1;
-      if ([outputStream_ hasSpaceAvailable]) {
-        flag |= 2;
-      }
-      // always try to write
-      try {
-        flag = handler_(recvBuffer_, flag);
-        if (flag == 2) {
-          [self onShutdownReceived];
-        }
-      } catch (const tvm::Error& e) {
-        [self close];
-      }
-    }
+- (void)viewDidLoad {
+  // To handle end editing events
+  self.proxyURL.delegate = self;
+  self.proxyPort.delegate = self;
+  self.proxyKey.delegate = self;
+
+  RPCArgs args = get_current_rpc_args();
+  self.proxyURL.text = @(args.host_url);
+  self.proxyPort.text = @(args.host_port).stringValue;
+  self.proxyKey.text = @(args.key);
+
+  self.ModeSelector.selectedSegmentIndex = args.server_mode;
+  self->verbose_ = args.verbose;
+  self->to_connect_ = true;
+
+  // Add border to button
+  void (^addBorder)(UIButton* btn) = ^(UIButton* btn) {
+    btn.layer.borderWidth = 2.0f;
+    btn.layer.borderColor = self.ConnectButton.currentTitleColor.CGColor;
+    btn.layer.cornerRadius = 10;
+  };
+  addBorder(self.ConnectButton);
+
+  // Connect to tracker immediately
+  if (args.immediate_connect) {
+    [self disableUIInteraction];
+    [self open];
   }
 }
 
-- (void)onShutdownReceived {
-  [self close];
-}
+/*!
+ * \brief Disable all UI elements
+ */
+- (void)disableUIInteraction {
+  void (^disable)(UITextField* field) = ^(UITextField* field) {
+    field.enabled = NO;
+    field.backgroundColor = [UIColor lightGrayColor];
+  };
 
-- (void)onWriteAvailable {
-  if (initSendPtr_ < initBytes_.length()) {
-    initSendPtr_ += [outputStream_ write:reinterpret_cast<uint8_t*>(&initBytes_[initSendPtr_])
-                               maxLength:(initBytes_.length() - initSendPtr_)];
-  }
-  if (initialized_) {
-    try {
-      std::string dummy;
-      int flag = handler_(dummy, 2);
-      if (flag == 2) {
-        [self onShutdownReceived];
-      }
-    } catch (const tvm::Error& e) {
-      [self close];
-    }
-  }
+  void (^disableButton)(UIButton* btn) = ^(UIButton* btn) {
+    btn.enabled = NO;
+    btn.layer.borderColor = btn.currentTitleColor.CGColor;
+  };
+
+  disable(self.proxyURL);
+  disable(self.proxyPort);
+  disable(self.proxyKey);
+  disableButton(self.ConnectButton);
+  self.ModeSelector.enabled = NO;
 }
 
+/*!
+ * \brief Start RPC server
+ */
 - (void)open {
-  constexpr int kRPCMagic = 0xff271;
-  NSLog(@"Connecting to the proxy server..");
-  // Initialize the data states.
-  key_ = [self.proxyKey.text UTF8String];
-  key_ = "server:" + key_;
-  std::ostringstream os;
-  int rpc_magic = kRPCMagic;
-  os.write(reinterpret_cast<char*>(&rpc_magic), sizeof(rpc_magic));
-  int keylen = static_cast<int>(key_.length());
-  os.write(reinterpret_cast<char*>(&keylen), sizeof(keylen));
-  os.write(key_.c_str(), key_.length());
-  initialized_ = false;
-  initBytes_ = os.str();
-  initSendPtr_ = 0;
-  // Initialize the network.
-  CFReadStreamRef readStream;
-  CFWriteStreamRef writeStream;
-  CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)self.proxyURL.text,
-                                     [self.proxyPort.text intValue], &readStream, &writeStream);
-  inputStream_ = (NSInputStream*)readStream;
-  outputStream_ = (NSOutputStream*)writeStream;
-  [inputStream_ setDelegate:self];
-  [outputStream_ setDelegate:self];
-  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [outputStream_ open];
-  [inputStream_ open];
-  handler_ = tvm::runtime::CreateServerEventHandler(outputStream_, key_, "%toinit");
-  ICHECK(handler_ != nullptr);
+  RPCServerMode server_mode = static_cast<RPCServerMode>(self.ModeSelector.selectedSegmentIndex);
+
+  server_ = [RPCServer serverWithMode:server_mode];
+  server_.host = self.proxyURL.text;
+  server_.port = self.proxyPort.text.intValue;
+  server_.key = self.proxyKey.text;
+  server_.verbose = self->verbose_;
+  server_.delegate = self;
+
+  [server_ start];
+
   self.infoText.text = @"";
   self.statusLabel.text = @"Connecting...";
 }
 
+/*!
+ * \brief Stop RPC server
+ */
 - (void)close {
-  NSLog(@"Closing the streams.");
-  [inputStream_ close];
-  [outputStream_ close];
-  [inputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [outputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [inputStream_ setDelegate:nil];
-  [outputStream_ setDelegate:nil];
-  inputStream_ = nil;
-  outputStream_ = nil;
-  handler_ = nullptr;
-  self.statusLabel.text = @"Disconnected";
+  [server_ stop];
+  self.statusLabel.text = @"Disconnecting...";

Review comment:
       The `stop` method is async operation, non blocking. Just submit a stop action to execute in worker thread. 
   Do not see any benefits from swap. 
   
   After `stop` was called, the status is "Disconnecting...", when completion callback is called it will be changed to "Disconnected".
   
   If you insist I can do that for both methods start and stop.

##########
File path: apps/ios_rpc/tvmrpc/RPCArgs.h
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#ifndef TVM_APPS_IOS_RPC_ARGS_H_
+#define TVM_APPS_IOS_RPC_ARGS_H_
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/*!
+ * \brief Struct representing arguments of iOS RPC app
+ */
+typedef struct RPCArgs_t {
+  /// Tracker or Proxy address (actually ip)
+  const char* host_url;
+
+  /// Tracker or Proxy port
+  int host_port;
+
+  /// device key to report
+  const char* key;
+
+  /// custom adress to report into Tracker. Ignored for other server modes.
+  const char* custom_addr;
+
+  /// Verbose mode. Will print status messages to std out.
+  /// 0 - no prints , 1 - print state to output
+  char verbose;
+
+  /// Immediate server launch. No UI interaction.
+  /// 0 - UI interaction, 1 - automatically connect on launch
+  char immediate_connect;

Review comment:
       You are right, bool should be used here. Will change.
   
   Originally it was a simplest way to share API between code written on Obj-C, Obj-C++, and C++ ("BOOL" and "bool" are not so compatible). But now there is no such need.

##########
File path: apps/ios_rpc/tvmrpc/RPCServer.mm
##########
@@ -0,0 +1,809 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file ViewController.mm
+ */
+
+#import "RPCServer.h"
+
+#include <tvm/runtime/packed_func.h>
+#include <tvm/runtime/registry.h>
+
+#include <random>
+#include <string>
+
+// To get device WiFi IP
+#include <arpa/inet.h>
+#include <ifaddrs.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+
+// TVM internal header to access Magic keys like kRPCMagic and others
+#include "../../../src/runtime/rpc/rpc_endpoint.h"
+
+namespace tvm {
+namespace runtime {
+
+/*!
+ * \brief Message handling function for event driven server.
+ *
+ * \param in_bytes The incoming bytes.
+ * \param event_flag  1: read_available, 2: write_avaiable.
+ * \return State flag.
+ *     1: continue running, no need to write,
+ *     2: need to write
+ *     0: shutdown
+ */
+using FEventHandler = PackedFunc;
+
+/*!
+ * \brief Create a server event handler.
+ *
+ * \param outputStream The output stream used to send outputs.
+ * \param name The name of the server.
+ * \param remote_key The remote key
+ * \return The event handler.
+ */
+FEventHandler CreateServerEventHandler(NSOutputStream* outputStream, std::string name,
+                                       std::string remote_key) {
+  const PackedFunc* event_handler_factor = Registry::Get("rpc.CreateEventDrivenServer");
+  ICHECK(event_handler_factor != nullptr)
+      << "You are using tvm_runtime module built without RPC support. "
+      << "Please rebuild it with USE_RPC flag.";
+
+  PackedFunc writer_func([outputStream](TVMArgs args, TVMRetValue* rv) {
+    TVMByteArray* data = args[0].ptr<TVMByteArray>();
+    int64_t nbytes = [outputStream write:reinterpret_cast<const uint8_t*>(data->data)
+                               maxLength:data->size];
+    if (nbytes < 0) {
+      NSLog(@"%@", [outputStream streamError].localizedDescription);
+      throw tvm::Error("Stream error");
+    }
+    *rv = nbytes;
+  });
+
+  return (*event_handler_factor)(writer_func, name, remote_key);
+}
+
+/*!
+ * \brief Helper function to query real IP of device in WiFi network
+ * \return string with IPv4 in format "192.168.0.1" or "unknown" if cannot detect
+ */
+static std::string getWiFiAddress() {
+  std::string address = "unknown";
+  ifaddrs* interfaces = nullptr;
+
+  int success = getifaddrs(&interfaces);
+  if (success == 0) {
+    ifaddrs* temp_addr = interfaces;
+    while (temp_addr != NULL) {
+      if (temp_addr->ifa_addr->sa_family == AF_INET) {
+        // Check if interface is en0 which is the wifi connection on the iPhone
+        if (std::string(temp_addr->ifa_name) == "en0") {
+          address = inet_ntoa(((sockaddr_in*)temp_addr->ifa_addr)->sin_addr);
+        }
+      }
+      temp_addr = temp_addr->ifa_next;
+    }
+  }
+
+  freeifaddrs(interfaces);
+  return address;
+}
+
+}  // namespace runtime
+}  // namespace tvm
+
+// Base class for any type of RPC servicing
+@interface RPCServerBase : RPCServer
+
+/*!
+ * Methods should be implemented in inherited classes
+ */
+- (bool)onReadHandler;   // return true - continue feeding, false - stop, try to drain output buffer
+- (bool)onWriteHandler;  // return true - continue draining, false - no data to write
+- (void)onEndEncountered;  // called on disconnect or session desided that it's shutdown time
+- (void)open;              // Initiate listening objects like i/o streams and other resources
+- (void)close;             // Deinitialize resources opend in "open" method
+@end
+
+@implementation RPCServerBase {
+  // Worker thread
+  NSThread* worker_thread_;
+  // Triger to continue RunLoop processing inside worker_thread_
+  BOOL shouldKeepRunning;
+  // Input socket stream
+ @protected
+  NSInputStream* inputStream_;
+  // Output socket stream
+  NSOutputStream* outputStream_;
+  // Temporal buffer with data to send
+  std::string sendBuffer_;
+  // Temporal receive buffer
+  std::string recvBuffer_;
+  // Requested data size to accumulate in recvBuffer_ before continue processing
+  int requiredToRecv_;
+}
+
+/*!
+ * Start internal worker thread with RunLoop and submit correspoding open handlers into it
+ * Not blocking
+ */
+- (void)start {
+  worker_thread_ = [[NSThread alloc] initWithBlock:^{
+    @autoreleasepool {
+      [self notifyState:RPCServerStatus_Launched];
+      [self open];
+      shouldKeepRunning = YES;
+      while (shouldKeepRunning && [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
+                                                           beforeDate:[NSDate distantFuture]])
+        ;
+      [self notifyState:RPCServerStatus_Stopped];
+    }
+  }];
+  [worker_thread_ start];
+}
+
+/*!
+ * Send message to workel thread runloop to finish processing
+ * Not blocking
+ */
+- (void)stop {
+  if (worker_thread_ == nil) return;
+
+  [self performSelector:@selector(stop_) onThread:worker_thread_ withObject:nil waitUntilDone:NO];
+  worker_thread_ = nil;  // TODO: is it valide? may be better to do that inside NSThread?
+}
+
+- (void)stop_ {
+  [self close];
+  shouldKeepRunning = NO;
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will connect to host and port specified in corresponding properties
+ */
+- (void)open {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)self.host, self.port, &readStream,
+                                     &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will assign i/o streams to provided socket connection.
+ */
+- (void)openWithSocket:(CFSocketNativeHandle)sock {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocket(NULL, sock, &readStream, &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Close i/o streams assosiated with connection
+ */
+- (void)close {
+  [inputStream_ close];
+  [outputStream_ close];
+  [inputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [inputStream_ setDelegate:nil];
+  [outputStream_ setDelegate:nil];
+  inputStream_ = nil;
+  outputStream_ = nil;
+}
+
+/// Unimplemented stubs
+- (bool)onReadHandler {
+  return false;
+}
+- (bool)onWriteHandler {
+  return false;
+}
+- (void)onEndEncountered {
+}
+
+/*!
+ * Try to read data from stream and call processing hadnler
+ */
+- (void)tryToRead {
+  const int kBufferSize = 4 << 10;  // 4kB buffer
+  const int prev_size = recvBuffer_.size();
+  recvBuffer_.resize(kBufferSize);
+  size_t nbytes = [inputStream_ read:(uint8_t*)recvBuffer_.data() + prev_size
+                           maxLength:recvBuffer_.size() - prev_size];
+  recvBuffer_.resize(nbytes + prev_size);
+
+  // feed while it accept or requested particulat buffer size
+  while (!recvBuffer_.empty() && requiredToRecv_ <= recvBuffer_.size() && [self onReadHandler])
+    ;
+}
+
+/*!
+ * Try to write remaining data to stream and call processing hadnler
+ */
+- (void)tryToWrite {
+  if (!sendBuffer_.empty()) {
+    size_t nbytes = [outputStream_ write:(uint8_t*)sendBuffer_.data() maxLength:sendBuffer_.size()];
+    sendBuffer_.erase(0, nbytes);
+  }
+  // call write handler while it want be called and space is available
+  while (sendBuffer_.empty() && [outputStream_ hasSpaceAvailable] && [self onWriteHandler])
+    ;
+}
+
+/*!
+ * Main event handler of socket stream events
+ */
+- (void)stream:(NSStream*)strm handleEvent:(NSStreamEvent)event {
+  std::string buffer;
+  switch (event) {
+    case NSStreamEventOpenCompleted: {
+      // Nothing
+      break;
+    }
+    case NSStreamEventHasBytesAvailable:
+      if (strm == inputStream_) {
+        [self tryToRead];
+        if ([outputStream_ hasSpaceAvailable]) [self tryToWrite];
+      }
+      break;
+    case NSStreamEventHasSpaceAvailable: {
+      if (strm == outputStream_) {
+        [self tryToWrite];
+        if ([inputStream_ hasBytesAvailable]) [self tryToRead];
+      }
+      break;
+    }
+    case NSStreamEventErrorOccurred: {
+      [self notifyError:[strm streamError].localizedDescription];
+      break;
+    }
+    case NSStreamEventEndEncountered: {
+      [self onEndEncountered];
+      break;
+    }
+    default: {
+      NSLog(@"Unknown event");
+    }
+  }
+}
+
+#pragma mark - Helpers
+
+/*!
+ * Set buffer to send into stream. Try to send immediatly or submit to lazy sending
+ * Non blocking operation
+ */
+- (void)toSend:(NSData*)data {
+  sendBuffer_.append(static_cast<const char*>(data.bytes), data.length);
+
+  // try to flush buffer
+  NSInteger sent_size = [outputStream_ write:(uint8_t*)sendBuffer_.data()
+                                   maxLength:sendBuffer_.size()];
+  sendBuffer_.erase(0, sent_size);
+}
+
+/*!
+ * Set buffer to send  in packet format [size, data]. Behaviour is same as for toSend.
+ */
+- (void)toSendPacked:(NSData*)data {
+  int packet_size = data.length;
+  [self toSend:[NSData dataWithBytes:&packet_size length:sizeof(packet_size)]];
+  [self toSend:data];
+}
+
+/*!
+ */
+- (NSData*)requestInputDataWithSize:(NSInteger)size {
+  if (recvBuffer_.size() < size) {
+    requiredToRecv_ = size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() length:size];
+  recvBuffer_.erase(0, size);
+  return res;
+}
+
+/*!
+ */
+- (NSData*)requestInputDataPacked {
+  int size;
+  if (recvBuffer_.size() < sizeof(size)) {
+    requiredToRecv_ = sizeof(size);
+    return nil;
+  }
+  size = *(int*)recvBuffer_.data();
+  if (recvBuffer_.size() < sizeof(size) + size) {
+    requiredToRecv_ = sizeof(size) + size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() + sizeof(size) length:size];
+  recvBuffer_.erase(0, sizeof(size) + size);
+  return res;
+};
+
+#pragma mark - Notifiers
+
+/*!
+ * Notify external listener about error.
+ * Also print error message to std out in case of Verbose mode
+ */
+- (void)notifyError:(NSString*)msg {
+  // Duplicate error message in std output. Host launcher script may listen it.
+  if (self.verbose) NSLog(@"[IOS-RPC] ERROR: %@", msg);
+  if (self.delegate) [self.delegate onError:msg];
+}
+
+/*!
+ * Notify external listener about server state changes.
+ * Also print information to std out in case of Verbose mode
+ */
+- (void)notifyState:(RPCServerStatus)state {
+  // Duplicate sattus changing in std output. Host launcher script may listen it.
+  if (self.verbose) NSLog(@"[IOS-RPC] STATE: %d", state);
+  if (self.delegate != nil) [self.delegate onStatusChanged:state];
+}
+
+@end
+
+@interface RPCServerProxy : RPCServerBase
+@end
+
+typedef enum {
+  RPCServerProxyState_Idle,
+  RPCServerProxyState_HandshakeToSend,
+  RPCServerProxyState_HandshakeToRecv,
+  RPCServerProxyState_Processing,
+} RPCServerProxyState;
+
+@implementation RPCServerProxy {
+  /// Original TVM RPC event handler
+  tvm::runtime::FEventHandler handler_;
+ @protected
+  /// Sate of Proxy client implementation
+  RPCServerProxyState state_;
+}
+
+- (instancetype)init {
+  if (self = [super init]) {
+    handler_ = nullptr;
+    state_ = RPCServerProxyState_Idle;
+  }
+  return self;
+}
+
+/*!
+ * Implement matching of internat state on state available for outside users
+ */
+- (void)setState:(RPCServerProxyState)new_state {
+  // Send Connected notification because Proxy doesn't responce until client connected.
+  if (new_state == RPCServerProxyState_HandshakeToRecv)
+    [self notifyState:RPCServerStatus_Connected];
+  if (new_state == RPCServerProxyState_Idle) [self notifyState:RPCServerStatus_Disconnected];
+  if (state_ == RPCServerProxyState_HandshakeToRecv && new_state == RPCServerProxyState_Processing)
+    [self notifyState:RPCServerStatus_RPCSessionStarted];
+  if (state_ == RPCServerProxyState_Processing && new_state == RPCServerProxyState_Idle)
+    [self notifyState:RPCServerStatus_RPCSessionStarted];
+
+  state_ = new_state;
+}
+
+- (bool)onWriteHandler {
+  switch (state_) {
+    case RPCServerProxyState_HandshakeToSend: {
+      // Send together kRPCMagic and server descriptor because of Proxy
+      int code = tvm::runtime::kRPCMagic;
+      [self toSend:[NSData dataWithBytes:&code length:sizeof(code)]];
+
+      std::string full_key = std::string("server:") + self.key.UTF8String;
+      [self toSendPacked:[NSData dataWithBytes:full_key.data() length:full_key.size()]];
+
+      self.state = RPCServerProxyState_HandshakeToRecv;
+      return TRUE;
+    }
+    case RPCServerProxyState_Processing: {
+      try {
+        TVMByteArray dummy{nullptr, 0};
+        int flag = handler_(dummy, 2);
+        if (flag == 0) {
+          [self onEndEncountered];
+        }
+        return flag == 2;
+      } catch (const tvm::Error& e) {
+        [self close];
+      }
+      break;
+    }
+    default:
+      // Nothing
+      break;
+  }
+  return FALSE;
+}
+
+- (bool)onReadHandler {
+  switch (state_) {
+    case RPCServerProxyState_HandshakeToRecv: {
+      NSData* data = [self requestInputDataWithSize:sizeof(int)];
+      if (data == nil) return FALSE;
+
+      if (*(int*)data.bytes != tvm::runtime::kRPCMagic) {
+        [self notifyError:@"Wrong responce, is not RPC client."];
+        [self close];
+        return FALSE;
+        break;
+      }
+
+      handler_ = tvm::runtime::CreateServerEventHandler(outputStream_, "iphone", "%toinit");
+
+      self.state = RPCServerProxyState_Processing;
+      return TRUE;
+      break;
+    }
+    case RPCServerProxyState_Processing: {
+      int flag = 1;
+      if ([outputStream_ hasSpaceAvailable]) {
+        flag |= 2;
+      }
+      // always try to write
+      try {
+        TVMByteArray arr{recvBuffer_.data(), recvBuffer_.size()};
+        flag = handler_(arr, flag);
+        recvBuffer_.clear();
+        if (flag == 0) {
+          [self onEndEncountered];
+        }
+        return flag == 1;
+      } catch (const tvm::Error& e) {
+        [self close];
+      }
+      break;
+    }
+    default:
+      // Nothing
+      break;
+  }
+  return FALSE;
+}
+
+- (void)onEndEncountered {
+  // Automatic reconnection when session is finished.

Review comment:
       close and open methods will do that automatically via setting new state. So it will be logged with next messages "RPCSessionFinished", "RPCServerStatus_Disconnected" and "RPCServerStatus_Connected".
   
   Not sure that we are interested in real reason of reconnection, was it broken TCP connection or end of RPC session.

##########
File path: apps/ios_rpc/tvmrpc/RPCArgs.mm
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#import "RPCArgs.h"
+
+#import <Foundation/Foundation.h>
+
+#import "../../../src/support/socket.h"
+#import "../../../src/support/utils.h"
+
+#import <string>
+
+using std::string;
+
+const char* kUsage =
+    "\n"
+    "iOS tvmrpc application supported flags:\n"
+    "--host_url      The tracker/proxy address, Default=0.0.0.0\n"
+    "--host_port     The tracker/proxy port, Default=9190\n"
+    "--key           The key used to identify the device type in tracker. Default=\"\"\n"
+    "--custom_addr   Custom IP Address to Report to RPC Tracker. Default=\"\"\n"
+    "--immediate_connect   No UI interconnection, connect to tracker immediately. Default=False\n"
+    "--verbose       Allow to print status info to std out. Default=False\n"
+    "--server_mode   Server mode. Can be \"pure_server\", \"proxy\" or \"tracker\". "
+    "Default=pure_server \n"
+    "\n";
+
+struct RPCArgs_cpp {
+  string host_url = "0.0.0.0";
+  int host_port = 9190;
+
+  string key;
+  string custom_addr = "";
+
+  bool immediate_connect = false;
+  bool verbose = false;
+  char server_mode = 0;

Review comment:
       Sure, it could be enum. I've changed it to use enum form RPCServer.h.  

##########
File path: apps/ios_rpc/tvmrpc/RPCArgs.mm
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#import "RPCArgs.h"
+
+#import <Foundation/Foundation.h>
+
+#import "../../../src/support/socket.h"
+#import "../../../src/support/utils.h"
+
+#import <string>
+
+using std::string;
+
+const char* kUsage =
+    "\n"
+    "iOS tvmrpc application supported flags:\n"
+    "--host_url      The tracker/proxy address, Default=0.0.0.0\n"
+    "--host_port     The tracker/proxy port, Default=9190\n"
+    "--key           The key used to identify the device type in tracker. Default=\"\"\n"
+    "--custom_addr   Custom IP Address to Report to RPC Tracker. Default=\"\"\n"
+    "--immediate_connect   No UI interconnection, connect to tracker immediately. Default=False\n"
+    "--verbose       Allow to print status info to std out. Default=False\n"
+    "--server_mode   Server mode. Can be \"pure_server\", \"proxy\" or \"tracker\". "
+    "Default=pure_server \n"
+    "\n";
+
+struct RPCArgs_cpp {
+  string host_url = "0.0.0.0";
+  int host_port = 9190;
+
+  string key;
+  string custom_addr = "";
+
+  bool immediate_connect = false;
+  bool verbose = false;
+  char server_mode = 0;
+
+  operator RPCArgs() const {
+    return RPCArgs{.host_url = host_url.c_str(),
+                   .host_port = host_port,
+                   .key = key.c_str(),
+                   .custom_addr = custom_addr.c_str(),
+                   .verbose = verbose,
+                   .immediate_connect = immediate_connect,
+                   .server_mode = server_mode};
+  };
+
+  RPCArgs_cpp& operator=(const RPCArgs& args) {
+    host_url = args.host_url;
+    host_port = args.host_port;
+    key = args.key;
+    custom_addr = args.custom_addr;
+    verbose = args.verbose;
+    immediate_connect = args.immediate_connect;
+    server_mode = args.server_mode;
+    return *this;
+  }
+};
+
+struct RPCArgs_cpp g_rpc_args;
+
+static void restore_from_cache() {
+  NSUserDefaults* defaults = [NSUserDefaults standardUserDefaults];
+
+  auto get_string_from_cache = [defaults](const char* key) {
+    NSString* ns_key = [NSString stringWithUTF8String:key];
+    NSString* ns_val = [defaults stringForKey:ns_key];
+    return std::string(ns_val != nil ? [ns_val UTF8String] : "");
+  };
+
+  auto get_int_from_cache = [defaults](const char* key) {
+    NSString* ns_key = [NSString stringWithUTF8String:key];
+    return static_cast<int>([defaults integerForKey:ns_key]);
+  };
+
+  g_rpc_args.host_url = get_string_from_cache("tmvrpc_url");
+  g_rpc_args.host_port = get_int_from_cache("tmvrpc_port");
+  g_rpc_args.key = get_string_from_cache("tmvrpc_key");
+}
+
+static void update_in_cache() {
+  NSUserDefaults* defaults = [NSUserDefaults standardUserDefaults];
+
+  [defaults setObject:[NSString stringWithUTF8String:g_rpc_args.host_url.c_str()]
+               forKey:@"tmvrpc_url"];
+  [defaults setInteger:g_rpc_args.host_port forKey:@"tmvrpc_port"];
+  [defaults setObject:[NSString stringWithUTF8String:g_rpc_args.key.c_str()] forKey:@"tmvrpc_key"];

Review comment:
       Fixed.

##########
File path: apps/ios_rpc/tvmrpc/RPCServer.mm
##########
@@ -0,0 +1,809 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file ViewController.mm
+ */
+
+#import "RPCServer.h"
+
+#include <tvm/runtime/packed_func.h>
+#include <tvm/runtime/registry.h>
+
+#include <random>
+#include <string>
+
+// To get device WiFi IP
+#include <arpa/inet.h>
+#include <ifaddrs.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+
+// TVM internal header to access Magic keys like kRPCMagic and others
+#include "../../../src/runtime/rpc/rpc_endpoint.h"
+
+namespace tvm {
+namespace runtime {
+
+/*!
+ * \brief Message handling function for event driven server.
+ *
+ * \param in_bytes The incoming bytes.
+ * \param event_flag  1: read_available, 2: write_avaiable.
+ * \return State flag.
+ *     1: continue running, no need to write,
+ *     2: need to write
+ *     0: shutdown
+ */
+using FEventHandler = PackedFunc;
+
+/*!
+ * \brief Create a server event handler.
+ *
+ * \param outputStream The output stream used to send outputs.
+ * \param name The name of the server.
+ * \param remote_key The remote key
+ * \return The event handler.
+ */
+FEventHandler CreateServerEventHandler(NSOutputStream* outputStream, std::string name,
+                                       std::string remote_key) {
+  const PackedFunc* event_handler_factor = Registry::Get("rpc.CreateEventDrivenServer");
+  ICHECK(event_handler_factor != nullptr)
+      << "You are using tvm_runtime module built without RPC support. "
+      << "Please rebuild it with USE_RPC flag.";
+
+  PackedFunc writer_func([outputStream](TVMArgs args, TVMRetValue* rv) {
+    TVMByteArray* data = args[0].ptr<TVMByteArray>();
+    int64_t nbytes = [outputStream write:reinterpret_cast<const uint8_t*>(data->data)
+                               maxLength:data->size];
+    if (nbytes < 0) {
+      NSLog(@"%@", [outputStream streamError].localizedDescription);
+      throw tvm::Error("Stream error");
+    }
+    *rv = nbytes;
+  });
+
+  return (*event_handler_factor)(writer_func, name, remote_key);
+}
+
+/*!
+ * \brief Helper function to query real IP of device in WiFi network
+ * \return string with IPv4 in format "192.168.0.1" or "unknown" if cannot detect
+ */
+static std::string getWiFiAddress() {
+  std::string address = "unknown";
+  ifaddrs* interfaces = nullptr;
+
+  int success = getifaddrs(&interfaces);
+  if (success == 0) {
+    ifaddrs* temp_addr = interfaces;
+    while (temp_addr != NULL) {
+      if (temp_addr->ifa_addr->sa_family == AF_INET) {
+        // Check if interface is en0 which is the wifi connection on the iPhone
+        if (std::string(temp_addr->ifa_name) == "en0") {
+          address = inet_ntoa(((sockaddr_in*)temp_addr->ifa_addr)->sin_addr);
+        }
+      }
+      temp_addr = temp_addr->ifa_next;
+    }
+  }
+
+  freeifaddrs(interfaces);
+  return address;
+}
+
+}  // namespace runtime
+}  // namespace tvm
+
+// Base class for any type of RPC servicing
+@interface RPCServerBase : RPCServer
+
+/*!
+ * Methods should be implemented in inherited classes
+ */
+- (bool)onReadHandler;   // return true - continue feeding, false - stop, try to drain output buffer
+- (bool)onWriteHandler;  // return true - continue draining, false - no data to write
+- (void)onEndEncountered;  // called on disconnect or session desided that it's shutdown time
+- (void)open;              // Initiate listening objects like i/o streams and other resources
+- (void)close;             // Deinitialize resources opend in "open" method
+@end
+
+@implementation RPCServerBase {
+  // Worker thread
+  NSThread* worker_thread_;
+  // Triger to continue RunLoop processing inside worker_thread_
+  BOOL shouldKeepRunning;
+  // Input socket stream
+ @protected
+  NSInputStream* inputStream_;
+  // Output socket stream
+  NSOutputStream* outputStream_;
+  // Temporal buffer with data to send
+  std::string sendBuffer_;
+  // Temporal receive buffer
+  std::string recvBuffer_;
+  // Requested data size to accumulate in recvBuffer_ before continue processing
+  int requiredToRecv_;
+}
+
+/*!
+ * Start internal worker thread with RunLoop and submit correspoding open handlers into it
+ * Not blocking
+ */
+- (void)start {
+  worker_thread_ = [[NSThread alloc] initWithBlock:^{
+    @autoreleasepool {
+      [self notifyState:RPCServerStatus_Launched];
+      [self open];
+      shouldKeepRunning = YES;
+      while (shouldKeepRunning && [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
+                                                           beforeDate:[NSDate distantFuture]])
+        ;
+      [self notifyState:RPCServerStatus_Stopped];
+    }
+  }];
+  [worker_thread_ start];
+}
+
+/*!
+ * Send message to workel thread runloop to finish processing
+ * Not blocking
+ */
+- (void)stop {
+  if (worker_thread_ == nil) return;
+
+  [self performSelector:@selector(stop_) onThread:worker_thread_ withObject:nil waitUntilDone:NO];
+  worker_thread_ = nil;  // TODO: is it valide? may be better to do that inside NSThread?
+}
+
+- (void)stop_ {
+  [self close];
+  shouldKeepRunning = NO;
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will connect to host and port specified in corresponding properties
+ */
+- (void)open {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)self.host, self.port, &readStream,
+                                     &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will assign i/o streams to provided socket connection.
+ */
+- (void)openWithSocket:(CFSocketNativeHandle)sock {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocket(NULL, sock, &readStream, &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Close i/o streams assosiated with connection
+ */
+- (void)close {
+  [inputStream_ close];
+  [outputStream_ close];
+  [inputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [inputStream_ setDelegate:nil];
+  [outputStream_ setDelegate:nil];
+  inputStream_ = nil;
+  outputStream_ = nil;
+}
+
+/// Unimplemented stubs
+- (bool)onReadHandler {
+  return false;
+}
+- (bool)onWriteHandler {
+  return false;
+}
+- (void)onEndEncountered {
+}
+
+/*!
+ * Try to read data from stream and call processing hadnler
+ */
+- (void)tryToRead {
+  const int kBufferSize = 4 << 10;  // 4kB buffer
+  const int prev_size = recvBuffer_.size();
+  recvBuffer_.resize(kBufferSize);
+  size_t nbytes = [inputStream_ read:(uint8_t*)recvBuffer_.data() + prev_size
+                           maxLength:recvBuffer_.size() - prev_size];
+  recvBuffer_.resize(nbytes + prev_size);
+
+  // feed while it accept or requested particulat buffer size
+  while (!recvBuffer_.empty() && requiredToRecv_ <= recvBuffer_.size() && [self onReadHandler])
+    ;
+}
+
+/*!
+ * Try to write remaining data to stream and call processing hadnler
+ */
+- (void)tryToWrite {
+  if (!sendBuffer_.empty()) {
+    size_t nbytes = [outputStream_ write:(uint8_t*)sendBuffer_.data() maxLength:sendBuffer_.size()];
+    sendBuffer_.erase(0, nbytes);
+  }
+  // call write handler while it want be called and space is available
+  while (sendBuffer_.empty() && [outputStream_ hasSpaceAvailable] && [self onWriteHandler])
+    ;
+}
+
+/*!
+ * Main event handler of socket stream events
+ */
+- (void)stream:(NSStream*)strm handleEvent:(NSStreamEvent)event {
+  std::string buffer;
+  switch (event) {
+    case NSStreamEventOpenCompleted: {
+      // Nothing
+      break;
+    }
+    case NSStreamEventHasBytesAvailable:
+      if (strm == inputStream_) {
+        [self tryToRead];
+        if ([outputStream_ hasSpaceAvailable]) [self tryToWrite];
+      }
+      break;
+    case NSStreamEventHasSpaceAvailable: {
+      if (strm == outputStream_) {
+        [self tryToWrite];
+        if ([inputStream_ hasBytesAvailable]) [self tryToRead];
+      }
+      break;
+    }
+    case NSStreamEventErrorOccurred: {
+      [self notifyError:[strm streamError].localizedDescription];
+      break;
+    }
+    case NSStreamEventEndEncountered: {
+      [self onEndEncountered];
+      break;
+    }
+    default: {
+      NSLog(@"Unknown event");
+    }
+  }
+}
+
+#pragma mark - Helpers
+
+/*!
+ * Set buffer to send into stream. Try to send immediatly or submit to lazy sending
+ * Non blocking operation
+ */
+- (void)toSend:(NSData*)data {
+  sendBuffer_.append(static_cast<const char*>(data.bytes), data.length);
+
+  // try to flush buffer
+  NSInteger sent_size = [outputStream_ write:(uint8_t*)sendBuffer_.data()
+                                   maxLength:sendBuffer_.size()];
+  sendBuffer_.erase(0, sent_size);
+}
+
+/*!
+ * Set buffer to send  in packet format [size, data]. Behaviour is same as for toSend.
+ */
+- (void)toSendPacked:(NSData*)data {
+  int packet_size = data.length;
+  [self toSend:[NSData dataWithBytes:&packet_size length:sizeof(packet_size)]];
+  [self toSend:data];
+}
+
+/*!
+ */
+- (NSData*)requestInputDataWithSize:(NSInteger)size {
+  if (recvBuffer_.size() < size) {
+    requiredToRecv_ = size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() length:size];
+  recvBuffer_.erase(0, size);
+  return res;
+}
+
+/*!
+ */
+- (NSData*)requestInputDataPacked {
+  int size;

Review comment:
       Agree. Moved to int32_t everywhere. 

##########
File path: apps/ios_rpc/tvmrpc/RPCServer.mm
##########
@@ -0,0 +1,809 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file ViewController.mm
+ */
+
+#import "RPCServer.h"
+
+#include <tvm/runtime/packed_func.h>
+#include <tvm/runtime/registry.h>
+
+#include <random>
+#include <string>
+
+// To get device WiFi IP
+#include <arpa/inet.h>
+#include <ifaddrs.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+
+// TVM internal header to access Magic keys like kRPCMagic and others
+#include "../../../src/runtime/rpc/rpc_endpoint.h"
+
+namespace tvm {
+namespace runtime {
+
+/*!
+ * \brief Message handling function for event driven server.
+ *
+ * \param in_bytes The incoming bytes.
+ * \param event_flag  1: read_available, 2: write_avaiable.
+ * \return State flag.
+ *     1: continue running, no need to write,
+ *     2: need to write
+ *     0: shutdown
+ */
+using FEventHandler = PackedFunc;
+
+/*!
+ * \brief Create a server event handler.
+ *
+ * \param outputStream The output stream used to send outputs.
+ * \param name The name of the server.
+ * \param remote_key The remote key
+ * \return The event handler.
+ */
+FEventHandler CreateServerEventHandler(NSOutputStream* outputStream, std::string name,
+                                       std::string remote_key) {
+  const PackedFunc* event_handler_factor = Registry::Get("rpc.CreateEventDrivenServer");
+  ICHECK(event_handler_factor != nullptr)
+      << "You are using tvm_runtime module built without RPC support. "
+      << "Please rebuild it with USE_RPC flag.";
+
+  PackedFunc writer_func([outputStream](TVMArgs args, TVMRetValue* rv) {
+    TVMByteArray* data = args[0].ptr<TVMByteArray>();
+    int64_t nbytes = [outputStream write:reinterpret_cast<const uint8_t*>(data->data)
+                               maxLength:data->size];
+    if (nbytes < 0) {
+      NSLog(@"%@", [outputStream streamError].localizedDescription);
+      throw tvm::Error("Stream error");
+    }
+    *rv = nbytes;
+  });
+
+  return (*event_handler_factor)(writer_func, name, remote_key);
+}
+
+/*!
+ * \brief Helper function to query real IP of device in WiFi network
+ * \return string with IPv4 in format "192.168.0.1" or "unknown" if cannot detect
+ */
+static std::string getWiFiAddress() {
+  std::string address = "unknown";
+  ifaddrs* interfaces = nullptr;
+
+  int success = getifaddrs(&interfaces);
+  if (success == 0) {
+    ifaddrs* temp_addr = interfaces;
+    while (temp_addr != NULL) {
+      if (temp_addr->ifa_addr->sa_family == AF_INET) {
+        // Check if interface is en0 which is the wifi connection on the iPhone
+        if (std::string(temp_addr->ifa_name) == "en0") {
+          address = inet_ntoa(((sockaddr_in*)temp_addr->ifa_addr)->sin_addr);
+        }
+      }
+      temp_addr = temp_addr->ifa_next;
+    }
+  }
+
+  freeifaddrs(interfaces);
+  return address;
+}
+
+}  // namespace runtime
+}  // namespace tvm
+
+// Base class for any type of RPC servicing
+@interface RPCServerBase : RPCServer
+
+/*!
+ * Methods should be implemented in inherited classes
+ */
+- (bool)onReadHandler;   // return true - continue feeding, false - stop, try to drain output buffer
+- (bool)onWriteHandler;  // return true - continue draining, false - no data to write
+- (void)onEndEncountered;  // called on disconnect or session desided that it's shutdown time
+- (void)open;              // Initiate listening objects like i/o streams and other resources
+- (void)close;             // Deinitialize resources opend in "open" method
+@end
+
+@implementation RPCServerBase {
+  // Worker thread
+  NSThread* worker_thread_;
+  // Triger to continue RunLoop processing inside worker_thread_
+  BOOL shouldKeepRunning;
+  // Input socket stream
+ @protected
+  NSInputStream* inputStream_;
+  // Output socket stream
+  NSOutputStream* outputStream_;
+  // Temporal buffer with data to send
+  std::string sendBuffer_;
+  // Temporal receive buffer
+  std::string recvBuffer_;
+  // Requested data size to accumulate in recvBuffer_ before continue processing
+  int requiredToRecv_;
+}
+
+/*!
+ * Start internal worker thread with RunLoop and submit correspoding open handlers into it
+ * Not blocking
+ */
+- (void)start {
+  worker_thread_ = [[NSThread alloc] initWithBlock:^{
+    @autoreleasepool {
+      [self notifyState:RPCServerStatus_Launched];
+      [self open];
+      shouldKeepRunning = YES;
+      while (shouldKeepRunning && [[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode
+                                                           beforeDate:[NSDate distantFuture]])
+        ;
+      [self notifyState:RPCServerStatus_Stopped];
+    }
+  }];
+  [worker_thread_ start];
+}
+
+/*!
+ * Send message to workel thread runloop to finish processing
+ * Not blocking
+ */
+- (void)stop {
+  if (worker_thread_ == nil) return;
+
+  [self performSelector:@selector(stop_) onThread:worker_thread_ withObject:nil waitUntilDone:NO];
+  worker_thread_ = nil;  // TODO: is it valide? may be better to do that inside NSThread?
+}
+
+- (void)stop_ {
+  [self close];
+  shouldKeepRunning = NO;
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will connect to host and port specified in corresponding properties
+ */
+- (void)open {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)self.host, self.port, &readStream,
+                                     &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Base implementation to selup i/o streams
+ * Will assign i/o streams to provided socket connection.
+ */
+- (void)openWithSocket:(CFSocketNativeHandle)sock {
+  CFReadStreamRef readStream;
+  CFWriteStreamRef writeStream;
+  CFStreamCreatePairWithSocket(NULL, sock, &readStream, &writeStream);
+  inputStream_ = (__bridge NSInputStream*)readStream;
+  outputStream_ = (__bridge NSOutputStream*)writeStream;
+  [inputStream_ setDelegate:self];
+  [outputStream_ setDelegate:self];
+  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ open];
+  [inputStream_ open];
+}
+
+/*!
+ * Close i/o streams assosiated with connection
+ */
+- (void)close {
+  [inputStream_ close];
+  [outputStream_ close];
+  [inputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [outputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
+  [inputStream_ setDelegate:nil];
+  [outputStream_ setDelegate:nil];
+  inputStream_ = nil;
+  outputStream_ = nil;
+}
+
+/// Unimplemented stubs
+- (bool)onReadHandler {
+  return false;
+}
+- (bool)onWriteHandler {
+  return false;
+}
+- (void)onEndEncountered {
+}
+
+/*!
+ * Try to read data from stream and call processing hadnler
+ */
+- (void)tryToRead {
+  const int kBufferSize = 4 << 10;  // 4kB buffer
+  const int prev_size = recvBuffer_.size();
+  recvBuffer_.resize(kBufferSize);
+  size_t nbytes = [inputStream_ read:(uint8_t*)recvBuffer_.data() + prev_size
+                           maxLength:recvBuffer_.size() - prev_size];
+  recvBuffer_.resize(nbytes + prev_size);
+
+  // feed while it accept or requested particulat buffer size
+  while (!recvBuffer_.empty() && requiredToRecv_ <= recvBuffer_.size() && [self onReadHandler])
+    ;
+}
+
+/*!
+ * Try to write remaining data to stream and call processing hadnler
+ */
+- (void)tryToWrite {
+  if (!sendBuffer_.empty()) {
+    size_t nbytes = [outputStream_ write:(uint8_t*)sendBuffer_.data() maxLength:sendBuffer_.size()];
+    sendBuffer_.erase(0, nbytes);
+  }
+  // call write handler while it want be called and space is available
+  while (sendBuffer_.empty() && [outputStream_ hasSpaceAvailable] && [self onWriteHandler])
+    ;
+}
+
+/*!
+ * Main event handler of socket stream events
+ */
+- (void)stream:(NSStream*)strm handleEvent:(NSStreamEvent)event {
+  std::string buffer;
+  switch (event) {
+    case NSStreamEventOpenCompleted: {
+      // Nothing
+      break;
+    }
+    case NSStreamEventHasBytesAvailable:
+      if (strm == inputStream_) {
+        [self tryToRead];
+        if ([outputStream_ hasSpaceAvailable]) [self tryToWrite];
+      }
+      break;
+    case NSStreamEventHasSpaceAvailable: {
+      if (strm == outputStream_) {
+        [self tryToWrite];
+        if ([inputStream_ hasBytesAvailable]) [self tryToRead];
+      }
+      break;
+    }
+    case NSStreamEventErrorOccurred: {
+      [self notifyError:[strm streamError].localizedDescription];
+      break;
+    }
+    case NSStreamEventEndEncountered: {
+      [self onEndEncountered];
+      break;
+    }
+    default: {
+      NSLog(@"Unknown event");
+    }
+  }
+}
+
+#pragma mark - Helpers
+
+/*!
+ * Set buffer to send into stream. Try to send immediatly or submit to lazy sending
+ * Non blocking operation
+ */
+- (void)toSend:(NSData*)data {
+  sendBuffer_.append(static_cast<const char*>(data.bytes), data.length);
+
+  // try to flush buffer
+  NSInteger sent_size = [outputStream_ write:(uint8_t*)sendBuffer_.data()
+                                   maxLength:sendBuffer_.size()];
+  sendBuffer_.erase(0, sent_size);
+}
+
+/*!
+ * Set buffer to send  in packet format [size, data]. Behaviour is same as for toSend.
+ */
+- (void)toSendPacked:(NSData*)data {
+  int packet_size = data.length;
+  [self toSend:[NSData dataWithBytes:&packet_size length:sizeof(packet_size)]];
+  [self toSend:data];
+}
+
+/*!
+ */
+- (NSData*)requestInputDataWithSize:(NSInteger)size {
+  if (recvBuffer_.size() < size) {
+    requiredToRecv_ = size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() length:size];
+  recvBuffer_.erase(0, size);
+  return res;
+}
+
+/*!
+ */
+- (NSData*)requestInputDataPacked {
+  int size;
+  if (recvBuffer_.size() < sizeof(size)) {
+    requiredToRecv_ = sizeof(size);
+    return nil;
+  }
+  size = *(int*)recvBuffer_.data();
+  if (recvBuffer_.size() < sizeof(size) + size) {
+    requiredToRecv_ = sizeof(size) + size;
+    return nil;
+  }
+  NSData* res = [NSData dataWithBytes:recvBuffer_.data() + sizeof(size) length:size];
+  recvBuffer_.erase(0, sizeof(size) + size);
+  return res;
+};
+
+#pragma mark - Notifiers
+
+/*!
+ * Notify external listener about error.
+ * Also print error message to std out in case of Verbose mode
+ */
+- (void)notifyError:(NSString*)msg {
+  // Duplicate error message in std output. Host launcher script may listen it.
+  if (self.verbose) NSLog(@"[IOS-RPC] ERROR: %@", msg);
+  if (self.delegate) [self.delegate onError:msg];
+}
+
+/*!
+ * Notify external listener about server state changes.
+ * Also print information to std out in case of Verbose mode
+ */
+- (void)notifyState:(RPCServerStatus)state {
+  // Duplicate sattus changing in std output. Host launcher script may listen it.
+  if (self.verbose) NSLog(@"[IOS-RPC] STATE: %d", state);
+  if (self.delegate != nil) [self.delegate onStatusChanged:state];
+}
+
+@end
+
+@interface RPCServerProxy : RPCServerBase
+@end
+
+typedef enum {
+  RPCServerProxyState_Idle,
+  RPCServerProxyState_HandshakeToSend,
+  RPCServerProxyState_HandshakeToRecv,
+  RPCServerProxyState_Processing,
+} RPCServerProxyState;
+
+@implementation RPCServerProxy {
+  /// Original TVM RPC event handler
+  tvm::runtime::FEventHandler handler_;
+ @protected
+  /// Sate of Proxy client implementation
+  RPCServerProxyState state_;
+}
+
+- (instancetype)init {
+  if (self = [super init]) {
+    handler_ = nullptr;
+    state_ = RPCServerProxyState_Idle;
+  }
+  return self;
+}
+
+/*!
+ * Implement matching of internat state on state available for outside users
+ */
+- (void)setState:(RPCServerProxyState)new_state {
+  // Send Connected notification because Proxy doesn't responce until client connected.
+  if (new_state == RPCServerProxyState_HandshakeToRecv)
+    [self notifyState:RPCServerStatus_Connected];
+  if (new_state == RPCServerProxyState_Idle) [self notifyState:RPCServerStatus_Disconnected];
+  if (state_ == RPCServerProxyState_HandshakeToRecv && new_state == RPCServerProxyState_Processing)
+    [self notifyState:RPCServerStatus_RPCSessionStarted];
+  if (state_ == RPCServerProxyState_Processing && new_state == RPCServerProxyState_Idle)
+    [self notifyState:RPCServerStatus_RPCSessionStarted];
+
+  state_ = new_state;
+}
+
+- (bool)onWriteHandler {
+  switch (state_) {
+    case RPCServerProxyState_HandshakeToSend: {
+      // Send together kRPCMagic and server descriptor because of Proxy
+      int code = tvm::runtime::kRPCMagic;

Review comment:
       Fixed

##########
File path: apps/ios_rpc/tvmrpc/RPCArgs.mm
##########
@@ -0,0 +1,197 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+#import "RPCArgs.h"
+
+#import <Foundation/Foundation.h>
+
+#import "../../../src/support/socket.h"
+#import "../../../src/support/utils.h"
+
+#import <string>
+
+using std::string;
+
+const char* kUsage =
+    "\n"
+    "iOS tvmrpc application supported flags:\n"
+    "--host_url      The tracker/proxy address, Default=0.0.0.0\n"
+    "--host_port     The tracker/proxy port, Default=9190\n"
+    "--key           The key used to identify the device type in tracker. Default=\"\"\n"
+    "--custom_addr   Custom IP Address to Report to RPC Tracker. Default=\"\"\n"
+    "--immediate_connect   No UI interconnection, connect to tracker immediately. Default=False\n"
+    "--verbose       Allow to print status info to std out. Default=False\n"
+    "--server_mode   Server mode. Can be \"pure_server\", \"proxy\" or \"tracker\". "
+    "Default=pure_server \n"
+    "\n";
+
+struct RPCArgs_cpp {
+  string host_url = "0.0.0.0";
+  int host_port = 9190;
+
+  string key;
+  string custom_addr = "";
+
+  bool immediate_connect = false;
+  bool verbose = false;
+  char server_mode = 0;
+
+  operator RPCArgs() const {
+    return RPCArgs{.host_url = host_url.c_str(),
+                   .host_port = host_port,
+                   .key = key.c_str(),
+                   .custom_addr = custom_addr.c_str(),
+                   .verbose = verbose,
+                   .immediate_connect = immediate_connect,
+                   .server_mode = server_mode};
+  };
+
+  RPCArgs_cpp& operator=(const RPCArgs& args) {
+    host_url = args.host_url;
+    host_port = args.host_port;
+    key = args.key;
+    custom_addr = args.custom_addr;
+    verbose = args.verbose;
+    immediate_connect = args.immediate_connect;
+    server_mode = args.server_mode;
+    return *this;
+  }
+};
+
+struct RPCArgs_cpp g_rpc_args;
+
+static void restore_from_cache() {
+  NSUserDefaults* defaults = [NSUserDefaults standardUserDefaults];
+
+  auto get_string_from_cache = [defaults](const char* key) {
+    NSString* ns_key = [NSString stringWithUTF8String:key];
+    NSString* ns_val = [defaults stringForKey:ns_key];
+    return std::string(ns_val != nil ? [ns_val UTF8String] : "");
+  };
+
+  auto get_int_from_cache = [defaults](const char* key) {
+    NSString* ns_key = [NSString stringWithUTF8String:key];
+    return static_cast<int>([defaults integerForKey:ns_key]);
+  };
+
+  g_rpc_args.host_url = get_string_from_cache("tmvrpc_url");
+  g_rpc_args.host_port = get_int_from_cache("tmvrpc_port");
+  g_rpc_args.key = get_string_from_cache("tmvrpc_key");
+}
+
+static void update_in_cache() {
+  NSUserDefaults* defaults = [NSUserDefaults standardUserDefaults];
+
+  [defaults setObject:[NSString stringWithUTF8String:g_rpc_args.host_url.c_str()]
+               forKey:@"tmvrpc_url"];
+  [defaults setInteger:g_rpc_args.host_port forKey:@"tmvrpc_port"];
+  [defaults setObject:[NSString stringWithUTF8String:g_rpc_args.key.c_str()] forKey:@"tmvrpc_key"];
+}
+
+string GetCmdOption(int argc, char* argv[], string option, bool key = false) {
+  string cmd;
+  for (int i = 1; i < argc; ++i) {
+    string arg = argv[i];
+    if (arg.find(option) == 0) {
+      if (key) {
+        cmd = argv[i];
+        return cmd;
+      }
+      // We assume "=" is the end of option.
+      ICHECK_EQ(*option.rbegin(), '=');
+      cmd = arg.substr(arg.find('=') + 1);
+      return cmd;
+    }
+  }
+  return cmd;
+}
+
+void update_rpc_args(int argc, char* argv[]) {
+  restore_from_cache();
+  RPCArgs_cpp& args = g_rpc_args;
+
+  using tvm::support::IsNumber;
+  using tvm::support::ValidateIP;
+  constexpr int MAX_PORT_NUM = 65535;
+
+  const string immediate_connect = GetCmdOption(argc, argv, "--immediate_connect", true);
+  args.immediate_connect = !immediate_connect.empty();
+
+  const string verbose = GetCmdOption(argc, argv, "--verbose", true);
+  args.verbose = !verbose.empty();
+
+  const string server_mode = GetCmdOption(argc, argv, "--server_mode=", false);
+  if (!server_mode.empty()) {
+    if (server_mode == "tracker") {
+      args.server_mode = 0;
+    } else if (server_mode == "proxy") {
+      args.server_mode = 1;
+    } else if (server_mode == "pure_server") {

Review comment:
       I've tried to answer in reply message.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r663210300



##########
File path: apps/ios_rpc/tvmrpc/RPCServer.mm
##########
@@ -0,0 +1,815 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file ViewController.mm
+ */
+
+#import "RPCServer.h"
+
+#include <tvm/runtime/packed_func.h>
+#include <tvm/runtime/registry.h>
+
+#include <random>
+#include <string>
+
+// To get device WiFi IP
+#include <arpa/inet.h>
+#include <ifaddrs.h>
+#include <netinet/in.h>
+#include <sys/socket.h>
+
+// TVM internal header to access Magic keys like kRPCMagic and others
+#include "../../../src/runtime/rpc/rpc_endpoint.h"
+
+namespace tvm {
+namespace runtime {
+
+/*!
+ * \brief Message handling function for event driven server.
+ *
+ * \param in_bytes The incoming bytes.
+ * \param event_flag  1: read_available, 2: write_avaiable.
+ * \return State flag.
+ *     1: continue running, no need to write,
+ *     2: need to write
+ *     0: shutdown
+ */
+using FEventHandler = PackedFunc;
+
+/*!
+ * \brief Create a server event handler.
+ *
+ * \param outputStream The output stream used to send outputs.
+ * \param name The name of the server.
+ * \param remote_key The remote key
+ * \return The event handler.
+ */
+FEventHandler CreateServerEventHandler(NSOutputStream* outputStream, std::string name,
+                                       std::string remote_key) {
+  const PackedFunc* event_handler_factor = Registry::Get("rpc.CreateEventDrivenServer");

Review comment:
       nit: event_handler_factory




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] jwfromm commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

jwfromm commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-856068307


   @apeskov Can you address @areusch's comments? It looks like you made a lot of his recommended changes but it'd be good to make sure you're both on the same page.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r711967785



##########
File path: apps/ios_rpc/README.md
##########
@@ -103,7 +40,13 @@ This custom `dlopen` mechanic is integrated into TVM RPC as plugin and registere
 application.
 
 The custom implementation of `dlopen` and other functions from `dlfcn.h` header are placed in separate repository,
-and will be downloaded automatically during cmake build for iOS. To run cmake build you may use next flags:
+and will be downloaded automatically during cmake build for iOS.
+
+Also, it is necessary to build `tvm_runtime.dylib` for our iOS device. The iOS
+TVM RPC application will be linked with this library. Use the following cmake
+flags:

Review comment:
       The cmake flags are specified below. Removed this sentence.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-914634081


   @apeskov just reviving this--please address the comment 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r651307207



##########
File path: apps/ios_rpc/tvmrpc/rpc_server.h
##########
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file rpc_server.h
+ * \brief RPC Server implementation.
+ */
+#ifndef TVM_APPS_IOS_RPC_SERVER_H_
+#define TVM_APPS_IOS_RPC_SERVER_H_
+
+#include <string>
+#include <future>
+#include <chrono>
+#include <dirent.h>
+
+#include "tvm/runtime/c_runtime_api.h"
+#include "runtime/rpc/rpc_endpoint.h"
+#include "runtime/rpc/rpc_socket_impl.h"
+#include "support/socket.h"
+#include "rpc_tracker_client.h"
+
+namespace tvm {
+namespace runtime {
+
+std::vector<std::string> ListDir(const std::string& dirname) {

Review comment:
       It was a copy past from `cpp_rpc` app. It returns a vector of paths to all files placed in pointed dir. It was using to clean temp folder without removing it.
   
   I've decided to remove this part of functionality from this PR.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r638354133



##########
File path: apps/ios_rpc/tvmrpc/rpc_server.h
##########
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file rpc_server.h
+ * \brief RPC Server implementation.
+ */
+#ifndef TVM_APPS_IOS_RPC_SERVER_H_
+#define TVM_APPS_IOS_RPC_SERVER_H_
+
+#include <string>
+#include <future>
+#include <chrono>
+#include <dirent.h>
+
+#include "tvm/runtime/c_runtime_api.h"
+#include "runtime/rpc/rpc_endpoint.h"
+#include "runtime/rpc/rpc_socket_impl.h"
+#include "support/socket.h"
+#include "rpc_tracker_client.h"
+
+namespace tvm {
+namespace runtime {
+
+std::vector<std::string> ListDir(const std::string& dirname) {

Review comment:
       could you comment this function to explain what is returned and what is not, plus any post-processing done?

##########
File path: apps/ios_rpc/tvmrpc/TVMRuntime.mm
##########
@@ -71,88 +73,19 @@ void LogMessageImpl(const std::string& file, int lineno, const std::string& mess
 namespace tvm {
 namespace runtime {
 
-class NSStreamChannel final : public RPCChannel {
- public:
-  explicit NSStreamChannel(NSOutputStream* stream) : stream_(stream) {}
-
-  size_t Send(const void* data, size_t size) final {
-    ssize_t nbytes = [stream_ write:reinterpret_cast<const uint8_t*>(data) maxLength:size];
-    if (nbytes < 0) {
-      NSLog(@"%@", [stream_ streamError].localizedDescription);
-      throw tvm::Error("Stream error");
-    }
-    return nbytes;
-  }
-
-  size_t Recv(void* data, size_t size) final {
-    LOG(FATAL) << "Do not allow explicit receive for";
-    return 0;
-  }
-
- private:
-  NSOutputStream* stream_;
-};
-
-FEventHandler CreateServerEventHandler(NSOutputStream* outputStream, std::string name,
-                                       std::string remote_key) {
-  std::unique_ptr<NSStreamChannel> ch(new NSStreamChannel(outputStream));
-  std::shared_ptr<RPCEndpoint> sess = RPCEndpoint::Create(std::move(ch), name, remote_key);
-  return [sess](const std::string& in_bytes, int flag) {
-    return sess->ServerAsyncIOEventHandler(in_bytes, flag);
-  };
-}
-
-// Runtime environment
-struct RPCEnv {
- public:
-  RPCEnv() {
-    NSString* path = NSTemporaryDirectory();
-    base_ = [path UTF8String];
-    if (base_[base_.length() - 1] != '/') {
-      base_ = base_ + '/';
-    }
-  }
-  // Get Path.
-  std::string GetPath(const std::string& file_name) { return base_ + file_name; }
-
- private:
-  std::string base_;
-};
-
-void LaunchSyncServer() {
-  // only load dylib from frameworks.
-  NSBundle* bundle = [NSBundle mainBundle];
-  NSString* base = [bundle privateFrameworksPath];
-  NSString* path = [base stringByAppendingPathComponent:@"tvm/rpc_config.txt"];
-  std::string name = [path UTF8String];
-  std::ifstream fs(name, std::ios::in);
-  std::string url, key;
-  int port;
-  ICHECK(fs >> url >> port >> key) << "Invalid RPC config file " << name;
-  RPCConnect(url, port, "server:" + key, TVMArgs(nullptr, nullptr, 0))->ServerLoop();
-}
-
 TVM_REGISTER_GLOBAL("tvm.rpc.server.workpath").set_body([](TVMArgs args, TVMRetValue* rv) {
-  static RPCEnv env;
-  *rv = env.GetPath(args[0]);
+  std::string name = args[0];
+  std::string base = [NSTemporaryDirectory() UTF8String];
+  *rv = base + "/" + name;
 });
 
 TVM_REGISTER_GLOBAL("tvm.rpc.server.load_module").set_body([](TVMArgs args, TVMRetValue* rv) {
   std::string name = args[0];
-  std::string fmt = GetFileFormat(name, "");

Review comment:
       could you comment why this is going away?

##########
File path: apps/ios_rpc/tvmrpc/rpc_server.h
##########
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file rpc_server.h
+ * \brief RPC Server implementation.
+ */
+#ifndef TVM_APPS_IOS_RPC_SERVER_H_
+#define TVM_APPS_IOS_RPC_SERVER_H_
+
+#include <string>
+#include <future>
+#include <chrono>
+#include <dirent.h>
+
+#include "tvm/runtime/c_runtime_api.h"
+#include "runtime/rpc/rpc_endpoint.h"
+#include "runtime/rpc/rpc_socket_impl.h"
+#include "support/socket.h"
+#include "rpc_tracker_client.h"
+
+namespace tvm {
+namespace runtime {
+
+std::vector<std::string> ListDir(const std::string& dirname) {
+  std::vector<std::string> vec;
+  DIR* dp = opendir(dirname.c_str());
+  if (dp == nullptr) {
+    int errsv = errno;
+    LOG(FATAL) << "ListDir " << dirname << " error: " << strerror(errsv);
+  }
+  dirent* d;
+  while ((d = readdir(dp)) != nullptr) {
+    std::string filename = d->d_name;
+    if (filename != "." && filename != "..") {
+      std::string f = dirname;
+      if (f[f.length() - 1] != '/') {
+        f += '/';
+      }
+      f += d->d_name;
+      vec.push_back(f);
+    }
+  }
+  closedir(dp);
+  return vec;
+}
+
+/*!
+ * \brief CleanDir Removes the files from the directory
+ * \param dirname The name of the directory
+ */
+void CleanDir(const std::string& dirname) {
+  auto files = ListDir(dirname);
+  for (const auto& filename : files) {
+    std::string file_path = dirname + "/";
+    file_path += filename;
+    const int ret = std::remove(filename.c_str());
+    if (ret != 0) {
+      LOG(WARNING) << "Remove file " << filename << " failed";
+    }
+  }
+}
+
+// Runtime environment
+struct RPCEnv {
+ public:
+  RPCEnv(const std::string &base):base_(base) {}
+  // Get Path.
+  std::string GetPath(const std::string& file_name) { return base_ + file_name; }
+
+  void CleanUp() const {
+    CleanDir(base_);
+  }
+ private:
+  std::string base_;
+};
+
+
+/*!
+ * \brief RPCServer RPC Server class.
+ * \param host The hostname of the server, Default=0.0.0.0
+ * \param port The port of the RPC, Default=9090
+ * \param port_end The end search port of the RPC, Default=9099
+ * \param tracker The address of RPC tracker in host:port format e.g. 10.77.1.234:9190 Default=""
+ * \param key The key used to identify the device type in tracker. Default=""
+ * \param custom_addr Custom IP Address to Report to RPC Tracker. Default=""
+ */
+class RPCServer {
+ public:
+  /*!
+   * \brief Constructor.
+   */
+  RPCServer(std::string host, int port, int port_end, std::string tracker_addr, std::string key,
+            std::string custom_addr, std::string work_dir)
+      : host_(std::move(host)),
+        port_(port),
+        my_port_(0),
+        port_end_(port_end),
+        tracker_addr_(std::move(tracker_addr)),
+        key_(std::move(key)),
+        custom_addr_(std::move(custom_addr)),
+        work_dir_(std::move(work_dir)),
+        tracker_(tracker_addr_, key_, custom_addr_) {}
+
+  /*!
+   * \brief Destructor.
+   */
+  ~RPCServer() {
+    try {
+      // Free the resources
+      listen_sock_.Close();
+      tracker_.Close();
+    } catch (...) {
+    }
+  }
+
+  /*!
+   * \brief Start Creates the RPC listen process and execution.
+   */
+  void Start() {
+    listen_sock_.Create();
+    my_port_ = listen_sock_.TryBindHost(host_, port_, port_end_);
+    LOG(INFO) << "bind to " << host_ << ":" << my_port_;
+    listen_sock_.Listen(1);
+    continue_processing = true;
+    proc_ = std::future<void>(std::async(std::launch::async, &RPCServer::ListenLoopProc, this));
+  }
+  
+  void Stop() {
+    continue_processing = false;
+    tracker_.Close();
+  }
+    
+  void setCompletionCallbacks(std::function<void()> callback_start, std::function<void()> callback_stop) {
+    completion_callback_start_ = callback_start;
+    completion_callback_stop_ = callback_stop;
+  }
+
+ private:
+  /*!
+   * \brief ListenLoopProc The listen process.
+   */
+  void ListenLoopProc() {
+    
+    while (continue_processing) {
+      support::TCPSocket conn;
+      support::SockAddr addr("0.0.0.0", 0);
+      std::string opts;
+      try {
+        // step 1: setup tracker and report to tracker
+        tracker_.TryConnect();
+        if (completion_callback_start_)
+          completion_callback_start_();
+        // step 2: wait for in-coming connections
+        AcceptConnection(&tracker_, &conn, &addr, &opts);
+      } catch (const char* msg) {
+        LOG(WARNING) << "Socket exception: " << msg;
+        // close tracker resource
+        tracker_.Close();
+        continue;
+      } catch (const std::exception& e) {
+        // close tracker resource
+        tracker_.Close();
+        LOG(WARNING) << "Exception standard: " << e.what();

Review comment:
       what's "standard" mean? should this come before `tracker_.Close()` in case that throws an error?

##########
File path: apps/ios_rpc/tvmrpc/TVMRuntime.mm
##########
@@ -71,88 +73,19 @@ void LogMessageImpl(const std::string& file, int lineno, const std::string& mess
 namespace tvm {
 namespace runtime {
 
-class NSStreamChannel final : public RPCChannel {

Review comment:
       just curious why this is going away? not needed if we use standard sockets?

##########
File path: apps/ios_rpc/tvmrpc/rpc_server.h
##########
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file rpc_server.h
+ * \brief RPC Server implementation.
+ */
+#ifndef TVM_APPS_IOS_RPC_SERVER_H_
+#define TVM_APPS_IOS_RPC_SERVER_H_
+
+#include <string>
+#include <future>
+#include <chrono>
+#include <dirent.h>
+
+#include "tvm/runtime/c_runtime_api.h"
+#include "runtime/rpc/rpc_endpoint.h"
+#include "runtime/rpc/rpc_socket_impl.h"
+#include "support/socket.h"
+#include "rpc_tracker_client.h"
+
+namespace tvm {
+namespace runtime {
+
+std::vector<std::string> ListDir(const std::string& dirname) {
+  std::vector<std::string> vec;
+  DIR* dp = opendir(dirname.c_str());
+  if (dp == nullptr) {
+    int errsv = errno;
+    LOG(FATAL) << "ListDir " << dirname << " error: " << strerror(errsv);
+  }
+  dirent* d;
+  while ((d = readdir(dp)) != nullptr) {
+    std::string filename = d->d_name;
+    if (filename != "." && filename != "..") {
+      std::string f = dirname;
+      if (f[f.length() - 1] != '/') {
+        f += '/';
+      }
+      f += d->d_name;
+      vec.push_back(f);
+    }
+  }
+  closedir(dp);
+  return vec;
+}
+
+/*!
+ * \brief CleanDir Removes the files from the directory
+ * \param dirname The name of the directory
+ */
+void CleanDir(const std::string& dirname) {
+  auto files = ListDir(dirname);
+  for (const auto& filename : files) {
+    std::string file_path = dirname + "/";
+    file_path += filename;
+    const int ret = std::remove(filename.c_str());
+    if (ret != 0) {
+      LOG(WARNING) << "Remove file " << filename << " failed";
+    }
+  }
+}
+
+// Runtime environment
+struct RPCEnv {
+ public:
+  RPCEnv(const std::string &base):base_(base) {}
+  // Get Path.
+  std::string GetPath(const std::string& file_name) { return base_ + file_name; }
+
+  void CleanUp() const {
+    CleanDir(base_);
+  }
+ private:
+  std::string base_;
+};
+
+
+/*!
+ * \brief RPCServer RPC Server class.
+ * \param host The hostname of the server, Default=0.0.0.0
+ * \param port The port of the RPC, Default=9090
+ * \param port_end The end search port of the RPC, Default=9099
+ * \param tracker The address of RPC tracker in host:port format e.g. 10.77.1.234:9190 Default=""
+ * \param key The key used to identify the device type in tracker. Default=""
+ * \param custom_addr Custom IP Address to Report to RPC Tracker. Default=""
+ */
+class RPCServer {
+ public:
+  /*!
+   * \brief Constructor.
+   */
+  RPCServer(std::string host, int port, int port_end, std::string tracker_addr, std::string key,
+            std::string custom_addr, std::string work_dir)
+      : host_(std::move(host)),
+        port_(port),
+        my_port_(0),
+        port_end_(port_end),
+        tracker_addr_(std::move(tracker_addr)),
+        key_(std::move(key)),
+        custom_addr_(std::move(custom_addr)),
+        work_dir_(std::move(work_dir)),
+        tracker_(tracker_addr_, key_, custom_addr_) {}
+
+  /*!
+   * \brief Destructor.
+   */
+  ~RPCServer() {
+    try {
+      // Free the resources
+      listen_sock_.Close();
+      tracker_.Close();
+    } catch (...) {
+    }
+  }
+
+  /*!
+   * \brief Start Creates the RPC listen process and execution.
+   */
+  void Start() {
+    listen_sock_.Create();
+    my_port_ = listen_sock_.TryBindHost(host_, port_, port_end_);
+    LOG(INFO) << "bind to " << host_ << ":" << my_port_;
+    listen_sock_.Listen(1);
+    continue_processing = true;
+    proc_ = std::future<void>(std::async(std::launch::async, &RPCServer::ListenLoopProc, this));
+  }
+  
+  void Stop() {
+    continue_processing = false;
+    tracker_.Close();
+  }
+    
+  void setCompletionCallbacks(std::function<void()> callback_start, std::function<void()> callback_stop) {
+    completion_callback_start_ = callback_start;
+    completion_callback_stop_ = callback_stop;
+  }
+
+ private:
+  /*!
+   * \brief ListenLoopProc The listen process.
+   */
+  void ListenLoopProc() {
+    
+    while (continue_processing) {
+      support::TCPSocket conn;
+      support::SockAddr addr("0.0.0.0", 0);
+      std::string opts;
+      try {
+        // step 1: setup tracker and report to tracker
+        tracker_.TryConnect();
+        if (completion_callback_start_)
+          completion_callback_start_();
+        // step 2: wait for in-coming connections
+        AcceptConnection(&tracker_, &conn, &addr, &opts);
+      } catch (const char* msg) {
+        LOG(WARNING) << "Socket exception: " << msg;
+        // close tracker resource
+        tracker_.Close();
+        continue;
+      } catch (const std::exception& e) {
+        // close tracker resource
+        tracker_.Close();
+        LOG(WARNING) << "Exception standard: " << e.what();
+        continue;
+      }
+
+      auto start_time = std::chrono::high_resolution_clock::now();
+      ServerLoopProc(conn, addr, work_dir_);
+      auto dur = std::chrono::high_resolution_clock::now() - start_time;
+
+      LOG(INFO) << "Serve Time " << std::chrono::duration_cast<std::chrono::milliseconds>(dur).count() << "ms";
+
+      // close from our side.
+      LOG(INFO) << "Socket Connection Closed";
+      conn.Close();
+    }
+    if (completion_callback_stop_)
+      completion_callback_stop_();
+
+  }
+
+  /*!
+   * \brief AcceptConnection Accepts the RPC Server connection.
+   * \param tracker Tracker details.
+   * \param conn_sock New connection information.
+   * \param addr New connection address information.
+   * \param opts Parsed options for socket
+   * \param ping_period Timeout for select call waiting
+   */
+  void AcceptConnection(TrackerClient* tracker, support::TCPSocket* conn_sock,
+                        support::SockAddr* addr, std::string* opts, int ping_period = 2) {
+    std::set<std::string> old_keyset;
+    std::string matchkey;
+
+    // Report resource to tracker and get key
+    tracker->ReportResourceAndGetKey(my_port_, &matchkey);
+
+    while (continue_processing) {
+      tracker->WaitConnectionAndUpdateKey(listen_sock_, my_port_, ping_period, &matchkey);
+      support::TCPSocket conn = listen_sock_.Accept(addr);
+
+      int code = kRPCMagic;
+      ICHECK_EQ(conn.RecvAll(&code, sizeof(code)), sizeof(code));
+      if (code != kRPCMagic) {
+        conn.Close();
+        LOG(FATAL) << "Client connected is not TVM RPC server";
+        continue;
+      }
+
+      int keylen = 0;
+      ICHECK_EQ(conn.RecvAll(&keylen, sizeof(keylen)), sizeof(keylen));
+
+      const char* CLIENT_HEADER = "client:";
+      const char* SERVER_HEADER = "server:";
+      std::string expect_header = CLIENT_HEADER + matchkey;
+      std::string server_key = SERVER_HEADER + key_;
+      if (size_t(keylen) < expect_header.length()) {
+        conn.Close();
+        LOG(INFO) << "Wrong client header length";
+        continue;
+      }
+
+      ICHECK_NE(keylen, 0);
+      std::string remote_key;
+      remote_key.resize(keylen);
+      ICHECK_EQ(conn.RecvAll(&remote_key[0], keylen), keylen);
+
+      std::stringstream ssin(remote_key);
+      std::string arg0;
+      ssin >> arg0;
+
+      if (arg0 != expect_header) {
+        code = kRPCMismatch;
+        ICHECK_EQ(conn.SendAll(&code, sizeof(code)), sizeof(code));
+        conn.Close();
+        LOG(WARNING) << "Mismatch key from" << addr->AsString();
+        continue;
+      } else {
+        code = kRPCSuccess;
+        ICHECK_EQ(conn.SendAll(&code, sizeof(code)), sizeof(code));
+        keylen = int(server_key.length());
+        ICHECK_EQ(conn.SendAll(&keylen, sizeof(keylen)), sizeof(keylen));
+        ICHECK_EQ(conn.SendAll(server_key.c_str(), keylen), keylen);
+        LOG(INFO) << "Connection success " << addr->AsString();
+        ssin >> *opts;
+        *conn_sock = conn;
+        return;
+      }
+    }
+  }
+
+  /*!
+   * \brief ServerLoopProc The Server loop process.
+   * \param sock The socket information
+   * \param addr The socket address information
+   */
+  static void ServerLoopProc(support::TCPSocket sock, support::SockAddr addr,
+                             std::string work_dir) {
+    // Server loop
+    const auto env = RPCEnv(work_dir);
+    RPCServerLoop(int(sock.sockfd));
+    LOG(INFO) << "Finish serving " << addr.AsString();
+    env.CleanUp();
+  }
+
+  /*!
+   * \brief GetTimeOutFromOpts Parse and get the timeout option.
+   * \param opts The option string
+   */
+  int GetTimeOutFromOpts(const std::string& opts) const {
+    const std::string option = "-timeout=";

Review comment:
       not sure if this should also have units? `-timeout-sec`

##########
File path: apps/ios_rpc/tvmrpc/TVMRuntime.mm
##########
@@ -22,36 +22,38 @@
  */
 #include "TVMRuntime.h"
 // Runtime API
-#include "../../../src/runtime/c_runtime_api.cc"
-#include "../../../src/runtime/cpu_device_api.cc"
-#include "../../../src/runtime/dso_library.cc"
-#include "../../../src/runtime/file_utils.cc"
-#include "../../../src/runtime/library_module.cc"
-#include "../../../src/runtime/metadata_module.cc"
-#include "../../../src/runtime/module.cc"
-#include "../../../src/runtime/ndarray.cc"
-#include "../../../src/runtime/object.cc"
-#include "../../../src/runtime/registry.cc"
-#include "../../../src/runtime/system_library.cc"
-#include "../../../src/runtime/thread_pool.cc"
-#include "../../../src/runtime/threading_backend.cc"
-#include "../../../src/runtime/workspace_pool.cc"
-
-// RPC server
-#include "../../../src/runtime/rpc/rpc_channel.cc"
-#include "../../../src/runtime/rpc/rpc_endpoint.cc"
-#include "../../../src/runtime/rpc/rpc_local_session.cc"
-#include "../../../src/runtime/rpc/rpc_module.cc"
-#include "../../../src/runtime/rpc/rpc_server_env.cc"
-#include "../../../src/runtime/rpc/rpc_session.cc"
-#include "../../../src/runtime/rpc/rpc_socket_impl.cc"
-// Graph executor
-#include "../../../src/runtime/graph_executor/graph_executor.cc"
-// Metal
-#include "../../../src/runtime/metal/metal_device_api.mm"
-#include "../../../src/runtime/metal/metal_module.mm"
-// CoreML
-#include "../../../src/runtime/contrib/coreml/coreml_runtime.mm"
+//#include "../../../src/runtime/c_runtime_api.cc"

Review comment:
       since i can't read XCode project files--is this change adjusting the XCode project to depend on libtvmruntime.so somehow?

##########
File path: apps/ios_rpc/tvmrpc/rpc_server.h
##########
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file rpc_server.h
+ * \brief RPC Server implementation.
+ */
+#ifndef TVM_APPS_IOS_RPC_SERVER_H_
+#define TVM_APPS_IOS_RPC_SERVER_H_
+
+#include <string>
+#include <future>
+#include <chrono>
+#include <dirent.h>
+
+#include "tvm/runtime/c_runtime_api.h"
+#include "runtime/rpc/rpc_endpoint.h"
+#include "runtime/rpc/rpc_socket_impl.h"
+#include "support/socket.h"
+#include "rpc_tracker_client.h"
+
+namespace tvm {
+namespace runtime {
+
+std::vector<std::string> ListDir(const std::string& dirname) {
+  std::vector<std::string> vec;
+  DIR* dp = opendir(dirname.c_str());
+  if (dp == nullptr) {
+    int errsv = errno;
+    LOG(FATAL) << "ListDir " << dirname << " error: " << strerror(errsv);
+  }
+  dirent* d;
+  while ((d = readdir(dp)) != nullptr) {
+    std::string filename = d->d_name;
+    if (filename != "." && filename != "..") {
+      std::string f = dirname;
+      if (f[f.length() - 1] != '/') {
+        f += '/';
+      }
+      f += d->d_name;
+      vec.push_back(f);
+    }
+  }
+  closedir(dp);
+  return vec;
+}
+
+/*!
+ * \brief CleanDir Removes the files from the directory
+ * \param dirname The name of the directory
+ */
+void CleanDir(const std::string& dirname) {
+  auto files = ListDir(dirname);
+  for (const auto& filename : files) {
+    std::string file_path = dirname + "/";
+    file_path += filename;
+    const int ret = std::remove(filename.c_str());
+    if (ret != 0) {
+      LOG(WARNING) << "Remove file " << filename << " failed";
+    }
+  }
+}
+
+// Runtime environment
+struct RPCEnv {
+ public:
+  RPCEnv(const std::string &base):base_(base) {}
+  // Get Path.
+  std::string GetPath(const std::string& file_name) { return base_ + file_name; }
+
+  void CleanUp() const {
+    CleanDir(base_);
+  }
+ private:
+  std::string base_;
+};
+
+
+/*!
+ * \brief RPCServer RPC Server class.
+ * \param host The hostname of the server, Default=0.0.0.0
+ * \param port The port of the RPC, Default=9090
+ * \param port_end The end search port of the RPC, Default=9099
+ * \param tracker The address of RPC tracker in host:port format e.g. 10.77.1.234:9190 Default=""
+ * \param key The key used to identify the device type in tracker. Default=""
+ * \param custom_addr Custom IP Address to Report to RPC Tracker. Default=""
+ */
+class RPCServer {
+ public:
+  /*!
+   * \brief Constructor.
+   */
+  RPCServer(std::string host, int port, int port_end, std::string tracker_addr, std::string key,
+            std::string custom_addr, std::string work_dir)
+      : host_(std::move(host)),
+        port_(port),
+        my_port_(0),
+        port_end_(port_end),
+        tracker_addr_(std::move(tracker_addr)),
+        key_(std::move(key)),
+        custom_addr_(std::move(custom_addr)),
+        work_dir_(std::move(work_dir)),
+        tracker_(tracker_addr_, key_, custom_addr_) {}
+
+  /*!
+   * \brief Destructor.
+   */
+  ~RPCServer() {
+    try {
+      // Free the resources
+      listen_sock_.Close();
+      tracker_.Close();
+    } catch (...) {
+    }
+  }
+
+  /*!
+   * \brief Start Creates the RPC listen process and execution.
+   */
+  void Start() {
+    listen_sock_.Create();
+    my_port_ = listen_sock_.TryBindHost(host_, port_, port_end_);
+    LOG(INFO) << "bind to " << host_ << ":" << my_port_;
+    listen_sock_.Listen(1);
+    continue_processing = true;
+    proc_ = std::future<void>(std::async(std::launch::async, &RPCServer::ListenLoopProc, this));
+  }
+  
+  void Stop() {
+    continue_processing = false;
+    tracker_.Close();
+  }
+    
+  void setCompletionCallbacks(std::function<void()> callback_start, std::function<void()> callback_stop) {
+    completion_callback_start_ = callback_start;
+    completion_callback_stop_ = callback_stop;
+  }
+
+ private:
+  /*!
+   * \brief ListenLoopProc The listen process.
+   */
+  void ListenLoopProc() {
+    
+    while (continue_processing) {
+      support::TCPSocket conn;
+      support::SockAddr addr("0.0.0.0", 0);
+      std::string opts;
+      try {
+        // step 1: setup tracker and report to tracker
+        tracker_.TryConnect();
+        if (completion_callback_start_)
+          completion_callback_start_();
+        // step 2: wait for in-coming connections
+        AcceptConnection(&tracker_, &conn, &addr, &opts);
+      } catch (const char* msg) {
+        LOG(WARNING) << "Socket exception: " << msg;
+        // close tracker resource
+        tracker_.Close();
+        continue;
+      } catch (const std::exception& e) {
+        // close tracker resource
+        tracker_.Close();
+        LOG(WARNING) << "Exception standard: " << e.what();
+        continue;
+      }
+
+      auto start_time = std::chrono::high_resolution_clock::now();
+      ServerLoopProc(conn, addr, work_dir_);
+      auto dur = std::chrono::high_resolution_clock::now() - start_time;
+
+      LOG(INFO) << "Serve Time " << std::chrono::duration_cast<std::chrono::milliseconds>(dur).count() << "ms";
+
+      // close from our side.
+      LOG(INFO) << "Socket Connection Closed";
+      conn.Close();
+    }
+    if (completion_callback_stop_)
+      completion_callback_stop_();
+
+  }
+
+  /*!
+   * \brief AcceptConnection Accepts the RPC Server connection.
+   * \param tracker Tracker details.
+   * \param conn_sock New connection information.
+   * \param addr New connection address information.
+   * \param opts Parsed options for socket
+   * \param ping_period Timeout for select call waiting
+   */
+  void AcceptConnection(TrackerClient* tracker, support::TCPSocket* conn_sock,
+                        support::SockAddr* addr, std::string* opts, int ping_period = 2) {

Review comment:
       specify units: `ping_period_sec`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r663192607



##########
File path: apps/ios_rpc/tvmrpc/ViewController.mm
##########
@@ -22,168 +22,144 @@
  */
 
 #import "ViewController.h"
-#include <string>
+#import "RPCArgs.h"
 
-@implementation ViewController
-
-- (void)stream:(NSStream*)strm handleEvent:(NSStreamEvent)event {
-  std::string buffer;
-  switch (event) {
-    case NSStreamEventOpenCompleted: {
-      self.statusLabel.text = @"Connected";
-      break;
-    }
-    case NSStreamEventHasBytesAvailable:
-      if (strm == inputStream_) {
-        [self onReadAvailable];
-      }
-      break;
-    case NSStreamEventHasSpaceAvailable: {
-      if (strm == outputStream_) {
-        [self onWriteAvailable];
-      }
-      break;
-    }
-    case NSStreamEventErrorOccurred: {
-      NSLog(@"%@", [strm streamError].localizedDescription);
-      break;
-    }
-    case NSStreamEventEndEncountered: {
-      [self close];
-      // auto reconnect when normal end.
-      [self open];
-      break;
-    }
-    default: {
-      NSLog(@"Unknown event");
-    }
-  }
+@implementation ViewController {
+  // server implementation
+  RPCServer* server_;
+  // verbose flag to print status info
+  bool verbose_;
+  // Button state. True - push will start connection, false - push will disconnect
+  bool to_connect_;
 }
 
-- (void)onReadAvailable {
-  constexpr int kRPCMagic = 0xff271;
-  if (!initialized_) {
-    int code;
-    size_t nbytes = [inputStream_ read:reinterpret_cast<uint8_t*>(&code) maxLength:sizeof(code)];
-    if (nbytes != sizeof(code)) {
-      self.infoText.text = @"Fail to receive remote confirmation code.";
-      [self close];
-    } else if (code == kRPCMagic + 2) {
-      self.infoText.text = @"Proxy server cannot find client that matches the key";
-      [self close];
-    } else if (code == kRPCMagic + 1) {
-      self.infoText.text = @"Proxy server already have another server with same key";
-      [self close];
-    } else if (code != kRPCMagic) {
-      self.infoText.text = @"Given address is not a TVM RPC Proxy";
-      [self close];
-    } else {
-      initialized_ = true;
-      self.statusLabel.text = @"Proxy connected.";
-      ICHECK(handler_ != nullptr);
-    }
-  }
-  const int kBufferSize = 4 << 10;
-  if (initialized_) {
-    while ([inputStream_ hasBytesAvailable]) {
-      recvBuffer_.resize(kBufferSize);
-      uint8_t* bptr = reinterpret_cast<uint8_t*>(&recvBuffer_[0]);
-      size_t nbytes = [inputStream_ read:bptr maxLength:kBufferSize];
-      recvBuffer_.resize(nbytes);
-      int flag = 1;
-      if ([outputStream_ hasSpaceAvailable]) {
-        flag |= 2;
-      }
-      // always try to write
-      try {
-        flag = handler_(recvBuffer_, flag);
-        if (flag == 2) {
-          [self onShutdownReceived];
-        }
-      } catch (const tvm::Error& e) {
-        [self close];
-      }
-    }
+- (void)viewDidLoad {
+  // To handle end editing events
+  self.proxyURL.delegate = self;
+  self.proxyPort.delegate = self;
+  self.proxyKey.delegate = self;
+
+  RPCArgs args = get_current_rpc_args();
+  self.proxyURL.text = @(args.host_url);
+  self.proxyPort.text = @(args.host_port).stringValue;
+  self.proxyKey.text = @(args.key);
+
+  self.ModeSelector.selectedSegmentIndex = args.server_mode;
+  self->verbose_ = args.verbose;
+  self->to_connect_ = true;
+
+  // Add border to button
+  void (^addBorder)(UIButton* btn) = ^(UIButton* btn) {
+    btn.layer.borderWidth = 2.0f;
+    btn.layer.borderColor = self.ConnectButton.currentTitleColor.CGColor;
+    btn.layer.cornerRadius = 10;
+  };
+  addBorder(self.ConnectButton);
+
+  // Connect to tracker immediately
+  if (args.immediate_connect) {
+    [self disableUIInteraction];
+    [self open];
   }
 }
 
-- (void)onShutdownReceived {
-  [self close];
-}
+/*!
+ * \brief Disable all UI elements
+ */
+- (void)disableUIInteraction {
+  void (^disable)(UITextField* field) = ^(UITextField* field) {
+    field.enabled = NO;
+    field.backgroundColor = [UIColor lightGrayColor];
+  };
 
-- (void)onWriteAvailable {
-  if (initSendPtr_ < initBytes_.length()) {
-    initSendPtr_ += [outputStream_ write:reinterpret_cast<uint8_t*>(&initBytes_[initSendPtr_])
-                               maxLength:(initBytes_.length() - initSendPtr_)];
-  }
-  if (initialized_) {
-    try {
-      std::string dummy;
-      int flag = handler_(dummy, 2);
-      if (flag == 2) {
-        [self onShutdownReceived];
-      }
-    } catch (const tvm::Error& e) {
-      [self close];
-    }
-  }
+  void (^disableButton)(UIButton* btn) = ^(UIButton* btn) {
+    btn.enabled = NO;
+    btn.layer.borderColor = btn.currentTitleColor.CGColor;
+  };
+
+  disable(self.proxyURL);
+  disable(self.proxyPort);
+  disable(self.proxyKey);
+  disableButton(self.ConnectButton);
+  self.ModeSelector.enabled = NO;
 }
 
+/*!
+ * \brief Start RPC server
+ */
 - (void)open {
-  constexpr int kRPCMagic = 0xff271;
-  NSLog(@"Connecting to the proxy server..");
-  // Initialize the data states.
-  key_ = [self.proxyKey.text UTF8String];
-  key_ = "server:" + key_;
-  std::ostringstream os;
-  int rpc_magic = kRPCMagic;
-  os.write(reinterpret_cast<char*>(&rpc_magic), sizeof(rpc_magic));
-  int keylen = static_cast<int>(key_.length());
-  os.write(reinterpret_cast<char*>(&keylen), sizeof(keylen));
-  os.write(key_.c_str(), key_.length());
-  initialized_ = false;
-  initBytes_ = os.str();
-  initSendPtr_ = 0;
-  // Initialize the network.
-  CFReadStreamRef readStream;
-  CFWriteStreamRef writeStream;
-  CFStreamCreatePairWithSocketToHost(NULL, (__bridge CFStringRef)self.proxyURL.text,
-                                     [self.proxyPort.text intValue], &readStream, &writeStream);
-  inputStream_ = (NSInputStream*)readStream;
-  outputStream_ = (NSOutputStream*)writeStream;
-  [inputStream_ setDelegate:self];
-  [outputStream_ setDelegate:self];
-  [inputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [outputStream_ scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [outputStream_ open];
-  [inputStream_ open];
-  handler_ = tvm::runtime::CreateServerEventHandler(outputStream_, key_, "%toinit");
-  ICHECK(handler_ != nullptr);
+  RPCServerMode server_mode = static_cast<RPCServerMode>(self.ModeSelector.selectedSegmentIndex);
+
+  server_ = [RPCServer serverWithMode:server_mode];
+  server_.host = self.proxyURL.text;
+  server_.port = self.proxyPort.text.intValue;
+  server_.key = self.proxyKey.text;
+  server_.verbose = self->verbose_;
+  server_.delegate = self;
+
+  [server_ start];
+
   self.infoText.text = @"";
   self.statusLabel.text = @"Connecting...";
 }
 
+/*!
+ * \brief Stop RPC server
+ */
 - (void)close {
-  NSLog(@"Closing the streams.");
-  [inputStream_ close];
-  [outputStream_ close];
-  [inputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [outputStream_ removeFromRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
-  [inputStream_ setDelegate:nil];
-  [outputStream_ setDelegate:nil];
-  inputStream_ = nil;
-  outputStream_ = nil;
-  handler_ = nullptr;
-  self.statusLabel.text = @"Disconnected";
+  [server_ stop];
+  self.statusLabel.text = @"Disconnecting...";

Review comment:
       ah understood, i thought i was sync




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r711965497



##########
File path: apps/ios_rpc/README.md
##########
@@ -17,84 +17,21 @@
 
 # iOS TVM RPC
 
-This folder contains iOS RPC app that allows us to launch an rpc server on a iOS device(e.g. ipython)
-and connect to it through python script and do testing on the python side as normal TVM RPC.
-You will need XCode and an iOS device to use this.
-
-## RPC proxy
-Start the RPC proxy by running in a terminal:
-
-    python -m tvm.exec.rpc_proxy
-
-On success, you should see something like this:
-
-    INFO:root:RPCProxy: client port bind to 0.0.0.0:9090
-    INFO:root:RPCProxy: Websock port bind to 8888
-
-IP-address of this machine will be used to initialize ```TVM_IOS_RPC_PROXY_HOST```
-environment variable (see below).
+This folder contains iOS RPC app that allows us to launch an rpc server on a iOS
+device. You will need XCode and an iOS device to use this.
+
+## Table of Contents
+* [Building](#building)
+    * [Building TVM runtime and custom DSO loader plugin](#building-tvm-runtime-and-custom-dso-loader-plugin)
+    * [Building iOS TVM RPC application](#building-ios-tvm-rpc-application)
+* [Workflow](#workflow)
+    * [Pure RPC](#pure-rpc)
+    * [iOS RPC App with proxy](#ios-rpc-app-with-proxy)
+    * [iOS RPC App with tracker](#ios-rpc-app-with-tracker)
+* [Communication without Wi-Fi and speed up in case of slow Wi-Fi](#communication-without-wi-fi-and-speed-up-in-case-of-slow-wi-fi)
 
 ## Building
-Before start, please run ```init_proj.py``` to update XCode developer metadata. After this step, open
-```tvmrpc.xcodeproj``` by using XCode, build the App and install the App on the phone. Usually, we
-**do not** use the iOS App directly.
-
-To test an App, you can fill ``Address`` field with IP-address of RPC proxy
-(see above), and press ``Connect to Proxy``.
-
-On success, "Disconnected" will change to "Connected".
-On RPC proxy side you can see the next message in a log:
-
-    INFO:root:Handler ready TCPSocketProxy:<iPhone IP address>:server:iphone
-
-Now App can be closed by pressing the home button (or even removed from a device).
-
-## Workflow
-Due to security restriction of iOS10. We cannot upload dynamic libraries to the App and load it from sandbox.
-Instead, we need to build a list of libraries, pack them into the app bundle, launch the RPC server and
-connect to test the bundled libraries. We use ```xcodebuild test``` to automate this process. There is also
-one more approach to workaround this limitation, for more details please take a look into section
-[Custom DSO loader integration](#custom-dso-loader-plugin).
-
-The test script [tests/ios_rpc_test.py](tests/ios_rpc_test.py) is a good template for the workflow. With this
-script, we don't need to manually operate the iOS App, this script will build the app, run it and collect the results 
-automatically.
-
- To run the script,  you need to configure the following environment variables
-
-- ```TVM_IOS_CODESIGN``` The signature you use to codesign the app and libraries (e.g. ```iPhone Developer: Name (XXXX)```)
-- ```TVM_IOS_TEAM_ID``` The developer Team ID available at https://developer.apple.com/account/#/membership     
-- ```TVM_IOS_RPC_ROOT``` The root directory of the iOS rpc project
-- ```TVM_IOS_RPC_PROXY_HOST``` The RPC proxy address (see above)
-- ```TVM_IOS_RPC_DESTINATION``` The Xcode target device (e.g. ```platform=iOS,id=xxxx```)
-
-See instructions of how to find UUID of the iOS device:
-
-- https://www.innerfence.com/howto/find-iphone-unique-device-identifier-udid
-
-## How it works
-Let us explain how it works, the project look for ```rpc_config.txt``` file in the project root folder.
-The ```rpc_config.txt``` file should be in the following format:
-```
-<url> <port> <key>
-[path to dylib1]
-[path to dylib2]
-...
-```
-The build script will copy all the dynamic libraries into bundle ```tvmrpc.app/Frameworks/tvm```,
-which you will be able to load via RPC using ```remote.load_module```.
-It will also create an ```tvmrpc.app/Frameworks/tvm/rpc_config.txt``` containing the first line.
-
-When we run the testcase, the testcase read the configuration from ```tvmrpc.app/Frameworks/tvm/rpc_config.txt```
-and connect to the specified RPC proxy, start serving loop.
-
-So if we want to start the RPC from XCode IDE, simply manually modify ```rpc_config.txt``` file and click test.
-Then connect to the proxy via the python script.
-
-We can also use the RPC App directly, by typing in the address and press connect to connect to the proxy.
-However, the restriction is we can only load the modules that are bundled to the App.
-
-## Custom DSO loader plugin
+### Building TVM runtime and custom DSO loader plugin
 While iOS platform itself doesn't allow us to run an unsigned binary, where is a partial ability to run JIT code

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r651294848



##########
File path: apps/ios_rpc/tvmrpc/rpc_server.h
##########
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file rpc_server.h
+ * \brief RPC Server implementation.
+ */
+#ifndef TVM_APPS_IOS_RPC_SERVER_H_
+#define TVM_APPS_IOS_RPC_SERVER_H_
+
+#include <string>
+#include <future>
+#include <chrono>
+#include <dirent.h>
+
+#include "tvm/runtime/c_runtime_api.h"
+#include "runtime/rpc/rpc_endpoint.h"
+#include "runtime/rpc/rpc_socket_impl.h"
+#include "support/socket.h"
+#include "rpc_tracker_client.h"
+
+namespace tvm {
+namespace runtime {
+
+std::vector<std::string> ListDir(const std::string& dirname) {
+  std::vector<std::string> vec;
+  DIR* dp = opendir(dirname.c_str());
+  if (dp == nullptr) {
+    int errsv = errno;
+    LOG(FATAL) << "ListDir " << dirname << " error: " << strerror(errsv);
+  }
+  dirent* d;
+  while ((d = readdir(dp)) != nullptr) {
+    std::string filename = d->d_name;
+    if (filename != "." && filename != "..") {
+      std::string f = dirname;
+      if (f[f.length() - 1] != '/') {
+        f += '/';
+      }
+      f += d->d_name;
+      vec.push_back(f);
+    }
+  }
+  closedir(dp);
+  return vec;
+}
+
+/*!
+ * \brief CleanDir Removes the files from the directory
+ * \param dirname The name of the directory
+ */
+void CleanDir(const std::string& dirname) {
+  auto files = ListDir(dirname);
+  for (const auto& filename : files) {
+    std::string file_path = dirname + "/";
+    file_path += filename;
+    const int ret = std::remove(filename.c_str());
+    if (ret != 0) {
+      LOG(WARNING) << "Remove file " << filename << " failed";
+    }
+  }
+}
+
+// Runtime environment
+struct RPCEnv {
+ public:
+  RPCEnv(const std::string &base):base_(base) {}
+  // Get Path.
+  std::string GetPath(const std::string& file_name) { return base_ + file_name; }
+
+  void CleanUp() const {
+    CleanDir(base_);
+  }
+ private:
+  std::string base_;
+};
+
+
+/*!
+ * \brief RPCServer RPC Server class.
+ * \param host The hostname of the server, Default=0.0.0.0
+ * \param port The port of the RPC, Default=9090
+ * \param port_end The end search port of the RPC, Default=9099
+ * \param tracker The address of RPC tracker in host:port format e.g. 10.77.1.234:9190 Default=""
+ * \param key The key used to identify the device type in tracker. Default=""
+ * \param custom_addr Custom IP Address to Report to RPC Tracker. Default=""
+ */
+class RPCServer {
+ public:
+  /*!
+   * \brief Constructor.
+   */
+  RPCServer(std::string host, int port, int port_end, std::string tracker_addr, std::string key,
+            std::string custom_addr, std::string work_dir)
+      : host_(std::move(host)),
+        port_(port),
+        my_port_(0),
+        port_end_(port_end),
+        tracker_addr_(std::move(tracker_addr)),
+        key_(std::move(key)),
+        custom_addr_(std::move(custom_addr)),
+        work_dir_(std::move(work_dir)),
+        tracker_(tracker_addr_, key_, custom_addr_) {}
+
+  /*!
+   * \brief Destructor.
+   */
+  ~RPCServer() {
+    try {
+      // Free the resources
+      listen_sock_.Close();
+      tracker_.Close();
+    } catch (...) {
+    }
+  }
+
+  /*!
+   * \brief Start Creates the RPC listen process and execution.
+   */
+  void Start() {
+    listen_sock_.Create();
+    my_port_ = listen_sock_.TryBindHost(host_, port_, port_end_);
+    LOG(INFO) << "bind to " << host_ << ":" << my_port_;
+    listen_sock_.Listen(1);
+    continue_processing = true;
+    proc_ = std::future<void>(std::async(std::launch::async, &RPCServer::ListenLoopProc, this));
+  }
+  
+  void Stop() {
+    continue_processing = false;
+    tracker_.Close();
+  }
+    
+  void setCompletionCallbacks(std::function<void()> callback_start, std::function<void()> callback_stop) {
+    completion_callback_start_ = callback_start;
+    completion_callback_stop_ = callback_stop;
+  }
+
+ private:
+  /*!
+   * \brief ListenLoopProc The listen process.
+   */
+  void ListenLoopProc() {
+    
+    while (continue_processing) {
+      support::TCPSocket conn;
+      support::SockAddr addr("0.0.0.0", 0);
+      std::string opts;
+      try {
+        // step 1: setup tracker and report to tracker
+        tracker_.TryConnect();
+        if (completion_callback_start_)
+          completion_callback_start_();
+        // step 2: wait for in-coming connections
+        AcceptConnection(&tracker_, &conn, &addr, &opts);
+      } catch (const char* msg) {
+        LOG(WARNING) << "Socket exception: " << msg;
+        // close tracker resource
+        tracker_.Close();
+        continue;
+      } catch (const std::exception& e) {
+        // close tracker resource
+        tracker_.Close();
+        LOG(WARNING) << "Exception standard: " << e.what();
+        continue;
+      }
+
+      auto start_time = std::chrono::high_resolution_clock::now();
+      ServerLoopProc(conn, addr, work_dir_);
+      auto dur = std::chrono::high_resolution_clock::now() - start_time;
+
+      LOG(INFO) << "Serve Time " << std::chrono::duration_cast<std::chrono::milliseconds>(dur).count() << "ms";
+
+      // close from our side.
+      LOG(INFO) << "Socket Connection Closed";
+      conn.Close();
+    }
+    if (completion_callback_stop_)
+      completion_callback_stop_();
+
+  }
+
+  /*!
+   * \brief AcceptConnection Accepts the RPC Server connection.
+   * \param tracker Tracker details.
+   * \param conn_sock New connection information.
+   * \param addr New connection address information.
+   * \param opts Parsed options for socket
+   * \param ping_period Timeout for select call waiting
+   */
+  void AcceptConnection(TrackerClient* tracker, support::TCPSocket* conn_sock,
+                        support::SockAddr* addr, std::string* opts, int ping_period = 2) {

Review comment:
       The same as previous comment. Copy-paste form `cpp_rpc`. Already removed. You may move this comment into original `cpp_rpc` application
   
   https://github.com/apache/tvm/blob/1c251f50ee616507bdfd8866408e7acf9888cc3f/apps/cpp_rpc/rpc_server.cc#L238




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r651293840



##########
File path: apps/ios_rpc/tvmrpc/rpc_server.h
##########
@@ -0,0 +1,318 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file rpc_server.h
+ * \brief RPC Server implementation.
+ */
+#ifndef TVM_APPS_IOS_RPC_SERVER_H_
+#define TVM_APPS_IOS_RPC_SERVER_H_
+
+#include <string>
+#include <future>
+#include <chrono>
+#include <dirent.h>
+
+#include "tvm/runtime/c_runtime_api.h"
+#include "runtime/rpc/rpc_endpoint.h"
+#include "runtime/rpc/rpc_socket_impl.h"
+#include "support/socket.h"
+#include "rpc_tracker_client.h"
+
+namespace tvm {
+namespace runtime {
+
+std::vector<std::string> ListDir(const std::string& dirname) {
+  std::vector<std::string> vec;
+  DIR* dp = opendir(dirname.c_str());
+  if (dp == nullptr) {
+    int errsv = errno;
+    LOG(FATAL) << "ListDir " << dirname << " error: " << strerror(errsv);
+  }
+  dirent* d;
+  while ((d = readdir(dp)) != nullptr) {
+    std::string filename = d->d_name;
+    if (filename != "." && filename != "..") {
+      std::string f = dirname;
+      if (f[f.length() - 1] != '/') {
+        f += '/';
+      }
+      f += d->d_name;
+      vec.push_back(f);
+    }
+  }
+  closedir(dp);
+  return vec;
+}
+
+/*!
+ * \brief CleanDir Removes the files from the directory
+ * \param dirname The name of the directory
+ */
+void CleanDir(const std::string& dirname) {
+  auto files = ListDir(dirname);
+  for (const auto& filename : files) {
+    std::string file_path = dirname + "/";
+    file_path += filename;
+    const int ret = std::remove(filename.c_str());
+    if (ret != 0) {
+      LOG(WARNING) << "Remove file " << filename << " failed";
+    }
+  }
+}
+
+// Runtime environment
+struct RPCEnv {
+ public:
+  RPCEnv(const std::string &base):base_(base) {}
+  // Get Path.
+  std::string GetPath(const std::string& file_name) { return base_ + file_name; }
+
+  void CleanUp() const {
+    CleanDir(base_);
+  }
+ private:
+  std::string base_;
+};
+
+
+/*!
+ * \brief RPCServer RPC Server class.
+ * \param host The hostname of the server, Default=0.0.0.0
+ * \param port The port of the RPC, Default=9090
+ * \param port_end The end search port of the RPC, Default=9099
+ * \param tracker The address of RPC tracker in host:port format e.g. 10.77.1.234:9190 Default=""
+ * \param key The key used to identify the device type in tracker. Default=""
+ * \param custom_addr Custom IP Address to Report to RPC Tracker. Default=""
+ */
+class RPCServer {
+ public:
+  /*!
+   * \brief Constructor.
+   */
+  RPCServer(std::string host, int port, int port_end, std::string tracker_addr, std::string key,
+            std::string custom_addr, std::string work_dir)
+      : host_(std::move(host)),
+        port_(port),
+        my_port_(0),
+        port_end_(port_end),
+        tracker_addr_(std::move(tracker_addr)),
+        key_(std::move(key)),
+        custom_addr_(std::move(custom_addr)),
+        work_dir_(std::move(work_dir)),
+        tracker_(tracker_addr_, key_, custom_addr_) {}
+
+  /*!
+   * \brief Destructor.
+   */
+  ~RPCServer() {
+    try {
+      // Free the resources
+      listen_sock_.Close();
+      tracker_.Close();
+    } catch (...) {
+    }
+  }
+
+  /*!
+   * \brief Start Creates the RPC listen process and execution.
+   */
+  void Start() {
+    listen_sock_.Create();
+    my_port_ = listen_sock_.TryBindHost(host_, port_, port_end_);
+    LOG(INFO) << "bind to " << host_ << ":" << my_port_;
+    listen_sock_.Listen(1);
+    continue_processing = true;
+    proc_ = std::future<void>(std::async(std::launch::async, &RPCServer::ListenLoopProc, this));
+  }
+  
+  void Stop() {
+    continue_processing = false;
+    tracker_.Close();
+  }
+    
+  void setCompletionCallbacks(std::function<void()> callback_start, std::function<void()> callback_stop) {
+    completion_callback_start_ = callback_start;
+    completion_callback_stop_ = callback_stop;
+  }
+
+ private:
+  /*!
+   * \brief ListenLoopProc The listen process.
+   */
+  void ListenLoopProc() {
+    
+    while (continue_processing) {
+      support::TCPSocket conn;
+      support::SockAddr addr("0.0.0.0", 0);
+      std::string opts;
+      try {
+        // step 1: setup tracker and report to tracker
+        tracker_.TryConnect();
+        if (completion_callback_start_)
+          completion_callback_start_();
+        // step 2: wait for in-coming connections
+        AcceptConnection(&tracker_, &conn, &addr, &opts);
+      } catch (const char* msg) {
+        LOG(WARNING) << "Socket exception: " << msg;
+        // close tracker resource
+        tracker_.Close();
+        continue;
+      } catch (const std::exception& e) {
+        // close tracker resource
+        tracker_.Close();
+        LOG(WARNING) << "Exception standard: " << e.what();

Review comment:
       It was a copy-paste form neighbour "cpp_rpc" application. At last version of this patch I decided to rewrite this part of code in nonblocking style with Obj-C streams, RunLoop and other stuff. There is no more this exception line in iOS RPC project.   
   
   https://github.com/apache/tvm/blob/1c251f50ee616507bdfd8866408e7acf9888cc3f/apps/cpp_rpc/rpc_server.cc#L160




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-922750354


   > @apeskov apologies for the delay. one clarifying question--when this was sent I googled PureRPC and saw [this](https://github.com/standy66/purerpc). On reading your README and the code further, it seems like in Pure RPC mode, it's just the standard TVM RPC server. Does that match your understanding? Apologies, I thought there was something I didn't understand in the iOS project configuration depending on gRPC.
   > 
   > I've also read through your README more thoroughly now and added a couple of suggestions for readability.
   
   You are right. In Pure RPC we mean just the standard RPC server. We name it Pure in the code and documentation only because it doesn't have any additional logic like connecting to tracker or proxy. If you are confused about the name, we could rename it. 
   
   Thank you for your comments about README. I updated almost all your comments.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r651296531



##########
File path: apps/ios_rpc/tvmrpc/TVMRuntime.mm
##########
@@ -71,88 +73,19 @@ void LogMessageImpl(const std::string& file, int lineno, const std::string& mess
 namespace tvm {
 namespace runtime {
 
-class NSStreamChannel final : public RPCChannel {

Review comment:
       Moved into "RPCServer.mm". And now it's wrapped into `PackedFunction` to meet API requirements of `rpc.CreateEventDrivenServer`.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-861041156


   @areusch I've tried to answer your comments. Could you please continue a review?
   
   Some part of code you pointed previously to improve was redesigned with latest commits, specially the code which was borrowed from `cpp_rpc`. I hope the new one will not cause a lot of questions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-924457092


   thanks @echuraev could we rename Pure to Standalone? or happy to hear other options. I think standalone makes sense to me--that would be `--server_mode standalone` and that would remove the name conflict with PureRPC.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] apeskov commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

apeskov commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r651304190



##########
File path: apps/ios_rpc/tvmrpc/TVMRuntime.mm
##########
@@ -22,36 +22,38 @@
  */
 #include "TVMRuntime.h"
 // Runtime API
-#include "../../../src/runtime/c_runtime_api.cc"
-#include "../../../src/runtime/cpu_device_api.cc"
-#include "../../../src/runtime/dso_library.cc"
-#include "../../../src/runtime/file_utils.cc"
-#include "../../../src/runtime/library_module.cc"
-#include "../../../src/runtime/metadata_module.cc"
-#include "../../../src/runtime/module.cc"
-#include "../../../src/runtime/ndarray.cc"
-#include "../../../src/runtime/object.cc"
-#include "../../../src/runtime/registry.cc"
-#include "../../../src/runtime/system_library.cc"
-#include "../../../src/runtime/thread_pool.cc"
-#include "../../../src/runtime/threading_backend.cc"
-#include "../../../src/runtime/workspace_pool.cc"
-
-// RPC server
-#include "../../../src/runtime/rpc/rpc_channel.cc"
-#include "../../../src/runtime/rpc/rpc_endpoint.cc"
-#include "../../../src/runtime/rpc/rpc_local_session.cc"
-#include "../../../src/runtime/rpc/rpc_module.cc"
-#include "../../../src/runtime/rpc/rpc_server_env.cc"
-#include "../../../src/runtime/rpc/rpc_session.cc"
-#include "../../../src/runtime/rpc/rpc_socket_impl.cc"
-// Graph executor
-#include "../../../src/runtime/graph_executor/graph_executor.cc"
-// Metal
-#include "../../../src/runtime/metal/metal_device_api.mm"
-#include "../../../src/runtime/metal/metal_module.mm"
-// CoreML
-#include "../../../src/runtime/contrib/coreml/coreml_runtime.mm"
+//#include "../../../src/runtime/c_runtime_api.cc"

Review comment:
       Yes, Xcode precept is using prebuilt _tvm_runtime.dylib_ by specifying custom build attribute `TVM_BUILD_DIR` (during build as a part of Cmake it will be set automatically).  I've tried to explain the reason of this change in PR description.  




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-914928241


   > @apeskov just reviving this--please address the comment
   
   Hello @areusch! @apeskov is busy with other activities. I'm working to finish this PR. What about your comment. I already fixed the typo, and now I'm working on adding a detailed instruction which will help to understand the new workflow and how it works.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r711989713



##########
File path: apps/ios_rpc/README.md
##########
@@ -114,19 +57,198 @@ cmake ..
   -DCMAKE_OSX_ARCHITECTURES=arm64
   -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0
   -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=ON
-  -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=XXXXXXXXXX  # insert your Team ID
   -DUSE_IOS_RPC=ON  # to enable build iOS RPC application from TVM project tree
-cmake --build . --target custom_dso_loader ios_rpc  # Will use custom DSO loader by default
-# Resulting iOS RPC app bundle will be placed in:
-# apps/ios_rpc/ios_rpc/src/ios_rpc-build/Build/Products/[CONFIG]-iphoneos/tvmrpc.app
+  -DUSE_METAL=ON    # to enable Metal runtime
+
+cmake --build . --target custom_dso_loader tvm_runtime
+```
+
+### Building iOS TVM RPC application
+Before start, please run [init_proj.py](./init_proj.py) to update XCode developer metadata:
+```shell
+python3 init_proj.py --team_id XXXXXXXXXX --tvm_build_dir "/path/to/tvm/ios/build/folder"
+```
+You can get value of your `team_id` in the following ways:
+- **You have registered Apple Developer Profile**. In this case you developer
+  Team ID available at https://developer.apple.com/account/#/membership
+- You are using your local developer profile. In this case run the command above
+  with `XXXXXXXXXX` instead of Team ID. Then open `tvmrpc.xcodeproj` by using
+  XCode, click on the project name (`tvmrpc`) on the left panel. Then select
+  target `tvmrpc`. At the bottom of this panel go to `Signing & Capabilities`
+  tab and in the field `Team` select your local developer profile
+  (`Your Name (Personal Team)`).
+  
+  On the first run of the application you may see message `Could not launch
+  "tvmrpc"` in the XCode and message `Untrusted Developer` on your device. In
+  this case it will be necessary to check the certificate. Open
+  `Settings -> General -> Device Management -> Apple Development: <your_email>
+  -> Trust "Apple Development: <your_email>"` and click `Trust`. After than you
+  should rerun your application in the XCode.
+
+After this step, open `tvmrpc.xcodeproj` by using XCode, build the App and
+install the App on the phone.
+
+## Workflow
+Due to security restriction of iOS10. We cannot upload dynamic libraries to the
+App and load it from sandbox.  Instead, we need to build a list of libraries,
+pack them into the app bundle, launch the RPC server and connect to test the
+bundled libraries.  We use one approach to workaround this limitation, for more
+details please take a look into section
+[Building TVM runtime and custom DSO loader integration](#building-tvm-runtime-and-custom-DSO-loader-plugin).
+
+The test script [tests/ios_rpc_test.py](tests/ios_rpc_test.py) and
+[tests/ios_rpc_mobilenet.py](tests/ios_rpc_mobilenet.py) are good templates for
+demonstrating the workflow.
+
+We have three different modes for iOS RPC server:
+- [Pure RPC](#pure-rpc): In this mode RPC server open port on the device and listening. Then
+  client connects to the server directly without any mediators.
+- [iOS RPC application with Proxy](#ios-rpc-app-with-proxy): RPC server and RPC client communicates through
+  `rpc_proxy`. The RPC server on iOS device notify `rpc_proxy` which was run on
+  host machine about itself and wait for incoming connections. Communications
+  between client and server works through `rpc_proxy`.
+- [iOS RPC application with Tracker](#ios-rpc-app-with-tracker): RPC server registered in the `rpc_tracker`
+  and client connects to the RPC server through `rpc_tracker`.
+
+### Pure RPC
+Start RPC server on your iOS device:
+- Enable verbose output.
+- Push on the `Connect` button.
+
+After that you supposed to see something like this in the app on the device:
+```
+IP: <device_ip>
+Port: <rpc_server_port>
 ```
 
-To enable using of Custom DSO Plugin during xcode build outsde of Cmake you should specify two additional variables.
-You can do it manually inside Xcode IDE or via command line args for `xcodebuild`. Make sure that `custom_dso_loader`
-target from previous step is already built.
-* TVM_BUILD_DIR=path-to-tvm-ios-build-dir
-* USE_CUSTOM_DSO_LOADER=1
+Printed `IP` is the IP address of your device and `PORT` is the number of port
+which was open for RPC connection. Next you should use them for connect your RPC
+client to the server.
 
-iOS RPC application with enabled custom DSO loader is able to process modules passed via regular
-`remote.upload("my_module.dylib")` mechanics. For example take a look inside `test_rpc_module_with_upload` test case
-of file [ios_rpc_test.py](tests/ios_rpc_test.py).
+Let's check that direct RPC connection works and we can upload a library with
+model and execute it on the device. For this purpose we will use
+[ios_rpc_test.py](tests/ios_rpc_test.py). Run it:
+```shell
+python3 tests/ios_rpc_test.py --host <device_ip> --port <rpc_server_port> --mode "pure_server"
+```
+This will compile TVM IR to shared libraries (CPU and Metal) and run vector
+addition on your iOS device. You are supposed to see something like this:
+```
+Metal: 0.000338692 secs/op
+CPU: 0.000219308 secs/op
+```
+
+### iOS RPC App with proxy
+Start the RPC proxy by running in a terminal:
+```shell
+python3 -m tvm.exec.rpc_proxy --host 0.0.0.0 --port 9090
+```
+
+On success, you should see something like this:
+```
+INFO:root:RPCProxy: client port bind to 0.0.0.0:9090
+INFO:root:RPCProxy: Websock port bind to 8888
+```
+Connect your iOS device to the RPC proxy via the iOS TVM RPC application. Set
+the `Address` and `Port` fields to the address and port of the RPC tracker
+respectively. Select mode `Proxy` and push `Connect` button. In success the
+text on the button will be changed to `Disconnect` and `Disconnected` in the top
+of the screen will be changed to `Connected`.
+On RPC proxy side you can see the next message in a log:
+```
+INFO:root:Handler ready TCPSocketProxy:<iPhone IP address>:server:iphone
+```
+Then we can check that RPC connection works and we can upload a library with
+model and execute it on the target device. For this purpose we will use
+[ios_rpc_test.py](tests/ios_rpc_test.py). Run it:
+```shell
+python3 tests/ios_rpc_test.py --host <host_ip_address> --port 9090 --mode "proxy"
+```
+The output should be the same as it was in previous section.
+
+### iOS RPC App with tracker
+First start an RPC tracker using
+```shell
+python3 -m tvm.exec.rpc_tracker --host 0.0.0.0 --port 9190
+```
+On success, you should see something like this:
+```
+INFO:RPCTracker:bind to 0.0.0.0:9190
+```
+Connect your iOS device to the RPC tracker via the iOS TVM RPC applcation. Set
+the `Address` and `Port` fields to the address and port of the RPC tracker
+respectively. Select mode `Tracker` and push `Connect` button. In success the
+text on the button will be changed to `Disconnect` and `Disconnected` in the top
+of the screen will be changed to `Connected`. On the host side you can check the
+connect by the following command:
+```shell
+python3 -m tvm.exec.query_rpc_tracker --port 9190
+```
+You are supposed to see something like this:
+```
+Tracker address 127.0.0.1:9190
+
+Server List
+----------------------------
+server-address  key
+----------------------------
+192.168.1.57:9190       server:iphone
+----------------------------
+
+Queue Status
+------------------------------
+key      total  free  pending
+------------------------------
+iphone   1      1     0
+------------------------------
+```
+
+Then we can check that RPC connection works and we can upload a library with
+model and execute it on the target device. For this purpose we will use
+[ios_rpc_test.py](tests/ios_rpc_test.py). Run it:
+```shell
+python3 tests/ios_rpc_test.py --host <host_ip_address> --port 9190 --mode "tracker"
+```
+The output will be the same as in section 
+[Pure RPC](#pure-rpc).
+
+## Communication without Wi-Fi and speed up in case of slow Wi-Fi
+Connection to the RPC server through `usbmux` can be used then you have slow,
+unstable or don't have any Wi-Fi connection. `usbmux` is used for binding local
+TCP port to port on the device and transfer packages between these ports by USB
+cable.
+
+First of all you should install `usbmux` to your system. You can do it with
+brew:
+```shell
+brew install usbmuxd
+```
+After that you can use `iproxy` program for binding ports. You can use it for
+all described workflows. Let's take a look how it works for [Pure RPC](#pure-rpc).

Review comment:
       It should work for all other modes also. I say about it in the sentence `You can use it for all described workflows.`. Just for example, I wrote about Pure RPC.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r711966709



##########
File path: apps/ios_rpc/README.md
##########
@@ -17,84 +17,21 @@
 
 # iOS TVM RPC
 
-This folder contains iOS RPC app that allows us to launch an rpc server on a iOS device(e.g. ipython)
-and connect to it through python script and do testing on the python side as normal TVM RPC.
-You will need XCode and an iOS device to use this.
-
-## RPC proxy
-Start the RPC proxy by running in a terminal:
-
-    python -m tvm.exec.rpc_proxy
-
-On success, you should see something like this:
-
-    INFO:root:RPCProxy: client port bind to 0.0.0.0:9090
-    INFO:root:RPCProxy: Websock port bind to 8888
-
-IP-address of this machine will be used to initialize ```TVM_IOS_RPC_PROXY_HOST```
-environment variable (see below).
+This folder contains iOS RPC app that allows us to launch an rpc server on a iOS
+device. You will need XCode and an iOS device to use this.
+
+## Table of Contents
+* [Building](#building)
+    * [Building TVM runtime and custom DSO loader plugin](#building-tvm-runtime-and-custom-dso-loader-plugin)
+    * [Building iOS TVM RPC application](#building-ios-tvm-rpc-application)
+* [Workflow](#workflow)
+    * [Pure RPC](#pure-rpc)
+    * [iOS RPC App with proxy](#ios-rpc-app-with-proxy)
+    * [iOS RPC App with tracker](#ios-rpc-app-with-tracker)
+* [Communication without Wi-Fi and speed up in case of slow Wi-Fi](#communication-without-wi-fi-and-speed-up-in-case-of-slow-wi-fi)
 
 ## Building
-Before start, please run ```init_proj.py``` to update XCode developer metadata. After this step, open
-```tvmrpc.xcodeproj``` by using XCode, build the App and install the App on the phone. Usually, we
-**do not** use the iOS App directly.
-
-To test an App, you can fill ``Address`` field with IP-address of RPC proxy
-(see above), and press ``Connect to Proxy``.
-
-On success, "Disconnected" will change to "Connected".
-On RPC proxy side you can see the next message in a log:
-
-    INFO:root:Handler ready TCPSocketProxy:<iPhone IP address>:server:iphone
-
-Now App can be closed by pressing the home button (or even removed from a device).
-
-## Workflow
-Due to security restriction of iOS10. We cannot upload dynamic libraries to the App and load it from sandbox.
-Instead, we need to build a list of libraries, pack them into the app bundle, launch the RPC server and
-connect to test the bundled libraries. We use ```xcodebuild test``` to automate this process. There is also
-one more approach to workaround this limitation, for more details please take a look into section
-[Custom DSO loader integration](#custom-dso-loader-plugin).
-
-The test script [tests/ios_rpc_test.py](tests/ios_rpc_test.py) is a good template for the workflow. With this
-script, we don't need to manually operate the iOS App, this script will build the app, run it and collect the results 
-automatically.
-
- To run the script,  you need to configure the following environment variables
-
-- ```TVM_IOS_CODESIGN``` The signature you use to codesign the app and libraries (e.g. ```iPhone Developer: Name (XXXX)```)
-- ```TVM_IOS_TEAM_ID``` The developer Team ID available at https://developer.apple.com/account/#/membership     
-- ```TVM_IOS_RPC_ROOT``` The root directory of the iOS rpc project
-- ```TVM_IOS_RPC_PROXY_HOST``` The RPC proxy address (see above)
-- ```TVM_IOS_RPC_DESTINATION``` The Xcode target device (e.g. ```platform=iOS,id=xxxx```)
-
-See instructions of how to find UUID of the iOS device:
-
-- https://www.innerfence.com/howto/find-iphone-unique-device-identifier-udid
-
-## How it works
-Let us explain how it works, the project look for ```rpc_config.txt``` file in the project root folder.
-The ```rpc_config.txt``` file should be in the following format:
-```
-<url> <port> <key>
-[path to dylib1]
-[path to dylib2]
-...
-```
-The build script will copy all the dynamic libraries into bundle ```tvmrpc.app/Frameworks/tvm```,
-which you will be able to load via RPC using ```remote.load_module```.
-It will also create an ```tvmrpc.app/Frameworks/tvm/rpc_config.txt``` containing the first line.
-
-When we run the testcase, the testcase read the configuration from ```tvmrpc.app/Frameworks/tvm/rpc_config.txt```
-and connect to the specified RPC proxy, start serving loop.
-
-So if we want to start the RPC from XCode IDE, simply manually modify ```rpc_config.txt``` file and click test.
-Then connect to the proxy via the python script.
-
-We can also use the RPC App directly, by typing in the address and press connect to connect to the proxy.
-However, the restriction is we can only load the modules that are bundled to the App.
-
-## Custom DSO loader plugin
+### Building TVM runtime and custom DSO loader plugin
 While iOS platform itself doesn't allow us to run an unsigned binary, where is a partial ability to run JIT code
 on real iOS devices. While application is running under debug session, system allows allocating memory with write
 and execute permissions (requirements of debugger). So we can use this feature to load binary on RPC side. For this

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r711970563



##########
File path: apps/ios_rpc/README.md
##########
@@ -114,19 +57,198 @@ cmake ..
   -DCMAKE_OSX_ARCHITECTURES=arm64
   -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0
   -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=ON
-  -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=XXXXXXXXXX  # insert your Team ID
   -DUSE_IOS_RPC=ON  # to enable build iOS RPC application from TVM project tree
-cmake --build . --target custom_dso_loader ios_rpc  # Will use custom DSO loader by default
-# Resulting iOS RPC app bundle will be placed in:
-# apps/ios_rpc/ios_rpc/src/ios_rpc-build/Build/Products/[CONFIG]-iphoneos/tvmrpc.app
+  -DUSE_METAL=ON    # to enable Metal runtime
+
+cmake --build . --target custom_dso_loader tvm_runtime
+```
+
+### Building iOS TVM RPC application
+Before start, please run [init_proj.py](./init_proj.py) to update XCode developer metadata:
+```shell
+python3 init_proj.py --team_id XXXXXXXXXX --tvm_build_dir "/path/to/tvm/ios/build/folder"
+```
+You can get value of your `team_id` in the following ways:
+- **You have registered Apple Developer Profile**. In this case you developer
+  Team ID available at https://developer.apple.com/account/#/membership
+- You are using your local developer profile. In this case run the command above
+  with `XXXXXXXXXX` instead of Team ID. Then open `tvmrpc.xcodeproj` by using
+  XCode, click on the project name (`tvmrpc`) on the left panel. Then select
+  target `tvmrpc`. At the bottom of this panel go to `Signing & Capabilities`
+  tab and in the field `Team` select your local developer profile
+  (`Your Name (Personal Team)`).
+  
+  On the first run of the application you may see message `Could not launch
+  "tvmrpc"` in the XCode and message `Untrusted Developer` on your device. In
+  this case it will be necessary to check the certificate. Open
+  `Settings -> General -> Device Management -> Apple Development: <your_email>
+  -> Trust "Apple Development: <your_email>"` and click `Trust`. After than you
+  should rerun your application in the XCode.
+
+After this step, open `tvmrpc.xcodeproj` by using XCode, build the App and
+install the App on the phone.
+
+## Workflow
+Due to security restriction of iOS10. We cannot upload dynamic libraries to the
+App and load it from sandbox.  Instead, we need to build a list of libraries,
+pack them into the app bundle, launch the RPC server and connect to test the
+bundled libraries.  We use one approach to workaround this limitation, for more
+details please take a look into section
+[Building TVM runtime and custom DSO loader integration](#building-tvm-runtime-and-custom-DSO-loader-plugin).

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r711969428



##########
File path: apps/ios_rpc/README.md
##########
@@ -114,19 +57,198 @@ cmake ..
   -DCMAKE_OSX_ARCHITECTURES=arm64
   -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0
   -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=ON
-  -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=XXXXXXXXXX  # insert your Team ID
   -DUSE_IOS_RPC=ON  # to enable build iOS RPC application from TVM project tree
-cmake --build . --target custom_dso_loader ios_rpc  # Will use custom DSO loader by default
-# Resulting iOS RPC app bundle will be placed in:
-# apps/ios_rpc/ios_rpc/src/ios_rpc-build/Build/Products/[CONFIG]-iphoneos/tvmrpc.app
+  -DUSE_METAL=ON    # to enable Metal runtime
+
+cmake --build . --target custom_dso_loader tvm_runtime
+```
+
+### Building iOS TVM RPC application
+Before start, please run [init_proj.py](./init_proj.py) to update XCode developer metadata:
+```shell
+python3 init_proj.py --team_id XXXXXXXXXX --tvm_build_dir "/path/to/tvm/ios/build/folder"
+```
+You can get value of your `team_id` in the following ways:
+- **You have registered Apple Developer Profile**. In this case you developer
+  Team ID available at https://developer.apple.com/account/#/membership
+- You are using your local developer profile. In this case run the command above
+  with `XXXXXXXXXX` instead of Team ID. Then open `tvmrpc.xcodeproj` by using

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch merged pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch merged pull request #7876:
URL: https://github.com/apache/tvm/pull/7876


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r711968450



##########
File path: apps/ios_rpc/README.md
##########
@@ -103,7 +40,13 @@ This custom `dlopen` mechanic is integrated into TVM RPC as plugin and registere
 application.
 
 The custom implementation of `dlopen` and other functions from `dlfcn.h` header are placed in separate repository,
-and will be downloaded automatically during cmake build for iOS. To run cmake build you may use next flags:
+and will be downloaded automatically during cmake build for iOS.
+
+Also, it is necessary to build `tvm_runtime.dylib` for our iOS device. The iOS
+TVM RPC application will be linked with this library. Use the following cmake
+flags:
+
+To run cmake build you may use next flags:

Review comment:
       Done




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r713477121



##########
File path: apps/ios_rpc/README.md
##########
@@ -114,19 +57,198 @@ cmake ..
   -DCMAKE_OSX_ARCHITECTURES=arm64
   -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0
   -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=ON
-  -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=XXXXXXXXXX  # insert your Team ID
   -DUSE_IOS_RPC=ON  # to enable build iOS RPC application from TVM project tree
-cmake --build . --target custom_dso_loader ios_rpc  # Will use custom DSO loader by default
-# Resulting iOS RPC app bundle will be placed in:
-# apps/ios_rpc/ios_rpc/src/ios_rpc-build/Build/Products/[CONFIG]-iphoneos/tvmrpc.app
+  -DUSE_METAL=ON    # to enable Metal runtime
+
+cmake --build . --target custom_dso_loader tvm_runtime
+```
+
+### Building iOS TVM RPC application
+Before start, please run [init_proj.py](./init_proj.py) to update XCode developer metadata:
+```shell
+python3 init_proj.py --team_id XXXXXXXXXX --tvm_build_dir "/path/to/tvm/ios/build/folder"
+```
+You can get value of your `team_id` in the following ways:
+- **You have registered Apple Developer Profile**. In this case you developer
+  Team ID available at https://developer.apple.com/account/#/membership
+- You are using your local developer profile. In this case run the command above
+  with `XXXXXXXXXX` instead of Team ID. Then open `tvmrpc.xcodeproj` by using
+  XCode, click on the project name (`tvmrpc`) on the left panel. Then select
+  target `tvmrpc`. At the bottom of this panel go to `Signing & Capabilities`
+  tab and in the field `Team` select your local developer profile
+  (`Your Name (Personal Team)`).
+  
+  On the first run of the application you may see message `Could not launch
+  "tvmrpc"` in the XCode and message `Untrusted Developer` on your device. In
+  this case it will be necessary to check the certificate. Open
+  `Settings -> General -> Device Management -> Apple Development: <your_email>
+  -> Trust "Apple Development: <your_email>"` and click `Trust`. After than you
+  should rerun your application in the XCode.
+
+After this step, open `tvmrpc.xcodeproj` by using XCode, build the App and
+install the App on the phone.
+
+## Workflow
+Due to security restriction of iOS10. We cannot upload dynamic libraries to the
+App and load it from sandbox.  Instead, we need to build a list of libraries,
+pack them into the app bundle, launch the RPC server and connect to test the
+bundled libraries.  We use one approach to workaround this limitation, for more
+details please take a look into section
+[Building TVM runtime and custom DSO loader integration](#building-tvm-runtime-and-custom-DSO-loader-plugin).
+
+The test script [tests/ios_rpc_test.py](tests/ios_rpc_test.py) and
+[tests/ios_rpc_mobilenet.py](tests/ios_rpc_mobilenet.py) are good templates for
+demonstrating the workflow.
+
+We have three different modes for iOS RPC server:
+- [Pure RPC](#pure-rpc): In this mode RPC server open port on the device and listening. Then
+  client connects to the server directly without any mediators.
+- [iOS RPC application with Proxy](#ios-rpc-app-with-proxy): RPC server and RPC client communicates through
+  `rpc_proxy`. The RPC server on iOS device notify `rpc_proxy` which was run on
+  host machine about itself and wait for incoming connections. Communications
+  between client and server works through `rpc_proxy`.
+- [iOS RPC application with Tracker](#ios-rpc-app-with-tracker): RPC server registered in the `rpc_tracker`
+  and client connects to the RPC server through `rpc_tracker`.
+
+### Pure RPC
+Start RPC server on your iOS device:
+- Enable verbose output.
+- Push on the `Connect` button.
+
+After that you supposed to see something like this in the app on the device:
+```
+IP: <device_ip>
+Port: <rpc_server_port>
 ```
 
-To enable using of Custom DSO Plugin during xcode build outsde of Cmake you should specify two additional variables.
-You can do it manually inside Xcode IDE or via command line args for `xcodebuild`. Make sure that `custom_dso_loader`
-target from previous step is already built.
-* TVM_BUILD_DIR=path-to-tvm-ios-build-dir
-* USE_CUSTOM_DSO_LOADER=1
+Printed `IP` is the IP address of your device and `PORT` is the number of port
+which was open for RPC connection. Next you should use them for connect your RPC
+client to the server.
 
-iOS RPC application with enabled custom DSO loader is able to process modules passed via regular
-`remote.upload("my_module.dylib")` mechanics. For example take a look inside `test_rpc_module_with_upload` test case
-of file [ios_rpc_test.py](tests/ios_rpc_test.py).
+Let's check that direct RPC connection works and we can upload a library with
+model and execute it on the device. For this purpose we will use
+[ios_rpc_test.py](tests/ios_rpc_test.py). Run it:
+```shell
+python3 tests/ios_rpc_test.py --host <device_ip> --port <rpc_server_port> --mode "pure_server"
+```
+This will compile TVM IR to shared libraries (CPU and Metal) and run vector
+addition on your iOS device. You are supposed to see something like this:
+```
+Metal: 0.000338692 secs/op
+CPU: 0.000219308 secs/op
+```
+
+### iOS RPC App with proxy
+Start the RPC proxy by running in a terminal:
+```shell
+python3 -m tvm.exec.rpc_proxy --host 0.0.0.0 --port 9090
+```
+
+On success, you should see something like this:
+```
+INFO:root:RPCProxy: client port bind to 0.0.0.0:9090
+INFO:root:RPCProxy: Websock port bind to 8888
+```
+Connect your iOS device to the RPC proxy via the iOS TVM RPC application. Set
+the `Address` and `Port` fields to the address and port of the RPC tracker
+respectively. Select mode `Proxy` and push `Connect` button. In success the
+text on the button will be changed to `Disconnect` and `Disconnected` in the top
+of the screen will be changed to `Connected`.
+On RPC proxy side you can see the next message in a log:
+```
+INFO:root:Handler ready TCPSocketProxy:<iPhone IP address>:server:iphone
+```
+Then we can check that RPC connection works and we can upload a library with
+model and execute it on the target device. For this purpose we will use
+[ios_rpc_test.py](tests/ios_rpc_test.py). Run it:
+```shell
+python3 tests/ios_rpc_test.py --host <host_ip_address> --port 9090 --mode "proxy"
+```
+The output should be the same as it was in previous section.
+
+### iOS RPC App with tracker
+First start an RPC tracker using
+```shell
+python3 -m tvm.exec.rpc_tracker --host 0.0.0.0 --port 9190
+```
+On success, you should see something like this:
+```
+INFO:RPCTracker:bind to 0.0.0.0:9190
+```
+Connect your iOS device to the RPC tracker via the iOS TVM RPC applcation. Set
+the `Address` and `Port` fields to the address and port of the RPC tracker
+respectively. Select mode `Tracker` and push `Connect` button. In success the
+text on the button will be changed to `Disconnect` and `Disconnected` in the top
+of the screen will be changed to `Connected`. On the host side you can check the
+connect by the following command:
+```shell
+python3 -m tvm.exec.query_rpc_tracker --port 9190
+```
+You are supposed to see something like this:
+```
+Tracker address 127.0.0.1:9190
+
+Server List
+----------------------------
+server-address  key
+----------------------------
+192.168.1.57:9190       server:iphone
+----------------------------
+
+Queue Status
+------------------------------
+key      total  free  pending
+------------------------------
+iphone   1      1     0
+------------------------------
+```
+
+Then we can check that RPC connection works and we can upload a library with
+model and execute it on the target device. For this purpose we will use
+[ios_rpc_test.py](tests/ios_rpc_test.py). Run it:
+```shell
+python3 tests/ios_rpc_test.py --host <host_ip_address> --port 9190 --mode "tracker"
+```
+The output will be the same as in section 
+[Pure RPC](#pure-rpc).
+
+## Communication without Wi-Fi and speed up in case of slow Wi-Fi
+Connection to the RPC server through `usbmux` can be used then you have slow,
+unstable or don't have any Wi-Fi connection. `usbmux` is used for binding local
+TCP port to port on the device and transfer packages between these ports by USB
+cable.
+
+First of all you should install `usbmux` to your system. You can do it with
+brew:
+```shell
+brew install usbmuxd
+```
+After that you can use `iproxy` program for binding ports. You can use it for
+all described workflows. Let's take a look how it works for [Pure RPC](#pure-rpc).

Review comment:
       ah my bad. that makes sense to me.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev edited a comment on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev edited a comment on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-914928241


   > @apeskov just reviving this--please address the comment
   
   Hello @areusch! @apeskov is busy with other activities. I'm working to finish this PR. What about your comment. I already fixed the typo, and now I'm working on adding a detailed instruction which will help to understand the new workflow and how it works.
   Also, I'm going to clean up the ios_rpc application and remove old code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] areusch commented on a change in pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

areusch commented on a change in pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#discussion_r711884712



##########
File path: apps/ios_rpc/README.md
##########
@@ -103,7 +40,13 @@ This custom `dlopen` mechanic is integrated into TVM RPC as plugin and registere
 application.
 
 The custom implementation of `dlopen` and other functions from `dlfcn.h` header are placed in separate repository,
-and will be downloaded automatically during cmake build for iOS. To run cmake build you may use next flags:
+and will be downloaded automatically during cmake build for iOS.
+
+Also, it is necessary to build `tvm_runtime.dylib` for our iOS device. The iOS
+TVM RPC application will be linked with this library. Use the following cmake
+flags:

Review comment:
       missing cmake flags?

##########
File path: apps/ios_rpc/README.md
##########
@@ -114,19 +57,198 @@ cmake ..
   -DCMAKE_OSX_ARCHITECTURES=arm64
   -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0
   -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=ON
-  -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=XXXXXXXXXX  # insert your Team ID
   -DUSE_IOS_RPC=ON  # to enable build iOS RPC application from TVM project tree
-cmake --build . --target custom_dso_loader ios_rpc  # Will use custom DSO loader by default
-# Resulting iOS RPC app bundle will be placed in:
-# apps/ios_rpc/ios_rpc/src/ios_rpc-build/Build/Products/[CONFIG]-iphoneos/tvmrpc.app
+  -DUSE_METAL=ON    # to enable Metal runtime
+
+cmake --build . --target custom_dso_loader tvm_runtime
+```
+
+### Building iOS TVM RPC application
+Before start, please run [init_proj.py](./init_proj.py) to update XCode developer metadata:
+```shell
+python3 init_proj.py --team_id XXXXXXXXXX --tvm_build_dir "/path/to/tvm/ios/build/folder"
+```
+You can get value of your `team_id` in the following ways:
+- **You have registered Apple Developer Profile**. In this case you developer
+  Team ID available at https://developer.apple.com/account/#/membership
+- You are using your local developer profile. In this case run the command above
+  with `XXXXXXXXXX` instead of Team ID. Then open `tvmrpc.xcodeproj` by using
+  XCode, click on the project name (`tvmrpc`) on the left panel. Then select
+  target `tvmrpc`. At the bottom of this panel go to `Signing & Capabilities`
+  tab and in the field `Team` select your local developer profile
+  (`Your Name (Personal Team)`).
+  
+  On the first run of the application you may see message `Could not launch
+  "tvmrpc"` in the XCode and message `Untrusted Developer` on your device. In
+  this case it will be necessary to check the certificate. Open
+  `Settings -> General -> Device Management -> Apple Development: <your_email>
+  -> Trust "Apple Development: <your_email>"` and click `Trust`. After than you
+  should rerun your application in the XCode.
+
+After this step, open `tvmrpc.xcodeproj` by using XCode, build the App and
+install the App on the phone.
+
+## Workflow
+Due to security restriction of iOS10. We cannot upload dynamic libraries to the
+App and load it from sandbox.  Instead, we need to build a list of libraries,
+pack them into the app bundle, launch the RPC server and connect to test the
+bundled libraries.  We use one approach to workaround this limitation, for more
+details please take a look into section
+[Building TVM runtime and custom DSO loader integration](#building-tvm-runtime-and-custom-DSO-loader-plugin).
+
+The test script [tests/ios_rpc_test.py](tests/ios_rpc_test.py) and
+[tests/ios_rpc_mobilenet.py](tests/ios_rpc_mobilenet.py) are good templates for
+demonstrating the workflow.
+
+We have three different modes for iOS RPC server:
+- [Pure RPC](#pure-rpc): In this mode RPC server open port on the device and listening. Then
+  client connects to the server directly without any mediators.
+- [iOS RPC application with Proxy](#ios-rpc-app-with-proxy): RPC server and RPC client communicates through
+  `rpc_proxy`. The RPC server on iOS device notify `rpc_proxy` which was run on
+  host machine about itself and wait for incoming connections. Communications
+  between client and server works through `rpc_proxy`.
+- [iOS RPC application with Tracker](#ios-rpc-app-with-tracker): RPC server registered in the `rpc_tracker`
+  and client connects to the RPC server through `rpc_tracker`.
+
+### Pure RPC
+Start RPC server on your iOS device:
+- Enable verbose output.
+- Push on the `Connect` button.
+
+After that you supposed to see something like this in the app on the device:
+```
+IP: <device_ip>
+Port: <rpc_server_port>
 ```
 
-To enable using of Custom DSO Plugin during xcode build outsde of Cmake you should specify two additional variables.
-You can do it manually inside Xcode IDE or via command line args for `xcodebuild`. Make sure that `custom_dso_loader`
-target from previous step is already built.
-* TVM_BUILD_DIR=path-to-tvm-ios-build-dir
-* USE_CUSTOM_DSO_LOADER=1
+Printed `IP` is the IP address of your device and `PORT` is the number of port
+which was open for RPC connection. Next you should use them for connect your RPC
+client to the server.
 
-iOS RPC application with enabled custom DSO loader is able to process modules passed via regular
-`remote.upload("my_module.dylib")` mechanics. For example take a look inside `test_rpc_module_with_upload` test case
-of file [ios_rpc_test.py](tests/ios_rpc_test.py).
+Let's check that direct RPC connection works and we can upload a library with
+model and execute it on the device. For this purpose we will use
+[ios_rpc_test.py](tests/ios_rpc_test.py). Run it:
+```shell
+python3 tests/ios_rpc_test.py --host <device_ip> --port <rpc_server_port> --mode "pure_server"
+```
+This will compile TVM IR to shared libraries (CPU and Metal) and run vector
+addition on your iOS device. You are supposed to see something like this:
+```
+Metal: 0.000338692 secs/op
+CPU: 0.000219308 secs/op
+```
+
+### iOS RPC App with proxy
+Start the RPC proxy by running in a terminal:
+```shell
+python3 -m tvm.exec.rpc_proxy --host 0.0.0.0 --port 9090
+```
+
+On success, you should see something like this:
+```
+INFO:root:RPCProxy: client port bind to 0.0.0.0:9090
+INFO:root:RPCProxy: Websock port bind to 8888
+```
+Connect your iOS device to the RPC proxy via the iOS TVM RPC application. Set
+the `Address` and `Port` fields to the address and port of the RPC tracker
+respectively. Select mode `Proxy` and push `Connect` button. In success the
+text on the button will be changed to `Disconnect` and `Disconnected` in the top
+of the screen will be changed to `Connected`.
+On RPC proxy side you can see the next message in a log:
+```
+INFO:root:Handler ready TCPSocketProxy:<iPhone IP address>:server:iphone
+```
+Then we can check that RPC connection works and we can upload a library with
+model and execute it on the target device. For this purpose we will use
+[ios_rpc_test.py](tests/ios_rpc_test.py). Run it:
+```shell
+python3 tests/ios_rpc_test.py --host <host_ip_address> --port 9090 --mode "proxy"
+```
+The output should be the same as it was in previous section.
+
+### iOS RPC App with tracker
+First start an RPC tracker using
+```shell
+python3 -m tvm.exec.rpc_tracker --host 0.0.0.0 --port 9190
+```
+On success, you should see something like this:
+```
+INFO:RPCTracker:bind to 0.0.0.0:9190
+```
+Connect your iOS device to the RPC tracker via the iOS TVM RPC applcation. Set
+the `Address` and `Port` fields to the address and port of the RPC tracker
+respectively. Select mode `Tracker` and push `Connect` button. In success the
+text on the button will be changed to `Disconnect` and `Disconnected` in the top
+of the screen will be changed to `Connected`. On the host side you can check the
+connect by the following command:
+```shell
+python3 -m tvm.exec.query_rpc_tracker --port 9190
+```
+You are supposed to see something like this:
+```
+Tracker address 127.0.0.1:9190
+
+Server List
+----------------------------
+server-address  key
+----------------------------
+192.168.1.57:9190       server:iphone
+----------------------------
+
+Queue Status
+------------------------------
+key      total  free  pending
+------------------------------
+iphone   1      1     0
+------------------------------
+```
+
+Then we can check that RPC connection works and we can upload a library with
+model and execute it on the target device. For this purpose we will use
+[ios_rpc_test.py](tests/ios_rpc_test.py). Run it:
+```shell
+python3 tests/ios_rpc_test.py --host <host_ip_address> --port 9190 --mode "tracker"
+```
+The output will be the same as in section 
+[Pure RPC](#pure-rpc).
+
+## Communication without Wi-Fi and speed up in case of slow Wi-Fi
+Connection to the RPC server through `usbmux` can be used then you have slow,
+unstable or don't have any Wi-Fi connection. `usbmux` is used for binding local
+TCP port to port on the device and transfer packages between these ports by USB
+cable.
+
+First of all you should install `usbmux` to your system. You can do it with
+brew:
+```shell
+brew install usbmuxd
+```
+After that you can use `iproxy` program for binding ports. You can use it for
+all described workflows. Let's take a look how it works for [Pure RPC](#pure-rpc).

Review comment:
       does it only work for Pure RPC, or can you use the other modes as well? (can you update README to clarify this?)

##########
File path: apps/ios_rpc/README.md
##########
@@ -114,19 +57,198 @@ cmake ..
   -DCMAKE_OSX_ARCHITECTURES=arm64
   -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0
   -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=ON
-  -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=XXXXXXXXXX  # insert your Team ID
   -DUSE_IOS_RPC=ON  # to enable build iOS RPC application from TVM project tree
-cmake --build . --target custom_dso_loader ios_rpc  # Will use custom DSO loader by default
-# Resulting iOS RPC app bundle will be placed in:
-# apps/ios_rpc/ios_rpc/src/ios_rpc-build/Build/Products/[CONFIG]-iphoneos/tvmrpc.app
+  -DUSE_METAL=ON    # to enable Metal runtime
+
+cmake --build . --target custom_dso_loader tvm_runtime
+```
+
+### Building iOS TVM RPC application
+Before start, please run [init_proj.py](./init_proj.py) to update XCode developer metadata:
+```shell
+python3 init_proj.py --team_id XXXXXXXXXX --tvm_build_dir "/path/to/tvm/ios/build/folder"
+```
+You can get value of your `team_id` in the following ways:
+- **You have registered Apple Developer Profile**. In this case you developer
+  Team ID available at https://developer.apple.com/account/#/membership
+- You are using your local developer profile. In this case run the command above
+  with `XXXXXXXXXX` instead of Team ID. Then open `tvmrpc.xcodeproj` by using
+  XCode, click on the project name (`tvmrpc`) on the left panel. Then select
+  target `tvmrpc`. At the bottom of this panel go to `Signing & Capabilities`
+  tab and in the field `Team` select your local developer profile
+  (`Your Name (Personal Team)`).
+  
+  On the first run of the application you may see message `Could not launch
+  "tvmrpc"` in the XCode and message `Untrusted Developer` on your device. In
+  this case it will be necessary to check the certificate. Open
+  `Settings -> General -> Device Management -> Apple Development: <your_email>
+  -> Trust "Apple Development: <your_email>"` and click `Trust`. After than you
+  should rerun your application in the XCode.
+
+After this step, open `tvmrpc.xcodeproj` by using XCode, build the App and
+install the App on the phone.
+
+## Workflow
+Due to security restriction of iOS10. We cannot upload dynamic libraries to the
+App and load it from sandbox.  Instead, we need to build a list of libraries,
+pack them into the app bundle, launch the RPC server and connect to test the
+bundled libraries.  We use one approach to workaround this limitation, for more
+details please take a look into section
+[Building TVM runtime and custom DSO loader integration](#building-tvm-runtime-and-custom-DSO-loader-plugin).

Review comment:
       ```suggestion
   bundled libraries. For more on the approach we use to work around this limitation, please 
   take a look into section
   [Building TVM runtime and custom DSO loader integration](#building-tvm-runtime-and-custom-DSO-loader-plugin).
   ```

##########
File path: apps/ios_rpc/README.md
##########
@@ -17,84 +17,21 @@
 
 # iOS TVM RPC
 
-This folder contains iOS RPC app that allows us to launch an rpc server on a iOS device(e.g. ipython)
-and connect to it through python script and do testing on the python side as normal TVM RPC.
-You will need XCode and an iOS device to use this.
-
-## RPC proxy
-Start the RPC proxy by running in a terminal:
-
-    python -m tvm.exec.rpc_proxy
-
-On success, you should see something like this:
-
-    INFO:root:RPCProxy: client port bind to 0.0.0.0:9090
-    INFO:root:RPCProxy: Websock port bind to 8888
-
-IP-address of this machine will be used to initialize ```TVM_IOS_RPC_PROXY_HOST```
-environment variable (see below).
+This folder contains iOS RPC app that allows us to launch an rpc server on a iOS
+device. You will need XCode and an iOS device to use this.
+
+## Table of Contents
+* [Building](#building)
+    * [Building TVM runtime and custom DSO loader plugin](#building-tvm-runtime-and-custom-dso-loader-plugin)
+    * [Building iOS TVM RPC application](#building-ios-tvm-rpc-application)
+* [Workflow](#workflow)
+    * [Pure RPC](#pure-rpc)
+    * [iOS RPC App with proxy](#ios-rpc-app-with-proxy)
+    * [iOS RPC App with tracker](#ios-rpc-app-with-tracker)
+* [Communication without Wi-Fi and speed up in case of slow Wi-Fi](#communication-without-wi-fi-and-speed-up-in-case-of-slow-wi-fi)
 
 ## Building
-Before start, please run ```init_proj.py``` to update XCode developer metadata. After this step, open
-```tvmrpc.xcodeproj``` by using XCode, build the App and install the App on the phone. Usually, we
-**do not** use the iOS App directly.
-
-To test an App, you can fill ``Address`` field with IP-address of RPC proxy
-(see above), and press ``Connect to Proxy``.
-
-On success, "Disconnected" will change to "Connected".
-On RPC proxy side you can see the next message in a log:
-
-    INFO:root:Handler ready TCPSocketProxy:<iPhone IP address>:server:iphone
-
-Now App can be closed by pressing the home button (or even removed from a device).
-
-## Workflow
-Due to security restriction of iOS10. We cannot upload dynamic libraries to the App and load it from sandbox.
-Instead, we need to build a list of libraries, pack them into the app bundle, launch the RPC server and
-connect to test the bundled libraries. We use ```xcodebuild test``` to automate this process. There is also
-one more approach to workaround this limitation, for more details please take a look into section
-[Custom DSO loader integration](#custom-dso-loader-plugin).
-
-The test script [tests/ios_rpc_test.py](tests/ios_rpc_test.py) is a good template for the workflow. With this
-script, we don't need to manually operate the iOS App, this script will build the app, run it and collect the results 
-automatically.
-
- To run the script,  you need to configure the following environment variables
-
-- ```TVM_IOS_CODESIGN``` The signature you use to codesign the app and libraries (e.g. ```iPhone Developer: Name (XXXX)```)
-- ```TVM_IOS_TEAM_ID``` The developer Team ID available at https://developer.apple.com/account/#/membership     
-- ```TVM_IOS_RPC_ROOT``` The root directory of the iOS rpc project
-- ```TVM_IOS_RPC_PROXY_HOST``` The RPC proxy address (see above)
-- ```TVM_IOS_RPC_DESTINATION``` The Xcode target device (e.g. ```platform=iOS,id=xxxx```)
-
-See instructions of how to find UUID of the iOS device:
-
-- https://www.innerfence.com/howto/find-iphone-unique-device-identifier-udid
-
-## How it works
-Let us explain how it works, the project look for ```rpc_config.txt``` file in the project root folder.
-The ```rpc_config.txt``` file should be in the following format:
-```
-<url> <port> <key>
-[path to dylib1]
-[path to dylib2]
-...
-```
-The build script will copy all the dynamic libraries into bundle ```tvmrpc.app/Frameworks/tvm```,
-which you will be able to load via RPC using ```remote.load_module```.
-It will also create an ```tvmrpc.app/Frameworks/tvm/rpc_config.txt``` containing the first line.
-
-When we run the testcase, the testcase read the configuration from ```tvmrpc.app/Frameworks/tvm/rpc_config.txt```
-and connect to the specified RPC proxy, start serving loop.
-
-So if we want to start the RPC from XCode IDE, simply manually modify ```rpc_config.txt``` file and click test.
-Then connect to the proxy via the python script.
-
-We can also use the RPC App directly, by typing in the address and press connect to connect to the proxy.
-However, the restriction is we can only load the modules that are bundled to the App.
-
-## Custom DSO loader plugin
+### Building TVM runtime and custom DSO loader plugin
 While iOS platform itself doesn't allow us to run an unsigned binary, where is a partial ability to run JIT code
 on real iOS devices. While application is running under debug session, system allows allocating memory with write
 and execute permissions (requirements of debugger). So we can use this feature to load binary on RPC side. For this

Review comment:
       ```suggestion
   and execute permissions (a debugger requirement). So we can use this feature to implement the `tvm.rpc.server.load_module` PackedFunc, used to load code over RPC. For this
   ```

##########
File path: apps/ios_rpc/README.md
##########
@@ -17,84 +17,21 @@
 
 # iOS TVM RPC
 
-This folder contains iOS RPC app that allows us to launch an rpc server on a iOS device(e.g. ipython)
-and connect to it through python script and do testing on the python side as normal TVM RPC.
-You will need XCode and an iOS device to use this.
-
-## RPC proxy
-Start the RPC proxy by running in a terminal:
-
-    python -m tvm.exec.rpc_proxy
-
-On success, you should see something like this:
-
-    INFO:root:RPCProxy: client port bind to 0.0.0.0:9090
-    INFO:root:RPCProxy: Websock port bind to 8888
-
-IP-address of this machine will be used to initialize ```TVM_IOS_RPC_PROXY_HOST```
-environment variable (see below).
+This folder contains iOS RPC app that allows us to launch an rpc server on a iOS
+device. You will need XCode and an iOS device to use this.
+
+## Table of Contents
+* [Building](#building)
+    * [Building TVM runtime and custom DSO loader plugin](#building-tvm-runtime-and-custom-dso-loader-plugin)
+    * [Building iOS TVM RPC application](#building-ios-tvm-rpc-application)
+* [Workflow](#workflow)
+    * [Pure RPC](#pure-rpc)
+    * [iOS RPC App with proxy](#ios-rpc-app-with-proxy)
+    * [iOS RPC App with tracker](#ios-rpc-app-with-tracker)
+* [Communication without Wi-Fi and speed up in case of slow Wi-Fi](#communication-without-wi-fi-and-speed-up-in-case-of-slow-wi-fi)
 
 ## Building
-Before start, please run ```init_proj.py``` to update XCode developer metadata. After this step, open
-```tvmrpc.xcodeproj``` by using XCode, build the App and install the App on the phone. Usually, we
-**do not** use the iOS App directly.
-
-To test an App, you can fill ``Address`` field with IP-address of RPC proxy
-(see above), and press ``Connect to Proxy``.
-
-On success, "Disconnected" will change to "Connected".
-On RPC proxy side you can see the next message in a log:
-
-    INFO:root:Handler ready TCPSocketProxy:<iPhone IP address>:server:iphone
-
-Now App can be closed by pressing the home button (or even removed from a device).
-
-## Workflow
-Due to security restriction of iOS10. We cannot upload dynamic libraries to the App and load it from sandbox.
-Instead, we need to build a list of libraries, pack them into the app bundle, launch the RPC server and
-connect to test the bundled libraries. We use ```xcodebuild test``` to automate this process. There is also
-one more approach to workaround this limitation, for more details please take a look into section
-[Custom DSO loader integration](#custom-dso-loader-plugin).
-
-The test script [tests/ios_rpc_test.py](tests/ios_rpc_test.py) is a good template for the workflow. With this
-script, we don't need to manually operate the iOS App, this script will build the app, run it and collect the results 
-automatically.
-
- To run the script,  you need to configure the following environment variables
-
-- ```TVM_IOS_CODESIGN``` The signature you use to codesign the app and libraries (e.g. ```iPhone Developer: Name (XXXX)```)
-- ```TVM_IOS_TEAM_ID``` The developer Team ID available at https://developer.apple.com/account/#/membership     
-- ```TVM_IOS_RPC_ROOT``` The root directory of the iOS rpc project
-- ```TVM_IOS_RPC_PROXY_HOST``` The RPC proxy address (see above)
-- ```TVM_IOS_RPC_DESTINATION``` The Xcode target device (e.g. ```platform=iOS,id=xxxx```)
-
-See instructions of how to find UUID of the iOS device:
-
-- https://www.innerfence.com/howto/find-iphone-unique-device-identifier-udid
-
-## How it works
-Let us explain how it works, the project look for ```rpc_config.txt``` file in the project root folder.
-The ```rpc_config.txt``` file should be in the following format:
-```
-<url> <port> <key>
-[path to dylib1]
-[path to dylib2]
-...
-```
-The build script will copy all the dynamic libraries into bundle ```tvmrpc.app/Frameworks/tvm```,
-which you will be able to load via RPC using ```remote.load_module```.
-It will also create an ```tvmrpc.app/Frameworks/tvm/rpc_config.txt``` containing the first line.
-
-When we run the testcase, the testcase read the configuration from ```tvmrpc.app/Frameworks/tvm/rpc_config.txt```
-and connect to the specified RPC proxy, start serving loop.
-
-So if we want to start the RPC from XCode IDE, simply manually modify ```rpc_config.txt``` file and click test.
-Then connect to the proxy via the python script.
-
-We can also use the RPC App directly, by typing in the address and press connect to connect to the proxy.
-However, the restriction is we can only load the modules that are bundled to the App.
-
-## Custom DSO loader plugin
+### Building TVM runtime and custom DSO loader plugin
 While iOS platform itself doesn't allow us to run an unsigned binary, where is a partial ability to run JIT code

Review comment:
       ```suggestion
   While iOS platform itself doesn't allow us to run an unsigned binary, there is a partial ability to run JIT code
   ```
   
   could you fix the typo?

##########
File path: apps/ios_rpc/README.md
##########
@@ -103,7 +40,13 @@ This custom `dlopen` mechanic is integrated into TVM RPC as plugin and registere
 application.
 
 The custom implementation of `dlopen` and other functions from `dlfcn.h` header are placed in separate repository,
-and will be downloaded automatically during cmake build for iOS. To run cmake build you may use next flags:
+and will be downloaded automatically during cmake build for iOS.
+
+Also, it is necessary to build `tvm_runtime.dylib` for our iOS device. The iOS

Review comment:
       would this be `libtvm_runtime.dylib`?

##########
File path: apps/ios_rpc/README.md
##########
@@ -114,19 +57,198 @@ cmake ..
   -DCMAKE_OSX_ARCHITECTURES=arm64
   -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0
   -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=ON
-  -DCMAKE_XCODE_ATTRIBUTE_DEVELOPMENT_TEAM=XXXXXXXXXX  # insert your Team ID
   -DUSE_IOS_RPC=ON  # to enable build iOS RPC application from TVM project tree
-cmake --build . --target custom_dso_loader ios_rpc  # Will use custom DSO loader by default
-# Resulting iOS RPC app bundle will be placed in:
-# apps/ios_rpc/ios_rpc/src/ios_rpc-build/Build/Products/[CONFIG]-iphoneos/tvmrpc.app
+  -DUSE_METAL=ON    # to enable Metal runtime
+
+cmake --build . --target custom_dso_loader tvm_runtime
+```
+
+### Building iOS TVM RPC application
+Before start, please run [init_proj.py](./init_proj.py) to update XCode developer metadata:
+```shell
+python3 init_proj.py --team_id XXXXXXXXXX --tvm_build_dir "/path/to/tvm/ios/build/folder"
+```
+You can get value of your `team_id` in the following ways:
+- **You have registered Apple Developer Profile**. In this case you developer
+  Team ID available at https://developer.apple.com/account/#/membership
+- You are using your local developer profile. In this case run the command above
+  with `XXXXXXXXXX` instead of Team ID. Then open `tvmrpc.xcodeproj` by using

Review comment:
       ```suggestion
   - You are using your local developer profile. In this case, leave `XXXXXXXXXX` in the command
     instead of substituting a Team ID. Then open `tvmrpc.xcodeproj` by using
   ```

##########
File path: apps/ios_rpc/README.md
##########
@@ -103,7 +40,13 @@ This custom `dlopen` mechanic is integrated into TVM RPC as plugin and registere
 application.
 
 The custom implementation of `dlopen` and other functions from `dlfcn.h` header are placed in separate repository,
-and will be downloaded automatically during cmake build for iOS. To run cmake build you may use next flags:
+and will be downloaded automatically during cmake build for iOS.
+
+Also, it is necessary to build `tvm_runtime.dylib` for our iOS device. The iOS
+TVM RPC application will be linked with this library. Use the following cmake
+flags:
+
+To run cmake build you may use next flags:

Review comment:
       ```suggestion
   Next, run the build using the following commands:
   ```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [tvm] echuraev commented on pull request #7876: [iOS] Add tracker support into ios-rpc application

[GitBox <gi...@apache.org>]

echuraev commented on pull request #7876:
URL: https://github.com/apache/tvm/pull/7876#issuecomment-924702051


   > thanks @echuraev could we rename Pure to Standalone? or happy to hear other options. I think standalone makes sense to me--that would be `--server_mode standalone` and that would remove the name conflict with PureRPC.
   
   Renamed to standalone


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org