[jira] [Comment Edited] (CRAIL-93) Using Crail with NVMf, the Default NQN also attaches the port number to the name.

["David Crespi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CRAIL-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16784558#comment-16784558 ] 

David Crespi edited comment on CRAIL-93 at 3/5/19 3:24 PM:
-----------------------------------------------------------

thanks for the answer Jonas, I think understand your issue.  Which is you'd like to have a single datanode attach to multiple subsystems... correct?  I don't know which target you're attaching to, but wouldn't using the combination of Host NQN & Host ID work for solving that problem?  If I remember correctly, that established a different QP.  Or in fact a host could have multiple subsystem NQN's to establish the connections using the same Host NQN. Or perhaps it isn't spdk you're attaching to?  Maybe make the NVMF_NQN variable a list "Subsystem1, Subsystem2, ... SubsystemN"?

For #2, I don't think I was clear on my post.  We generate the subsystem NQN, so I'd like to be able to use it.  We can set spdk to allow all hosts access to that subsystem.  If I used our subsystem NQN with crail, it demands that it's host NQN be used, even if the option on the target is completely open.  I would agree that most likely in practice users wouldn't leave this wide open, but they could... and it would cause Crail not to attach.  Along with that, what I noticed is since this is coming up in a container, I don't know the Host NQN Crail is generating (I have no issue with the format of the NQN), and it appears that if it fails initially, it will not attempt to reconnect. And if I restart the container, then we get a new Host NQN.... so how can I capture that NQN to add to the subsystem a priori?

 

As long as I have you, I have another question that you can most likely clear up.  In reading the documentation I'm a bit fuzzy on the functionality of the NVMf interface.  Is it truly just for the 

metadata, or can we also move the data there?  It appears the answer is no.  In my limited testing, It seems that the metadata isn't completely there either.  When I restarted both the namenode and datnode containers, the directory structure was invisible, but it did complain that directories/files were still there.  When deleting these and adding these back, all appeared to be okay again.  Weird eh.  I may have something set up incorrectly, so I thought I ask about specific expected behavior.

 

Thanks much for the support!

      david


was (Author: dcrespi):
thanks for the answer Jonas, I think understand your issue.  Which is you'd like to have a single datanode attach to multiple subsystems... correct?  I don't know which target you're attaching to, but wouldn't using the combination of Host NQN & Host ID work for solving that problem?  If I remember correctly, that established a different QP.  Or in fact a host could have multiple subsystem NQN's to establish the connections using the same Host NQN. Or perhaps it isn't spdk you're attaching to?  Maybe make the NVMF_NQN variable a list "Subsystem1, Subsystem2, ... SubsystemN"?

For #2, I don't think I was clear on my post.  We generate the subsystem NQN, so I'd like to be able to use it.  We can set spdk to allow all hosts access to that subsystem.  If I used our subsystem NQN with crail, it demands that it's host NQN be used, even if the option on the target it completely open.  I would agree that most likely in practice users wouldn't leave this wide open, but they could... and it would cause Crail not to attach.  Along with that what I noticed is since this is coming up in a container, I don't know the Host NQN Crail is generating (I have no issue with the format of the NQN), and it appears that if it fails initially, it will not attempt to reconnect. And if I restart the container, then we get a new Host NQN.... so how can I capture that NQN to add to the subsystem a priori?

 

As long as I have you, I have another question that you can most likely clear up.  In reading the documentation I'm a bit fuzzy on the functionality of the NVMf interface.  Is it truly just for the 

metadata, or can we also move the data there?  It appears the answer is no.  In my limited testing, It seems that the metadata isn't completely there either.  When I restarted both the namenode and datnode containers, the directory structure was invisible, but it did complain that directories/files were still there.  When deleting these and adding these back, all appeared to be okay again.  Weird eh.  I may have something set up incorrectly, so I thought I ask about specific expected behavior.

 

Thanks much for the support!

      david

> Using Crail with NVMf, the Default NQN also attaches the port number to the name.
> ---------------------------------------------------------------------------------
>
>                 Key: CRAIL-93
>                 URL: https://issues.apache.org/jira/browse/CRAIL-93
>             Project: Apache Crail
>          Issue Type: Bug
>    Affects Versions: 1.2
>         Environment: I'm running crail nodes with docker containers, on a Ubuntu 18.04 base.
>            Reporter: David Crespi
>            Assignee: Jonas Pfefferle
>            Priority: Critical
>
> This is the version I'm actually using: v1.1-2-gf0afadc
> I'm set up to use spdk on the backend to crail.  When attempting to attach, it appears that crail needs two subsystems to achieve the connection. Instead of allowing the default name, I have the system variable set: -e NVMF_NQN="nqn.2017-06.io.crail:cnode"
> 1) subsystem NQN: nqn.2017-06.io.crail:cnode
> 2) subsystem NQN: nqn.2017-06.io.crail:cnode4420
> 19/03/01 17:46:45 INFO crail: CrailHadoopFileSystem fs initialization done..
>  19/03/01 17:46:45 INFO crail: Connecting to NVMf target at Transport address = /192.168.2.104:4420, subsystem NQN = nqn.2017-06.io.crail:cnode4420
> It appears that the initial connect/discovery of the subsystem uses #1, but using the 
> crail commands (crail fs -mkdir /test) uses #2. 
> Both have to have a valid namespace attached as well.
>  
> It also appears that when using my own subsystem NQN (NVMF_NQN) name, crail wants to generate its own Host NQN.  A new one every time.  First, how do you learn of that
> NQN, and 2nd, it would be great to disable it if spdk has "allow any hosts" set.  It refuses
> to connect to spdk.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)