[GitHub] nifi issue #2561: NIFI-4035 Implement record-based Solr processors

[bbende <gi...@git.apache.org>]

Github user bbende commented on the issue:

    https://github.com/apache/nifi/pull/2561
  
    Nested records and arrays of nested records are not working correctly...
    
    **Scenario 1 - Nested Record**
    
    Schema:
    ```
    {
        "type": "record",
        "name": "exams",
        "fields" : [
          { "name": "first", "type": "string" },
          { "name": "last", "type": "string" },
          { "name": "grade", "type": "int" },
          {
            "name": "exam",
            "type": {
              "name" : "exam",
              "type" : "record",
              "fields" : [
                { "name": "subject", "type": "string" },
                { "name": "test", "type": "string" },
                { "name": "marks", "type": "int" }
              ]
            }
          }
        ]
    }
    ```
    
    Input:
    ```
    {
      "first": "Abhi",
      "last": "R",
      "grade": 8,
      "exam": {
        "subject": "Maths",
        "test" : "term1",
        "marks" : 90
      }
    }
    ```
    
    Result:
    ```
    java.util.NoSuchElementException: No value present
    	at java.util.Optional.get(Optional.java:135)
    	at org.apache.nifi.processors.solr.SolrUtils.writeRecord(SolrUtils.java:313)
    	at org.apache.nifi.processors.solr.SolrUtils.writeValue(SolrUtils.java:384)
    	at org.apache.nifi.processors.solr.SolrUtils.writeRecord(SolrUtils.java:314)
    	at org.apache.nifi.processors.solr.PutSolrRecord.onTrigger(PutSolrRecord.java:247)
    	at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
    ```
    
    **Scenario 2 - Array of Records**
    
    Schema:
    ```
    {
        "type": "record",
        "name": "exams",
        "fields" : [
          { "name": "first", "type": "string" },
          { "name": "last", "type": "string" },
          { "name": "grade", "type": "int" },
          {
            "name": "exams",
            "type": {
              "type" : "array",
              "items" : {
                "name" : "exam",
                "type" : "record",
                "fields" : [
                    { "name": "subject", "type": "string" },
                    { "name": "test", "type": "string" },
                    { "name": "marks", "type": "int" }
                ]
              }
            }
          }
        ]
    }
    ```
    
    Input:
    ```
    {
    "first": "Abhi",
    "last": "R",
    "grade": 8,
    "exams": [
        {
            "subject": "Maths",
            "test" : "term1",
            "marks" : 90
        },
        {
            "subject": "Physics",
            "test" : "term1",
            "marks" : 95
        }
    ]
    }
    ```
    
    Result:
    
    Solr Document with multi-valued field exams where the values are the toString of a MapRecord:
    ```
     "exams":["org.apache.nifi.serialization.record.MapRecord:MapRecord[{marks=90, test=term1, subject=Maths}]",
              "org.apache.nifi.serialization.record.MapRecord:MapRecord[{marks=95, test=term1, subject=Physics}]"],
    ```
    Should have created fields like exams_marks, exams_test, exams_subject.
    
    Here is a full template for the two scenarios:
    
    https://gist.githubusercontent.com/bbende/edc2e7d61db83b29533ac3fc520de30f/raw/8764d50ed5e14d876c53a0b84b3af5741d910b3b/PutSolrRecordTesting.xml
    
    There needs to be unit tests that cover both these cases.


---