[GitHub] nifi issue #2292: NIFI-4639: fresh writer for each output record

[markap14 <gi...@git.apache.org>]

Github user markap14 commented on the issue:

    https://github.com/apache/nifi/pull/2292
  
    @joewitt I agree that this seems correct but have concerns about performance as well. I built this PR and then copied in the kafka-0.11 processors from NiFi 1.4.0 in order to do a side-by-side comparison. The results are shown below (left-hand-side being the new one):
    
    ![screen shot 2017-12-07 at 1 33 41 pm](https://user-images.githubusercontent.com/184268/33732137-6fdf77ee-db53-11e7-94e8-1640b516644b.png)
    
    So we can see that the performance dropped by at 15%. This was the JSON writer, specifically. Haven't tried with the Avro writer, but in any case we need to ensure that this has high performance for all cases.
    
    I wonder if a better option may be to add some sort of `rebind(OuptutStream)` method to RecordWriter, so that it would reset itself to write to a new Output Stream. This would avoid having to recreate potentially expensive objects for every single record. Then, instead of creating a new writer each time, we can just re-bind the existing writer to a new OutputStream, write the record, flush, and repeat.


---