[GitHub] flink issue #4358: [FLINK-7068][blob] change BlobService sub-classes for per...

[NicoK <gi...@git.apache.org>]

Github user NicoK commented on the issue:

    https://github.com/apache/flink/pull/4358
  
    After an offline discussion with @tillrohrmann, we agreed to have more of the transient vs. permanent BLOB handling inside the `BlobServer` by including the type into the `BlobKey`. The caches, however, remain separate (with a common base class for shared code) because of the different cache guarantees they give. In addition, we decided upon having a get-and-delete functionality for transient BLOBs (as planned by a TODO item previously).
    
    This means the following changes:
    - the `BlobServer` deletes transient BLOBs after they have been successfully downloaded to a BLOB cache (the client acknowledges that)
    - the `TransientBlobCache` and `PermanentBlobCache` differ in the functionality they provide:
      - `PermanentBlobCache` uses ref-counting for cleaning up its resources
      - `TransientBlobCache` imposes the user to manually delete requested BLOB files (from the cache!)
      - `TransientBlobCache` allows uploading (transient) BLOBs
      - `TransientBlobCache` allows non-job-related BLOBs
      - both have a GET method: `#getTransientFile()` vs. `#getPermanentFile()`
      - irrespective of the actual cache being used, the `BlobServer` decides whether to delete the BLOB (transient BLOBs only) or not (permanent BLOBs)
      - theoretically, a permanent BLOB could also be retrieved from the `TransientBlobCache` (and the other way around if job-related) - this behaviour is not used though but basic tests exist to cover it


---