You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streampipes.apache.org by "RedC9 (via GitHub)" <gi...@apache.org> on 2026/09/19 20:40:12 UTC

[PR] feat(#4991): add image upload for assets

RedC9 opened a new pull request, #4997:
URL: https://github.com/apache/streampipes/pull/4997

   <!--
     ~ Licensed to the Apache Software Foundation (ASF) under one or more
     ~ contributor license agreements.  See the NOTICE file distributed with
     ~ this work for additional information regarding copyright ownership.
     ~ The ASF licenses this file to You under the Apache License, Version 2.0
     ~ (the "License"); you may not use this file except in compliance with
     ~ the License.  You may obtain a copy of the License at
     ~
     ~    http://www.apache.org/licenses/LICENSE-2.0
     ~
     ~ Unless required by applicable law or agreed to in writing, software
     ~ distributed under the License is distributed on an "AS IS" BASIS,
     ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     ~ See the License for the specific language governing permissions and
     ~ limitations under the License.
     ~
     -->
   
     <!--
   Thanks for contributing! Here are some tips you can follow to help us incorporate your contribution quickly and easily:
   1. If this is your first time, please read our contributor guidelines:
       - https://streampipes.apache.org/community/get-involved/
       - https://cwiki.apache.org/confluence/display/STREAMPIPES/Getting+Started
   2. Make sure the PR title is formatted like: `[#<GitHub issue id>] PR title ...`
   3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., `[WIP][#<GitHub issue id>] PR title ...`.
   4. Please write your PR title to summarize what this PR proposes/fixes.
   5. Link the PR to the corresponding GitHub issue (if present) in the `Development` section in the right menu bar.
   6. Be sure to keep the PR description updated to reflect all changes.
   7. If possible, provide a concise example to reproduce the issue for a faster review.
   8. Make sure tests pass via `mvn clean install`.
   9. (Optional) If the contribution is large, please file an Apache ICLA
       - http://apache.org/licenses/icla.pdf
   -->
   
   ### Purpose
   Closes https://github.com/apache/streampipes/issues/4991 . Users can upload images and have it displayed on the individual asset view screen. An example for the usage would be an image of a physical machine representing the asset.
   
   **### How it works**
   
   - The feature was created using only the existing File Management interactions as it already supports image file types. As such images are treated simply as files with no unique fields, but displayed as such.
   - During asset creation or editing a new section under Sites was added to allow the upload of images.
   - The image can be uploaded, deleted through confirmation or alternatively only have its file link removed and preserved in the database able to be removed in the Files section in the Settings.
   - The image can also be added though the Asset links sections and displayed in the edit page.
   - The uploaded and saved image can be viewed in the asset overview.
   - Upon click the image is enhanced for better viewing during both the edit Asset and view Asset pages.
   <img width="1270" height="581" alt="image" src="https://github.com/user-attachments/assets/14a3cf70-920d-4f78-af33-250731fcdad0" />
   <img width="1273" height="583" alt="image" src="https://github.com/user-attachments/assets/c834c5a0-3d17-44e3-a8b4-6be7332ec1a9" />
   <img width="1278" height="584" alt="image" src="https://github.com/user-attachments/assets/8f15f0ea-1c37-4108-9b67-e0ee15bedccf" />
   <img width="1279" height="589" alt="image" src="https://github.com/user-attachments/assets/36e8658d-7c5a-4927-ae05-f78f67de3b0e" />
   
   ### Remarks
   <!--
   Is there anything left we need to pay attention on?
   Are there some references that might be important? E.g. links to Confluence, or discussions
   on the mailing list or GitHub.
   -->
   The change primarily affects frontend files and reuses the existing file deletion function in the backend to avoid CouchDB revision conflicts.
   PR introduces (a) breaking change(s): No
   
   PR introduces (a) deprecation(s): No
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(#4991): add image upload for assets

Posted by "dominikriemer (via GitHub) via dev" <de...@streampipes.apache.org>.
dominikriemer commented on PR #4997:
URL: https://github.com/apache/streampipes/pull/4997#issuecomment-6607777536

   Hi @RedC9 thanks for the changes.
   
   Separating images from general file management is a useful step, and I think we can now simplify this quite a bit, because an asset image doesn't need to be a managed file of its own.
   
   **Proposal**
   
   - **Model:** Replace `List<FileMetadata> assetImages` with two fields on `SpAsset`: `imageId` and `filetype`. The backend sets both on upload (a generated UUID and the validated file type).
   - **Storage:** Keep writing the file to the filesystem under `<filesDir>/images/`, as you already do. We don't need the `imagemetadata` database or to store `FileMetadata` anywhere.
   - **REST:** Add dedicated endpoints for asset images to `AssetManagementResource`, backed by the same permission as the asset (`hasPermission(#elementId, 'READ')` / `'WRITE'`). Please verify that the requested `imageId` is actually referenced in that asset document, otherwise access to one asset would allow reading any image. With this, the info banner about image access can go too.
   - **UI:** Handle everything in the asset view and remove the Images settings page. Images uploaded there can't be linked to an asset anyway.
   
   **What this lets us remove**
   
   - `ImageResource`, `IImageMetadataStorage` and `ImageMetadataStorageImpl`
   - The `asset-images/by-image` view, its setup task and its migration, since each image belongs to exactly one asset and we no longer need to look up references
   - The one-image-per-asset validation, since a single field can't hold more than one
   - The duplicate filename check and rename flow, since files are stored by UUID
   
   **Image lifecycle**
   
   - Please hold a newly selected image in the browser and upload it when the user saves, rather than immediately. That removes the tracking of pending and replaced uploads, and the cleanup on discard. For a new asset, the upload has to happen after the asset is created, since the permission check needs it to exist.
   - Delete image files explicitly through the new endpoint rather than by comparing old and new asset documents on update. Moving a sub-asset to another asset removes it from the source document, and the image must survive that.
   - Deleting an asset should also delete the images of all its sub-assets on the backend.
   - If "Delete image" is applied on save like every other edit, the revision handling and the retry on 409/500 in `updateExistingAsset` can go as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(#4991): add image upload for assets

Posted by "RedC9 (via GitHub)" <gi...@apache.org>.
RedC9 commented on PR #4997:
URL: https://github.com/apache/streampipes/pull/4997#issuecomment-6013360480

   Hello @dominikriemer. I have made the changes according your comment, but before committing them here I would like to clarify a few points
   
   - The png were not added, because there were not originally supported in the Filetypes file and I was unsure if I should modify it. I have since done so and tested it, I could also add webp or other file types if needed, but I have only done so for png for the time being.
   - I have done the changes for adding assetImages as a list in SpAssetModel, but currently all assets that are part of a tree share the same images regardless of where uploaded. I could resolve this using a hash  map in SpAssetModel or try and implement images directly into SpAsset instead. If associated assets should share images, I could also leave it as is. Otherwise I could wait for a confirmation.
   - Added info banner as requested. Something I would like to add in regards to permissions is that for the deletion route both asset and file write permissions are needed and the upload requires the file write permission. Asset Admin default role does not have the file write permission so both will not be allowed for said role. I could remove this permission from the deletion route I created and make a new specific upload that requires only asset based permission. If file permissions are necessary even for these actions taken for assets then I could wait for a confirmation on this too.
   
   <img width="379" height="385" alt="image" src="https://github.com/user-attachments/assets/f3d8cd3a-f34b-4ce4-ba89-7e7e140ffb58" />
   
   - I have also added missing translation from phrases made by me.
   - I have added a warning on the file overview, that image files still present in assets have to be removed from there before deletion. Similar to how Site warning behaves,
   <img width="542" height="149" alt="image" src="https://github.com/user-attachments/assets/3d13ff7b-2fa6-4818-b971-d6c05f310b31" />
    
    
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(#4991): add image upload for assets

Posted by "dominikriemer (via GitHub)" <gi...@apache.org>.
dominikriemer commented on PR #4997:
URL: https://github.com/apache/streampipes/pull/4997#issuecomment-5885212963

   Hi @RedC9 thanks for the PR!
   
   Before merging the PR I have a few questions/suggestions:
   
   * Is there a reason why the upload only support JPG and not other image types, e.g. png?
   * If I understand correctly, the image upload adds an asset link of type file for each uploaded file. For displaying the file, all file metadata objects are fetched and then filtered based on the added asset links, which doesn't scale well for a large number of uploaded files. Also in case other files are associated with the asset which are images, but not meant to be an asset image, would be displayed. I suggest to add an explicit field in the `SpAssetModel` that contains a reference to the file metadata and manages the image.
   * File management does not support resource-based privilege management, so asset images would be publicly accessible by registered users. We should at least add an info banner to the upload dialog so that users are aware.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(#4991): add image upload for assets

Posted by "dominikriemer (via GitHub) via dev" <de...@streampipes.apache.org>.
dominikriemer commented on PR #4997:
URL: https://github.com/apache/streampipes/pull/4997#issuecomment-6780741376

   Hi @RedC9 if `filetype` is not needed you don't have to include it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(#4991): add image upload for assets

Posted by "RedC9 (via GitHub)" <gi...@apache.org>.
RedC9 commented on PR #4997:
URL: https://github.com/apache/streampipes/pull/4997#issuecomment-6349223402

   **Changelog**
   
   - Moved `assetImages `from `SpAssetModel ` to `SpAsset` so hierarchy issue on child tree is solved.
   - Restricted images to only one per asset. Frontend modified to accomodate.
   - Uploading a new image now prompts to replace the old one.
   - Kept using `FileMetadata` to avoid redundant repetition, but created a separate `imagemetadata `database and image REST endpoints to keep logic separate.
   - Applied only Asset permissions to Image actions. Logic and data is now separate from Files so this flow is more similar to Sites and Labels. This helps avoid the need to give Asset Admin seed role, access to files and keeps responsibilities within the current limits. 
   - Extended `FileHandler `to store image files in an images/ subdirectory.
   - Added indexed database checks to prevent deleting images referenced by assets.
   - Updated translations.
   - Added Java tests: `SpAssetImagesTest`, `ImageManagerTest `and `AssetResourceManagerTest`.
   - Added an Images settings page with search, pagination, upload, download and deletion. Accessible with the asset permissions.
   <img width="1900" height="814" alt="Screenshot 2026-10-22 203837" src="https://github.com/user-attachments/assets/6b011892-b43b-44e7-b77e-f9dd9e47d57a" />
   
   **Remarks**
   
   - Explored the idea of creating an abstract class for the current `FileMetadata` and  a possible `ImageMetadata`, but it seems to be to repetitive. However if different requirements are needed in the future for both then this option could be explored again.
   - It is possible to upload an image in the the new Image view at settings, however currently there is no why to link an image added in such way. Kept the functionality regardless allowing the opportunity to expand upon it if needed.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(#4991): add image upload for assets

Posted by "dominikriemer (via GitHub) via dev" <de...@streampipes.apache.org>.
dominikriemer merged PR #4997:
URL: https://github.com/apache/streampipes/pull/4997


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(#4991): add image upload for assets

Posted by "RedC9 (via GitHub) via dev" <de...@streampipes.apache.org>.
RedC9 commented on PR #4997:
URL: https://github.com/apache/streampipes/pull/4997#issuecomment-6657677492

   Hello @dominikriemer.
   I am working on the changes you suggested. I have a question regarding `filetype`. I understand that is created without added effort, but in the scope of the `SpAsset` it does not affect anything. In addition to the fact that images uploaded this way are not part of the `files` display it seems rather unnecessary. Should I keep it for possible filtering/actions in the future or can I remove it from `SpAsset` ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] feat(#4991): add image upload for assets

Posted by "RedC9 (via GitHub) via dev" <de...@streampipes.apache.org>.
RedC9 commented on PR #4997:
URL: https://github.com/apache/streampipes/pull/4997#issuecomment-6853048220

   Hi , I have tested upload without `filetype` and everything works as expected. Beside the function that sets it as a variable there were no other applications for it.
   Furthermore, only image files can be uploaded and other filetypes are not accepted. However, there currently is nothing to prevent a non-image file with a .png extension from being uploaded. This is an issue present at the other file upload operations as well. I tested a `png` image with `.csv` extension and it is accepted. Even at the Datasets it is accepted, but there is no data to extract, so the creation of a Dataset cannot be not completed from it.
   I tested a function to validate the pixels to make sure that it is an image before the import is finished. It works, but it does make the upload heavier briefly until completed. There is no notable delay and since image upload is not an action that will be done frequently it should not cause problems. Regardless, I am making this comment to confirm before committing it. 
   I have also added a limit to 10MB for the image upload.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@streampipes.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org