You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@buildstream.apache.org by GitBox <gi...@apache.org> on 2022/03/03 09:06:09 UTC

[GitHub] [buildstream] gtristan opened a new pull request #1607: source.py: Split up staging functions into separate codepaths.

gtristan opened a new pull request #1607:
URL: https://github.com/apache/buildstream/pull/1607


   Currently we pass a file path string or Directory object to these methods
   depending on whether BST_STAGE_VIRTUAL_DIRECTORY is set, which is not very
   pretty for plugin developers especially if they are using type annotations
   and need to handle `Union[str, Directory]` in these methods.
   
   Instead, created Source.stage_directory() and Source.init_workspace_directory()
   to handle the Directory variants of these methods separately.
   
   Also updated `local` and `workspace` plugins to use the new methods.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [buildstream] gtristan commented on pull request #1607: source.py: Split up staging functions into separate codepaths.

Posted by GitBox <gi...@apache.org>.
gtristan commented on pull request #1607:
URL: https://github.com/apache/buildstream/pull/1607#issuecomment-1060178913


   > I have been thinking of this too (in the context of #1317). What I was thinking to do is to always pass a Directory to `stage()` (I didn't look into `init_workspace()`, so I can't say anything). The plugins would need to change indeed, but I think it's not that hard.
   > 
   > ```
   > with self.tempdir() as directory:
   >     ...
   >     vdir.import_files(directory)
   > ```
   
   It is desirable for plugins to stage directly into cas using the Directory API and avoid an additional disk copy, so I can see the desire to make the API more rigid and have more direct messaging to plugin authors that they ultimately have to get the data into a cas Directory, however I don't think it's a realistic goal for the majority of source plugins, and so I would much prefer to favor a simpler and more convenient API for the majority of source plugins.
   
   Regarding `init_workspace()`, it's main purpose is to prepare a special case staging ground for a workspace - plugins like `git` and `bzr` which tend to carry non-deterministic data in them because they carry project history (i.e. the `.git/` directory) will normally omit such non-deterministic data at normal `stage()` time, but will include the `.git/` directory for the convenience of a workspace.
   
   In the end what we're really talking about is whether or not to provide a more natural/convenient API for most plugins rather then forcing them to do the `with self.tempdir()` trick themselves, I think that since the majority of plugins will have to do it, we aught to do it for them and offer a more convenient API.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



[GitHub] [buildstream] abderrahim commented on pull request #1607: source.py: Split up staging functions into separate codepaths.

Posted by GitBox <gi...@apache.org>.
abderrahim commented on pull request #1607:
URL: https://github.com/apache/buildstream/pull/1607#issuecomment-1059019265


   I have been thinking of this too (in the context of #1317). What I was thinking to do is to always pass a Directory to `stage()` (I didn't look into `init_workspace()`, so I can't say anything). The plugins would need to change indeed, but I think it's not that hard.
   
   ```
   with self.tempdir() as directory:
       ...
       vdir.import_files(directory)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@buildstream.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org