You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by GitBox <gi...@apache.org> on 2022/07/25 07:08:47 UTC
[GitHub] [incubator-devlake] klesh opened a new issue, #2078: Relocate files/dirs
klesh opened a new issue, #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078
## What and why to refactor
Many folders were randomly located in the wrong folder, and sometimes, they caused `cycle import` error,
and the root folder is bloated
## Describe the solution you'd like
- #2156
- remove `scripts` folder
- move `CONTRIBUTING.md` to `lake-website`
- move `plugins/*.md` to `lake-website`
- move `k8s-deploy.yml` to `releases` and update documentation
- rename `plugin_db_migration.go` to `migratable`
- remove `e2e` folder
- remove `plugins/helper/common/callbacks.go`
To reorganize the packages/folders to the following structure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1346975147
> Good thinking, I would like to share my 2cents as well:
>
> 1. both go/python belongs to the Backend, while `go` acting as the dominating role controlling the whole operation and `python` serving as a optional helper for plugins, it makes more sense to put `python` inside the folder containing our `go` code.
> 2. should we keep the backend/frontend code inside the same repo? there were discussion talking about extracting config-ui to another repo.
>
> It is big, let's do it step by step, finish the go module relocation first.
1. Yeah, it's just kind of awkward to mix go.* files with, say, requirements.txt or *.toml in the same top directory. If in the future we decide to add a third language as well it'll just get messier. How about backend/go/* and backend/python/*? backend-go/* and backend-py/* would achieve the same thing as well. I guess it's a matter of preference.
2. I agree, we should do that in another PR if we decide that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1347436480
Here's what I have at the moment:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] mindlesscloud closed issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
mindlesscloud closed issue #2078: Relocate files/dirs
URL: https://github.com/apache/incubator-devlake/issues/2078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
klesh commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1311186179
> @klesh
>
> 1. Do you have any suggestions for where any framework-level python code should go (from [[Feature][Python plugin SDK] Support plugin development in Python #3701](https://github.com/apache/incubator-devlake/issues/3701))? Should we create a top level directory called "python" and throw all *.py files in there? Or should we place them next to the *.go files at the appropriate places? (I personally prefer the first option because it keeps things well-isolated)
> 2. I've used the scripts folder for the model generator of Singer taps ([[Feature][Framework][POC] Streamed data collection using Singer Spec #3123](https://github.com/apache/incubator-devlake/issues/3123)). Do we still want to remove it?
>
> cc @CamilleTeruel
@keon94
1. I think the top-level directory makes sense. Everything related to python-support should be put into it including *.go files if they are meant for python plugin support
2. I don't have an opinion on this, it is up to you to judge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1350369880
Alright, I've moved everything under backend/. Question: there's no interface for Config - we just use Viper directly everywhere, so I can't split it into cfg and vipercfg. I assume this ought to be a separate refactor for later? For now, just keep what we have in the config package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] hezyin commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
hezyin commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1347034363
As you guys mentioned, it's really a matter of preference. Personally, I'd vote for having both backend/go/* and backend/python/* and there's a plugin folder under each language folder (e.g., backend/go/plugins and backend/python/plugins).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1309137318
@klesh
1. Do you have any suggestions for where any framework-level python code should go (from #3701)? Should we create a top level directory called "python" and throw all *.py files in there? Or should we place them next to the *.go files at the appropriate places? (I personally prefer the first option because it keeps things well-isolated)
2. I've used the scripts folder for the model generator of Singer taps (#3123). Do we still want to remove it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1179833328
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1350372249
I think we should eventually move to use a dependency injection framework, e.g. https://github.com/uber-go/fx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] closed issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #2078: Relocate files/dirs
URL: https://github.com/apache/incubator-devlake/issues/2078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1347134487
Another solution, to keep the directory levels fewer, is to just move everything Go to backend/. And the future Python code to backend-py/. Having a simple "backend" directory without a suffix would imply that this is the main backend directory (Go).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1345157669
@klesh I was thinking, how about we put everything go-related under a broader "go" directory as well? Since config-ui contains all frontend related logic, I think a dedicated folder for backend will also help with the organization. It also makes more since as we are using a top-level python/ directory as well in the Python SDK branch. So something like go-backend/ which contains all the modules, the go.mod/sum etc. Maybe py-backend in that other branch for python stuff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1261600460
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1307950407
bump
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1188462688
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
klesh commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1350241151
I would prefer to keep `backend` for **all backend code** including go/python and move all python into `backend/python` folder including `requirements.txt`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #2078: Relocate files/dirs
Posted by GitBox <gi...@apache.org>.
klesh commented on issue #2078:
URL: https://github.com/apache/incubator-devlake/issues/2078#issuecomment-1346030056
> @klesh I was thinking, how about we put everything go-related under a broader "go" directory as well? Since config-ui contains all frontend related logic, I think a dedicated folder for backend will also help with the organization. It also makes more since as we are using a top-level python/ directory as well in the Python SDK branch. So something like go-backend/ which contains all the modules/packages, the go.mod/sum etc. Maybe py-backend in that other branch for python stuff.
Good thinking, I would like to share my 2cents as well:
1. both go/python belongs to the Backend, while `go` acting as the dominating role controlling the whole operation and `python` serving as a optional helper for plugins, it makes more sense to put `python` inside the folder containing our `go` code.
2. should we keep the backend/frontend code inside the same repo? there were discussion talking about extracting config-ui to another repo.
It is big, let's do it step by step, finish the go module relocation first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org