You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@devlake.apache.org by GitBox <gi...@apache.org> on 2022/05/18 10:58:55 UTC
[GitHub] [incubator-devlake] warren830 opened a new issue, #1914: Refactor `parser.run` to several smaller funcs
warren830 opened a new issue, #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914
## Description
`parser.run` is a huge func, it's hard to do the unit test, usually we have to check if it's ok after collect all the data.
## Describe the solution you'd like
1. extract every `Foreach` to a single func
2. write unit test for every derived func
3. in the unit test, you should check the data source on the website first, then compare the result of the funcs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh closed issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
klesh closed issue #1914: Refactor `run` to several smaller funcs
URL: https://github.com/apache/incubator-devlake/issues/1914
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
klesh commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1142953434
@keon94 Happy Coding
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1142679904
I can take this one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
klesh commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1144321030
@keon94 I'm not very sure what @warren830 meant either, maybe you can submit your PR as a draft, and let's see how can we do unit-test?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
klesh commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1178421082
Closed by #2050
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1178397042
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1144155712
Hi @klesh, I'm not clear on step 3. Do you not want the tests to connect to a mocked git.Repository that returns mocked objects? By the way, to mock the repo, I'll have to wrap it in an interface so we can use something like mockery on it to generate the mock. Please let me know your expectation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] hezyin commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
hezyin commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1145365942
@keon94 My guess for what @warren830 meant in step 3 is to verify the data correctness of the git extractor plugin after data collection. Historically, we have been comparing the number of records in the database after running the plugin with the number of records displayed on the data source's website. For example, you can check how many commits/tags are there in a project on GitHub against how many git extractor collected in the database.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1144353119
@klesh @warren830 I have a draft PR here:
https://github.com/apache/incubator-devlake/pull/2050
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
keon94 commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1145467642
@hezyin, okay so we'd want to do integration/manual testing for that. I'm just not sure how exactly we want to perform unit testing here as we're hitting a third party API. We'd have to mock the API calls to the repo instance if that's what we want. Do we have any instance of go-mock/mockery usage in our code-base that I can use as reference? At first glance, I couldn't find any.
@warren830 please chime in when you get a chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] klesh commented on issue #1914: Refactor `run` to several smaller funcs
Posted by GitBox <gi...@apache.org>.
klesh commented on issue #1914:
URL: https://github.com/apache/incubator-devlake/issues/1914#issuecomment-1147142417
> @keon94 My guess for what @warren830 meant in step 3 is to verify the data correctness of the git extractor plugin after data collection. Historically, we have been comparing the number of records in the database after running the plugin with the number of records displayed on the data source's website. For example, you can check how many commits/tags are there in a project on GitHub against how many git extractor collected in the database.
That sounds like an `e2e-test` with actual git repo and database involved and all.
> @hezyin, okay so we'd want to do integration/manual testing for that. I'm just not sure how exactly we want to perform unit testing here as we're hitting a third party API. We'd have to mock the API calls to the repo instance if that's what we want. Do we have any instance of go-mock/mockery usage in our code-base that I can use as reference? At first glance, I couldn't find any. @warren830 please chime in when you get a chance.
I have a PR with mockery setup https://github.com/apache/incubator-devlake/pull/2053, but for what we want to test against this plugin, I think `e2e-test` would be a better fit, that we don't need mockery at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org