You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@devlake.apache.org by GitBox <gi...@apache.org> on 2022/10/29 02:28:16 UTC
[GitHub] [incubator-devlake] keon94 opened a new issue, #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
keon94 opened a new issue, #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616
## What and why to refactor
Extract functions use raw objects which they manually have to json-deserialize into predefined structs. This is needless boilerplate now that Go supports generics. Refactor the api_extractor.go (and other relevant framework files) to performs these deserializations under the hood and send the resulting Go structs to the user (Extract) functions.
The Converters suffer from this as well, although there's less boilerplate there. Refactor accordingly.
## Describe the solution you'd like
Example:
currently we have:
```go
Extract: func(row *helper.RawData) ([]interface{}, errors.Error) {
apiAccountOrgs := &[]GithubAccountOrgsResponse{}
err := json.Unmarshal(row.Data, apiAccountOrgs)
if err != nil {
return nil, errors.Convert(err)
}
simpleAccount := &SimpleAccountWithId{}
err = json.Unmarshal(row.Input, simpleAccount)
if err != nil {
return nil, errors.Convert(err)
}
// more logic...
}
```
it should become like:
```go
Extract: func(orgs []GithubAccountOrgsResponse, input *SimpleAccountWithId) ([]interface{}, errors.Error) {
// more logic...
}
```
(The API extractor will need to take two generic parameters: for the row data and the row input)
## Related issues
n/a
## Additional context
n/a
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1732713413
This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "keon94 (via GitHub)" <gi...@apache.org>.
keon94 commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1598074698
@Dev79844 I don't believe so. Sure, you can have it. Thanks for taking it on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] Dev79844 commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "Dev79844 (via GitHub)" <gi...@apache.org>.
Dev79844 commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1598070703
Hey @keon94 ! Is anyone working on this issue? If not can you assign it to me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] coldgust commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "coldgust (via GitHub)" <gi...@apache.org>.
coldgust commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1462475310
It seems that most extractors just need to json-deserialize `row.Data` field, such as `backend/plugins/github/tasks/account_extractor.go`.
I want to define like this
```go
type ApiExtractorArgs[T any] struct {
RawDataSubTaskArgs
Params interface{}
Extract func(data *T, row *RawData) ([]interface{}, errors.Error)
BatchSize int
}
```
It still pass `*RawData` parameter, and the user dicide to whether json-deserialize `row.Input`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1650736895
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1329912662
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] keon94 commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "keon94 (via GitHub)" <gi...@apache.org>.
keon94 commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1457107004
@coldgust Thanks for your interest. I've assigned this to you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Refactor][Framework] Refactor Extractors and Converters to use Generics [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
URL: https://github.com/apache/incubator-devlake/issues/3616
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1436145079
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] Startrekzky commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "Startrekzky (via GitHub)" <gi...@apache.org>.
Startrekzky commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1437044556
Bump
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Refactor][Framework] Refactor Extractors and Converters to use Generics [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1826158934
This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Refactor][Framework] Refactor Extractors and Converters to use Generics [incubator-devlake]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1836949091
This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1367652334
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] coldgust commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "coldgust (via GitHub)" <gi...@apache.org>.
coldgust commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1455880725
Hello, I would like to finish this taskļ¼could you assign this to me?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] github-actions[bot] commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1555390631
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-devlake] safeoy commented on issue #3616: [Refactor][Framework] Refactor Extractors and Converters to use Generics
Posted by "safeoy (via GitHub)" <gi...@apache.org>.
safeoy commented on issue #3616:
URL: https://github.com/apache/incubator-devlake/issues/3616#issuecomment-1571987879
Hello, I will take it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org