You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Sreehari Hegden <sh...@mathworks.com> on 2021/07/13 17:13:19 UTC

[MATLAB] arrow_13202 Enabling CI on Debian for MATLAB submissions

Hello Team,

I have been working to enable CI and testing for the MATLAB code on the Apache Arrow project.

matlab-actions
matlab-actions<https://github.com/matlab-actions/overview> is essentially MATLAB with GitHub Actions in which we can run MATLAB scripts/ functions including tests.
I have been experimenting with this in the branch arrow_13202<https://github.com/mathworks/arrow/tree/arrow_13202/> in mathworks/arrow<https://github.com/mathworks/arrow> fork, and we target to get MATLAB installed and execute tests when a GitHub push is done.

Current Status and Steps Forward
The prototype with matlab-actions cannot build the Arrow CPP libraries and MATLAB CPP sources on the fly as matlab-actions installs specified MATLAB release but doesn't build any source.
I am trying to automate the building of the latest CPP sources in the MATLAB and kickstart MATLAB with this latest build, hence need additional CI steps along with matlab-actions.

I started by adding arrow/.github/workflows/matlab.yml<https://github.com/mathworks/arrow/blob/arrow_13202/.github/workflows/matlab.yml> and arrow/ci/scripts/matlab_build.sh<https://github.com/mathworks/arrow/blob/arrow_13202/ci/scripts/matlab_build.sh>.
If there are any existing documents on best practices for CI with the arrow project and information on set of boilerplate to be used, please guide me to the documentation.

Once we have this CI for MATLAB working for Debian, it can be extended to Mac and Windows platforms.
I will keep the group posted on updates we make on this.

Thanks & Regards
Sreehari Hegden
MATLAB I/O QE
shegden@mathworks.com<ma...@mathworks.com>

RE: [MATLAB] arrow_13202 Enabling CI on Debian for MATLAB submissions

Posted by Sreehari Hegden <sh...@mathworks.com>.
Thank you, Kou!
I missed the mails earlier as my subscription to the mailing list was not completed.

Thanks for notifying me about adding matlab-actions to allow list.
I have created INFRA-22155<https://issues.apache.org/jira/browse/INFRA-22155> for the same. It's left Unassigned.
Let me know if any further action is needed.


Thanks & Regards
Sreehari Hegden
MATLAB I/O QE
shegden@mathworks.com<ma...@mathworks.com>

From: Sutou Kouhei <ko...@clear-code.com>>
Date: Tuesday, July 13, 2021 at 10:57 PM
To: dev@arrow.apache.org<ma...@arrow.apache.org> <de...@arrow.apache.org>>
Subject: Re: [MATLAB] arrow_13202 Enabling CI on Debian for MATLAB submissions
Hi,

Thanks for sharing the current status.

We need to add matlab-actions/* GitHub actions to allow list
to use them in apache/arrow.

Could you request it by creating a new JIRA issue for INFRA?
See also: https://issues.apache.org/jira/browse/INFRA-21239<https://issues.apache.org/jira/browse/INFRA-21239>

> I started by adding arrow/.github/workflows/matlab.yml<https://github.com/mathworks/arrow/blob/arrow_13202/.github/workflows/matlab.yml<https://github.com/mathworks/arrow/blob/arrow_13202/.github/workflows/matlab.yml>> and arrow/ci/scripts/matlab_build.sh<https://github.com/mathworks/arrow/blob/arrow_13202/ci/scripts/matlab_build.sh<https://github.com/mathworks/arrow/blob/arrow_13202/ci/scripts/matlab_build.sh>>.
> If there are any existing documents on best practices for CI with the arrow project and information on set of boilerplate to be used, please guide me to the documentation.

I think that we don't have any documents about it but your
approach looks good to me.


Thanks,
--
kou

In <BL...@BLAPR05MB7489.namprd05.prod.outlook.com>>
"[MATLAB] arrow_13202 Enabling CI on Debian for MATLAB submissions" on Tue, 13 Jul 2021 17:13:19 +0000,
Sreehari Hegden <sh...@mathworks.com>> wrote:

> Hello Team,
>
> I have been working to enable CI and testing for the MATLAB code on the Apache Arrow project.
>
> matlab-actions
> matlab-actions<https://github.com/matlab-actions/overview<https://github.com/matlab-actions/overview>> is essentially MATLAB with GitHub Actions in which we can run MATLAB scripts/ functions including tests.
> I have been experimenting with this in the branch arrow_13202<https://github.com/mathworks/arrow/tree/arrow_13202/<https://github.com/mathworks/arrow/tree/arrow_13202>> in mathworks/arrow<https://github.com/mathworks/arrow<https://github.com/mathworks/arrow>> fork, and we target to get MATLAB installed and execute tests when a GitHub push is done.
>
> Current Status and Steps Forward
> The prototype with matlab-actions cannot build the Arrow CPP libraries and MATLAB CPP sources on the fly as matlab-actions installs specified MATLAB release but doesn't build any source.
> I am trying to automate the building of the latest CPP sources in the MATLAB and kickstart MATLAB with this latest build, hence need additional CI steps along with matlab-actions.
>
> I started by adding arrow/.github/workflows/matlab.yml<https://github.com/mathworks/arrow/blob/arrow_13202/.github/workflows/matlab.yml<https://github.com/mathworks/arrow/blob/arrow_13202/.github/workflows/matlab.yml>> and arrow/ci/scripts/matlab_build.sh<https://github.com/mathworks/arrow/blob/arrow_13202/ci/scripts/matlab_build.sh<https://github.com/mathworks/arrow/blob/arrow_13202/ci/scripts/matlab_build.sh>>.
> If there are any existing documents on best practices for CI with the arrow project and information on set of boilerplate to be used, please guide me to the documentation.
>
> Once we have this CI for MATLAB working for Debian, it can be extended to Mac and Windows platforms.
> I will keep the group posted on updates we make on this.
>
> Thanks & Regards
> Sreehari Hegden
> MATLAB I/O QE
> shegden@mathworks.com<ma...@mathworks.com>>
>

Re: [MATLAB] arrow_13202 Enabling CI on Debian for MATLAB submissions

Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,

Thanks for sharing the current status.

We need to add matlab-actions/* GitHub actions to allow list
to use them in apache/arrow.

Could you request it by creating a new JIRA issue for INFRA?
See also: https://issues.apache.org/jira/browse/INFRA-21239

> I started by adding arrow/.github/workflows/matlab.yml<https://github.com/mathworks/arrow/blob/arrow_13202/.github/workflows/matlab.yml> and arrow/ci/scripts/matlab_build.sh<https://github.com/mathworks/arrow/blob/arrow_13202/ci/scripts/matlab_build.sh>.
> If there are any existing documents on best practices for CI with the arrow project and information on set of boilerplate to be used, please guide me to the documentation.

I think that we don't have any documents about it but your
approach looks good to me.


Thanks,
-- 
kou

In <BL...@BLAPR05MB7489.namprd05.prod.outlook.com>
  "[MATLAB] arrow_13202 Enabling CI on Debian for MATLAB submissions" on Tue, 13 Jul 2021 17:13:19 +0000,
  Sreehari Hegden <sh...@mathworks.com> wrote:

> Hello Team,
> 
> I have been working to enable CI and testing for the MATLAB code on the Apache Arrow project.
> 
> matlab-actions
> matlab-actions<https://github.com/matlab-actions/overview> is essentially MATLAB with GitHub Actions in which we can run MATLAB scripts/ functions including tests.
> I have been experimenting with this in the branch arrow_13202<https://github.com/mathworks/arrow/tree/arrow_13202/> in mathworks/arrow<https://github.com/mathworks/arrow> fork, and we target to get MATLAB installed and execute tests when a GitHub push is done.
> 
> Current Status and Steps Forward
> The prototype with matlab-actions cannot build the Arrow CPP libraries and MATLAB CPP sources on the fly as matlab-actions installs specified MATLAB release but doesn't build any source.
> I am trying to automate the building of the latest CPP sources in the MATLAB and kickstart MATLAB with this latest build, hence need additional CI steps along with matlab-actions.
> 
> I started by adding arrow/.github/workflows/matlab.yml<https://github.com/mathworks/arrow/blob/arrow_13202/.github/workflows/matlab.yml> and arrow/ci/scripts/matlab_build.sh<https://github.com/mathworks/arrow/blob/arrow_13202/ci/scripts/matlab_build.sh>.
> If there are any existing documents on best practices for CI with the arrow project and information on set of boilerplate to be used, please guide me to the documentation.
> 
> Once we have this CI for MATLAB working for Debian, it can be extended to Mac and Windows platforms.
> I will keep the group posted on updates we make on this.
> 
> Thanks & Regards
> Sreehari Hegden
> MATLAB I/O QE
> shegden@mathworks.com<ma...@mathworks.com>
>