You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stratos.apache.org by Imesh Gunaratne <im...@apache.org> on 2015/02/13 03:01:15 UTC
[Discuss] Fixing Python Cartridge Agent (PCA) Issues
Hi Devs,
Yesterday we fixed several issues in PCA, now I'm seeing the below error
when trying to receive artifact updated event:
[2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error processing
'ArtifactUpdatedEvent' event
Traceback (most recent call last):
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
line 103, in run
handler(event_msg)
File "agent.py", line 185, in on_artifact_updated
CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
line 122, in on_artifact_updated_event
subscribe_run, updated =
agentgithandler.AgentGitHandler.checkout(repo_info)
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
line 89, in checkout
git_repo = AgentGitHandler.clone(git_repo)
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
line 190, in clone
if os.path.isdir(git_repo.local_repo_path):
File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
st = os.stat(s)
TypeError: coercing to Unicode: need string or buffer, NoneType foun
Thanks
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Chamila De Alwis <ch...@wso2.com>.
Hi Imesh,
I've opened a PR [1] for the GitPython work done so far. Please merge.
[1] - https://github.com/apache/stratos/pull/224
Regards,
Chamila de Alwis
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com
On Fri, Feb 13, 2015 at 10:08 AM, Imesh Gunaratne <im...@apache.org> wrote:
> Hi Chamila,
>
> Great stuff, please send a PR. We will investigate this. For me git clone
> works fine with authentication.
>
> Thanks
>
> On Fri, Feb 13, 2015 at 10:01 AM, Chamila De Alwis <ch...@wso2.com>
> wrote:
>
>> Hi Imesh,
>>
>> I have almost completed the work to switch to gitpython. However there is
>> an issue with the type of the auth url, that is given to GitPython. Other
>> than that, public repo cloning now works with the library.
>>
>> When a url with credentials is passed, I'm getting the following error.
>> Investigating further.
>>
>> [2015-02-13 09:59:13,200] ERROR {eventsubscriber.py:run} - Error
>> processing 'ArtifactUpdatedEvent' event
>> Traceback (most recent call last):
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/subscriber/eventsubscriber.py",
>> line 103, in run
>> handler(event_msg)
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/test-classes/../10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/agent.py",
>> line 185, in on_artifact_updated
>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/extensions/extensionhandler.py",
>> line 122, in on_artifact_updated_event
>> subscribe_run, updated =
>> agentgithandler.AgentGitHandler.checkout(repo_info)
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
>> line 89, in checkout
>> git_repo = AgentGitHandler.clone(git_repo)
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
>> line 192, in clone
>> Repo.clone_from(unicode(git_repo.repo_url, "utf-8"),
>> unicode(git_repo.local_repo_path, "utf-8"))
>> File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line
>> 844, in clone_from
>> return cls._clone(Git(os.getcwd()), url, to_path, GitCmdObjectDB,
>> progress, **kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line
>> 791, in _clone
>> v=True, **add_progress(kwargs, git, progress))
>> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 431, in
>> <lambda>
>> return lambda *args, **kwargs: self._call_process(name, *args,
>> **kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 802, in
>> _call_process
>> return self.execute(make_call(), **_kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 556, in
>> execute
>> **subprocess_kwargs
>> File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
>> errread, errwrite)
>> File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
>> raise child_exception
>> TypeError: execve() arg 2 must contain only strings
>>
>>
>> Regards,
>> Chamila de Alwis
>> Software Engineer | WSO2 | +94772207163
>> Blog: code.chamiladealwis.com
>>
>>
>>
>> On Fri, Feb 13, 2015 at 9:30 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> I analyzed the issue the NoneType error has occurred due to the EOF
>>> exception which has been raised while executing git clone shell command:
>>>
>>> The problem is in the following line:
>>>
>>> git_repo = AgentGitHandler.clone(git_repo)
>>> AgentGitHandler.log.debug("Git clone executed: [tenant-id] %s [repo-url] %s",
>>> git_repo.tenant_id, git_repo.repo_url)
>>>
>>> Here clone() method returns NoneType if clone fails. Anyway the root cause of the problem is with the Git client implementation. Will switch to GitPython to avoid this intermittent issue of EOF.
>>>
>>> Thanks
>>>
>>>
>>> On Fri, Feb 13, 2015 at 9:20 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> I can see the tenant id attribute in received message, may be it was
>>>> not properly set:
>>>>
>>>> [2015-02-13 03:44:06,394] DEBUG {eventsubscriber.py:on_message} -
>>>> Message received: instance/notifier/ArtifactUpdatedEvent:
>>>>
>>>> {"clusterId":"php1.php.domain","repoUserName":"","repoPassword":"","repoURL":"
>>>> https://github.com/imesh/stratos-php-applications.git
>>>> ","tenantId":"-1234","commitEnabled":false}
>>>>
>>>> On Fri, Feb 13, 2015 at 9:19 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> Also the following error was there in the log:
>>>>>
>>>>> [2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception
>>>>> while executing git clone command on git binary : EOF("End Of File (EOF).
>>>>> Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion:
>>>>> 3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', '
>>>>> https://github.com/imesh/stratos-php-applications.git',
>>>>> '/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer
>>>>> (last 100 chars): ''\nbefore (last 100 chars): 'ng deltas: 98% (71/72)
>>>>> \\rResolving deltas: 100% (72/72) \\rResolving deltas: 100% (72/72),
>>>>> done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index:
>>>>> None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed:
>>>>> False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile:
>>>>> None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase:
>>>>> False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose:
>>>>> 0.1\ndelayafterterminate: 0.1",)
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>> line 200, in clone
>>>>> result = clone_op.expect(["Username for .*", "Password for .*",
>>>>> "Checking connectivity... done."])
>>>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>>>> line 1451, in expect
>>>>> timeout, searchwindowsize)
>>>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>>>> line 1466, in expect_list
>>>>> timeout, searchwindowsize)
>>>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>>>> line 1554, in expect_loop
>>>>> raise EOF(str(err) + '\n' + str(self))
>>>>> EOF: End Of File (EOF). Exception style platform.
>>>>> <pexpect.spawn object at 0x25c17d0>
>>>>>
>>>>> On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> The problem was in populate-user-data.sh file [1], APPLICATION_PATH
>>>>>> has been removed from it. I just fixed it, pushed the new PHP docker image
>>>>>> to docker hub and did another flow test and found the below error:
>>>>>>
>>>>>> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error
>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>> Traceback (most recent call last):
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>> line 103, in run
>>>>>> handler(event_msg)
>>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>>
>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>> line 122, in on_artifact_updated_event
>>>>>> subscribe_run, updated =
>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>> line 91, in checkout
>>>>>> git_repo.tenant_id, git_repo.repo_url)
>>>>>> AttributeError: 'NoneType' object has no attribute 'tenant_id'
>>>>>>
>>>>>> [1]
>>>>>> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <im...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks Lakmal, yes as I found basic features we need works with Git
>>>>>>> python. Will do that.
>>>>>>> Regarding the above error, seems like it has occurred due to the
>>>>>>> APPLICATION_PATH not being sent in payload, there was no None check or a
>>>>>>> log saying its not found:
>>>>>>>
>>>>>>> def clone(git_repo):
>>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>>> # delete and recreate local repo path if exists
>>>>>>> GitUtils.delete_folder_tree(git_repo.local_repo_path)
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <
>>>>>>> lakmal@wso2.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> As it looks like this error has been caused by the agent git
>>>>>>>>> handler while executing git clone, IMO its better if we can switch to
>>>>>>>>> GitPython.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> +1, If GitPython has full functionality what we expect, lets move
>>>>>>>> to it. I was under impression that it has some limitation. If so please not
>>>>>>>> waist time to fix current one.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <imesh@apache.org
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> Hi Devs,
>>>>>>>>>>
>>>>>>>>>> Yesterday we fixed several issues in PCA, now I'm seeing the
>>>>>>>>>> below error when trying to receive artifact updated event:
>>>>>>>>>>
>>>>>>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>>>>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File
>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>>>> line 103, in run
>>>>>>>>>> handler(event_msg)
>>>>>>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>>>>>>
>>>>>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>>>>>> File
>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>>>>>> line 122, in on_artifact_updated_event
>>>>>>>>>> subscribe_run, updated =
>>>>>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>>>>>> File
>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>>>> line 89, in checkout
>>>>>>>>>> git_repo = AgentGitHandler.clone(git_repo)
>>>>>>>>>> File
>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>>>> line 190, in clone
>>>>>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>>>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>>>>>>>> st = os.stat(s)
>>>>>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType
>>>>>>>>>> foun
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>
>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Imesh Gunaratne
>>>>>>>>>
>>>>>>>>> Technical Lead, WSO2
>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Lakmal Warusawithana
>>>>>>>> Vice President, Apache Stratos
>>>>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>>>>> Mobile : +94714289692
>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Imesh Gunaratne
>>>>>>>
>>>>>>> Technical Lead, WSO2
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
Hi Lakmal,
That's a great news! :-) Very nice to hear that.
Yes definitely we were lacking port information on the UI/API method. I
will add them and also add the public IP addresses to the Kubernetes
cluster model ASAP.
Thanks
On Fri, Feb 13, 2015 at 4:24 PM, Lakmal Warusawithana <la...@wso2.com>
wrote:
> HI Imesh,
>
> Thanks, I have tested on EC2 and single-cartridge kubernetes application
> deployed and activated successfully.
>
> Few more things need to complete end to end.
>
> - In the case of running docker/kubernetes on top of of EC2/IaaSs we
> need to provide Kubernetes host's private and public IPs with the
> kubernetes cluster registration to Stratos. Make public IP optional,
> because if we deploy Kubernetes in bare metal machine, it may only have
> single IP.
> - In the case of having public IPs for minions we should show that IP
> as default public IP (see attachment)
> - Also we need to list all private port list (which export in the
> container) and also need to show all public port lists which mapped into
> kubernetes service proxy
>
>
> On Fri, Feb 13, 2015 at 2:35 PM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> No, docker image is stratos/php:4.1.0-beta, source tag for this fix
>> is 4.1.0-beta-kubernetes-v1.
>>
>> I have now made the above Thrift receiver hostname 0.0.0.0 by default to
>> make it much easier to use the product.
>>
>> Thanks
>>
>> On Fri, Feb 13, 2015 at 1:56 PM, Lakmal Warusawithana <la...@wso2.com>
>> wrote:
>>
>>> Do you mean docker image with tag 4.1.0-beta-kubernetes-v1 ?
>>>
>>> On Fri, Feb 13, 2015 at 1:53 PM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> I have created a tag: 4.1.0-beta-kubernetes-v1 to mark this fix.
>>>>
>>>> Thanks
>>>>
>>>> On Fri, Feb 13, 2015 at 1:48 PM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> Kubernetes workflow works fine now! :-)
>>>>>
>>>>> There was an issue in the single-cartridge application and fixed.
>>>>> Please take a pull and update data-bridge-config.xml to bind CEP thrift
>>>>> receiver to 0.0.0.0:
>>>>>
>>>>>
>>>>> apache-stratos-4.1.0-SNAPSHOT/repository/conf/data-bridge/data-bridge-config.xml
>>>>> <thriftDataReceiver>
>>>>> <hostName>0.0.0.0</hostName>
>>>>> <port>7611</port>
>>>>> <securePort>7711</securePort>
>>>>> </thriftDataReceiver>
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Fri, Feb 13, 2015 at 1:43 PM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> I just tested Kubernetes workflow with the lastest fixes in PCA and
>>>>>> PCA works fine with an unprotected Git repository.
>>>>>>
>>>>>> Will update the status of the Kubernetes workflow soon. PHP docker
>>>>>> image has been pushed to docker hub with the latest PCA
>>>>>> (stratos/php:4.1.0-beta).
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 11:50 AM, Imesh Gunaratne <im...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks for the pull request Chamila, now git clone works properly
>>>>>>> with GitPython (without user authentication). I have now updated the PHP
>>>>>>> docker image with this and pushing it to docker hub.
>>>>>>>
>>>>>>> In addition to this I fixed few issues in PCA live test:
>>>>>>> - Terminating processes at the end of the test
>>>>>>> - Stoping server sockets used for simulating CEP and server
>>>>>>> - Asserting instance activated event
>>>>>>> - Cleaning temp folder created for running PCA
>>>>>>>
>>>>>>> We will try to fix issue with cloning private repos soon.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> On Fri, Feb 13, 2015 at 10:13 AM, Chamila De Alwis <
>>>>>>> chamilad@wso2.com> wrote:
>>>>>>>
>>>>>>>> Hi Imesh,
>>>>>>>>
>>>>>>>> On Fri, Feb 13, 2015 at 10:09 AM, Imesh Gunaratne <imesh@apache.org
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> According to the error, seems like the checkout path parameter is
>>>>>>>>> not String:
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yes, this seems to be the case, however, when the type of the
>>>>>>>> parameters that were sent to GitPython was printed, all of them were of
>>>>>>>> `str` type. I've tried converting the strings to utf-8 encoding, with no
>>>>>>>> better result. The only issue here might be the encoding of the decrypted
>>>>>>>> password that is included in the credentialled url.
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Chamila de Alwis
>>>>>>>> Software Engineer | WSO2 | +94772207163
>>>>>>>> Blog: code.chamiladealwis.com
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Imesh Gunaratne
>>>>>>>
>>>>>>> Technical Lead, WSO2
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Vice President, Apache Stratos
>>> Director - Cloud Architecture; WSO2 Inc.
>>> Mobile : +94714289692
>>> Blog : http://lakmalsview.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Lakmal Warusawithana
> Vice President, Apache Stratos
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Lakmal Warusawithana <la...@wso2.com>.
HI Imesh,
Thanks, I have tested on EC2 and single-cartridge kubernetes application
deployed and activated successfully.
Few more things need to complete end to end.
- In the case of running docker/kubernetes on top of of EC2/IaaSs we
need to provide Kubernetes host's private and public IPs with the
kubernetes cluster registration to Stratos. Make public IP optional,
because if we deploy Kubernetes in bare metal machine, it may only have
single IP.
- In the case of having public IPs for minions we should show that IP as
default public IP (see attachment)
- Also we need to list all private port list (which export in the
container) and also need to show all public port lists which mapped into
kubernetes service proxy
On Fri, Feb 13, 2015 at 2:35 PM, Imesh Gunaratne <im...@apache.org> wrote:
> No, docker image is stratos/php:4.1.0-beta, source tag for this fix
> is 4.1.0-beta-kubernetes-v1.
>
> I have now made the above Thrift receiver hostname 0.0.0.0 by default to
> make it much easier to use the product.
>
> Thanks
>
> On Fri, Feb 13, 2015 at 1:56 PM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
>> Do you mean docker image with tag 4.1.0-beta-kubernetes-v1 ?
>>
>> On Fri, Feb 13, 2015 at 1:53 PM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> I have created a tag: 4.1.0-beta-kubernetes-v1 to mark this fix.
>>>
>>> Thanks
>>>
>>> On Fri, Feb 13, 2015 at 1:48 PM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> Kubernetes workflow works fine now! :-)
>>>>
>>>> There was an issue in the single-cartridge application and fixed.
>>>> Please take a pull and update data-bridge-config.xml to bind CEP thrift
>>>> receiver to 0.0.0.0:
>>>>
>>>>
>>>> apache-stratos-4.1.0-SNAPSHOT/repository/conf/data-bridge/data-bridge-config.xml
>>>> <thriftDataReceiver>
>>>> <hostName>0.0.0.0</hostName>
>>>> <port>7611</port>
>>>> <securePort>7711</securePort>
>>>> </thriftDataReceiver>
>>>>
>>>> Thanks
>>>>
>>>> On Fri, Feb 13, 2015 at 1:43 PM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> I just tested Kubernetes workflow with the lastest fixes in PCA and
>>>>> PCA works fine with an unprotected Git repository.
>>>>>
>>>>> Will update the status of the Kubernetes workflow soon. PHP docker
>>>>> image has been pushed to docker hub with the latest PCA
>>>>> (stratos/php:4.1.0-beta).
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Fri, Feb 13, 2015 at 11:50 AM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Thanks for the pull request Chamila, now git clone works properly
>>>>>> with GitPython (without user authentication). I have now updated the PHP
>>>>>> docker image with this and pushing it to docker hub.
>>>>>>
>>>>>> In addition to this I fixed few issues in PCA live test:
>>>>>> - Terminating processes at the end of the test
>>>>>> - Stoping server sockets used for simulating CEP and server
>>>>>> - Asserting instance activated event
>>>>>> - Cleaning temp folder created for running PCA
>>>>>>
>>>>>> We will try to fix issue with cloning private repos soon.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 10:13 AM, Chamila De Alwis <chamilad@wso2.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Hi Imesh,
>>>>>>>
>>>>>>> On Fri, Feb 13, 2015 at 10:09 AM, Imesh Gunaratne <im...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> According to the error, seems like the checkout path parameter is
>>>>>>>> not String:
>>>>>>>>
>>>>>>>
>>>>>>> Yes, this seems to be the case, however, when the type of the
>>>>>>> parameters that were sent to GitPython was printed, all of them were of
>>>>>>> `str` type. I've tried converting the strings to utf-8 encoding, with no
>>>>>>> better result. The only issue here might be the encoding of the decrypted
>>>>>>> password that is included in the credentialled url.
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Chamila de Alwis
>>>>>>> Software Engineer | WSO2 | +94772207163
>>>>>>> Blog: code.chamiladealwis.com
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Vice President, Apache Stratos
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Lakmal Warusawithana
Vice President, Apache Stratos
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
No, docker image is stratos/php:4.1.0-beta, source tag for this fix
is 4.1.0-beta-kubernetes-v1.
I have now made the above Thrift receiver hostname 0.0.0.0 by default to
make it much easier to use the product.
Thanks
On Fri, Feb 13, 2015 at 1:56 PM, Lakmal Warusawithana <la...@wso2.com>
wrote:
> Do you mean docker image with tag 4.1.0-beta-kubernetes-v1 ?
>
> On Fri, Feb 13, 2015 at 1:53 PM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> I have created a tag: 4.1.0-beta-kubernetes-v1 to mark this fix.
>>
>> Thanks
>>
>> On Fri, Feb 13, 2015 at 1:48 PM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> Kubernetes workflow works fine now! :-)
>>>
>>> There was an issue in the single-cartridge application and fixed. Please
>>> take a pull and update data-bridge-config.xml to bind CEP thrift receiver
>>> to 0.0.0.0:
>>>
>>>
>>> apache-stratos-4.1.0-SNAPSHOT/repository/conf/data-bridge/data-bridge-config.xml
>>> <thriftDataReceiver>
>>> <hostName>0.0.0.0</hostName>
>>> <port>7611</port>
>>> <securePort>7711</securePort>
>>> </thriftDataReceiver>
>>>
>>> Thanks
>>>
>>> On Fri, Feb 13, 2015 at 1:43 PM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> I just tested Kubernetes workflow with the lastest fixes in PCA and PCA
>>>> works fine with an unprotected Git repository.
>>>>
>>>> Will update the status of the Kubernetes workflow soon. PHP docker
>>>> image has been pushed to docker hub with the latest PCA
>>>> (stratos/php:4.1.0-beta).
>>>>
>>>> Thanks
>>>>
>>>> On Fri, Feb 13, 2015 at 11:50 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> Thanks for the pull request Chamila, now git clone works properly with
>>>>> GitPython (without user authentication). I have now updated the PHP docker
>>>>> image with this and pushing it to docker hub.
>>>>>
>>>>> In addition to this I fixed few issues in PCA live test:
>>>>> - Terminating processes at the end of the test
>>>>> - Stoping server sockets used for simulating CEP and server
>>>>> - Asserting instance activated event
>>>>> - Cleaning temp folder created for running PCA
>>>>>
>>>>> We will try to fix issue with cloning private repos soon.
>>>>>
>>>>> Thanks
>>>>>
>>>>> On Fri, Feb 13, 2015 at 10:13 AM, Chamila De Alwis <ch...@wso2.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Imesh,
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 10:09 AM, Imesh Gunaratne <im...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> According to the error, seems like the checkout path parameter is
>>>>>>> not String:
>>>>>>>
>>>>>>
>>>>>> Yes, this seems to be the case, however, when the type of the
>>>>>> parameters that were sent to GitPython was printed, all of them were of
>>>>>> `str` type. I've tried converting the strings to utf-8 encoding, with no
>>>>>> better result. The only issue here might be the encoding of the decrypted
>>>>>> password that is included in the credentialled url.
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Chamila de Alwis
>>>>>> Software Engineer | WSO2 | +94772207163
>>>>>> Blog: code.chamiladealwis.com
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Lakmal Warusawithana
> Vice President, Apache Stratos
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Lakmal Warusawithana <la...@wso2.com>.
Do you mean docker image with tag 4.1.0-beta-kubernetes-v1 ?
On Fri, Feb 13, 2015 at 1:53 PM, Imesh Gunaratne <im...@apache.org> wrote:
> I have created a tag: 4.1.0-beta-kubernetes-v1 to mark this fix.
>
> Thanks
>
> On Fri, Feb 13, 2015 at 1:48 PM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> Kubernetes workflow works fine now! :-)
>>
>> There was an issue in the single-cartridge application and fixed. Please
>> take a pull and update data-bridge-config.xml to bind CEP thrift receiver
>> to 0.0.0.0:
>>
>>
>> apache-stratos-4.1.0-SNAPSHOT/repository/conf/data-bridge/data-bridge-config.xml
>> <thriftDataReceiver>
>> <hostName>0.0.0.0</hostName>
>> <port>7611</port>
>> <securePort>7711</securePort>
>> </thriftDataReceiver>
>>
>> Thanks
>>
>> On Fri, Feb 13, 2015 at 1:43 PM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> I just tested Kubernetes workflow with the lastest fixes in PCA and PCA
>>> works fine with an unprotected Git repository.
>>>
>>> Will update the status of the Kubernetes workflow soon. PHP docker image
>>> has been pushed to docker hub with the latest PCA (stratos/php:4.1.0-beta).
>>>
>>> Thanks
>>>
>>> On Fri, Feb 13, 2015 at 11:50 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> Thanks for the pull request Chamila, now git clone works properly with
>>>> GitPython (without user authentication). I have now updated the PHP docker
>>>> image with this and pushing it to docker hub.
>>>>
>>>> In addition to this I fixed few issues in PCA live test:
>>>> - Terminating processes at the end of the test
>>>> - Stoping server sockets used for simulating CEP and server
>>>> - Asserting instance activated event
>>>> - Cleaning temp folder created for running PCA
>>>>
>>>> We will try to fix issue with cloning private repos soon.
>>>>
>>>> Thanks
>>>>
>>>> On Fri, Feb 13, 2015 at 10:13 AM, Chamila De Alwis <ch...@wso2.com>
>>>> wrote:
>>>>
>>>>> Hi Imesh,
>>>>>
>>>>> On Fri, Feb 13, 2015 at 10:09 AM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> According to the error, seems like the checkout path parameter is not
>>>>>> String:
>>>>>>
>>>>>
>>>>> Yes, this seems to be the case, however, when the type of the
>>>>> parameters that were sent to GitPython was printed, all of them were of
>>>>> `str` type. I've tried converting the strings to utf-8 encoding, with no
>>>>> better result. The only issue here might be the encoding of the decrypted
>>>>> password that is included in the credentialled url.
>>>>>
>>>>>
>>>>> Regards,
>>>>> Chamila de Alwis
>>>>> Software Engineer | WSO2 | +94772207163
>>>>> Blog: code.chamiladealwis.com
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Lakmal Warusawithana
Vice President, Apache Stratos
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
I have created a tag: 4.1.0-beta-kubernetes-v1 to mark this fix.
Thanks
On Fri, Feb 13, 2015 at 1:48 PM, Imesh Gunaratne <im...@apache.org> wrote:
> Kubernetes workflow works fine now! :-)
>
> There was an issue in the single-cartridge application and fixed. Please
> take a pull and update data-bridge-config.xml to bind CEP thrift receiver
> to 0.0.0.0:
>
>
> apache-stratos-4.1.0-SNAPSHOT/repository/conf/data-bridge/data-bridge-config.xml
> <thriftDataReceiver>
> <hostName>0.0.0.0</hostName>
> <port>7611</port>
> <securePort>7711</securePort>
> </thriftDataReceiver>
>
> Thanks
>
> On Fri, Feb 13, 2015 at 1:43 PM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> I just tested Kubernetes workflow with the lastest fixes in PCA and PCA
>> works fine with an unprotected Git repository.
>>
>> Will update the status of the Kubernetes workflow soon. PHP docker image
>> has been pushed to docker hub with the latest PCA (stratos/php:4.1.0-beta).
>>
>> Thanks
>>
>> On Fri, Feb 13, 2015 at 11:50 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> Thanks for the pull request Chamila, now git clone works properly with
>>> GitPython (without user authentication). I have now updated the PHP docker
>>> image with this and pushing it to docker hub.
>>>
>>> In addition to this I fixed few issues in PCA live test:
>>> - Terminating processes at the end of the test
>>> - Stoping server sockets used for simulating CEP and server
>>> - Asserting instance activated event
>>> - Cleaning temp folder created for running PCA
>>>
>>> We will try to fix issue with cloning private repos soon.
>>>
>>> Thanks
>>>
>>> On Fri, Feb 13, 2015 at 10:13 AM, Chamila De Alwis <ch...@wso2.com>
>>> wrote:
>>>
>>>> Hi Imesh,
>>>>
>>>> On Fri, Feb 13, 2015 at 10:09 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> According to the error, seems like the checkout path parameter is not
>>>>> String:
>>>>>
>>>>
>>>> Yes, this seems to be the case, however, when the type of the
>>>> parameters that were sent to GitPython was printed, all of them were of
>>>> `str` type. I've tried converting the strings to utf-8 encoding, with no
>>>> better result. The only issue here might be the encoding of the decrypted
>>>> password that is included in the credentialled url.
>>>>
>>>>
>>>> Regards,
>>>> Chamila de Alwis
>>>> Software Engineer | WSO2 | +94772207163
>>>> Blog: code.chamiladealwis.com
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
Kubernetes workflow works fine now! :-)
There was an issue in the single-cartridge application and fixed. Please
take a pull and update data-bridge-config.xml to bind CEP thrift receiver
to 0.0.0.0:
apache-stratos-4.1.0-SNAPSHOT/repository/conf/data-bridge/data-bridge-config.xml
<thriftDataReceiver>
<hostName>0.0.0.0</hostName>
<port>7611</port>
<securePort>7711</securePort>
</thriftDataReceiver>
Thanks
On Fri, Feb 13, 2015 at 1:43 PM, Imesh Gunaratne <im...@apache.org> wrote:
> I just tested Kubernetes workflow with the lastest fixes in PCA and PCA
> works fine with an unprotected Git repository.
>
> Will update the status of the Kubernetes workflow soon. PHP docker image
> has been pushed to docker hub with the latest PCA (stratos/php:4.1.0-beta).
>
> Thanks
>
> On Fri, Feb 13, 2015 at 11:50 AM, Imesh Gunaratne <im...@apache.org>
> wrote:
>
>> Thanks for the pull request Chamila, now git clone works properly with
>> GitPython (without user authentication). I have now updated the PHP docker
>> image with this and pushing it to docker hub.
>>
>> In addition to this I fixed few issues in PCA live test:
>> - Terminating processes at the end of the test
>> - Stoping server sockets used for simulating CEP and server
>> - Asserting instance activated event
>> - Cleaning temp folder created for running PCA
>>
>> We will try to fix issue with cloning private repos soon.
>>
>> Thanks
>>
>> On Fri, Feb 13, 2015 at 10:13 AM, Chamila De Alwis <ch...@wso2.com>
>> wrote:
>>
>>> Hi Imesh,
>>>
>>> On Fri, Feb 13, 2015 at 10:09 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> According to the error, seems like the checkout path parameter is not
>>>> String:
>>>>
>>>
>>> Yes, this seems to be the case, however, when the type of the parameters
>>> that were sent to GitPython was printed, all of them were of `str` type.
>>> I've tried converting the strings to utf-8 encoding, with no better result.
>>> The only issue here might be the encoding of the decrypted password that is
>>> included in the credentialled url.
>>>
>>>
>>> Regards,
>>> Chamila de Alwis
>>> Software Engineer | WSO2 | +94772207163
>>> Blog: code.chamiladealwis.com
>>>
>>>
>>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
I just tested Kubernetes workflow with the lastest fixes in PCA and PCA
works fine with an unprotected Git repository.
Will update the status of the Kubernetes workflow soon. PHP docker image
has been pushed to docker hub with the latest PCA (stratos/php:4.1.0-beta).
Thanks
On Fri, Feb 13, 2015 at 11:50 AM, Imesh Gunaratne <im...@apache.org> wrote:
> Thanks for the pull request Chamila, now git clone works properly with
> GitPython (without user authentication). I have now updated the PHP docker
> image with this and pushing it to docker hub.
>
> In addition to this I fixed few issues in PCA live test:
> - Terminating processes at the end of the test
> - Stoping server sockets used for simulating CEP and server
> - Asserting instance activated event
> - Cleaning temp folder created for running PCA
>
> We will try to fix issue with cloning private repos soon.
>
> Thanks
>
> On Fri, Feb 13, 2015 at 10:13 AM, Chamila De Alwis <ch...@wso2.com>
> wrote:
>
>> Hi Imesh,
>>
>> On Fri, Feb 13, 2015 at 10:09 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> According to the error, seems like the checkout path parameter is not
>>> String:
>>>
>>
>> Yes, this seems to be the case, however, when the type of the parameters
>> that were sent to GitPython was printed, all of them were of `str` type.
>> I've tried converting the strings to utf-8 encoding, with no better result.
>> The only issue here might be the encoding of the decrypted password that is
>> included in the credentialled url.
>>
>>
>> Regards,
>> Chamila de Alwis
>> Software Engineer | WSO2 | +94772207163
>> Blog: code.chamiladealwis.com
>>
>>
>>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
Thanks for the pull request Chamila, now git clone works properly with
GitPython (without user authentication). I have now updated the PHP docker
image with this and pushing it to docker hub.
In addition to this I fixed few issues in PCA live test:
- Terminating processes at the end of the test
- Stoping server sockets used for simulating CEP and server
- Asserting instance activated event
- Cleaning temp folder created for running PCA
We will try to fix issue with cloning private repos soon.
Thanks
On Fri, Feb 13, 2015 at 10:13 AM, Chamila De Alwis <ch...@wso2.com>
wrote:
> Hi Imesh,
>
> On Fri, Feb 13, 2015 at 10:09 AM, Imesh Gunaratne <im...@apache.org>
> wrote:
>
>> According to the error, seems like the checkout path parameter is not
>> String:
>>
>
> Yes, this seems to be the case, however, when the type of the parameters
> that were sent to GitPython was printed, all of them were of `str` type.
> I've tried converting the strings to utf-8 encoding, with no better result.
> The only issue here might be the encoding of the decrypted password that is
> included in the credentialled url.
>
>
> Regards,
> Chamila de Alwis
> Software Engineer | WSO2 | +94772207163
> Blog: code.chamiladealwis.com
>
>
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Chamila De Alwis <ch...@wso2.com>.
Hi Imesh,
On Fri, Feb 13, 2015 at 10:09 AM, Imesh Gunaratne <im...@apache.org> wrote:
> According to the error, seems like the checkout path parameter is not
> String:
>
Yes, this seems to be the case, however, when the type of the parameters
that were sent to GitPython was printed, all of them were of `str` type.
I've tried converting the strings to utf-8 encoding, with no better result.
The only issue here might be the encoding of the decrypted password that is
included in the credentialled url.
Regards,
Chamila de Alwis
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
According to the error, seems like the checkout path parameter is not
String:
TypeError: execve() arg 2 must contain only strings
On Fri, Feb 13, 2015 at 10:08 AM, Imesh Gunaratne <im...@apache.org> wrote:
> Hi Chamila,
>
> Great stuff, please send a PR. We will investigate this. For me git clone
> works fine with authentication.
>
> Thanks
>
> On Fri, Feb 13, 2015 at 10:01 AM, Chamila De Alwis <ch...@wso2.com>
> wrote:
>
>> Hi Imesh,
>>
>> I have almost completed the work to switch to gitpython. However there is
>> an issue with the type of the auth url, that is given to GitPython. Other
>> than that, public repo cloning now works with the library.
>>
>> When a url with credentials is passed, I'm getting the following error.
>> Investigating further.
>>
>> [2015-02-13 09:59:13,200] ERROR {eventsubscriber.py:run} - Error
>> processing 'ArtifactUpdatedEvent' event
>> Traceback (most recent call last):
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/subscriber/eventsubscriber.py",
>> line 103, in run
>> handler(event_msg)
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/test-classes/../10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/agent.py",
>> line 185, in on_artifact_updated
>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/extensions/extensionhandler.py",
>> line 122, in on_artifact_updated_event
>> subscribe_run, updated =
>> agentgithandler.AgentGitHandler.checkout(repo_info)
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
>> line 89, in checkout
>> git_repo = AgentGitHandler.clone(git_repo)
>> File
>> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
>> line 192, in clone
>> Repo.clone_from(unicode(git_repo.repo_url, "utf-8"),
>> unicode(git_repo.local_repo_path, "utf-8"))
>> File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line
>> 844, in clone_from
>> return cls._clone(Git(os.getcwd()), url, to_path, GitCmdObjectDB,
>> progress, **kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line
>> 791, in _clone
>> v=True, **add_progress(kwargs, git, progress))
>> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 431, in
>> <lambda>
>> return lambda *args, **kwargs: self._call_process(name, *args,
>> **kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 802, in
>> _call_process
>> return self.execute(make_call(), **_kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 556, in
>> execute
>> **subprocess_kwargs
>> File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
>> errread, errwrite)
>> File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
>> raise child_exception
>> TypeError: execve() arg 2 must contain only strings
>>
>>
>> Regards,
>> Chamila de Alwis
>> Software Engineer | WSO2 | +94772207163
>> Blog: code.chamiladealwis.com
>>
>>
>>
>> On Fri, Feb 13, 2015 at 9:30 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> I analyzed the issue the NoneType error has occurred due to the EOF
>>> exception which has been raised while executing git clone shell command:
>>>
>>> The problem is in the following line:
>>>
>>> git_repo = AgentGitHandler.clone(git_repo)
>>> AgentGitHandler.log.debug("Git clone executed: [tenant-id] %s [repo-url] %s",
>>> git_repo.tenant_id, git_repo.repo_url)
>>>
>>> Here clone() method returns NoneType if clone fails. Anyway the root cause of the problem is with the Git client implementation. Will switch to GitPython to avoid this intermittent issue of EOF.
>>>
>>> Thanks
>>>
>>>
>>> On Fri, Feb 13, 2015 at 9:20 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> I can see the tenant id attribute in received message, may be it was
>>>> not properly set:
>>>>
>>>> [2015-02-13 03:44:06,394] DEBUG {eventsubscriber.py:on_message} -
>>>> Message received: instance/notifier/ArtifactUpdatedEvent:
>>>>
>>>> {"clusterId":"php1.php.domain","repoUserName":"","repoPassword":"","repoURL":"
>>>> https://github.com/imesh/stratos-php-applications.git
>>>> ","tenantId":"-1234","commitEnabled":false}
>>>>
>>>> On Fri, Feb 13, 2015 at 9:19 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> Also the following error was there in the log:
>>>>>
>>>>> [2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception
>>>>> while executing git clone command on git binary : EOF("End Of File (EOF).
>>>>> Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion:
>>>>> 3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', '
>>>>> https://github.com/imesh/stratos-php-applications.git',
>>>>> '/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer
>>>>> (last 100 chars): ''\nbefore (last 100 chars): 'ng deltas: 98% (71/72)
>>>>> \\rResolving deltas: 100% (72/72) \\rResolving deltas: 100% (72/72),
>>>>> done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index:
>>>>> None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed:
>>>>> False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile:
>>>>> None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase:
>>>>> False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose:
>>>>> 0.1\ndelayafterterminate: 0.1",)
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>> line 200, in clone
>>>>> result = clone_op.expect(["Username for .*", "Password for .*",
>>>>> "Checking connectivity... done."])
>>>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>>>> line 1451, in expect
>>>>> timeout, searchwindowsize)
>>>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>>>> line 1466, in expect_list
>>>>> timeout, searchwindowsize)
>>>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>>>> line 1554, in expect_loop
>>>>> raise EOF(str(err) + '\n' + str(self))
>>>>> EOF: End Of File (EOF). Exception style platform.
>>>>> <pexpect.spawn object at 0x25c17d0>
>>>>>
>>>>> On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> The problem was in populate-user-data.sh file [1], APPLICATION_PATH
>>>>>> has been removed from it. I just fixed it, pushed the new PHP docker image
>>>>>> to docker hub and did another flow test and found the below error:
>>>>>>
>>>>>> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error
>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>> Traceback (most recent call last):
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>> line 103, in run
>>>>>> handler(event_msg)
>>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>>
>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>> line 122, in on_artifact_updated_event
>>>>>> subscribe_run, updated =
>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>> line 91, in checkout
>>>>>> git_repo.tenant_id, git_repo.repo_url)
>>>>>> AttributeError: 'NoneType' object has no attribute 'tenant_id'
>>>>>>
>>>>>> [1]
>>>>>> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <im...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks Lakmal, yes as I found basic features we need works with Git
>>>>>>> python. Will do that.
>>>>>>> Regarding the above error, seems like it has occurred due to the
>>>>>>> APPLICATION_PATH not being sent in payload, there was no None check or a
>>>>>>> log saying its not found:
>>>>>>>
>>>>>>> def clone(git_repo):
>>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>>> # delete and recreate local repo path if exists
>>>>>>> GitUtils.delete_folder_tree(git_repo.local_repo_path)
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <
>>>>>>> lakmal@wso2.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> As it looks like this error has been caused by the agent git
>>>>>>>>> handler while executing git clone, IMO its better if we can switch to
>>>>>>>>> GitPython.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> +1, If GitPython has full functionality what we expect, lets move
>>>>>>>> to it. I was under impression that it has some limitation. If so please not
>>>>>>>> waist time to fix current one.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <imesh@apache.org
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> Hi Devs,
>>>>>>>>>>
>>>>>>>>>> Yesterday we fixed several issues in PCA, now I'm seeing the
>>>>>>>>>> below error when trying to receive artifact updated event:
>>>>>>>>>>
>>>>>>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>>>>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File
>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>>>> line 103, in run
>>>>>>>>>> handler(event_msg)
>>>>>>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>>>>>>
>>>>>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>>>>>> File
>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>>>>>> line 122, in on_artifact_updated_event
>>>>>>>>>> subscribe_run, updated =
>>>>>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>>>>>> File
>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>>>> line 89, in checkout
>>>>>>>>>> git_repo = AgentGitHandler.clone(git_repo)
>>>>>>>>>> File
>>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>>>> line 190, in clone
>>>>>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>>>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>>>>>>>> st = os.stat(s)
>>>>>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType
>>>>>>>>>> foun
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Imesh Gunaratne
>>>>>>>>>>
>>>>>>>>>> Technical Lead, WSO2
>>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Imesh Gunaratne
>>>>>>>>>
>>>>>>>>> Technical Lead, WSO2
>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Lakmal Warusawithana
>>>>>>>> Vice President, Apache Stratos
>>>>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>>>>> Mobile : +94714289692
>>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Imesh Gunaratne
>>>>>>>
>>>>>>> Technical Lead, WSO2
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
Hi Chamila,
Great stuff, please send a PR. We will investigate this. For me git clone
works fine with authentication.
Thanks
On Fri, Feb 13, 2015 at 10:01 AM, Chamila De Alwis <ch...@wso2.com>
wrote:
> Hi Imesh,
>
> I have almost completed the work to switch to gitpython. However there is
> an issue with the type of the auth url, that is given to GitPython. Other
> than that, public repo cloning now works with the library.
>
> When a url with credentials is passed, I'm getting the following error.
> Investigating further.
>
> [2015-02-13 09:59:13,200] ERROR {eventsubscriber.py:run} - Error
> processing 'ArtifactUpdatedEvent' event
> Traceback (most recent call last):
> File
> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/subscriber/eventsubscriber.py",
> line 103, in run
> handler(event_msg)
> File
> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/test-classes/../10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/agent.py",
> line 185, in on_artifact_updated
> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
> File
> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/extensions/extensionhandler.py",
> line 122, in on_artifact_updated_event
> subscribe_run, updated =
> agentgithandler.AgentGitHandler.checkout(repo_info)
> File
> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
> line 89, in checkout
> git_repo = AgentGitHandler.clone(git_repo)
> File
> "/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
> line 192, in clone
> Repo.clone_from(unicode(git_repo.repo_url, "utf-8"),
> unicode(git_repo.local_repo_path, "utf-8"))
> File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line
> 844, in clone_from
> return cls._clone(Git(os.getcwd()), url, to_path, GitCmdObjectDB,
> progress, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line
> 791, in _clone
> v=True, **add_progress(kwargs, git, progress))
> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 431, in
> <lambda>
> return lambda *args, **kwargs: self._call_process(name, *args,
> **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 802, in
> _call_process
> return self.execute(make_call(), **_kwargs)
> File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 556, in
> execute
> **subprocess_kwargs
> File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
> errread, errwrite)
> File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
> raise child_exception
> TypeError: execve() arg 2 must contain only strings
>
>
> Regards,
> Chamila de Alwis
> Software Engineer | WSO2 | +94772207163
> Blog: code.chamiladealwis.com
>
>
>
> On Fri, Feb 13, 2015 at 9:30 AM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> I analyzed the issue the NoneType error has occurred due to the EOF
>> exception which has been raised while executing git clone shell command:
>>
>> The problem is in the following line:
>>
>> git_repo = AgentGitHandler.clone(git_repo)
>> AgentGitHandler.log.debug("Git clone executed: [tenant-id] %s [repo-url] %s",
>> git_repo.tenant_id, git_repo.repo_url)
>>
>> Here clone() method returns NoneType if clone fails. Anyway the root cause of the problem is with the Git client implementation. Will switch to GitPython to avoid this intermittent issue of EOF.
>>
>> Thanks
>>
>>
>> On Fri, Feb 13, 2015 at 9:20 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> I can see the tenant id attribute in received message, may be it was not
>>> properly set:
>>>
>>> [2015-02-13 03:44:06,394] DEBUG {eventsubscriber.py:on_message} -
>>> Message received: instance/notifier/ArtifactUpdatedEvent:
>>>
>>> {"clusterId":"php1.php.domain","repoUserName":"","repoPassword":"","repoURL":"
>>> https://github.com/imesh/stratos-php-applications.git
>>> ","tenantId":"-1234","commitEnabled":false}
>>>
>>> On Fri, Feb 13, 2015 at 9:19 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> Also the following error was there in the log:
>>>>
>>>> [2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception
>>>> while executing git clone command on git binary : EOF("End Of File (EOF).
>>>> Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion:
>>>> 3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', '
>>>> https://github.com/imesh/stratos-php-applications.git',
>>>> '/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer
>>>> (last 100 chars): ''\nbefore (last 100 chars): 'ng deltas: 98% (71/72)
>>>> \\rResolving deltas: 100% (72/72) \\rResolving deltas: 100% (72/72),
>>>> done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index:
>>>> None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed:
>>>> False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile:
>>>> None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase:
>>>> False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose:
>>>> 0.1\ndelayafterterminate: 0.1",)
>>>> Traceback (most recent call last):
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>> line 200, in clone
>>>> result = clone_op.expect(["Username for .*", "Password for .*",
>>>> "Checking connectivity... done."])
>>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>>> line 1451, in expect
>>>> timeout, searchwindowsize)
>>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>>> line 1466, in expect_list
>>>> timeout, searchwindowsize)
>>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>>> line 1554, in expect_loop
>>>> raise EOF(str(err) + '\n' + str(self))
>>>> EOF: End Of File (EOF). Exception style platform.
>>>> <pexpect.spawn object at 0x25c17d0>
>>>>
>>>> On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> The problem was in populate-user-data.sh file [1], APPLICATION_PATH
>>>>> has been removed from it. I just fixed it, pushed the new PHP docker image
>>>>> to docker hub and did another flow test and found the below error:
>>>>>
>>>>> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error
>>>>> processing 'ArtifactUpdatedEvent' event
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>> line 103, in run
>>>>> handler(event_msg)
>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>
>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>> line 122, in on_artifact_updated_event
>>>>> subscribe_run, updated =
>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>> line 91, in checkout
>>>>> git_repo.tenant_id, git_repo.repo_url)
>>>>> AttributeError: 'NoneType' object has no attribute 'tenant_id'
>>>>>
>>>>> [1]
>>>>> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
>>>>>
>>>>> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Thanks Lakmal, yes as I found basic features we need works with Git
>>>>>> python. Will do that.
>>>>>> Regarding the above error, seems like it has occurred due to the
>>>>>> APPLICATION_PATH not being sent in payload, there was no None check or a
>>>>>> log saying its not found:
>>>>>>
>>>>>> def clone(git_repo):
>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>> # delete and recreate local repo path if exists
>>>>>> GitUtils.delete_folder_tree(git_repo.local_repo_path)
>>>>>>
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <
>>>>>> lakmal@wso2.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> As it looks like this error has been caused by the agent git
>>>>>>>> handler while executing git clone, IMO its better if we can switch to
>>>>>>>> GitPython.
>>>>>>>>
>>>>>>>>
>>>>>>> +1, If GitPython has full functionality what we expect, lets move to
>>>>>>> it. I was under impression that it has some limitation. If so please not
>>>>>>> waist time to fix current one.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <im...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Devs,
>>>>>>>>>
>>>>>>>>> Yesterday we fixed several issues in PCA, now I'm seeing the below
>>>>>>>>> error when trying to receive artifact updated event:
>>>>>>>>>
>>>>>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>>>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>>> line 103, in run
>>>>>>>>> handler(event_msg)
>>>>>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>>>>>
>>>>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>>>>> line 122, in on_artifact_updated_event
>>>>>>>>> subscribe_run, updated =
>>>>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>>> line 89, in checkout
>>>>>>>>> git_repo = AgentGitHandler.clone(git_repo)
>>>>>>>>> File
>>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>>> line 190, in clone
>>>>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>>>>>>> st = os.stat(s)
>>>>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType
>>>>>>>>> foun
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Imesh Gunaratne
>>>>>>>>>
>>>>>>>>> Technical Lead, WSO2
>>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Lakmal Warusawithana
>>>>>>> Vice President, Apache Stratos
>>>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>>>> Mobile : +94714289692
>>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Chamila De Alwis <ch...@wso2.com>.
Hi Imesh,
I have almost completed the work to switch to gitpython. However there is
an issue with the type of the auth url, that is given to GitPython. Other
than that, public repo cloning now works with the library.
When a url with credentials is passed, I'm getting the following error.
Investigating further.
[2015-02-13 09:59:13,200] ERROR {eventsubscriber.py:run} - Error processing
'ArtifactUpdatedEvent' event
Traceback (most recent call last):
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/subscriber/eventsubscriber.py",
line 103, in run
handler(event_msg)
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/test-classes/../10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/agent.py",
line 185, in on_artifact_updated
CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/extensions/extensionhandler.py",
line 122, in on_artifact_updated_event
subscribe_run, updated =
agentgithandler.AgentGitHandler.checkout(repo_info)
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
line 89, in checkout
git_repo = AgentGitHandler.clone(git_repo)
File
"/home/chamilad/dev/stratos/components/org.apache.stratos.python.cartridge.agent/target/10c657f1-a3a4-4aee-bfe3-949fda7346a9/cartridge.agent/modules/artifactmgt/git/agentgithandler.py",
line 192, in clone
Repo.clone_from(unicode(git_repo.repo_url, "utf-8"),
unicode(git_repo.local_repo_path, "utf-8"))
File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line 844,
in clone_from
return cls._clone(Git(os.getcwd()), url, to_path, GitCmdObjectDB,
progress, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/git/repo/base.py", line 791,
in _clone
v=True, **add_progress(kwargs, git, progress))
File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 431, in
<lambda>
return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 802, in
_call_process
return self.execute(make_call(), **_kwargs)
File "/usr/local/lib/python2.7/dist-packages/git/cmd.py", line 556, in
execute
**subprocess_kwargs
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
TypeError: execve() arg 2 must contain only strings
Regards,
Chamila de Alwis
Software Engineer | WSO2 | +94772207163
Blog: code.chamiladealwis.com
On Fri, Feb 13, 2015 at 9:30 AM, Imesh Gunaratne <im...@apache.org> wrote:
> I analyzed the issue the NoneType error has occurred due to the EOF
> exception which has been raised while executing git clone shell command:
>
> The problem is in the following line:
>
> git_repo = AgentGitHandler.clone(git_repo)
> AgentGitHandler.log.debug("Git clone executed: [tenant-id] %s [repo-url] %s",
> git_repo.tenant_id, git_repo.repo_url)
>
> Here clone() method returns NoneType if clone fails. Anyway the root cause of the problem is with the Git client implementation. Will switch to GitPython to avoid this intermittent issue of EOF.
>
> Thanks
>
>
> On Fri, Feb 13, 2015 at 9:20 AM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> I can see the tenant id attribute in received message, may be it was not
>> properly set:
>>
>> [2015-02-13 03:44:06,394] DEBUG {eventsubscriber.py:on_message} - Message
>> received: instance/notifier/ArtifactUpdatedEvent:
>>
>> {"clusterId":"php1.php.domain","repoUserName":"","repoPassword":"","repoURL":"
>> https://github.com/imesh/stratos-php-applications.git
>> ","tenantId":"-1234","commitEnabled":false}
>>
>> On Fri, Feb 13, 2015 at 9:19 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> Also the following error was there in the log:
>>>
>>> [2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception
>>> while executing git clone command on git binary : EOF("End Of File (EOF).
>>> Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion:
>>> 3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', '
>>> https://github.com/imesh/stratos-php-applications.git',
>>> '/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer
>>> (last 100 chars): ''\nbefore (last 100 chars): 'ng deltas: 98% (71/72)
>>> \\rResolving deltas: 100% (72/72) \\rResolving deltas: 100% (72/72),
>>> done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index:
>>> None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed:
>>> False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile:
>>> None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase:
>>> False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose:
>>> 0.1\ndelayafterterminate: 0.1",)
>>> Traceback (most recent call last):
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>> line 200, in clone
>>> result = clone_op.expect(["Username for .*", "Password for .*",
>>> "Checking connectivity... done."])
>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>> line 1451, in expect
>>> timeout, searchwindowsize)
>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>> line 1466, in expect_list
>>> timeout, searchwindowsize)
>>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py",
>>> line 1554, in expect_loop
>>> raise EOF(str(err) + '\n' + str(self))
>>> EOF: End Of File (EOF). Exception style platform.
>>> <pexpect.spawn object at 0x25c17d0>
>>>
>>> On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> The problem was in populate-user-data.sh file [1], APPLICATION_PATH has
>>>> been removed from it. I just fixed it, pushed the new PHP docker image to
>>>> docker hub and did another flow test and found the below error:
>>>>
>>>> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error
>>>> processing 'ArtifactUpdatedEvent' event
>>>> Traceback (most recent call last):
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>> line 103, in run
>>>> handler(event_msg)
>>>> File "agent.py", line 185, in on_artifact_updated
>>>>
>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>> line 122, in on_artifact_updated_event
>>>> subscribe_run, updated =
>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>> line 91, in checkout
>>>> git_repo.tenant_id, git_repo.repo_url)
>>>> AttributeError: 'NoneType' object has no attribute 'tenant_id'
>>>>
>>>> [1]
>>>> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
>>>>
>>>> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> Thanks Lakmal, yes as I found basic features we need works with Git
>>>>> python. Will do that.
>>>>> Regarding the above error, seems like it has occurred due to the
>>>>> APPLICATION_PATH not being sent in payload, there was no None check or a
>>>>> log saying its not found:
>>>>>
>>>>> def clone(git_repo):
>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>> # delete and recreate local repo path if exists
>>>>> GitUtils.delete_folder_tree(git_repo.local_repo_path)
>>>>>
>>>>>
>>>>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <lakmal@wso2.com
>>>>> > wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> As it looks like this error has been caused by the agent git handler
>>>>>>> while executing git clone, IMO its better if we can switch to GitPython.
>>>>>>>
>>>>>>>
>>>>>> +1, If GitPython has full functionality what we expect, lets move to
>>>>>> it. I was under impression that it has some limitation. If so please not
>>>>>> waist time to fix current one.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <im...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Devs,
>>>>>>>>
>>>>>>>> Yesterday we fixed several issues in PCA, now I'm seeing the below
>>>>>>>> error when trying to receive artifact updated event:
>>>>>>>>
>>>>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>>>> Traceback (most recent call last):
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>>> line 103, in run
>>>>>>>> handler(event_msg)
>>>>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>>>>
>>>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>>>> line 122, in on_artifact_updated_event
>>>>>>>> subscribe_run, updated =
>>>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>> line 89, in checkout
>>>>>>>> git_repo = AgentGitHandler.clone(git_repo)
>>>>>>>> File
>>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>>> line 190, in clone
>>>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>>>>>> st = os.stat(s)
>>>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType foun
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Imesh Gunaratne
>>>>>>>>
>>>>>>>> Technical Lead, WSO2
>>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Imesh Gunaratne
>>>>>>>
>>>>>>> Technical Lead, WSO2
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Lakmal Warusawithana
>>>>>> Vice President, Apache Stratos
>>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>>> Mobile : +94714289692
>>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
I analyzed the issue the NoneType error has occurred due to the EOF
exception which has been raised while executing git clone shell command:
The problem is in the following line:
git_repo = AgentGitHandler.clone(git_repo)
AgentGitHandler.log.debug("Git clone executed: [tenant-id] %s [repo-url] %s",
git_repo.tenant_id, git_repo.repo_url)
Here clone() method returns NoneType if clone fails. Anyway the root
cause of the problem is with the Git client implementation. Will
switch to GitPython to avoid this intermittent issue of EOF.
Thanks
On Fri, Feb 13, 2015 at 9:20 AM, Imesh Gunaratne <im...@apache.org> wrote:
> I can see the tenant id attribute in received message, may be it was not
> properly set:
>
> [2015-02-13 03:44:06,394] DEBUG {eventsubscriber.py:on_message} - Message
> received: instance/notifier/ArtifactUpdatedEvent:
>
> {"clusterId":"php1.php.domain","repoUserName":"","repoPassword":"","repoURL":"
> https://github.com/imesh/stratos-php-applications.git
> ","tenantId":"-1234","commitEnabled":false}
>
> On Fri, Feb 13, 2015 at 9:19 AM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> Also the following error was there in the log:
>>
>> [2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception
>> while executing git clone command on git binary : EOF("End Of File (EOF).
>> Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion:
>> 3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', '
>> https://github.com/imesh/stratos-php-applications.git',
>> '/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer
>> (last 100 chars): ''\nbefore (last 100 chars): 'ng deltas: 98% (71/72)
>> \\rResolving deltas: 100% (72/72) \\rResolving deltas: 100% (72/72),
>> done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index:
>> None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed:
>> False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile:
>> None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase:
>> False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose:
>> 0.1\ndelayafterterminate: 0.1",)
>> Traceback (most recent call last):
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>> line 200, in clone
>> result = clone_op.expect(["Username for .*", "Password for .*",
>> "Checking connectivity... done."])
>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", line
>> 1451, in expect
>> timeout, searchwindowsize)
>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", line
>> 1466, in expect_list
>> timeout, searchwindowsize)
>> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", line
>> 1554, in expect_loop
>> raise EOF(str(err) + '\n' + str(self))
>> EOF: End Of File (EOF). Exception style platform.
>> <pexpect.spawn object at 0x25c17d0>
>>
>> On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> The problem was in populate-user-data.sh file [1], APPLICATION_PATH has
>>> been removed from it. I just fixed it, pushed the new PHP docker image to
>>> docker hub and did another flow test and found the below error:
>>>
>>> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error
>>> processing 'ArtifactUpdatedEvent' event
>>> Traceback (most recent call last):
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>> line 103, in run
>>> handler(event_msg)
>>> File "agent.py", line 185, in on_artifact_updated
>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>> line 122, in on_artifact_updated_event
>>> subscribe_run, updated =
>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>> line 91, in checkout
>>> git_repo.tenant_id, git_repo.repo_url)
>>> AttributeError: 'NoneType' object has no attribute 'tenant_id'
>>>
>>> [1]
>>> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
>>>
>>> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> Thanks Lakmal, yes as I found basic features we need works with Git
>>>> python. Will do that.
>>>> Regarding the above error, seems like it has occurred due to the
>>>> APPLICATION_PATH not being sent in payload, there was no None check or a
>>>> log saying its not found:
>>>>
>>>> def clone(git_repo):
>>>> if os.path.isdir(git_repo.local_repo_path):
>>>> # delete and recreate local repo path if exists
>>>> GitUtils.delete_folder_tree(git_repo.local_repo_path)
>>>>
>>>>
>>>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <la...@wso2.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> As it looks like this error has been caused by the agent git handler
>>>>>> while executing git clone, IMO its better if we can switch to GitPython.
>>>>>>
>>>>>>
>>>>> +1, If GitPython has full functionality what we expect, lets move to
>>>>> it. I was under impression that it has some limitation. If so please not
>>>>> waist time to fix current one.
>>>>>
>>>>>
>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <im...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Devs,
>>>>>>>
>>>>>>> Yesterday we fixed several issues in PCA, now I'm seeing the below
>>>>>>> error when trying to receive artifact updated event:
>>>>>>>
>>>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>>> Traceback (most recent call last):
>>>>>>> File
>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>>> line 103, in run
>>>>>>> handler(event_msg)
>>>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>>>
>>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>>> File
>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>>> line 122, in on_artifact_updated_event
>>>>>>> subscribe_run, updated =
>>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>>> File
>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>> line 89, in checkout
>>>>>>> git_repo = AgentGitHandler.clone(git_repo)
>>>>>>> File
>>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>>> line 190, in clone
>>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>>>>> st = os.stat(s)
>>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType foun
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Imesh Gunaratne
>>>>>>>
>>>>>>> Technical Lead, WSO2
>>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lakmal Warusawithana
>>>>> Vice President, Apache Stratos
>>>>> Director - Cloud Architecture; WSO2 Inc.
>>>>> Mobile : +94714289692
>>>>> Blog : http://lakmalsview.blogspot.com/
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
I can see the tenant id attribute in received message, may be it was not
properly set:
[2015-02-13 03:44:06,394] DEBUG {eventsubscriber.py:on_message} - Message
received: instance/notifier/ArtifactUpdatedEvent:
{"clusterId":"php1.php.domain","repoUserName":"","repoPassword":"","repoURL":"
https://github.com/imesh/stratos-php-applications.git
","tenantId":"-1234","commitEnabled":false}
On Fri, Feb 13, 2015 at 9:19 AM, Imesh Gunaratne <im...@apache.org> wrote:
> Also the following error was there in the log:
>
> [2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception
> while executing git clone command on git binary : EOF("End Of File (EOF).
> Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion:
> 3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', '
> https://github.com/imesh/stratos-php-applications.git',
> '/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer
> (last 100 chars): ''\nbefore (last 100 chars): 'ng deltas: 98% (71/72)
> \\rResolving deltas: 100% (72/72) \\rResolving deltas: 100% (72/72),
> done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index:
> None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed:
> False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile:
> None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase:
> False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose:
> 0.1\ndelayafterterminate: 0.1",)
> Traceback (most recent call last):
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
> line 200, in clone
> result = clone_op.expect(["Username for .*", "Password for .*",
> "Checking connectivity... done."])
> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", line
> 1451, in expect
> timeout, searchwindowsize)
> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", line
> 1466, in expect_list
> timeout, searchwindowsize)
> File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", line
> 1554, in expect_loop
> raise EOF(str(err) + '\n' + str(self))
> EOF: End Of File (EOF). Exception style platform.
> <pexpect.spawn object at 0x25c17d0>
>
> On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> The problem was in populate-user-data.sh file [1], APPLICATION_PATH has
>> been removed from it. I just fixed it, pushed the new PHP docker image to
>> docker hub and did another flow test and found the below error:
>>
>> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error
>> processing 'ArtifactUpdatedEvent' event
>> Traceback (most recent call last):
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>> line 103, in run
>> handler(event_msg)
>> File "agent.py", line 185, in on_artifact_updated
>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>> line 122, in on_artifact_updated_event
>> subscribe_run, updated =
>> agentgithandler.AgentGitHandler.checkout(repo_info)
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>> line 91, in checkout
>> git_repo.tenant_id, git_repo.repo_url)
>> AttributeError: 'NoneType' object has no attribute 'tenant_id'
>>
>> [1]
>> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
>>
>> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> Thanks Lakmal, yes as I found basic features we need works with Git
>>> python. Will do that.
>>> Regarding the above error, seems like it has occurred due to the
>>> APPLICATION_PATH not being sent in payload, there was no None check or a
>>> log saying its not found:
>>>
>>> def clone(git_repo):
>>> if os.path.isdir(git_repo.local_repo_path):
>>> # delete and recreate local repo path if exists
>>> GitUtils.delete_folder_tree(git_repo.local_repo_path)
>>>
>>>
>>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <la...@wso2.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> As it looks like this error has been caused by the agent git handler
>>>>> while executing git clone, IMO its better if we can switch to GitPython.
>>>>>
>>>>>
>>>> +1, If GitPython has full functionality what we expect, lets move to
>>>> it. I was under impression that it has some limitation. If so please not
>>>> waist time to fix current one.
>>>>
>>>>
>>>>
>>>>> Thanks
>>>>>
>>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <im...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Devs,
>>>>>>
>>>>>> Yesterday we fixed several issues in PCA, now I'm seeing the below
>>>>>> error when trying to receive artifact updated event:
>>>>>>
>>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>>>>> processing 'ArtifactUpdatedEvent' event
>>>>>> Traceback (most recent call last):
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>>> line 103, in run
>>>>>> handler(event_msg)
>>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>>
>>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>>> line 122, in on_artifact_updated_event
>>>>>> subscribe_run, updated =
>>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>> line 89, in checkout
>>>>>> git_repo = AgentGitHandler.clone(git_repo)
>>>>>> File
>>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>>> line 190, in clone
>>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>>>> st = os.stat(s)
>>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType foun
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Imesh Gunaratne
>>>>>>
>>>>>> Technical Lead, WSO2
>>>>>> Committer & PMC Member, Apache Stratos
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Lakmal Warusawithana
>>>> Vice President, Apache Stratos
>>>> Director - Cloud Architecture; WSO2 Inc.
>>>> Mobile : +94714289692
>>>> Blog : http://lakmalsview.blogspot.com/
>>>>
>>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
Also the following error was there in the log:
[2015-02-13 03:44:28,498] ERROR {agentgithandler.py:clone} - Exception
while executing git clone command on git binary : EOF("End Of File (EOF).
Exception style platform.\n<pexpect.spawn object at 0x25c17d0>\nversion:
3.3\ncommand: /usr/bin/git\nargs: ['/usr/bin/git', 'clone', '
https://github.com/imesh/stratos-php-applications.git',
'/var/www']\nsearcher: <pexpect.searcher_re object at 0x25c18d0>\nbuffer
(last 100 chars): ''\nbefore (last 100 chars): 'ng deltas: 98% (71/72)
\\rResolving deltas: 100% (72/72) \\rResolving deltas: 100% (72/72),
done.\\r\\n'\nafter: <class 'pexpect.EOF'>\nmatch: None\nmatch_index:
None\nexitstatus: None\nflag_eof: True\npid: 63\nchild_fd: 11\nclosed:
False\ntimeout: 120\ndelimiter: <class 'pexpect.EOF'>\nlogfile:
None\nlogfile_read: None\nlogfile_send: None\nmaxread: 2000\nignorecase:
False\nsearchwindowsize: None\ndelaybeforesend: 0.05\ndelayafterclose:
0.1\ndelayafterterminate: 0.1",)
Traceback (most recent call last):
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
line 200, in clone
result = clone_op.expect(["Username for .*", "Password for .*",
"Checking connectivity... done."])
File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", line
1451, in expect
timeout, searchwindowsize)
File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", line
1466, in expect_list
timeout, searchwindowsize)
File "/usr/local/lib/python2.7/dist-packages/pexpect/__init__.py", line
1554, in expect_loop
raise EOF(str(err) + '\n' + str(self))
EOF: End Of File (EOF). Exception style platform.
<pexpect.spawn object at 0x25c17d0>
On Fri, Feb 13, 2015 at 9:17 AM, Imesh Gunaratne <im...@apache.org> wrote:
> The problem was in populate-user-data.sh file [1], APPLICATION_PATH has
> been removed from it. I just fixed it, pushed the new PHP docker image to
> docker hub and did another flow test and found the below error:
>
> [2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error
> processing 'ArtifactUpdatedEvent' event
> Traceback (most recent call last):
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
> line 103, in run
> handler(event_msg)
> File "agent.py", line 185, in on_artifact_updated
> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
> line 122, in on_artifact_updated_event
> subscribe_run, updated =
> agentgithandler.AgentGitHandler.checkout(repo_info)
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
> line 91, in checkout
> git_repo.tenant_id, git_repo.repo_url)
> AttributeError: 'NoneType' object has no attribute 'tenant_id'
>
> [1]
> https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
>
> On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> Thanks Lakmal, yes as I found basic features we need works with Git
>> python. Will do that.
>> Regarding the above error, seems like it has occurred due to the
>> APPLICATION_PATH not being sent in payload, there was no None check or a
>> log saying its not found:
>>
>> def clone(git_repo):
>> if os.path.isdir(git_repo.local_repo_path):
>> # delete and recreate local repo path if exists
>> GitUtils.delete_folder_tree(git_repo.local_repo_path)
>>
>>
>> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <la...@wso2.com>
>> wrote:
>>
>>>
>>>
>>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> As it looks like this error has been caused by the agent git handler
>>>> while executing git clone, IMO its better if we can switch to GitPython.
>>>>
>>>>
>>> +1, If GitPython has full functionality what we expect, lets move to it.
>>> I was under impression that it has some limitation. If so please not waist
>>> time to fix current one.
>>>
>>>
>>>
>>>> Thanks
>>>>
>>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <im...@apache.org>
>>>> wrote:
>>>>
>>>>> Hi Devs,
>>>>>
>>>>> Yesterday we fixed several issues in PCA, now I'm seeing the below
>>>>> error when trying to receive artifact updated event:
>>>>>
>>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>>>> processing 'ArtifactUpdatedEvent' event
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>>> line 103, in run
>>>>> handler(event_msg)
>>>>> File "agent.py", line 185, in on_artifact_updated
>>>>>
>>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>>> line 122, in on_artifact_updated_event
>>>>> subscribe_run, updated =
>>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>> line 89, in checkout
>>>>> git_repo = AgentGitHandler.clone(git_repo)
>>>>> File
>>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>>> line 190, in clone
>>>>> if os.path.isdir(git_repo.local_repo_path):
>>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>>> st = os.stat(s)
>>>>> TypeError: coercing to Unicode: need string or buffer, NoneType foun
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> --
>>>>> Imesh Gunaratne
>>>>>
>>>>> Technical Lead, WSO2
>>>>> Committer & PMC Member, Apache Stratos
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Lakmal Warusawithana
>>> Vice President, Apache Stratos
>>> Director - Cloud Architecture; WSO2 Inc.
>>> Mobile : +94714289692
>>> Blog : http://lakmalsview.blogspot.com/
>>>
>>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
The problem was in populate-user-data.sh file [1], APPLICATION_PATH has
been removed from it. I just fixed it, pushed the new PHP docker image to
docker hub and did another flow test and found the below error:
[2015-02-13 03:44:28,603] ERROR {eventsubscriber.py:run} - Error processing
'ArtifactUpdatedEvent' event
Traceback (most recent call last):
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
line 103, in run
handler(event_msg)
File "agent.py", line 185, in on_artifact_updated
CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
line 122, in on_artifact_updated_event
subscribe_run, updated =
agentgithandler.AgentGitHandler.checkout(repo_info)
File
"/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
line 91, in checkout
git_repo.tenant_id, git_repo.repo_url)
AttributeError: 'NoneType' object has no attribute 'tenant_id'
[1]
https://github.com/apache/stratos/blob/master/tools/docker-images/cartridge-docker-images/base-image/files/populate-user-data.sh
On Fri, Feb 13, 2015 at 7:43 AM, Imesh Gunaratne <im...@apache.org> wrote:
> Thanks Lakmal, yes as I found basic features we need works with Git
> python. Will do that.
> Regarding the above error, seems like it has occurred due to the
> APPLICATION_PATH not being sent in payload, there was no None check or a
> log saying its not found:
>
> def clone(git_repo):
> if os.path.isdir(git_repo.local_repo_path):
> # delete and recreate local repo path if exists
> GitUtils.delete_folder_tree(git_repo.local_repo_path)
>
>
> On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <la...@wso2.com>
> wrote:
>
>>
>>
>> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> As it looks like this error has been caused by the agent git handler
>>> while executing git clone, IMO its better if we can switch to GitPython.
>>>
>>>
>> +1, If GitPython has full functionality what we expect, lets move to it.
>> I was under impression that it has some limitation. If so please not waist
>> time to fix current one.
>>
>>
>>
>>> Thanks
>>>
>>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <im...@apache.org>
>>> wrote:
>>>
>>>> Hi Devs,
>>>>
>>>> Yesterday we fixed several issues in PCA, now I'm seeing the below
>>>> error when trying to receive artifact updated event:
>>>>
>>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>>> processing 'ArtifactUpdatedEvent' event
>>>> Traceback (most recent call last):
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>>> line 103, in run
>>>> handler(event_msg)
>>>> File "agent.py", line 185, in on_artifact_updated
>>>>
>>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>>> line 122, in on_artifact_updated_event
>>>> subscribe_run, updated =
>>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>> line 89, in checkout
>>>> git_repo = AgentGitHandler.clone(git_repo)
>>>> File
>>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>>> line 190, in clone
>>>> if os.path.isdir(git_repo.local_repo_path):
>>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>>> st = os.stat(s)
>>>> TypeError: coercing to Unicode: need string or buffer, NoneType foun
>>>>
>>>> Thanks
>>>>
>>>>
>>>> --
>>>> Imesh Gunaratne
>>>>
>>>> Technical Lead, WSO2
>>>> Committer & PMC Member, Apache Stratos
>>>>
>>>
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Lakmal Warusawithana
>> Vice President, Apache Stratos
>> Director - Cloud Architecture; WSO2 Inc.
>> Mobile : +94714289692
>> Blog : http://lakmalsview.blogspot.com/
>>
>>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
Thanks Lakmal, yes as I found basic features we need works with Git python.
Will do that.
Regarding the above error, seems like it has occurred due to the
APPLICATION_PATH not being sent in payload, there was no None check or a
log saying its not found:
def clone(git_repo):
if os.path.isdir(git_repo.local_repo_path):
# delete and recreate local repo path if exists
GitUtils.delete_folder_tree(git_repo.local_repo_path)
On Fri, Feb 13, 2015 at 7:37 AM, Lakmal Warusawithana <la...@wso2.com>
wrote:
>
>
> On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> As it looks like this error has been caused by the agent git handler
>> while executing git clone, IMO its better if we can switch to GitPython.
>>
>>
> +1, If GitPython has full functionality what we expect, lets move to it. I
> was under impression that it has some limitation. If so please not waist
> time to fix current one.
>
>
>
>> Thanks
>>
>> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <im...@apache.org>
>> wrote:
>>
>>> Hi Devs,
>>>
>>> Yesterday we fixed several issues in PCA, now I'm seeing the below error
>>> when trying to receive artifact updated event:
>>>
>>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>>> processing 'ArtifactUpdatedEvent' event
>>> Traceback (most recent call last):
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>>> line 103, in run
>>> handler(event_msg)
>>> File "agent.py", line 185, in on_artifact_updated
>>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>>> line 122, in on_artifact_updated_event
>>> subscribe_run, updated =
>>> agentgithandler.AgentGitHandler.checkout(repo_info)
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>> line 89, in checkout
>>> git_repo = AgentGitHandler.clone(git_repo)
>>> File
>>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>>> line 190, in clone
>>> if os.path.isdir(git_repo.local_repo_path):
>>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>>> st = os.stat(s)
>>> TypeError: coercing to Unicode: need string or buffer, NoneType foun
>>>
>>> Thanks
>>>
>>>
>>> --
>>> Imesh Gunaratne
>>>
>>> Technical Lead, WSO2
>>> Committer & PMC Member, Apache Stratos
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Lakmal Warusawithana
> Vice President, Apache Stratos
> Director - Cloud Architecture; WSO2 Inc.
> Mobile : +94714289692
> Blog : http://lakmalsview.blogspot.com/
>
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Lakmal Warusawithana <la...@wso2.com>.
On Fri, Feb 13, 2015 at 7:34 AM, Imesh Gunaratne <im...@apache.org> wrote:
> As it looks like this error has been caused by the agent git handler while
> executing git clone, IMO its better if we can switch to GitPython.
>
>
+1, If GitPython has full functionality what we expect, lets move to it. I
was under impression that it has some limitation. If so please not waist
time to fix current one.
> Thanks
>
> On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <im...@apache.org> wrote:
>
>> Hi Devs,
>>
>> Yesterday we fixed several issues in PCA, now I'm seeing the below error
>> when trying to receive artifact updated event:
>>
>> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
>> processing 'ArtifactUpdatedEvent' event
>> Traceback (most recent call last):
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
>> line 103, in run
>> handler(event_msg)
>> File "agent.py", line 185, in on_artifact_updated
>> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
>> line 122, in on_artifact_updated_event
>> subscribe_run, updated =
>> agentgithandler.AgentGitHandler.checkout(repo_info)
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>> line 89, in checkout
>> git_repo = AgentGitHandler.clone(git_repo)
>> File
>> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
>> line 190, in clone
>> if os.path.isdir(git_repo.local_repo_path):
>> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
>> st = os.stat(s)
>> TypeError: coercing to Unicode: need string or buffer, NoneType foun
>>
>> Thanks
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Lakmal Warusawithana
Vice President, Apache Stratos
Director - Cloud Architecture; WSO2 Inc.
Mobile : +94714289692
Blog : http://lakmalsview.blogspot.com/
Re: [Discuss] Fixing Python Cartridge Agent (PCA) Issues
Posted by Imesh Gunaratne <im...@apache.org>.
As it looks like this error has been caused by the agent git handler while
executing git clone, IMO its better if we can switch to GitPython.
Thanks
On Fri, Feb 13, 2015 at 7:31 AM, Imesh Gunaratne <im...@apache.org> wrote:
> Hi Devs,
>
> Yesterday we fixed several issues in PCA, now I'm seeing the below error
> when trying to receive artifact updated event:
>
> [2015-02-13 01:56:07,311] ERROR {eventsubscriber.py:run} - Error
> processing 'ArtifactUpdatedEvent' event
> Traceback (most recent call last):
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/subscriber/eventsubscriber.py",
> line 103, in run
> handler(event_msg)
> File "agent.py", line 185, in on_artifact_updated
> CartridgeAgent.extension_handler.on_artifact_updated_event(event_obj)
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/extensions/extensionhandler.py",
> line 122, in on_artifact_updated_event
> subscribe_run, updated =
> agentgithandler.AgentGitHandler.checkout(repo_info)
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
> line 89, in checkout
> git_repo = AgentGitHandler.clone(git_repo)
> File
> "/mnt/apache-stratos-python-cartridge-agent-4.1.0-SNAPSHOT/modules/artifactmgt/git/agentgithandler.py",
> line 190, in clone
> if os.path.isdir(git_repo.local_repo_path):
> File "/usr/lib/python2.7/genericpath.py", line 41, in isdir
> st = os.stat(s)
> TypeError: coercing to Unicode: need string or buffer, NoneType foun
>
> Thanks
>
>
> --
> Imesh Gunaratne
>
> Technical Lead, WSO2
> Committer & PMC Member, Apache Stratos
>
--
Imesh Gunaratne
Technical Lead, WSO2
Committer & PMC Member, Apache Stratos