You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Lior Chaga <li...@taboola.com> on 2021/08/10 07:29:16 UTC
loading interpreter repositories from interpreter settings
I'm facing an issue loading repositories from the settings file.
Zeppelin uses gson to load the saved repositories, and as RemoteRepository
doesn't have a setter for protocol, it instantiates a RemoteRepository with
null protocol (not going through the Builder which resolves the protocol
from the url), resulting in not being able to fetch dependencies.
For now, my workaround is to patch InterpreterSettingManager, by cloning
the RemoteRepository and forcing it going through the Builder to resolve
protocol (see below).
Wonder if there's a patchless solution. couldn't find an open issue. With
zeppelin 0.8.2 this doesn't happen...
Lior
for (RemoteRepository repo : infoSaving.interpreterRepositories) {
if (!dependencyResolver.getRepos().contains(repo)) {
RemoteRepository clonedRepo = new
RemoteRepository.Builder(repo.getId(), repo.getContentType(),
repo.getUrl()).build();
this.interpreterRepositories.add(clonedRepo);
}
}
Re: loading interpreter repositories from interpreter settings
Posted by Lior Chaga <li...@taboola.com>.
https://issues.apache.org/jira/browse/ZEPPELIN-5485
On Wed, Aug 11, 2021 at 9:06 AM Lior Chaga <li...@taboola.com> wrote:
> Gladly
>
> On Wed, Aug 11, 2021 at 9:01 AM Jeff Zhang <zj...@gmail.com> wrote:
>
>> It seems a bug that additional configured repo doesn't work, could you
>> create a ticket for it ?
>>
>>
>> Lior Chaga <li...@taboola.com> 于2021年8月11日周三 下午1:50写道：
>>
>>> Default central repo doesn't fail, but behind VPN the zeppelin cannot
>>> access central.
>>> With zeppelin 0.8.2 we just configured an additional artifactory, and it
>>> worked. For some reason (not sure why, as it uses the same
>>> maven-resolver-api version, and I didn't notice any related code changes),
>>> with zeppelin master branch the additional configured repo is not loaded
>>> properly (protocol not resolved). So by just replacing the default central
>>> repo with our artifactory, everything works fine.
>>>
>>> On Wed, Aug 11, 2021 at 8:40 AM Jeff Zhang <zj...@gmail.com> wrote:
>>>
>>>> Hi Lior,
>>>>
>>>> Thanks for the update, not sure how default central repo fails. Is it
>>>> due to you unable to access the default maven repo ?
>>>>
>>>> Lior Chaga <li...@taboola.com> 于2021年8月11日周三 下午12:50写道：
>>>>
>>>>> I found a non-patch solution, just setting
>>>>> "zeppelin.interpreter.dep.mvnRepo" to desired repo (replacing default
>>>>> central repo)
>>>>> But IMO there's still a bug (if someone simply cannot replace default
>>>>> repo and have to add one)
>>>>>
>>>>> On Tue, Aug 10, 2021 at 10:29 AM Lior Chaga <li...@taboola.com>
>>>>> wrote:
>>>>>
>>>>>> I'm facing an issue loading repositories from the settings file.
>>>>>> Zeppelin uses gson to load the saved repositories, and
>>>>>> as RemoteRepository doesn't have a setter for protocol, it instantiates a
>>>>>> RemoteRepository with null protocol (not going through the Builder which
>>>>>> resolves the protocol from the url), resulting in not being able to fetch
>>>>>> dependencies.
>>>>>>
>>>>>> For now, my workaround is to patch InterpreterSettingManager, by
>>>>>> cloning the RemoteRepository and forcing it going through the Builder to
>>>>>> resolve protocol (see below).
>>>>>> Wonder if there's a patchless solution. couldn't find an open issue.
>>>>>> With zeppelin 0.8.2 this doesn't happen...
>>>>>>
>>>>>> Lior
>>>>>>
>>>>>> for (RemoteRepository repo : infoSaving.interpreterRepositories) {
>>>>>> if (!dependencyResolver.getRepos().contains(repo)) {
>>>>>> RemoteRepository clonedRepo = new RemoteRepository.Builder(repo.getId(), repo.getContentType(), repo.getUrl()).build();
>>>>>> this.interpreterRepositories.add(clonedRepo);
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>
>>>>
>>>> --
>>>> Best Regards
>>>>
>>>> Jeff Zhang
>>>>
>>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
Re: loading interpreter repositories from interpreter settings
Posted by Lior Chaga <li...@taboola.com>.
Gladly
On Wed, Aug 11, 2021 at 9:01 AM Jeff Zhang <zj...@gmail.com> wrote:
> It seems a bug that additional configured repo doesn't work, could you
> create a ticket for it ?
>
>
> Lior Chaga <li...@taboola.com> 于2021年8月11日周三 下午1:50写道：
>
>> Default central repo doesn't fail, but behind VPN the zeppelin cannot
>> access central.
>> With zeppelin 0.8.2 we just configured an additional artifactory, and it
>> worked. For some reason (not sure why, as it uses the same
>> maven-resolver-api version, and I didn't notice any related code changes),
>> with zeppelin master branch the additional configured repo is not loaded
>> properly (protocol not resolved). So by just replacing the default central
>> repo with our artifactory, everything works fine.
>>
>> On Wed, Aug 11, 2021 at 8:40 AM Jeff Zhang <zj...@gmail.com> wrote:
>>
>>> Hi Lior,
>>>
>>> Thanks for the update, not sure how default central repo fails. Is it
>>> due to you unable to access the default maven repo ?
>>>
>>> Lior Chaga <li...@taboola.com> 于2021年8月11日周三 下午12:50写道：
>>>
>>>> I found a non-patch solution, just setting
>>>> "zeppelin.interpreter.dep.mvnRepo" to desired repo (replacing default
>>>> central repo)
>>>> But IMO there's still a bug (if someone simply cannot replace default
>>>> repo and have to add one)
>>>>
>>>> On Tue, Aug 10, 2021 at 10:29 AM Lior Chaga <li...@taboola.com> wrote:
>>>>
>>>>> I'm facing an issue loading repositories from the settings file.
>>>>> Zeppelin uses gson to load the saved repositories, and
>>>>> as RemoteRepository doesn't have a setter for protocol, it instantiates a
>>>>> RemoteRepository with null protocol (not going through the Builder which
>>>>> resolves the protocol from the url), resulting in not being able to fetch
>>>>> dependencies.
>>>>>
>>>>> For now, my workaround is to patch InterpreterSettingManager, by
>>>>> cloning the RemoteRepository and forcing it going through the Builder to
>>>>> resolve protocol (see below).
>>>>> Wonder if there's a patchless solution. couldn't find an open issue.
>>>>> With zeppelin 0.8.2 this doesn't happen...
>>>>>
>>>>> Lior
>>>>>
>>>>> for (RemoteRepository repo : infoSaving.interpreterRepositories) {
>>>>> if (!dependencyResolver.getRepos().contains(repo)) {
>>>>> RemoteRepository clonedRepo = new RemoteRepository.Builder(repo.getId(), repo.getContentType(), repo.getUrl()).build();
>>>>> this.interpreterRepositories.add(clonedRepo);
>>>>> }
>>>>> }
>>>>>
>>>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: loading interpreter repositories from interpreter settings
Posted by Jeff Zhang <zj...@gmail.com>.
It seems a bug that additional configured repo doesn't work, could you
create a ticket for it ?
Lior Chaga <li...@taboola.com> 于2021年8月11日周三 下午1:50写道：
> Default central repo doesn't fail, but behind VPN the zeppelin cannot
> access central.
> With zeppelin 0.8.2 we just configured an additional artifactory, and it
> worked. For some reason (not sure why, as it uses the same
> maven-resolver-api version, and I didn't notice any related code changes),
> with zeppelin master branch the additional configured repo is not loaded
> properly (protocol not resolved). So by just replacing the default central
> repo with our artifactory, everything works fine.
>
> On Wed, Aug 11, 2021 at 8:40 AM Jeff Zhang <zj...@gmail.com> wrote:
>
>> Hi Lior,
>>
>> Thanks for the update, not sure how default central repo fails. Is it due
>> to you unable to access the default maven repo ?
>>
>> Lior Chaga <li...@taboola.com> 于2021年8月11日周三 下午12:50写道：
>>
>>> I found a non-patch solution, just setting
>>> "zeppelin.interpreter.dep.mvnRepo" to desired repo (replacing default
>>> central repo)
>>> But IMO there's still a bug (if someone simply cannot replace default
>>> repo and have to add one)
>>>
>>> On Tue, Aug 10, 2021 at 10:29 AM Lior Chaga <li...@taboola.com> wrote:
>>>
>>>> I'm facing an issue loading repositories from the settings file.
>>>> Zeppelin uses gson to load the saved repositories, and
>>>> as RemoteRepository doesn't have a setter for protocol, it instantiates a
>>>> RemoteRepository with null protocol (not going through the Builder which
>>>> resolves the protocol from the url), resulting in not being able to fetch
>>>> dependencies.
>>>>
>>>> For now, my workaround is to patch InterpreterSettingManager, by
>>>> cloning the RemoteRepository and forcing it going through the Builder to
>>>> resolve protocol (see below).
>>>> Wonder if there's a patchless solution. couldn't find an open issue.
>>>> With zeppelin 0.8.2 this doesn't happen...
>>>>
>>>> Lior
>>>>
>>>> for (RemoteRepository repo : infoSaving.interpreterRepositories) {
>>>> if (!dependencyResolver.getRepos().contains(repo)) {
>>>> RemoteRepository clonedRepo = new RemoteRepository.Builder(repo.getId(), repo.getContentType(), repo.getUrl()).build();
>>>> this.interpreterRepositories.add(clonedRepo);
>>>> }
>>>> }
>>>>
>>>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
--
Best Regards
Jeff Zhang
Re: loading interpreter repositories from interpreter settings
Posted by Lior Chaga <li...@taboola.com>.
Default central repo doesn't fail, but behind VPN the zeppelin cannot
access central.
With zeppelin 0.8.2 we just configured an additional artifactory, and it
worked. For some reason (not sure why, as it uses the same
maven-resolver-api version, and I didn't notice any related code changes),
with zeppelin master branch the additional configured repo is not loaded
properly (protocol not resolved). So by just replacing the default central
repo with our artifactory, everything works fine.
On Wed, Aug 11, 2021 at 8:40 AM Jeff Zhang <zj...@gmail.com> wrote:
> Hi Lior,
>
> Thanks for the update, not sure how default central repo fails. Is it due
> to you unable to access the default maven repo ?
>
> Lior Chaga <li...@taboola.com> 于2021年8月11日周三 下午12:50写道：
>
>> I found a non-patch solution, just setting
>> "zeppelin.interpreter.dep.mvnRepo" to desired repo (replacing default
>> central repo)
>> But IMO there's still a bug (if someone simply cannot replace default
>> repo and have to add one)
>>
>> On Tue, Aug 10, 2021 at 10:29 AM Lior Chaga <li...@taboola.com> wrote:
>>
>>> I'm facing an issue loading repositories from the settings file.
>>> Zeppelin uses gson to load the saved repositories, and
>>> as RemoteRepository doesn't have a setter for protocol, it instantiates a
>>> RemoteRepository with null protocol (not going through the Builder which
>>> resolves the protocol from the url), resulting in not being able to fetch
>>> dependencies.
>>>
>>> For now, my workaround is to patch InterpreterSettingManager, by cloning
>>> the RemoteRepository and forcing it going through the Builder to resolve
>>> protocol (see below).
>>> Wonder if there's a patchless solution. couldn't find an open issue.
>>> With zeppelin 0.8.2 this doesn't happen...
>>>
>>> Lior
>>>
>>> for (RemoteRepository repo : infoSaving.interpreterRepositories) {
>>> if (!dependencyResolver.getRepos().contains(repo)) {
>>> RemoteRepository clonedRepo = new RemoteRepository.Builder(repo.getId(), repo.getContentType(), repo.getUrl()).build();
>>> this.interpreterRepositories.add(clonedRepo);
>>> }
>>> }
>>>
>>>
>
> --
> Best Regards
>
> Jeff Zhang
>
Re: loading interpreter repositories from interpreter settings
Posted by Jeff Zhang <zj...@gmail.com>.
Hi Lior,
Thanks for the update, not sure how default central repo fails. Is it due
to you unable to access the default maven repo ?
Lior Chaga <li...@taboola.com> 于2021年8月11日周三 下午12:50写道：
> I found a non-patch solution, just setting
> "zeppelin.interpreter.dep.mvnRepo" to desired repo (replacing default
> central repo)
> But IMO there's still a bug (if someone simply cannot replace default repo
> and have to add one)
>
> On Tue, Aug 10, 2021 at 10:29 AM Lior Chaga <li...@taboola.com> wrote:
>
>> I'm facing an issue loading repositories from the settings file.
>> Zeppelin uses gson to load the saved repositories, and
>> as RemoteRepository doesn't have a setter for protocol, it instantiates a
>> RemoteRepository with null protocol (not going through the Builder which
>> resolves the protocol from the url), resulting in not being able to fetch
>> dependencies.
>>
>> For now, my workaround is to patch InterpreterSettingManager, by cloning
>> the RemoteRepository and forcing it going through the Builder to resolve
>> protocol (see below).
>> Wonder if there's a patchless solution. couldn't find an open issue. With
>> zeppelin 0.8.2 this doesn't happen...
>>
>> Lior
>>
>> for (RemoteRepository repo : infoSaving.interpreterRepositories) {
>> if (!dependencyResolver.getRepos().contains(repo)) {
>> RemoteRepository clonedRepo = new RemoteRepository.Builder(repo.getId(), repo.getContentType(), repo.getUrl()).build();
>> this.interpreterRepositories.add(clonedRepo);
>> }
>> }
>>
>>
--
Best Regards
Jeff Zhang
Re: loading interpreter repositories from interpreter settings
Posted by Lior Chaga <li...@taboola.com>.
I found a non-patch solution, just setting
"zeppelin.interpreter.dep.mvnRepo" to desired repo (replacing default
central repo)
But IMO there's still a bug (if someone simply cannot replace default repo
and have to add one)
On Tue, Aug 10, 2021 at 10:29 AM Lior Chaga <li...@taboola.com> wrote:
> I'm facing an issue loading repositories from the settings file.
> Zeppelin uses gson to load the saved repositories, and as RemoteRepository
> doesn't have a setter for protocol, it instantiates a RemoteRepository with
> null protocol (not going through the Builder which resolves the protocol
> from the url), resulting in not being able to fetch dependencies.
>
> For now, my workaround is to patch InterpreterSettingManager, by cloning
> the RemoteRepository and forcing it going through the Builder to resolve
> protocol (see below).
> Wonder if there's a patchless solution. couldn't find an open issue. With
> zeppelin 0.8.2 this doesn't happen...
>
> Lior
>
> for (RemoteRepository repo : infoSaving.interpreterRepositories) {
> if (!dependencyResolver.getRepos().contains(repo)) {
> RemoteRepository clonedRepo = new RemoteRepository.Builder(repo.getId(), repo.getContentType(), repo.getUrl()).build();
> this.interpreterRepositories.add(clonedRepo);
> }
> }
>
>