You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@marmotta.apache.org by Encolpe DEGOUTE <en...@free.fr> on 2014/12/11 10:40:20 UTC
Massive import from the REST interface
Hello,
These days we are migrating our LOM metadata to MLR to store them in marmotta
3.2.0 (the last war file available via the debian repository).
To test our conversion process we try to import data in turtle via the REST
import interface.
A file that works via the HTML interface raises an RDF error in the first case
and seem works in the second case:
- https://sources.colpi.org/_admin/gists/4
- https://sources.colpi.org/_admin/gists/5
The source file:
https://sources.colpi.org/_admin/gists/6
Have you any clue to explain this difference in the two process ?
Best Regards
--
Encolpe Degoute
http://encolpe.wordpress.com
http://encolpe.degoute.free.fr
Logiciels libres, hockey sur glace et autres activités cérébrales
Re: Massive import from the REST interface
Posted by Sergio Fernández <wi...@apache.org>.
Hi Encolpe,
On 11/12/14 16:09, Encolpe DEGOUTE wrote:
> Le jeudi 11 décembre 2014, 11:30:04 Sergio Fernández a écrit :
>> From my experience with requests in Python (for instance in ldpy [1]),
>> I always used the parameter 'data' to send the payload [2] using the
>> following construction:
>>
>> requests.post(url, data=open(path,'rb'), headers=headers)
>
> Fine, that was the point.
Perfect, that's what I suspected.
> The import is damn fast for all our old metadata database.
;-)
>> BTW, if there is a need for our community, we could add to our roadmap
>> the addition of a Python version to our Client Library [4].
>
> Well, I'm not sure that a Python client will fit to all projets.
The currently Client Library has been built using an "on demand"
approach: whenever we had the need, instead of individually code for
specific projects, we tried to contribute back as a new client library.
That also explain why some time is a bit abandoned, but we have a small
community of developers to maintain some languages.
> As marmotta can be requested in REST,
> we need code examples. The ldpy project can help on
> this way. We can also propose a project for the GSoC 2015.
Sure. Even if I find it cool, I personally don't have the need of such
version of the Client Library. So, unless you have the real need of
using Marmotta from Python and you have the time or resource for
contributing there, I'd suggest to keep the idea on hold untik we find
room for it, GSoC or whatever.
> Thanks for your reactivity.
No problem. And welcome to the Marmotta community :-)
> Is there some Marmotta workshop or tutorial scheduled in 2015 like it was in
> 2014?
Well, not really... In 2014 we hosted a half-day tutorial at ISWC2014
[1]. Personally I was very happy with the result, it was a very
productive day and we got a lot of feedback from the scientific
community. Therefore, if there is a real interest we can try another
shot, maybe not in another event like that, but a more industrial one,
even a meetup could work. The question is where... This week in the LDP
working group at W3C discussed the idea of organizing a LDP working in
Spring [2], maybe could be an option.
Cheers,
[1] http://marmotta.apache.org/events/iswc2014
[2] http://www.w3.org/2013/meeting/ldp/2014-12-08#resolution_5
--
Sergio Fernández
Partner Technology Manager
Redlink GmbH
m: +43 660 2747 925
e: sergio.fernandez@redlink.co
w: http://redlink.co
Re: Massive import from the REST interface
Posted by Encolpe DEGOUTE <en...@free.fr>.
Le jeudi 11 décembre 2014, 11:30:04 Sergio Fernández a écrit :
> Hi,
>
> On 11/12/14 10:40, Encolpe DEGOUTE wrote:
> > These days we are migrating our LOM metadata to MLR to store them in
> > marmotta 3.2.0 (the last war file available via the debian repository).
>
> 3.3.0 release is being vote currently. If it passes it, you would have a
> better version to use, also as .deb. Although that part of the system
> does not come with significant improvements, I'd recommend you to
> upgrade as soon as it gets out.
Perfect, I will. :)
> > To test our conversion process we try to import data in turtle via the
> > REST
> > import interface.
> >
> > A file that works via the HTML interface raises an RDF error in the first
> > case and seem works in the second case:
> > - https://sources.colpi.org/_admin/gists/4
> > - https://sources.colpi.org/_admin/gists/5
> >
> > The source file:
> > https://sources.colpi.org/_admin/gists/6
>
> The Turtle file looks syntactically correct, that's fine.
>
> > Have you any clue to explain this difference in the two process ?
>
> Well, as far as I see both request use the same rest web service. The
> difference is that the first one sends the file from its path, while the
> second one tries to send the file attached to the request, which I think
> is not supported (files' data must be send in the payload).
>
> From my experience with requests in Python (for instance in ldpy [1]),
> I always used the parameter 'data' to send the payload [2] using the
> following construction:
>
> requests.post(url, data=open(path,'rb'), headers=headers)
Fine, that was the point. The import is damn fast for all our old metadata
database.
>
> You could find more details about the different method available to
> import data in the wiki [3].
>
> BTW, if there is a need for our community, we could add to our roadmap
> the addition of a Python version to our Client Library [4].
Well, I'm not sure that a Python client will fit to all projets. As marmotta
can be requested in REST, we need code examples. The ldpy project can help on
this way. We can also propose a project for the GSoC 2015.
Thanks for your reactivity.
Is there some Marmotta workshop or tutorial scheduled in 2015 like it was in
2014?
Best regards.
--
Encolpe Degoute
http://encolpe.wordpress.com
http://encolpe.degoute.free.fr
Logiciels libres, hockey sur glace et autres activités cérébrales
Re: Massive import from the REST interface
Posted by Sergio Fernández <wi...@apache.org>.
Hi,
On 11/12/14 10:40, Encolpe DEGOUTE wrote:
> These days we are migrating our LOM metadata to MLR to store them in marmotta
> 3.2.0 (the last war file available via the debian repository).
3.3.0 release is being vote currently. If it passes it, you would have a
better version to use, also as .deb. Although that part of the system
does not come with significant improvements, I'd recommend you to
upgrade as soon as it gets out.
> To test our conversion process we try to import data in turtle via the REST
> import interface.
>
> A file that works via the HTML interface raises an RDF error in the first case
> and seem works in the second case:
> - https://sources.colpi.org/_admin/gists/4
> - https://sources.colpi.org/_admin/gists/5
>
> The source file:
> https://sources.colpi.org/_admin/gists/6
The Turtle file looks syntactically correct, that's fine.
> Have you any clue to explain this difference in the two process ?
Well, as far as I see both request use the same rest web service. The
difference is that the first one sends the file from its path, while the
second one tries to send the file attached to the request, which I think
is not supported (files' data must be send in the payload).
From my experience with requests in Python (for instance in ldpy [1]),
I always used the parameter 'data' to send the payload [2] using the
following construction:
requests.post(url, data=open(path,'rb'), headers=headers)
You could find more details about the different method available to
import data in the wiki [3].
BTW, if there is a need for our community, we could add to our roadmap
the addition of a Python version to our Client Library [4].
Cheers,
[1] https://github.com/wikier/ldpy/blob/master/ldpy/client.py#L70
[2]
http://docs.python-requests.org/en/latest/user/quickstart/#more-complicated-post-requests
[3] http://wiki.apache.org/marmotta/ImportData
[4] http://marmotta.apache.org/client-library
--
Sergio Fernández
Partner Technology Manager
Redlink GmbH
m: +43 660 2747 925
e: sergio.fernandez@redlink.co
w: http://redlink.co