You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Nadav Hoze <Na...@dbMotion.com> on 2011/11/10 15:15:12 UTC

declare a script and run it with jena TDB

Hi,

I want to update triples that I have in my model with datetime now.
Is it possible to define a script and run it with jena TDB programmatically , or just write a delete and insert query statement with datetime now.

Thanks,


Nadav Hoze
Senior Software Engineer
dbMotion Ltd.
Direct: +972-9-7699000
Extension number: 9037
Mobile: 972-54-4821606
E-mail: nadav.hoze@dbMotion.com<ma...@dbMotion.com>
[Description: Logo + Tagline]

This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. Please note that any disclosure, copying or distribution of the content of this information is strictly forbidden. If you have received this email message in error please notify its sender and then delete it from your files.

Re: [Fuseki] Java heap space

Posted by Andy Seaborne <an...@apache.org>.
On 14/11/11 17:51, Camel Christophe wrote:
> Hi,
>
> I’m facing an already discussed issue about the memory usage of Fuseki.
> I’m using version 0.2.0 of Fuseky, with a DatasetTDB database
> (unionDefaultGraph to true) running on a 32 bits JVM on windows Seven,
> started with “-Xmx1200M” java option.
>
> The test is pretty simple. It consists in sending an rdf document of
> about 1.5Mb using the file endpoint of Fuski, then performing a removal
> of all the database (clear) content via the sparql update endpoint of
> Fuseki. The test is repeated many times (x1500).
>
> If you are interested, here is the code:
>
> public class Test {
>
> public static void main(String[] args) {
>
> File f = new File("C:\\rdf\\test.rdf");
>
> WebResource uploadWebResource =
> Client.create().resource("http://localhost:3030/my-dataset/upload");
>
> WebResource updateWebResource =
> Client.create().resource("http://localhost:3030/ my-dataset /update");
>
> ClientResponse response;
>
> for (int i = 0; i < 1500; ++i) {
>
> System.out.println(i + "\t\t ingesting");
>
> // Upload
>
> FormDataMultiPart fdmp = new FormDataMultiPart();
>
> fdmp.bodyPart(new FileDataBodyPart("test.rdf", f,
> MediaType.valueOf("application/rdf+xml")));

What's in test.rdf?

>
> fdmp.bodyPart(new FormDataBodyPart("graph", "http://graph/exemple"));
>
> FormDataMultiPart form = new FormDataMultiPart().field("UNSET FILE
> NAME", f, MediaType.MULTIPART_FORM_DATA_TYPE).field("graph",
> "http://graph/exemple");
>
> response =
> uploadWebResource.type(MediaType.MULTIPART_FORM_DATA_TYPE).post(ClientResponse.class,
> fdmp);
>
> System.out.println(i + "\t\t deleting");
>
> // Delete
>
> MultivaluedMap formData = new MultivaluedMapImpl();
>
> formData.add("update", "clear all");
>
> response =
> updateWebResource.type("application/x-www-form-urlencoded").post(ClientResponse.class,
> formData);

This does not seem get the statuc code - does it close the connection 
afterwards? Otherwise your client is holding http: connections open and 
the server needs space at it's end.

>
> System.out.println(response.getEntity(String.class));
>
> }
>
> }
>
> }
>
> After a while, Fuseki hangs and throws a Java.lang.OutOfMemoryError:
> Java heap space.
>
> The following graph is the capture of the memory consumption during the test

The graph didn't make it through email - could you fwd the original 
message to me directly please (andy AT apache.org)

Linear growth upto the JVM heap limit isbn't unexpected.

> As you can see, the memory consumption is growing up linearly until it
> reaches a threshold. At this stage (I don’t know exactly when), Fuseki
> starts to throw out of memory exceptions.
>
> I wonder how to fix this issue. Will the increase of the maximum heap
> size of the jvm (xmx setting) be sufficient ?

Unlikely:

1/ There is a Java limitation of1.5Gb on 32 bit regardless
2/ I would expect GC to trgigger

>
> Is there something else to do in order to master the memory consumption
> of Fuseki ?

Use jvisualvm to see which objects are holding on to the memory.


	Andy

[Fuseki] Java heap space

Posted by Camel Christophe <cc...@cls.fr>.
Hi,

 

I'm facing an already discussed issue about the memory usage of Fuseki.
I'm using version 0.2.0 of Fuseky, with a DatasetTDB database
(unionDefaultGraph to true) running on a 32 bits JVM on windows Seven,
started with "-Xmx1200M" java option.

 

The test is pretty simple. It consists in sending an rdf document of
about 1.5Mb using the file endpoint of Fuski, then performing a removal
of all the database (clear) content via the sparql update endpoint of
Fuseki. The test is repeated many times (x1500).

 

If you are interested, here is the code:

 

public class Test {

    public static void main(String[] args) {

        File f = new File("C:\\rdf\\test.rdf");

        WebResource uploadWebResource =
Client.create().resource("http://localhost:3030/my-dataset/upload");

        WebResource updateWebResource =
Client.create().resource("http://localhost:3030/ my-dataset /update");

        ClientResponse response;

 

        for (int i = 0; i < 1500; ++i) {

            System.out.println(i + "\t\t ingesting");

            // Upload

            FormDataMultiPart fdmp = new FormDataMultiPart();

            fdmp.bodyPart(new FileDataBodyPart("test.rdf", f,
MediaType.valueOf("application/rdf+xml")));

            fdmp.bodyPart(new FormDataBodyPart("graph",
"http://graph/exemple"));

            FormDataMultiPart form = new
FormDataMultiPart().field("UNSET FILE NAME", f,
MediaType.MULTIPART_FORM_DATA_TYPE).field("graph",
"http://graph/exemple");

            response =
uploadWebResource.type(MediaType.MULTIPART_FORM_DATA_TYPE).post(ClientRe
sponse.class, fdmp);

 

            System.out.println(i + "\t\t deleting");

            // Delete

            MultivaluedMap formData = new MultivaluedMapImpl();

            formData.add("update", "clear all");

            response =
updateWebResource.type("application/x-www-form-urlencoded").post(ClientR
esponse.class, formData);

            System.out.println(response.getEntity(String.class));

        }

    }

}

 

After a while, Fuseki hangs and throws a Java.lang.OutOfMemoryError:
Java heap space.

 

The following graph is the capture of the memory consumption during the
test

 

 

As you can see, the memory consumption is growing up linearly until it
reaches a threshold. At this stage (I don't know exactly when), Fuseki
starts to throw out of memory exceptions.

 

I wonder how to fix this issue. Will the increase of the maximum heap
size of the jvm (xmx setting) be sufficient ?

Is there something else to do in order to master the memory consumption
of Fuseki ?

Re: declare a script and run it with jena TDB

Posted by Andy Seaborne <an...@apache.org>.
On 10/11/11 14:15, Nadav Hoze wrote:
> Hi,
>
> I want to update triples that I have in my model with datetime now.
>
> Is it possible to define a script and run it with jena TDB
> programmatically , or just write a delete and insert query statement
> with datetime now.
>
> Thanks,
>
> *Nadav Hoze*

You could do something like:

PREFIX : <http://example/>


DELETE { ?x :p ?then }
INSERT { ?x :p ?now }
WHERE
{
    BIND (now() AS ?now)
}

now() is a SPARQL 1.1 function that returns an xsd:dateTime for some 
point in time for the execution.

	Andy