You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Sören Brunk <so...@tu-dresden.de> on 2015/03/03 14:28:37 UTC
Apache Jena for Android
Hi all,
We are working on some Android app prototypes that deal with RDF data.
To make things easier, we have created a version of Apache Jena that
runs on Android. It basically consists of a bunch of Maven build files
that do some repackaging of the original Jena jars in order to make them
run on Android's Dalvik VM. As the port might be interesting to others,
we have published it to Github for everyone to use.
From the project description:
This project aims to make the Apache Jena Framework usable on Android.
While Jena is written in pure Java, it can't be used on Android as is,
due to multiple package conflict issues. To overcome these issues, we
have created an Android port enabling developers to use Apache Jena in
Android apps. In contrast to other ports we don't just publish binaries,
but make the Maven build files of the port available. We also stay as
close as possible to the original (i.e. no source code changes). This
should make it easier to stay in sync with upcoming Jena releases.
The project, including usage instructions, is available here:
https://github.com/seus-inf/jena-android
See here for an older discussion about the issues:
http://mail-archives.apache.org/mod_mbox/jena-dev/201304.mbox/%3C5163E9A0.2030307%40apache.org%3E
Best regards,
Sören
--
Dipl. Inf. Sören Brunk
Research Associate
Technische Universität Dresden
Faculty of Computer Science
Institute for Software- and Multimedia-Technology
Junior Professorship in Software Engineering of Ubiquitous Systems
01062 Dresden
Re: Apache Jena for Android
Posted by Fabio Ricci <fa...@semweb.ch>.
Thank you very much for all these hints!
Yes it is SOLR and the hint on the text/search integration was helpful!!!
Fabio
Am 10.3.15 um 16:39 schrieb Stian Soiland-Reyes:
> SPARQL results are HTTP-streamed back from Fuseki (and in TDB
> programmatically), as long as you don't use ORDER and select a
> streamable serialization format like JSON (not JSON-LD)
>
> That is exactly what I do in
> https://github.com/openphacts/openphacts-irs
> to populate an ElasticSearch instance, by generating SPARQL queries to select
> certain properties, then iterating
> over the JSON response in a streaming way.
>
> As I register one JSON document in ES per SPARQL result (or more
> correctly, per subject URI), then I use the bulk loading feature of ES
> to chunk of 1000 in one HTTP request to ES.
>
> I don't know which search engine you have, but I guess you could do
> something similar.
>
>
> If you do it through TDBs API then you should not need to worry about
> the serialization as you can just use the Iterator normally.
>
>
> Note that Jena also has its own text-search integration with Solr
> and/or ElasticSearch which could be of interest -
> https://jena.apache.org/documentation/query/text-query.html
>
> (in my case I need to customize lots of ElasticSearch indexes (and
> need the option to populate from Virtuoso instead of Jena) and so
> have rather done it with the streaming script.)
>
>
> On 10 March 2015 at 14:58, Fabio Ricci <fa...@semweb.ch> wrote:
>> Dear community
>>
>> does anyone know how to query RDF Data with SPARQL in huge graphs using Jena
>> so, that a portion of rdf triples is processed - say 1000 triples at a time
>> ?
>>
>> I need this in order to INDEX triples in a search engine, need therefore to
>> access every triple but not all at a time ...
>>
>> Thank you in advance
>> Regards
>> Fabio
>
>
Re: Apache Jena for Android
Posted by Stian Soiland-Reyes <st...@apache.org>.
SPARQL results are HTTP-streamed back from Fuseki (and in TDB
programmatically), as long as you don't use ORDER and select a
streamable serialization format like JSON (not JSON-LD)
That is exactly what I do in
https://github.com/openphacts/openphacts-irs
to populate an ElasticSearch instance, by generating SPARQL queries to select
certain properties, then iterating
over the JSON response in a streaming way.
As I register one JSON document in ES per SPARQL result (or more
correctly, per subject URI), then I use the bulk loading feature of ES
to chunk of 1000 in one HTTP request to ES.
I don't know which search engine you have, but I guess you could do
something similar.
If you do it through TDBs API then you should not need to worry about
the serialization as you can just use the Iterator normally.
Note that Jena also has its own text-search integration with Solr
and/or ElasticSearch which could be of interest -
https://jena.apache.org/documentation/query/text-query.html
(in my case I need to customize lots of ElasticSearch indexes (and
need the option to populate from Virtuoso instead of Jena) and so
have rather done it with the streaming script.)
On 10 March 2015 at 14:58, Fabio Ricci <fa...@semweb.ch> wrote:
> Dear community
>
> does anyone know how to query RDF Data with SPARQL in huge graphs using Jena
> so, that a portion of rdf triples is processed - say 1000 triples at a time
> ?
>
> I need this in order to INDEX triples in a search engine, need therefore to
> access every triple but not all at a time ...
>
> Thank you in advance
> Regards
> Fabio
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718
Re: Apache Jena for Android
Posted by Fabio Ricci <fa...@semweb.ch>.
Dear community
does anyone know how to query RDF Data with SPARQL in huge graphs using
Jena so, that a portion of rdf triples is processed - say 1000 triples
at a time ?
I need this in order to INDEX triples in a search engine, need therefore
to access every triple but not all at a time ...
Thank you in advance
Regards
Fabio
Re: Apache Jena for Android
Posted by Stian Soiland-Reyes <st...@apache.org>.
The JSON-LD support through JSON-LD-Java also uses HTTPClient (to load
@contexts), but it can be statically overridden, if needed:
https://github.com/jsonld-java/jsonld-java#customizing-the-apache-httpclient
https://github.com/jsonld-java/jsonld-java/blob/master/core/src/main/java/com/github/jsonldjava/core/DocumentLoader.java#L155
On 2 April 2015 at 12:03, Andy Seaborne <an...@apache.org> wrote:
> Hi Sören,
>
> Thanks for that assessment.
>
> Re: Xerces
> Jena uses Xerces for datatypes - java.xml stiff provides Java types to
> handle xml values but jena uses Xerces internals for literal parsing and for
> the knowledge contained in the derived types hierarchy. It's not an
> insignificant amount of work.
>
> This has come up before. Xerces isn't modular so we can't just pick out the
> dv bytes - we may be able to extract the source code and incorporate that
> way and hence not depend on Xerces for that. That's clearly a bit uck but I
> don't see another way to have the datatype support.
>
> I've long had a wish that there was a separate XSD library for datatypes,
> separate from XML processing, but the body of knowledge captured by teh
> Xerces work is pretty huge so the effort needed to get it right (and fast)
> is quite daunting.
>
> Of all the syntax forms supported, TriX is not one I personally worry about
> much :-)
>
> Re: httpclient
>
> Trackign back ...
> https://issues.apache.org/jira/browse/JENA-576
>
> Solr4j depends on httpclient 4.2 or did at the time someone last looked.
>
> Jena's use of httpclient is fairly contained - having the authenticator
> stuff work is quite important though.
>
> Andy
>
>
> On 31/03/15 21:41, Sören Brunk wrote:
>>
>> Hi Andy,
>>
>> thanks for your feedback and sorry for taking so long to reply. It would
>> be great to get Android support into mainline Jena and I'm willing to
>> help. It might be necessary to build an extra jar package for Android
>> though, similar to what jena-osgi does.
>>>
>>>
>>> Is there are anything the Jena project can do that would make the
>>> conversion? There may be things that Jena does, or the way is it
>>> packaged, that are inconvenient for you but really make no differnce
>>> to the project - sometimes things are just the way they are because it
>>> was done that way but could easily be done another way. If you have
>>> any such points, do email thsilist or raise a JIRA.
>>
>>
>> The httpclient issue might resolve itself as soon as Jena is able to
>> switch to httpclient 4.3. I've looked into the javax.xml issues in more
>> detail and the good news is that all the xsd datatype classes are
>> actually there. What's missing are the StAX classes (everything in
>> java.xml.stream) which seem to be used in ARQ and core (for SPARQL XML
>> result sets, RDF/XML and TriX) and of course in Xerces. That means it's
>> enough to repackage the StAX classes from xml-apis and to modify
>> jena-arq, jena-core and xercesImpl. I've changed my build accordingly.
>>
>> Android uses XmlPullParser for stream parsing. So another solution could
>> be to replace StAX with a XmlPull based parsing but I guess that would
>> take some effort.
>>>
>>>
>>> On Android, how does TDB work well? TDB uses fairly traditional file
>>> handling for 32 bit machines - "direct mode" - and uses memory mapped
>>> I/O for 64 bit machines - "mapped mode" - if it can detect the mode
>>> correctly. Detection is not subtle, it looks in system property
>>> "java.vm.info" and default to 32 bit. In mapped mode, it does try to
>>> use as much memory as possible which is not friendly to co-resident
>>> apps (you can force "direct" mode programmatically).
>>
>>
>> Since most Android devices nowadays run on a 32 bit ARM architecture I
>> guess it will run almost always in direct mode.
>> But I realized that TDB isn't working at all at the moment, because it
>> depends on JDK classes that aren't available on Android for things like
>> getting the PID. I think it is possible to replace those calls by using
>> similar classes provided by the Android SDK. I will try to get it
>> working. A dependency on Android specific code would probably make
>> mainline integration more difficult though.
>>
>> Sören
>>
>
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718
Re: Apache Jena for Android
Posted by Andy Seaborne <an...@apache.org>.
Hi Sören,
Thanks for that assessment.
Re: Xerces
Jena uses Xerces for datatypes - java.xml stiff provides Java types to
handle xml values but jena uses Xerces internals for literal parsing and
for the knowledge contained in the derived types hierarchy. It's not an
insignificant amount of work.
This has come up before. Xerces isn't modular so we can't just pick out
the dv bytes - we may be able to extract the source code and incorporate
that way and hence not depend on Xerces for that. That's clearly a bit
uck but I don't see another way to have the datatype support.
I've long had a wish that there was a separate XSD library for
datatypes, separate from XML processing, but the body of knowledge
captured by teh Xerces work is pretty huge so the effort needed to get
it right (and fast) is quite daunting.
Of all the syntax forms supported, TriX is not one I personally worry
about much :-)
Re: httpclient
Trackign back ...
https://issues.apache.org/jira/browse/JENA-576
Solr4j depends on httpclient 4.2 or did at the time someone last looked.
Jena's use of httpclient is fairly contained - having the authenticator
stuff work is quite important though.
Andy
On 31/03/15 21:41, Sören Brunk wrote:
> Hi Andy,
>
> thanks for your feedback and sorry for taking so long to reply. It would
> be great to get Android support into mainline Jena and I'm willing to
> help. It might be necessary to build an extra jar package for Android
> though, similar to what jena-osgi does.
>>
>> Is there are anything the Jena project can do that would make the
>> conversion? There may be things that Jena does, or the way is it
>> packaged, that are inconvenient for you but really make no differnce
>> to the project - sometimes things are just the way they are because it
>> was done that way but could easily be done another way. If you have
>> any such points, do email thsilist or raise a JIRA.
>
> The httpclient issue might resolve itself as soon as Jena is able to
> switch to httpclient 4.3. I've looked into the javax.xml issues in more
> detail and the good news is that all the xsd datatype classes are
> actually there. What's missing are the StAX classes (everything in
> java.xml.stream) which seem to be used in ARQ and core (for SPARQL XML
> result sets, RDF/XML and TriX) and of course in Xerces. That means it's
> enough to repackage the StAX classes from xml-apis and to modify
> jena-arq, jena-core and xercesImpl. I've changed my build accordingly.
>
> Android uses XmlPullParser for stream parsing. So another solution could
> be to replace StAX with a XmlPull based parsing but I guess that would
> take some effort.
>>
>> On Android, how does TDB work well? TDB uses fairly traditional file
>> handling for 32 bit machines - "direct mode" - and uses memory mapped
>> I/O for 64 bit machines - "mapped mode" - if it can detect the mode
>> correctly. Detection is not subtle, it looks in system property
>> "java.vm.info" and default to 32 bit. In mapped mode, it does try to
>> use as much memory as possible which is not friendly to co-resident
>> apps (you can force "direct" mode programmatically).
>
> Since most Android devices nowadays run on a 32 bit ARM architecture I
> guess it will run almost always in direct mode.
> But I realized that TDB isn't working at all at the moment, because it
> depends on JDK classes that aren't available on Android for things like
> getting the PID. I think it is possible to replace those calls by using
> similar classes provided by the Android SDK. I will try to get it
> working. A dependency on Android specific code would probably make
> mainline integration more difficult though.
>
> Sören
>
Re: Apache Jena for Android
Posted by Rob Vesse <rv...@dotnetrdf.org>.
Soren
It is also worth noting that ProcessUtils is only used to provide the
dataset locking functionality which helps prevents databases being used
from multiple JVMs at a time which may be a non-issue on Android since how
likely is is that more than one app will try and use the same TDB database?
If this isn't an issue a simpler alternative would just be to disable this
feature on Android via the setting:
SystemTDB.DiskLocationMultiJvmUsagePrevention = false;
Rob
On 20/04/2015 03:57, "Andy Seaborne" <an...@apache.org> wrote:
>On 15/04/15 14:49, Sören Brunk wrote:
>> Thanks for your suggestions. I got TDB running now. It only required a
>> small change in ProcessUtils to get the process id. This adds a
>> dependency on the Android sdk though, but it is possible to add a maven
>> dependency on the sdk with "provided" scope so I think it's feasible.
>>
>> According to the log TDB runs in direct mode on my test device as well
>> as x68_64 emulator so perhaps detection doesn't work on right on Android
>> yet. What would be a good way to test performance?
>>
>> I think a JIRA is more appropriate for further discussion so I created
>> one: https://issues.apache.org/jira/browse/JENA-914
>>
>> Cheers,
>> Sören
>
>
>Hi Sören,
>
>Detection is "unsubtle" - it looks for "64" in system property
>"java.vm.info" (see SystemTDB.determineIf64Bit()). Adding a "isAndroid"
>flag would be good - what's the best way to detect Android that does not
>rely on Android code?
>
>It might be better to run in direct mode on Android anyway as mapped
>mode can end up competing for memory resource from other processes while
>direct is more controllable. It's a server-style approach where any and
>all available free RAM is used for the database.
>
>In terms of performance, for smallish datasets, or at least
>small-RAM-footprint, the performance might be similar. Mapped mode (on
>Linux) comes into it's own when queries are repeatedly doing large index
>access (e.g. heavy FILTERing).
>
> Andy
>
>
Re: Apache Jena for Android
Posted by Andy Seaborne <an...@apache.org>.
On 15/04/15 14:49, Sören Brunk wrote:
> Thanks for your suggestions. I got TDB running now. It only required a
> small change in ProcessUtils to get the process id. This adds a
> dependency on the Android sdk though, but it is possible to add a maven
> dependency on the sdk with "provided" scope so I think it's feasible.
>
> According to the log TDB runs in direct mode on my test device as well
> as x68_64 emulator so perhaps detection doesn't work on right on Android
> yet. What would be a good way to test performance?
>
> I think a JIRA is more appropriate for further discussion so I created
> one: https://issues.apache.org/jira/browse/JENA-914
>
> Cheers,
> Sören
Hi Sören,
Detection is "unsubtle" - it looks for "64" in system property
"java.vm.info" (see SystemTDB.determineIf64Bit()). Adding a "isAndroid"
flag would be good - what's the best way to detect Android that does not
rely on Android code?
It might be better to run in direct mode on Android anyway as mapped
mode can end up competing for memory resource from other processes while
direct is more controllable. It's a server-style approach where any and
all available free RAM is used for the database.
In terms of performance, for smallish datasets, or at least
small-RAM-footprint, the performance might be similar. Mapped mode (on
Linux) comes into it's own when queries are repeatedly doing large index
access (e.g. heavy FILTERing).
Andy
Re: Apache Jena for Android
Posted by Sören Brunk <so...@tu-dresden.de>.
Thanks for your suggestions. I got TDB running now. It only required a
small change in ProcessUtils to get the process id. This adds a
dependency on the Android sdk though, but it is possible to add a maven
dependency on the sdk with "provided" scope so I think it's feasible.
According to the log TDB runs in direct mode on my test device as well
as x68_64 emulator so perhaps detection doesn't work on right on Android
yet. What would be a good way to test performance?
I think a JIRA is more appropriate for further discussion so I created
one: https://issues.apache.org/jira/browse/JENA-914
Cheers,
Sören
On 02.04.2015 09:47, Andy Seaborne wrote:
> You can configure the footprint of TDB in 32 bit "direct" mode for
> smaller datasets:
>
> http://jena.staging.apache.org/documentation/tdb/store-parameters.html
>
> Andy
>
> On 01/04/15 22:38, Stian Soiland-Reyes wrote:
>> +1 - and how well does the memory mapping of TDB work on Dalvik?
>>
>>
>> Would jena-sdb work with SQLite?
>>
>> https://developer.android.com/guide/topics/data/data-storage.html#db
>>
>> You would probably need https://github.com/SQLDroid/SQLDroid so you
>> get a JDBC driver.
>>
>>
>> On 1 April 2015 at 18:02, Claude Warren <cl...@xenei.com> wrote:
>>> Might it make sense to use a different persistence engine on Android
>>> since
>>> it probably will not have the huge datasets that TDB is designed
>>> for. It
>>> might make sense to have a small store available for small devices.
>>> Just a
>>> thought.
>>>
>>> Claude
>>>
>>> On Tue, Mar 31, 2015 at 9:41 PM, Sören Brunk
>>> <so...@tu-dresden.de>
>>> wrote:
>>>
>>>> Hi Andy,
>>>>
>>>> thanks for your feedback and sorry for taking so long to reply. It
>>>> would
>>>> be great to get Android support into mainline Jena and I'm willing
>>>> to help.
>>>> It might be necessary to build an extra jar package for Android
>>>> though,
>>>> similar to what jena-osgi does.
>>>>
>>>>>
>>>>> Is there are anything the Jena project can do that would make the
>>>>> conversion? There may be things that Jena does, or the way is it
>>>>> packaged,
>>>>> that are inconvenient for you but really make no differnce to the
>>>>> project -
>>>>> sometimes things are just the way they are because it was done
>>>>> that way but
>>>>> could easily be done another way. If you have any such points, do
>>>>> email
>>>>> thsilist or raise a JIRA.
>>>>>
>>>>
>>>> The httpclient issue might resolve itself as soon as Jena is able to
>>>> switch to httpclient 4.3. I've looked into the javax.xml issues in
>>>> more
>>>> detail and the good news is that all the xsd datatype classes are
>>>> actually
>>>> there. What's missing are the StAX classes (everything in
>>>> java.xml.stream)
>>>> which seem to be used in ARQ and core (for SPARQL XML result sets,
>>>> RDF/XML
>>>> and TriX) and of course in Xerces. That means it's enough to
>>>> repackage the
>>>> StAX classes from xml-apis and to modify jena-arq, jena-core and
>>>> xercesImpl. I've changed my build accordingly.
>>>>
>>>> Android uses XmlPullParser for stream parsing. So another solution
>>>> could
>>>> be to replace StAX with a XmlPull based parsing but I guess that
>>>> would take
>>>> some effort.
>>>>
>>>>>
>>>>> On Android, how does TDB work well? TDB uses fairly traditional file
>>>>> handling for 32 bit machines - "direct mode" - and uses memory
>>>>> mapped I/O
>>>>> for 64 bit machines - "mapped mode" - if it can detect the mode
>>>>> correctly.
>>>>> Detection is not subtle, it looks in system property
>>>>> "java.vm.info" and
>>>>> default to 32 bit. In mapped mode, it does try to use as much
>>>>> memory as
>>>>> possible which is not friendly to co-resident apps (you can force
>>>>> "direct"
>>>>> mode programmatically).
>>>>>
>>>>
>>>> Since most Android devices nowadays run on a 32 bit ARM architecture I
>>>> guess it will run almost always in direct mode.
>>>> But I realized that TDB isn't working at all at the moment, because it
>>>> depends on JDK classes that aren't available on Android for things
>>>> like
>>>> getting the PID. I think it is possible to replace those calls by
>>>> using
>>>> similar classes provided by the Android SDK. I will try to get it
>>>> working.
>>>> A dependency on Android specific code would probably make mainline
>>>> integration more difficult though.
>>>>
>>>> Sören
>>>>
>>>>
Re: Apache Jena for Android
Posted by Andy Seaborne <an...@apache.org>.
You can configure the footprint of TDB in 32 bit "direct" mode for
smaller datasets:
http://jena.staging.apache.org/documentation/tdb/store-parameters.html
Andy
On 01/04/15 22:38, Stian Soiland-Reyes wrote:
> +1 - and how well does the memory mapping of TDB work on Dalvik?
>
>
> Would jena-sdb work with SQLite?
>
> https://developer.android.com/guide/topics/data/data-storage.html#db
>
> You would probably need https://github.com/SQLDroid/SQLDroid so you
> get a JDBC driver.
>
>
> On 1 April 2015 at 18:02, Claude Warren <cl...@xenei.com> wrote:
>> Might it make sense to use a different persistence engine on Android since
>> it probably will not have the huge datasets that TDB is designed for. It
>> might make sense to have a small store available for small devices. Just a
>> thought.
>>
>> Claude
>>
>> On Tue, Mar 31, 2015 at 9:41 PM, Sören Brunk <so...@tu-dresden.de>
>> wrote:
>>
>>> Hi Andy,
>>>
>>> thanks for your feedback and sorry for taking so long to reply. It would
>>> be great to get Android support into mainline Jena and I'm willing to help.
>>> It might be necessary to build an extra jar package for Android though,
>>> similar to what jena-osgi does.
>>>
>>>>
>>>> Is there are anything the Jena project can do that would make the
>>>> conversion? There may be things that Jena does, or the way is it packaged,
>>>> that are inconvenient for you but really make no differnce to the project -
>>>> sometimes things are just the way they are because it was done that way but
>>>> could easily be done another way. If you have any such points, do email
>>>> thsilist or raise a JIRA.
>>>>
>>>
>>> The httpclient issue might resolve itself as soon as Jena is able to
>>> switch to httpclient 4.3. I've looked into the javax.xml issues in more
>>> detail and the good news is that all the xsd datatype classes are actually
>>> there. What's missing are the StAX classes (everything in java.xml.stream)
>>> which seem to be used in ARQ and core (for SPARQL XML result sets, RDF/XML
>>> and TriX) and of course in Xerces. That means it's enough to repackage the
>>> StAX classes from xml-apis and to modify jena-arq, jena-core and
>>> xercesImpl. I've changed my build accordingly.
>>>
>>> Android uses XmlPullParser for stream parsing. So another solution could
>>> be to replace StAX with a XmlPull based parsing but I guess that would take
>>> some effort.
>>>
>>>>
>>>> On Android, how does TDB work well? TDB uses fairly traditional file
>>>> handling for 32 bit machines - "direct mode" - and uses memory mapped I/O
>>>> for 64 bit machines - "mapped mode" - if it can detect the mode correctly.
>>>> Detection is not subtle, it looks in system property "java.vm.info" and
>>>> default to 32 bit. In mapped mode, it does try to use as much memory as
>>>> possible which is not friendly to co-resident apps (you can force "direct"
>>>> mode programmatically).
>>>>
>>>
>>> Since most Android devices nowadays run on a 32 bit ARM architecture I
>>> guess it will run almost always in direct mode.
>>> But I realized that TDB isn't working at all at the moment, because it
>>> depends on JDK classes that aren't available on Android for things like
>>> getting the PID. I think it is possible to replace those calls by using
>>> similar classes provided by the Android SDK. I will try to get it working.
>>> A dependency on Android specific code would probably make mainline
>>> integration more difficult though.
>>>
>>> Sören
>>>
>>> --
>>> Dipl. Inf. Sören Brunk
>>> Research Associate
>>>
>>> Technische Universität Dresden
>>> Faculty of Computer Science
>>> Institute for Software- and Multimedia-Technology
>>> Junior Professorship in Software Engineering of Ubiquitous Systems
>>> 01062 Dresden
>>>
>>>
>>
>>
>> --
>> I like: Like Like - The likeliest place on the web
>> <http://like-like.xenei.com>
>> LinkedIn: http://www.linkedin.com/in/claudewarren
>
>
>
Re: Apache Jena for Android
Posted by Stian Soiland-Reyes <st...@apache.org>.
+1 - and how well does the memory mapping of TDB work on Dalvik?
Would jena-sdb work with SQLite?
https://developer.android.com/guide/topics/data/data-storage.html#db
You would probably need https://github.com/SQLDroid/SQLDroid so you
get a JDBC driver.
On 1 April 2015 at 18:02, Claude Warren <cl...@xenei.com> wrote:
> Might it make sense to use a different persistence engine on Android since
> it probably will not have the huge datasets that TDB is designed for. It
> might make sense to have a small store available for small devices. Just a
> thought.
>
> Claude
>
> On Tue, Mar 31, 2015 at 9:41 PM, Sören Brunk <so...@tu-dresden.de>
> wrote:
>
>> Hi Andy,
>>
>> thanks for your feedback and sorry for taking so long to reply. It would
>> be great to get Android support into mainline Jena and I'm willing to help.
>> It might be necessary to build an extra jar package for Android though,
>> similar to what jena-osgi does.
>>
>>>
>>> Is there are anything the Jena project can do that would make the
>>> conversion? There may be things that Jena does, or the way is it packaged,
>>> that are inconvenient for you but really make no differnce to the project -
>>> sometimes things are just the way they are because it was done that way but
>>> could easily be done another way. If you have any such points, do email
>>> thsilist or raise a JIRA.
>>>
>>
>> The httpclient issue might resolve itself as soon as Jena is able to
>> switch to httpclient 4.3. I've looked into the javax.xml issues in more
>> detail and the good news is that all the xsd datatype classes are actually
>> there. What's missing are the StAX classes (everything in java.xml.stream)
>> which seem to be used in ARQ and core (for SPARQL XML result sets, RDF/XML
>> and TriX) and of course in Xerces. That means it's enough to repackage the
>> StAX classes from xml-apis and to modify jena-arq, jena-core and
>> xercesImpl. I've changed my build accordingly.
>>
>> Android uses XmlPullParser for stream parsing. So another solution could
>> be to replace StAX with a XmlPull based parsing but I guess that would take
>> some effort.
>>
>>>
>>> On Android, how does TDB work well? TDB uses fairly traditional file
>>> handling for 32 bit machines - "direct mode" - and uses memory mapped I/O
>>> for 64 bit machines - "mapped mode" - if it can detect the mode correctly.
>>> Detection is not subtle, it looks in system property "java.vm.info" and
>>> default to 32 bit. In mapped mode, it does try to use as much memory as
>>> possible which is not friendly to co-resident apps (you can force "direct"
>>> mode programmatically).
>>>
>>
>> Since most Android devices nowadays run on a 32 bit ARM architecture I
>> guess it will run almost always in direct mode.
>> But I realized that TDB isn't working at all at the moment, because it
>> depends on JDK classes that aren't available on Android for things like
>> getting the PID. I think it is possible to replace those calls by using
>> similar classes provided by the Android SDK. I will try to get it working.
>> A dependency on Android specific code would probably make mainline
>> integration more difficult though.
>>
>> Sören
>>
>> --
>> Dipl. Inf. Sören Brunk
>> Research Associate
>>
>> Technische Universität Dresden
>> Faculty of Computer Science
>> Institute for Software- and Multimedia-Technology
>> Junior Professorship in Software Engineering of Ubiquitous Systems
>> 01062 Dresden
>>
>>
>
>
> --
> I like: Like Like - The likeliest place on the web
> <http://like-like.xenei.com>
> LinkedIn: http://www.linkedin.com/in/claudewarren
--
Stian Soiland-Reyes
Apache Taverna (incubating), Apache Commons RDF (incubating)
http://orcid.org/0000-0001-9842-9718
Re: Apache Jena for Android
Posted by Claude Warren <cl...@xenei.com>.
Might it make sense to use a different persistence engine on Android since
it probably will not have the huge datasets that TDB is designed for. It
might make sense to have a small store available for small devices. Just a
thought.
Claude
On Tue, Mar 31, 2015 at 9:41 PM, Sören Brunk <so...@tu-dresden.de>
wrote:
> Hi Andy,
>
> thanks for your feedback and sorry for taking so long to reply. It would
> be great to get Android support into mainline Jena and I'm willing to help.
> It might be necessary to build an extra jar package for Android though,
> similar to what jena-osgi does.
>
>>
>> Is there are anything the Jena project can do that would make the
>> conversion? There may be things that Jena does, or the way is it packaged,
>> that are inconvenient for you but really make no differnce to the project -
>> sometimes things are just the way they are because it was done that way but
>> could easily be done another way. If you have any such points, do email
>> thsilist or raise a JIRA.
>>
>
> The httpclient issue might resolve itself as soon as Jena is able to
> switch to httpclient 4.3. I've looked into the javax.xml issues in more
> detail and the good news is that all the xsd datatype classes are actually
> there. What's missing are the StAX classes (everything in java.xml.stream)
> which seem to be used in ARQ and core (for SPARQL XML result sets, RDF/XML
> and TriX) and of course in Xerces. That means it's enough to repackage the
> StAX classes from xml-apis and to modify jena-arq, jena-core and
> xercesImpl. I've changed my build accordingly.
>
> Android uses XmlPullParser for stream parsing. So another solution could
> be to replace StAX with a XmlPull based parsing but I guess that would take
> some effort.
>
>>
>> On Android, how does TDB work well? TDB uses fairly traditional file
>> handling for 32 bit machines - "direct mode" - and uses memory mapped I/O
>> for 64 bit machines - "mapped mode" - if it can detect the mode correctly.
>> Detection is not subtle, it looks in system property "java.vm.info" and
>> default to 32 bit. In mapped mode, it does try to use as much memory as
>> possible which is not friendly to co-resident apps (you can force "direct"
>> mode programmatically).
>>
>
> Since most Android devices nowadays run on a 32 bit ARM architecture I
> guess it will run almost always in direct mode.
> But I realized that TDB isn't working at all at the moment, because it
> depends on JDK classes that aren't available on Android for things like
> getting the PID. I think it is possible to replace those calls by using
> similar classes provided by the Android SDK. I will try to get it working.
> A dependency on Android specific code would probably make mainline
> integration more difficult though.
>
> Sören
>
> --
> Dipl. Inf. Sören Brunk
> Research Associate
>
> Technische Universität Dresden
> Faculty of Computer Science
> Institute for Software- and Multimedia-Technology
> Junior Professorship in Software Engineering of Ubiquitous Systems
> 01062 Dresden
>
>
--
I like: Like Like - The likeliest place on the web
<http://like-like.xenei.com>
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: Apache Jena for Android
Posted by Sören Brunk <so...@tu-dresden.de>.
Hi Andy,
thanks for your feedback and sorry for taking so long to reply. It would
be great to get Android support into mainline Jena and I'm willing to
help. It might be necessary to build an extra jar package for Android
though, similar to what jena-osgi does.
>
> Is there are anything the Jena project can do that would make the
> conversion? There may be things that Jena does, or the way is it
> packaged, that are inconvenient for you but really make no differnce
> to the project - sometimes things are just the way they are because it
> was done that way but could easily be done another way. If you have
> any such points, do email thsilist or raise a JIRA.
The httpclient issue might resolve itself as soon as Jena is able to
switch to httpclient 4.3. I've looked into the javax.xml issues in more
detail and the good news is that all the xsd datatype classes are
actually there. What's missing are the StAX classes (everything in
java.xml.stream) which seem to be used in ARQ and core (for SPARQL XML
result sets, RDF/XML and TriX) and of course in Xerces. That means it's
enough to repackage the StAX classes from xml-apis and to modify
jena-arq, jena-core and xercesImpl. I've changed my build accordingly.
Android uses XmlPullParser for stream parsing. So another solution could
be to replace StAX with a XmlPull based parsing but I guess that would
take some effort.
>
> On Android, how does TDB work well? TDB uses fairly traditional file
> handling for 32 bit machines - "direct mode" - and uses memory mapped
> I/O for 64 bit machines - "mapped mode" - if it can detect the mode
> correctly. Detection is not subtle, it looks in system property
> "java.vm.info" and default to 32 bit. In mapped mode, it does try to
> use as much memory as possible which is not friendly to co-resident
> apps (you can force "direct" mode programmatically).
Since most Android devices nowadays run on a 32 bit ARM architecture I
guess it will run almost always in direct mode.
But I realized that TDB isn't working at all at the moment, because it
depends on JDK classes that aren't available on Android for things like
getting the PID. I think it is possible to replace those calls by using
similar classes provided by the Android SDK. I will try to get it
working. A dependency on Android specific code would probably make
mainline integration more difficult though.
Sören
--
Dipl. Inf. Sören Brunk
Research Associate
Technische Universität Dresden
Faculty of Computer Science
Institute for Software- and Multimedia-Technology
Junior Professorship in Software Engineering of Ubiquitous Systems
01062 Dresden
Re: Apache Jena for Android
Posted by Andy Seaborne <an...@apache.org>.
Hi Sören,
Interesting approach - having something that will be able to track more
easily the evolution of the Jena codebase is great to see. There seem
to have been an increase in questions on StackOverflow and on
answer.semanticweb about Jena on Android over the last few months so
great timing.
Is there are anything the Jena project can do that would make the
conversion? There may be things that Jena does, or the way is it
packaged, that are inconvenient for you but really make no differnce to
the project - sometimes things are just the way they are because it was
done that way but could easily be done another way. If you have any
such points, do email thsilist or raise a JIRA.
On Android, how does TDB work well? TDB uses fairly traditional file
handling for 32 bit machines - "direct mode" - and uses memory mapped
I/O for 64 bit machines - "mapped mode" - if it can detect the mode
correctly. Detection is not subtle, it looks in system property
"java.vm.info" and default to 32 bit. In mapped mode, it does try to
use as much memory as possible which is not friendly to co-resident apps
(you can force "direct" mode programmatically).
Andy
On 03/03/15 13:28, Sören Brunk wrote:
> Hi all,
>
> We are working on some Android app prototypes that deal with RDF data.
> To make things easier, we have created a version of Apache Jena that
> runs on Android. It basically consists of a bunch of Maven build files
> that do some repackaging of the original Jena jars in order to make them
> run on Android's Dalvik VM. As the port might be interesting to others,
> we have published it to Github for everyone to use.
>
> From the project description:
> This project aims to make the Apache Jena Framework usable on Android.
> While Jena is written in pure Java, it can't be used on Android as is,
> due to multiple package conflict issues. To overcome these issues, we
> have created an Android port enabling developers to use Apache Jena in
> Android apps. In contrast to other ports we don't just publish binaries,
> but make the Maven build files of the port available. We also stay as
> close as possible to the original (i.e. no source code changes). This
> should make it easier to stay in sync with upcoming Jena releases.
>
> The project, including usage instructions, is available here:
> https://github.com/seus-inf/jena-android
>
> See here for an older discussion about the issues:
> http://mail-archives.apache.org/mod_mbox/jena-dev/201304.mbox/%3C5163E9A0.2030307%40apache.org%3E
>
>
> Best regards,
> Sören
>