Performance question

["Kunnen, J. (Johan)" <jo...@tno.nl>]

Hi all,

I already had a look at the Performance Tuning section in the Marmotta Wiki, so I intend to setup PostgreSQL to see if I can gain some performance there. I am not really familiar with PostgreSQL so I need to investigate some time in it. Currently I am just testing with the default marmotta setup and I notice that it takes a really long time to import an owl file of say 80 MB. During the import my (Windows) machine has only two cores working at about 70%. It doesn't look as if it is very busy performing its task. After the import, the queries on the database still perform very well. So I wonder what is going on during an import. Can I somehow speed up this process by allocating more memory or processor capacity? Importing this owl file took about 8 hours. Is there any hope for me that I could do it within an hour by using postgreSQL?

Kind regards,
Johan Kunnen.



Dit bericht kan informatie bevatten die niet voor u is bestemd. Indien u niet de geadresseerde bent of dit bericht abusievelijk aan u is toegezonden, wordt u verzocht dat aan de afzender te melden en het bericht te verwijderen. TNO aanvaardt geen aansprakelijkheid voor de inhoud van deze e-mail, de wijze waarop u deze gebruikt en voor schade, van welke aard ook, die verband houdt met risico's verbonden aan het elektronisch verzenden van berichten.

 

This message may contain information that is not intended for you. If you are not the addressee or if this message was sent to you by mistake, you are requested to inform the sender and delete the message. TNO accepts no liability for the content of this e-mail, for the manner in which you use it and for damage of any kind resulting from the risks inherent to the electronic transmission of messages.

Re: Performance question

[Sergio Fernández <wi...@apache.org>]

Hi

On 15/09/14 17:29, Kunnen, J. (Johan) wrote:
> I already had a look at the Performance Tuning section in the Marmotta Wiki, so I intend to setup PostgreSQL to see if I can gain some performance there. I am not really familiar with PostgreSQL so I need to investigate some time in it. Currently I am just testing with the default marmotta setup and I notice that it takes a really long time to import an owl file of say 80 MB. During the import my (Windows) machine has only two cores working at about 70%. It doesn't look as if it is very busy performing its task. After the import, the queries on the database still perform very well. So I wonder what is going on during an import. Can I somehow speed up this process by allocating more memory or processor capacity? Importing this owl file took about 8 hours. Is there any hope for me that I could do it within an hour by using postgreSQL?

Some hints:

* H2, the default database, is not very performance
* The import takes more time on purpose, to later give more performance 
on query time
* For bulks importing it's recommended to use a native loader 
<http://marmotta.apache.org/kiwi/loader> where indexes are drop

Hope that helps.

-- 
Sergio Fernández
Partner Technology Manager
Redlink GmbH
m: +43 660 2747 925
e: sergio.fernandez@redlink.co
w: http://redlink.co