Content creation strategy

[Frédéric Esnault <fe...@legisway.com>]

Hi all !

 

I'm wondering what's the best strategy to create a lot of contents.

 

I tried creating 33 333 contents, one by one (random date generation, node/properties creation using data, session.save() ). The process stopped after 28000 contents had been created and saved, because hard drive was full. Here is the mySql values I saw in mySql manager :

 

mySql tables :

* default_node

  > rows : 35 820 969 !!!!!!

  > size : 22,5 GB !!!!!!

* default_prop

  > rows : 676 591

  > size : 177,1 MB

* default_refs

  > rows : 12002

  > size : 13,6 MB

 

I retried today, and after "only" 2 200" contents node had been created and saved, the default_node table was already more than 280 MB...

 

The thing is before I created 18 000 contents, but using another strategy : Creation of one third of the nodes ( 6 000), saving them, then 6 000 more, then 6000 more.

The result is quite ok, and the tables are quite correct. Today I reimported the 18 000 contents xml file and my mySql tables are like this :

*	default_node 

	*	rows : 56 787 
	*	size : 38.6 MB 

*	default_prop 

	*	rows : 389 841 
	*	size : 102 MB 

*	default_refs 

	*	rows : 12 054 
	*	size : 13.5 MB 

*	default_node 

	*	rows : 56 787 
	*	size : 38.6 MB 

*	default_prop 

	*	rows : 389 841 
	*	size : 102 MB 

*	default_refs 

	*	rows : 12 054 
	*	size : 13.5 MB 

 

I'm wondering why such a difference, what are those tables exactly, and of course, what is the best strategy of mass content creation (speaking of time and table size).

Frederic Esnault