[Moose-dev] Re: [Pharo-dev] Growing large images: the case of Moose models

Thanks Eliot for the implementation of the new compactor! Just to add a piece of information: the mse imported have a size of 40Mb. Cheers, Vincent De : Pharo-dev [mailto:pharo-dev-bounces@lists.pharo.org] De la part de Clément Bera Envoyé : vendredi 3 mars 2017 11:56 À : Discusses Development of Pharo Objet : [Pharo-dev] Growing large images: the case of Moose models Hello everyone, This morning I investigated with Vincent Blondeau a problem reported by the Moose community a while ago: loading Moose model is slower in Spur (Pharo 5+) than in pre-Spur (Pharo 4 and older). In general, this problem was present for anyone growing images to a significant size. To investigate the problem, we loaded a 200Mb[3] Moose model on a 250Mb image, growing the image to 450Mb. Loading such a model takes 2 minutes in Spur and 1m30s in pre-Spur VMs. Using the stable Pharo VM, the analysis results were the following: - total time spent to load the Model: 2 minutes - time spent in full GC: 1 minute (4 fullGCs) - time spent in scavenges[1]: 15 seconds On the 2 minutes spent, we have 50% of the time spent in full GCs, 12.5% in scavenges, 37.5% executing code. We then used the latest VM that features the new compactor (VM from beginning of March 2017 and over). The full GC execution time went down from 1 minute to 2 seconds. In addition, we increased the size of Eden[2] from 4Mb to 12Mb. Time spent in scavenges decreased from 15 seconds to 5 seconds. Overall, loading the model is now taking ~50 seconds instead of 2 minutes. To increase Eden size, one needs to run a script similar to: | currentEdenSize desiredEdenSize | currentEdenSize := Smalltalk vm parameterAt: 44. desiredEdenSize := currentEdenSize * 4. Smalltalk vm parameterAt: 45 put: desiredEdenSize. And then restart the image. I hope this report can be useful for some of you. I will try to make a blog post out of it, detailing other GC settings one can change from the image to improve performance. Best, Clement [1] A scavenge is basically the garbage collection of only young objects [2] Eden is basically the space where objects are initially allocated. [3] All numbers in the report are order of magnitudes and not precise numbers !!!************************************************************************************* "Ce message et les pièces jointes sont confidentiels et réservés à l'usage exclusif de ses destinataires. Il peut également être protégé par le secret professionnel. Si vous recevez ce message par erreur, merci d'en avertir immédiatement l'expéditeur et de le détruire. L'intégrité du message ne pouvant être assurée sur Internet, la responsabilité de Worldline ne pourra être recherchée quant au contenu de ce message. Bien que les meilleurs efforts soient faits pour maintenir cette transmission exempte de tout virus, l'expéditeur ne donne aucune garantie à cet égard et sa responsabilité ne saurait être recherchée pour tout dommage résultant d'un virus transmis. This e-mail and the documents attached are confidential and intended solely for the addressee; it may also be privileged. If you receive this e-mail in error, please notify the sender immediately and destroy it. As its integrity cannot be secured on the Internet, the Worldline liability cannot be triggered for the message content. Although the sender endeavours to maintain a computer virus-free network, the sender does not warrant that this transmission is virus-free and will not be liable for any damages resulting from any virus transmitted.!!!"