Hi,<div><br></div><div>Magritte 3 is now available for Gemstone - with tests green. The configuration has been updated and I loaded Magritte 3 with:</div><div><br></div><div><div>Gofer it</div><div><span class="Apple-tab-span" style="white-space:pre">	</span>squeaksource: &#39;MetacelloRepository&#39;;</div>
<div><span class="Apple-tab-span" style="white-space:pre">	</span>package: &#39;ConfigurationOfMagritte3&#39;;</div><div><span class="Apple-tab-span" style="white-space:pre">	</span>load.</div></div><div><br></div><div><div>
MCPlatformSupport commitOnAlmostOutOfMemoryDuring: [</div><div><span class="Apple-tab-span" style="white-space:pre">	</span>ConfigurationOfMagritte3 project stableVersion load: #(&#39;Seaside Tests&#39;) ].</div></div><div>
<br></div><div><br></div><div>However the tests run extremely slowly - in the test suite I&#39;ve measured 20x slowdown between Magritte 2 and Magritte 3 - making the tests a pain to execute.</div><div><br></div><div>I&#39;ve run a series of tests to characterise where the problem might be. The tests are as follows:</div>
<div><br></div><div>-----</div><div>1) Test-suite benchmark:</div><div><div>| testSuite |</div><div>testSuite := (OBClassCategoryNode on: &#39;Magritte-Tests-Model-Description&#39;) classes</div><div><span class="Apple-tab-span" style="white-space:pre">		</span>inject: TestSuite new</div>
<div><span class="Apple-tab-span" style="white-space:pre">		</span>into: [:suite :classNode | </div><div><span class="Apple-tab-span" style="white-space:pre">				</span>(classNode theClass inheritsFrom: TestCase)</div><div>
<span class="Apple-tab-span" style="white-space:pre">						</span>ifTrue: [suite addTests: classNode theClass suite tests].</div><div><span class="Apple-tab-span" style="white-space:pre">				</span>suite].</div><div><span class="Apple-tab-span" style="white-space:pre">			</span></div>
<div>Time millisecondsToRun:  [ testSuite run ].</div></div><div><br></div><div>2) Description benchmark:</div><div><p style="margin:0px 0px 0px 0px;font:12.0px Helvetica;min-height:14.0px"><br>Magritte 2:</p><p style="margin:0px 0px 0px 0px;font:12.0px Helvetica;min-height:14.0px">
</p><p style="margin:0px 0px 0px 0px;font:12.0px Helvetica;min-height:14.0px">Time millisecondsToRun: </p><p style="margin:0px 0px 0px 0px;font:12.0px Helvetica;min-height:14.0px"><span class="Apple-tab-span" style="white-space:pre">	</span>   [100 timesRepeat: [ MABooleanDescription  description ] ].  </p>
<p></p><p style="margin:0px 0px 0px 0px;font:12.0px Helvetica;min-height:14.0px"><br></p><p style="margin:0px 0px 0px 0px;font:12.0px Helvetica;min-height:14.0px">Magritte 3:</p>
</div><div><div>Time millisecondsToRun: </div><div><span class="Apple-tab-span" style="white-space:pre">	</span>   [100 timesRepeat: [ MABooleanDescription new magritteDescription ] ].</div></div><div><br></div><div>3) Micro-benchmark</div>
<div><br></div><div>Magritte 2:</div><div><div>Time millisecondsToRun: </div><div><span class="Apple-tab-span" style="white-space:pre">	</span>   [100 timesRepeat: [ MABooleanDescription class allSelectors select: [ :each | each isDescriptionSelector ] ] ].</div>
</div><div><br></div><div>Magritte 3:</div><div><div>Time millisecondsToRun: </div><div><span class="Apple-tab-span" style="white-space:pre">	</span>   [100 timesRepeat: [ Pragma allNamed: MAPragmaBuilder magritteDescriptionPragma from: MABooleanDescription to: Object ] ].</div>
</div><div><br></div><div>* The micro-benchmark is illustrative of the difference between Magritte 2 and Magritte 3 - Magritte 2 uses a naming convention to retrieve descriptions, whereas Magritte 3 uses Pragmas.</div><div>
<br></div><div>-----</div><div><br></div><div>Results:</div><div><br></div><div>Pharo Magritte 2: (1) 436 (2) 0 (3) 85</div><div>Pharo Magritte 3: (1) 743 (2) 58 (3) 12</div><div>Gemstone Magritte 2: (1) 988 (2) 30 (3) 2229</div>
<div>Gemstone Magritte 3: (1) 21663 (2) 1575 (3) 444</div><div><br></div><div>Cavet: Although Gemstone and Pharo are running on the same machine - Gemstone is running virtualised in a VMWare Linux installation, so the results are not directly comparable, although the 30x performance decrease between Pharo and Gemstone surprised me.</div>
<div><br></div><div>First the good news: </div><div>For Magritte the pragma description look-up is more performant than the named-based lookup (micro-benchmark (3)).</div><div><br></div><div>Now the bad news: </div><div>In Pharo Magritte 2 is faster then Magritte 3 when running a description lookup based test-suite containing 1416 tests. My feeling is that in the context of a real application performing a smaller number of lookups this is probably not significant.</div>
<div>However in Gemstone the performance degradation is more noticeable and if nothing else makes running the tests painful.</div><div><br></div><div>How to proceed:</div><div>Magritte 2&#39;s performance advantage is gained from caching the descriptions. However I think the benefits of having dynamic (non-caching) descriptions outweigh the disadvantages and I&#39;d be reluctant to add a generic layer of caching. In Pharo I don&#39;t think this is an issue. However in Gemstone I feel that these numbers need some further investigation. </div>
<div><br></div><div>Any thoughts how to proceed?</div><div><br></div><div>Nick</div>