[Moose-dev] Re: [Pharo-users] Re: Script of the day

On Jul 31, 2015, at 5:06 AM, Tudor Girba &lt;tudor(a)tudorgirba.com&gt; wrote: Hi, I use it quite often. The builder is Ok, but I would improve some things to make it elegant: 1. A treemap works well when there is a nested structure of one kind of nodes and the leaves are of a different kind. So, folder/file, packages/classes. Thus, having only one traversal will inevitably lead to an if inside. I would rather be interested in having a higher level constructs that allow us to state what is the nested structure, and what are the leaves. Perhaps like this: b buildFrom: aRootNestingObject nesting: [ :aNestingObject | aNestingObject childNestedObjects ] leaves: [ :aNestingObject | aNestingObject childLeaves] 2. The aggregation of metrics should be built in. Typically, you want the size of the nested structure to be deduced by the aggregate size of the inner leaves. Thus, ideally, there should be a way to specify the weight only for the leaves and let the engine do the rest of calculations. For example, in the case of the file system visualization, the weight block looks like weight: [ :each | each isFile ifTrue: [each size max: 1] ifFalse: [(each allChildren sumNumbers: #size) max: 1 ] ]. This means that I will traverse and compute the size of files needlessly multiple times for each of the parent folder. To make it faster, I would have to build a cache for the size of files and look it up in the block. This should be offered by the engine because it is the default use case. So, I should only say: b leafWeight: [:file | file size] 3. Talking about weights, if the weight is 0, the builder crashes. This should be made more robust. In my scripts, I always add a "max: 1", but this is clumsy. This should be built in. The same remarks apply for the RTTreeMapBuilder. Cheers, Doru > On Fri, Jul 31, 2015 at 12:38 AM, Alexandre Bergel &lt;alexandre.bergel(a)me.com&gt; wrote: > Excellent! > > By the way, you told me you use the treemap builder quite a lot. > This means you are happy about it right ? Do you have anything to change? > > Alexandre > > > > On Jul 30, 2015, at 5:35 PM, Tudor Girba &lt;tudor(a)tudorgirba.com&gt; wrote: > > > > Of course, it is. In fact, a treemap is a better suited representation for understanding the size of a file system. > > > > Here it is: > > http://ws.stfx.eu/C9RM2EURHKMI > > > > <file-treemap.png> > > > > Cheers, > > Doru > > > > On Thu, Jul 30, 2015 at 8:40 PM, H. Hirzel &lt;hannes.hirzel(a)gmail.com&gt; wrote: > > Is it possible to use a tree map? > > > > --Hannes > > > > On 7/30/15, Alexandre Bergel &lt;alexandre.bergel(a)me.com&gt; wrote: > > > I gave a try, and I did not find anything weird. > > > I guess your document folder contains many files, as Doru said > > > > > > Alexandre > > > -- > > > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > > > Alexandre Bergel http://www.bergel.eu > > > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > > > > > > > > >> On Jul 30, 2015, at 5:51 AM, Dimitris Chloupis &lt;kilon.alios(a)gmail.com&gt; > > >> wrote: > > >> > > >> Frankly I dont mind big images or big data , neither I share the obsession > > >> to shrink things down to few mbs in a time that we are talking in TBs . > > >> > > >> Anyway you asked for the image and this is a link to it > > >> > > >> https://www.dropbox.com/s/wh1071xgoo54fq7/1%20Ephestos%2027-05-15.zip?dl=0 > > >> <https://www.dropbox.com/s/wh1071xgoo54fq7/1%20Ephestos%2027-05-15.zip?dl=0> > > >> > > >> I dont think you will find anything abnormal about it though. > > >> > > >> I was wondering whether it would worth the effort beyond the unit tests > > >> that check for behaviour of the code to have also benchmark tests that > > >> will have to pass specific standards so that specific method must perform > > >> under a strict timetable specific tasks, this way CI may alert not only > > >> unit tests that fail but also benchmarks that fail , automagically. > > >> > > >> Maybe some food for thought. > > >> > > >> Personally I am far more worried how much CPU Pharo consumes than how much > > >> RAM. > > >> > > >> On Wed, Jul 29, 2015 at 11:54 PM Peter Uhnák &lt;i.uhnak(a)gmail.com > > >> <mailto:i.uhnak@gmail.com>> wrote: > > >> Not sure if there is any downside (I've been using it for a long time > > >> without problems), but image cleanup often does wonders for me. > > >> > > >> Moose image > > >> 410 MB => cleanup (World -> System -> Do Image Cleanup) => save => 237 > > >> MB > > >> and I even had situations where it went from ~600 MB to ~60 MB. > > >> (As to why my images are so massive... right now there are 19 windows > > >> opened inside most of them with opened roassal visualization... and I am > > >> saving like every five minutes... but that is just a guess.) > > >> > > >> But image cleanup is supposed to break things, so better to backup the > > >> folder first. > > >> Plus is cache really necessary? I mean it's a cache. .image + .changes > > >> should usually suffice. > > >> > > >> Peter > > >> > > >> On Wed, Jul 29, 2015 at 10:45 PM, Dimitris Chloupis &lt;kilon.alios(a)gmail.com > > >> <mailto:kilon.alios@gmail.com>> wrote: > > >> Tudor I dont know if you received my message but I already answered that > > >> with this: > > >> > > >> "ah ok then its normal. Yes the structure is extensive, its 13 folders > > >> (the rest 71 are files some of them very big), I would not be surprised if > > >> each contains 100 files in its own subfolders." > > >> > > >> > > >> > > >> On Wed, Jul 29, 2015 at 11:30 PM Tudor Girba &lt;tudor(a)tudorgirba.com > > >> <mailto:tudor@tudorgirba.com>> wrote: > > >> Hi, > > >> > > >> But, do you confirm that you have many files under the mentioned folder? > > >> > > >> Cheers, > > >> Doru > > >> > > >> > > >> > > >> On Wed, Jul 29, 2015 at 10:25 PM, Dimitris Chloupis &lt;kilon.alios(a)gmail.com > > >> <mailto:kilon.alios@gmail.com>> wrote: > > >> the image is 180 mb, changes are another 15, there are several other > > >> folders that raise the total size to 250 mb, monticello-cache and > > >> github-cache prodominately. > > >> > > >> Hmm ok my bad, the size of the zip is 89 MB not 200 mb. Still will take me > > >> 1-2 hours to upload on this connection. So if you guys still need the > > >> image i can upload tommorow from work, though it seems my slow down is > > >> normal. > > >> > > >> Afterall I am sure you can test it yourselves on complex file structures > > >> to see if it has delays as long as 20 minutes. > > >> > > >> > > >> > > >> On Wed, Jul 29, 2015 at 11:01 PM Sven Van Caekenberghe &lt;sven(a)stfx.eu > > >> <mailto:sven@stfx.eu>> wrote: > > >> > > >> > On 29 Jul 2015, at 21:33, Dimitris Chloupis &lt;kilon.alios(a)gmail.com > > >> > <mailto:kilon.alios@gmail.com>> wrote: > > >> > > > >> > well i can send you the image tommorow because its huge (200 MB ziped) > > >> > for my pathetically slow internet connection at home. I will send it > > >> > from work which is 8 times faster. > > >> > > >> If the image is 200 MB zipped, how large is it in its normal state ? > > >> > > >> For me, a 100 MB image file is already quite large, that would compress to > > >> maybe 20/30 MB. > > >> > > >> This doesn't sound right to me, unless you explicitly tried to store a lot > > >> of data. > > >> > > >> Maybe something else is wrong that would explain the slowness. > > >> > > >> > I can tell you that I am on MacOS Yosemite , I get my image with > > >> > pharolauncher, the image was dowloaded at 27-07-15 some of the 84 items > > >> > are folders that contain many sub folders. No idea if that matters. > > >> > > > >> > I got Roassal 2 using the Package Browser in Pharo 5 (the new > > >> > configuration browser). > > >> > > > >> > On Wed, Jul 29, 2015 at 8:17 PM Alexandre Bergel > > >> > &lt;alexandre.bergel(a)me.com <mailto:alexandre.bergel@me.com>> wrote: > > >> > Are you using windows? I know it may has problems regarding the source > > >> > code. You have 84 items and the visualization is slow? How is this > > >> > possible. > > >> > > > >> > Can I have a look at your image? > > >> > > > >> > Cheers, > > >> > Alexandre > > >> > -- > > >> > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > > >> > Alexandre Bergel http://www.bergel.eu <http://www.bergel.eu/> > > >> > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > >> > > > >> > > > >> > > > >> >> On Jul 29, 2015, at 12:26 PM, Dimitris Chloupis &lt;kilon.alios(a)gmail.com > > >> >> <mailto:kilon.alios@gmail.com>> wrote: > > >> >> > > >> >> thats ok it unfroze after 20 minutes or so . I managed to inspect the b > > >> >> but its incredible slow to navigate and use . Even moving the inspector > > >> >> around is incredible slow. My Documents folder contains 84 items, maybe > > >> >> it cant handle so well this amount of datea ? > > >> >> > > >> >> On Wed, Jul 29, 2015 at 5:55 PM Alexandre Bergel > > >> >> &lt;alexandre.bergel(a)me.com <mailto:alexandre.bergel@me.com>> wrote: > > >> >> Oh… sorry. Was not my intention > > >> >> > > >> >> Alexandre > > >> >> -- > > >> >> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > > >> >> Alexandre Bergel http://www.bergel.eu <http://www.bergel.eu/> > > >> >> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > >> >> > > >> >> > > >> >> > > >> >>> On Jul 29, 2015, at 11:24 AM, Dimitris Chloupis &lt;kilon.alios(a)gmail.com > > >> >>> <mailto:kilon.alios@gmail.com>> wrote: > > >> >>> > > >> >>> that was a nice way to freeze my image, thank you :D > > >> >>> > > >> >>> On Wed, Jul 29, 2015 at 4:27 PM Alexandre Bergel > > >> >>> &lt;alexandre.bergel(a)me.com <mailto:alexandre.bergel@me.com>> wrote: > > >> >>> Hi! > > >> >>> > > >> >>> Just to share a couple of experiment. I have tried the following in a > > >> >>> playground: > > >> >>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > > >> >>> b := RTMondrian new. > > >> >>> > > >> >>> b shape rectangle > > >> >>> if: [ :aFile | aFile path basename beginsWith: '.' ] color: Color > > >> >>> red. > > >> >>> b nodes: FileLocator documents allChildren. > > >> >>> b edges connectFrom: #parent. > > >> >>> b normalizer > > >> >>> normalizeSize: #size using: #sqrt. > > >> >>> b layout tree. > > >> >>> b > > >> >>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > > >> >>> > > >> >>> It shows a tree of the file system. Thanks to GT, simply clicking on a > > >> >>> file may show you the content and other things. > > >> >>> <Screen Shot 2015-07-29 at 10.23.51 AM.png> > > >> >>> > > >> >>> You can also use the cluster layout: > > >> >>> <Screen Shot 2015-07-29 at 10.25.11 AM.png> > > >> >>> > > >> >>> I have tried this on OS X, since the pointed folder is ~/Documents. > > >> >>> > > >> >>> Cheers, > > >> >>> Alexandre > > >> >>> -- > > >> >>> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > > >> >>> Alexandre Bergel http://www.bergel.eu <http://www.bergel.eu/> > > >> >>> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > >> >>> > > >> >>> > > >> >>> > > >> >>> _______________________________________________ > > >> >>> Moose-dev mailing list > > >> >>> Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> >>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> >>> <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> >> > > >> >> _______________________________________________ > > >> >> Moose-dev mailing list > > >> >> Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> >> <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> >> _______________________________________________ > > >> >> Moose-dev mailing list > > >> >> Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> >> <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> > > > >> > _______________________________________________ > > >> > Moose-dev mailing list > > >> > Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> > https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> > <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> > _______________________________________________ > > >> > Moose-dev mailing list > > >> > Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> > https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> > <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> > > >> > > >> _______________________________________________ > > >> Moose-dev mailing list > > >> Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> > > >> _______________________________________________ > > >> Moose-dev mailing list > > >> Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> > > >> > > >> > > >> > > >> -- > > >> www.tudorgirba.com <http://www.tudorgirba.com/> > > >> > > >> "Every thing has its own flow" > > >> _______________________________________________ > > >> Moose-dev mailing list > > >> Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> > > >> _______________________________________________ > > >> Moose-dev mailing list > > >> Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> > > >> > > >> _______________________________________________ > > >> Moose-dev mailing list > > >> Moose-dev(a)iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch> > > >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > >> <https://www.iam.unibe.ch/mailman/listinfo/moose-dev> > > >> _______________________________________________ > > >> Moose-dev mailing list > > >> Moose-dev(a)iam.unibe.ch > > >> https://www.iam.unibe.ch/mailman/listinfo/moose-dev > > > > > > > > > > > > > > > > -- > > www.tudorgirba.com > > > > "Every thing has its own flow" > > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > > _______________________________________________ > Moose-dev mailing list > Moose-dev(a)iam.unibe.ch > https://www.iam.unibe.ch/mailman/listinfo/moose-dev -- www.tudorgirba.com "Every thing has its own flow" _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev