Reviewing Fuel export/import utility - Moose-dev

Mariano Martinez Peck

4 Jan 4 Jan

10:47 p.m.

ping? On Thu, Dec 29, 2011 at 4:31 AM, Martin Dias <tinchodias(a)gmail.com> wrote:

Hello, I'm looking to improve current implementation, and maybe you can help me. The current procedure is based in a custom serialization of each MooseElement. It's inspired on MSE, I describe this with the following pseudo-code: serialize: aMooseElement (MooseModel meta descriptionOf: aMooseElement class) allAttributes do: [ :anAttribute | | values | values := anAttribute getFrom: aMooseElement. (self shouldIgnore: anAttribute withAll: values) ifFalse: [ self serialize: anAttribute withAll: values ] ]. where: shouldIgnore: anAttribute withAll: values ^ values isEmpty or: [ anAttribute isDerived or: [ anAttribute type == FM3MetaDescription boolean and: [ values size == 1 and: [ values first == false ]]]] The advantage of serializing the MooseElements in this way (and not just as a normal object) is to avoid storing unnecessary stuff that aMooseElement references. It's a disadvantage using FM3PropertyDescription>>getFrom: (and then to import, FM3PropertyDescription>>setOn:values:) which ends sending #perform: of the corresponding accessor selector. It'd be better to use #instVarAt: (and #instVarAt:put:) as Fuel normally does. I hope I've been clear enough to explain up to this point. Now my question: Do you think Fuel can do something on each MooseElement like - clean up some unnecessary references - declare some instance variables as transient - if it's not good idea to modify the elements, create a method like MooseElement>>copyWithoutDerivedValues, and so actually serialize such copy instead of the original element - any other ??? and thus, serialize the MooseElements as normal Fuel objects, removing the custom procedure. I'll be happy to receive some help from Moose and Fame experts! Thanks in advance. Martín _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- Mariano http://marianopeck.wordpress.com

Reply

Tudor Girba

12:41 p.m.

Hi, Thanks for re-pinging us. I somehow did not see it before. Strange :). As far as I understand, you are worried about speed. Is that right? In any case, the problem with your proposition is that some properties are not stored in explicit instance variables, but in a dictionary in EntityState. That means that only looking at the instance variables you can miss important things. The ultimate truth of what needs to be serialized lies in the meta-description. I would not know how to go around that. Cheers, Doru On Fri, Mar 2, 2012 at 12:16 PM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote:

...

re-ping? On Wed, Jan 4, 2012 at 10:47 PM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote:

ping? On Thu, Dec 29, 2011 at 4:31 AM, Martin Dias <tinchodias(a)gmail.com> wrote:

Hello, I'm looking to improve current implementation, and maybe you can help me. The current procedure is based in a custom serialization of each MooseElement. It's inspired on MSE, I describe this with the following pseudo-code: serialize: aMooseElement (MooseModel meta descriptionOf: aMooseElement class) allAttributes do: [ :anAttribute | | values | values := anAttribute getFrom: aMooseElement. (self shouldIgnore: anAttribute withAll: values) ifFalse: [ self serialize: anAttribute withAll: values ] ]. where: shouldIgnore: anAttribute withAll: values ^ values isEmpty or: [ anAttribute isDerived or: [ anAttribute type == FM3MetaDescription boolean and: [ values size == 1 and: [ values first == false ]]]] The advantage of serializing the MooseElements in this way (and not just as a normal object) is to avoid storing unnecessary stuff that aMooseElement references. It's a disadvantage using FM3PropertyDescription>>getFrom: (and then to import, FM3PropertyDescription>>setOn:values:) which ends sending #perform: of the corresponding accessor selector. It'd be better to use #instVarAt: (and #instVarAt:put:) as Fuel normally does. I hope I've been clear enough to explain up to this point. Now my question: Do you think Fuel can do something on each MooseElement like - clean up some unnecessary references - declare some instance variables as transient - if it's not good idea to modify the elements, create a method like MooseElement>>copyWithoutDerivedValues, and so actually serialize such copy instead of the original element - any other ??? and thus, serialize the MooseElements as normal Fuel objects, removing the custom procedure. I'll be happy to receive some help from Moose and Fame experts! Thanks in advance. Martín _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- Mariano http://marianopeck.wordpress.com

-- Mariano http://marianopeck.wordpress.com _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com "Every thing has its own flow"

Reply

Mariano Martinez Peck

28 May 28 May

2:56 p.m.

On Fri, Mar 2, 2012 at 12:41 PM, Tudor Girba <tudor(a)tudorgirba.com> wrote:

...

Hi, Thanks for re-pinging us. I somehow did not see it before. Strange :). As far as I understand, you are worried about speed. Is that right?

I was playing a bit (updating Moose extensions to latest Fuel) and I also noticed the same as Martin. The 70% of the export time is taken by these 4 methods: #numberOfLinesOfCode, #cyclomaticComplexity, #numberOfConditionals and #numberOfStatements which seem to be computed during the export. I don't really know much about Moose, but I attach a screenshot of the profile. So, is that correct/expect? are those values already calculated/stored somewhere? thanks

...

In any case, the problem with your proposition is that some properties are not stored in explicit instance variables, but in a dictionary in EntityState. That means that only looking at the instance variables you can miss important things. The ultimate truth of what needs to be serialized lies in the meta-description. I would not know how to go around that. Cheers, Doru On Fri, Mar 2, 2012 at 12:16 PM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote:

re-ping? On Wed, Jan 4, 2012 at 10:47 PM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > > ping? > > On Thu, Dec 29, 2011 at 4:31 AM, Martin Dias <tinchodias(a)gmail.com>

wrote:

>> >> Hello, >> >> I'm looking to improve current implementation, and maybe you can help

me.

>> >> The current procedure is based in a custom serialization of each >> MooseElement. It's inspired on MSE, I describe this with the following >> pseudo-code: >> >> serialize: aMooseElement >> (MooseModel meta descriptionOf: aMooseElement class) allAttributes >> do: [ :anAttribute | >> | values | >> values := anAttribute getFrom: aMooseElement. >> (self shouldIgnore: anAttribute withAll: values) >> ifFalse: [ self serialize: anAttribute withAll: values ] ]. >> >> where: >> >> shouldIgnore: anAttribute withAll: values >> ^ values isEmpty or: [ >> anAttribute isDerived or: [ >> anAttribute type == FM3MetaDescription boolean and: [ >> values size == 1 and: [ >> values first == false ]]]] >> >> >> The advantage of serializing the MooseElements in this way (and not

just

>> as a normal object) is to avoid storing unnecessary stuff that

aMooseElement

>> references. >> >> It's a disadvantage using FM3PropertyDescription>>getFrom: (and then to >> import, FM3PropertyDescription>>setOn:values:) which ends sending

#perform:

>> of the corresponding accessor selector. It'd be better to use

#instVarAt:

>> (and #instVarAt:put:) as Fuel normally does. >> >> I hope I've been clear enough to explain up to this point. Now my >> question: >> >> Do you think Fuel can do something on each MooseElement like >> >> - clean up some unnecessary references >> - declare some instance variables as transient >> - if it's not good idea to modify the elements, create a method like >> MooseElement>>copyWithoutDerivedValues, and so actually serialize such

copy

instead of the original element - any other ??? and thus, serialize the MooseElements as normal Fuel objects, removing the custom procedure. I'll be happy to receive some help from Moose and Fame experts! Thanks in advance. Martín _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- Mariano http://marianopeck.wordpress.com

-- Mariano http://marianopeck.wordpress.com _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com "Every thing has its own flow" _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- Mariano http://marianopeck.wordpress.com

Reply

Tudor Girba

3:09 p.m.

Hi, Thanks for looking into this. These properties are mandatory only for FAMIX models. However, they are computed lazily: if they exist, they do not get computed again. So, this is great news :). And here is another thing: the FuelMooseExtension works at the level of Fame, which is a meta-engine that, like Magritte, can describe any meta-models (not just FAMIX). So, knowing that we can export those meta-models for free and quite fast is great. As I mentioned at PharoConf, the uber-cool thing would be to have the Fuel serialization available in Java :). That is because we already have Fame there, and thus, we could have a fast data route between the two worlds. Cheers, Doru On 28 May 2012, at 14:57, Mariano Martinez Peck wrote:

...

On Mon, May 28, 2012 at 2:56 PM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: On Fri, Mar 2, 2012 at 12:41 PM, Tudor Girba <tudor(a)tudorgirba.com> wrote: Hi, Thanks for re-pinging us. I somehow did not see it before. Strange :). As far as I understand, you are worried about speed. Is that right? I was playing a bit (updating Moose extensions to latest Fuel) and I also noticed the same as Martin. The 70% of the export time is taken by these 4 methods: #numberOfLinesOfCode, #cyclomaticComplexity, #numberOfConditionals and #numberOfStatements which seem to be computed during the export. I don't really know much about Moose, but I attach a screenshot of the profile. sorry, here it is the attach. So, is that correct/expect? are those values already calculated/stored somewhere? thanks In any case, the problem with your proposition is that some properties are not stored in explicit instance variables, but in a dictionary in EntityState. That means that only looking at the instance variables you can miss important things. The ultimate truth of what needs to be serialized lies in the meta-description. I would not know how to go around that. Cheers, Doru On Fri, Mar 2, 2012 at 12:16 PM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote:

re-ping? On Wed, Jan 4, 2012 at 10:47 PM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote:

ping? On Thu, Dec 29, 2011 at 4:31 AM, Martin Dias <tinchodias(a)gmail.com> wrote:

Hello, I'm looking to improve current implementation, and maybe you can help me. The current procedure is based in a custom serialization of each MooseElement. It's inspired on MSE, I describe this with the following pseudo-code: serialize: aMooseElement (MooseModel meta descriptionOf: aMooseElement class) allAttributes do: [ :anAttribute | | values | values := anAttribute getFrom: aMooseElement. (self shouldIgnore: anAttribute withAll: values) ifFalse: [ self serialize: anAttribute withAll: values ] ]. where: shouldIgnore: anAttribute withAll: values ^ values isEmpty or: [ anAttribute isDerived or: [ anAttribute type == FM3MetaDescription boolean and: [ values size == 1 and: [ values first == false ]]]] The advantage of serializing the MooseElements in this way (and not just as a normal object) is to avoid storing unnecessary stuff that aMooseElement references. It's a disadvantage using FM3PropertyDescription>>getFrom: (and then to import, FM3PropertyDescription>>setOn:values:) which ends sending #perform: of the corresponding accessor selector. It'd be better to use #instVarAt: (and #instVarAt:put:) as Fuel normally does. I hope I've been clear enough to explain up to this point. Now my question: Do you think Fuel can do something on each MooseElement like - clean up some unnecessary references - declare some instance variables as transient - if it's not good idea to modify the elements, create a method like MooseElement>>copyWithoutDerivedValues, and so actually serialize such copy instead of the original element - any other ??? and thus, serialize the MooseElements as normal Fuel objects, removing the custom procedure. I'll be happy to receive some help from Moose and Fame experts! Thanks in advance. Martín _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- Mariano http://marianopeck.wordpress.com

-- Mariano http://marianopeck.wordpress.com _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com "Every thing has its own flow" _______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev -- Mariano http://marianopeck.wordpress.com -- Mariano http://marianopeck.wordpress.com <Screen shot 2012-05-28 at 2.55.49 PM.png>_______________________________________________ Moose-dev mailing list Moose-dev(a)iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com "What we can governs what we wish."

Reply

Martin Dias

8:32 a.m.

I have a question about the screenshot: FAMIXType >> numberOfStatements <MSEProperty: #numberOfStatements type: #Number> <derived> <MSEComment: 'The number of statements in a class'> ^self lookUpPropertyNamed: #numberOfStatements computedAs: [self methodsGroup sum: #numberOfStatements] is *derived*, so (look at the in the code in the bottom of the picture): self shouldIgnore: 'numberOfStatements' withAll: values will return false, and thus "values" (which was so expensive to calculate) will be discarded (not encoded). I mean, our code could be optimized. Right? Martín

Reply