Hey,

I can confirm the slowdown. The latest change (JanKurs.238, committed today) should return most of the performance back. Please, let me know, if you have any other issues

Cheers, Jan

On 22 August 2014 16:42, Tudor Girba tudor@tudorgirba.com wrote:

Hi,

Thanks for raising this.

We did not have time to announce these changes properly. We integrated Jan's work on supporting island parsing and enhanced traceability. The features are really cool. To this end, the stream of petit parser was replaced with a first class context that holds the stream.

We did expect a slight slowdown, but not a factor of 7 (which is what I get from your measurements) though.

Jan? :)

Doru

On Fri, Aug 22, 2014 at 1:30 PM, Usman Bhatti usman.bhatti@gmail.com wrote:

...

Hi,

Recent changes in PetitParser have caused parsing to slow down by a factor of 15-20. Here is an example run on freshly downloaded Moose image:

[|string allStyles text| string := (PharoFilesOpener default changesFileOrNil contents copyFrom: 1 to: 60000). text := string asText. allStyles := { 'Announcement' -> TextColor green. 'Collections' -> TextColor blue. 'File' -> TextColor blue. 'Metacello' -> TextColor red. 'Monticello' -> TextColor magenta. 'Morphic' -> TextColor orange. 'Mooose' -> TextColor green. 'FAMIX' -> TextColor green. 'Roassal' -> TextColor green. }. allStyles do: [ :assoc | ((assoc key asParser, #newline asParser negate star) matchingRangesIn: string) do: [:aRange | text addAttribute: (assoc value) from: aRange first to: aRange last] ]. text] timeToRun ==> 15 sec

============== now do this:

Gofer new url: 'http://smalltalkhub.com/mc/Moose/PetitParser/main'; version:'PetitParser-DiegoLont.233'; load.

and run the example again. timeToRun ===> 2 sec

So, we need to have a look at what is added recently in PetitParser that is making it slow down quite significantly because the for the larger systems, this can be enormous.

regards,

Usman

P.S: Tests run on OS X 10.9.4, 2.3 GHz, 8 GB RAM, HD SSD.

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com

"Every thing has its own flow"

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Hey,

sorry for a complication, the branching system is a too tricky for me. The version JanKurs.240 should do the thing. I just checked in a build Moose-50 #2222 (https://ci.inria.fr/moose/job/moose-5.0/2222/)

Cheers, Jan

On 24 August 2014 15:02, Tudor Girba tudor@tudorgirba.com wrote:

Hi Jan,

Thanks a lot for looking into this. I checked, but I think the improvement you mention did not make it in the repo of PetitParser.

Version JanKurs.238 is from August 20 (before the slowdown report):

Name: PetitParser-JanKurs.238 Author: JanKurs Time: 20 August 2014, 10:13:55.838908 am UUID: a7316ed3-0751-4703-80c9-2790add1cc00 Ancestors: PetitParser-JanKurs.237

The latest version is JanKurs.239 and it is also from the same day:

Name: PetitParser-JanKurs.239 Author: JanKurs Time: 20 August 2014, 12:13:26.196695 pm UUID: 8985ce0d-8d40-47ef-bd82-ead0d8695be2 Ancestors: PetitParser-JanKurs.238

Could you please double check?

Cheers, Doru

On Sun, Aug 24, 2014 at 3:56 PM, Jan Kurš kurs.jan@gmail.com wrote:

...

Hey,

I can confirm the slowdown. The latest change (JanKurs.238, committed today) should return most of the performance back. Please, let me know, if you have any other issues

Cheers, Jan

On 22 August 2014 16:42, Tudor Girba tudor@tudorgirba.com wrote:

...

Hi,

Thanks for raising this.

We did not have time to announce these changes properly. We integrated Jan's work on supporting island parsing and enhanced traceability. The features are really cool. To this end, the stream of petit parser was replaced with a first class context that holds the stream.

We did expect a slight slowdown, but not a factor of 7 (which is what I get from your measurements) though.

Jan? :)

Doru

On Fri, Aug 22, 2014 at 1:30 PM, Usman Bhatti usman.bhatti@gmail.com wrote:

...

Hi,

Recent changes in PetitParser have caused parsing to slow down by a factor of 15-20. Here is an example run on freshly downloaded Moose image:

[|string allStyles text| string := (PharoFilesOpener default changesFileOrNil contents copyFrom: 1 to: 60000). text := string asText. allStyles := { 'Announcement' -> TextColor green. 'Collections' -> TextColor blue. 'File' -> TextColor blue. 'Metacello' -> TextColor red. 'Monticello' -> TextColor magenta. 'Morphic' -> TextColor orange. 'Mooose' -> TextColor green. 'FAMIX' -> TextColor green. 'Roassal' -> TextColor green. }. allStyles do: [ :assoc | ((assoc key asParser, #newline asParser negate star) matchingRangesIn: string) do: [:aRange | text addAttribute: (assoc value) from: aRange first to: aRange last] ]. text] timeToRun ==> 15 sec

============== now do this:

Gofer new url: 'http://smalltalkhub.com/mc/Moose/PetitParser/main'; version:'PetitParser-DiegoLont.233'; load.

and run the example again. timeToRun ===> 2 sec

So, we need to have a look at what is added recently in PetitParser that is making it slow down quite significantly because the for the larger systems, this can be enormous.

regards,

Usman

P.S: Tests run on OS X 10.9.4, 2.3 GHz, 8 GB RAM, HD SSD.

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com

"Every thing has its own flow"

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com

"Every thing has its own flow"

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Hi,

On Sun, Aug 24, 2014 at 5:30 PM, Tudor Girba tudor@tudorgirba.com wrote:

Great!

I tested it. The example of Usman went from 15s to 3s.

@Usman: could you test and see if it is acceptable?

I tested the example and we are down to roughly 3-4 seconds. Thank you Doru and Jan for looking into it.

Cheers, Doru

On Sun, Aug 24, 2014 at 5:16 PM, Jan Kurš kurs.jan@gmail.com wrote:

...

Hey,

sorry for a complication, the branching system is a too tricky for me. The version JanKurs.240 should do the thing. I just checked in a build Moose-50 #2222 (https://ci.inria.fr/moose/job/moose-5.0/2222/)

Cheers, Jan

On 24 August 2014 15:02, Tudor Girba tudor@tudorgirba.com wrote:

...

Hi Jan,

Thanks a lot for looking into this. I checked, but I think the improvement you mention did not make it in the repo of PetitParser.

Version JanKurs.238 is from August 20 (before the slowdown report):

Name: PetitParser-JanKurs.238 Author: JanKurs Time: 20 August 2014, 10:13:55.838908 am UUID: a7316ed3-0751-4703-80c9-2790add1cc00 Ancestors: PetitParser-JanKurs.237

The latest version is JanKurs.239 and it is also from the same day:

Name: PetitParser-JanKurs.239 Author: JanKurs Time: 20 August 2014, 12:13:26.196695 pm UUID: 8985ce0d-8d40-47ef-bd82-ead0d8695be2 Ancestors: PetitParser-JanKurs.238

Could you please double check?

Cheers, Doru

On Sun, Aug 24, 2014 at 3:56 PM, Jan Kurš kurs.jan@gmail.com wrote:

...

Hey,

I can confirm the slowdown. The latest change (JanKurs.238, committed today) should return most of the performance back. Please, let me know, if you have any other issues

Cheers, Jan

On 22 August 2014 16:42, Tudor Girba tudor@tudorgirba.com wrote:

...

Hi,

Thanks for raising this.

We did not have time to announce these changes properly. We integrated Jan's work on supporting island parsing and enhanced traceability. The features are really cool. To this end, the stream of petit parser was replaced with a first class context that holds the stream.

We did expect a slight slowdown, but not a factor of 7 (which is what I get from your measurements) though.

Jan? :)

Doru

On Fri, Aug 22, 2014 at 1:30 PM, Usman Bhatti usman.bhatti@gmail.com wrote:

...

Hi,

Recent changes in PetitParser have caused parsing to slow down by a factor of 15-20. Here is an example run on freshly downloaded Moose image:

[|string allStyles text| string := (PharoFilesOpener default changesFileOrNil contents copyFrom: 1 to: 60000). text := string asText. allStyles := { 'Announcement' -> TextColor green. 'Collections' -> TextColor blue. 'File' -> TextColor blue. 'Metacello' -> TextColor red. 'Monticello' -> TextColor magenta. 'Morphic' -> TextColor orange. 'Mooose' -> TextColor green. 'FAMIX' -> TextColor green. 'Roassal' -> TextColor green. }. allStyles do: [ :assoc | ((assoc key asParser, #newline asParser negate star) matchingRangesIn: string) do: [:aRange | text addAttribute: (assoc value) from: aRange first to: aRange last] ]. text] timeToRun ==> 15 sec

============== now do this:

Gofer new url: 'http://smalltalkhub.com/mc/Moose/PetitParser/main'; version:'PetitParser-DiegoLont.233'; load.

and run the example again. timeToRun ===> 2 sec

So, we need to have a look at what is added recently in PetitParser that is making it slow down quite significantly because the for the larger systems, this can be enormous.

regards,

Usman

P.S: Tests run on OS X 10.9.4, 2.3 GHz, 8 GB RAM, HD SSD.

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com

"Every thing has its own flow"

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com

"Every thing has its own flow"

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

-- www.tudorgirba.com

"Every thing has its own flow"

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

2014-08-25 11:12 GMT+02:00 Usman Bhatti usman.bhatti@gmail.com:

I understand that context is important when parsing a program for certain parsing related information.

In my case, I am using PetitParser more as a syntax highlighter on the lines of PPTextHighlighter and in that case you dont need much of contextual information. All you need is token positions. Can we envisage, for such usage, a simplistic but more robust parser?

If you're more into that, then a lexer may be enough and a very efficient one can be built with SmaCC.

What you're doing doesn't seems to require a parser: it is just matching

Keyword [^\n] *

isn't it?

Thierry

[|string allStyles text| string := (PharoFilesOpener default changesFileOrNil contents copyFrom: 1 to: 60000). text := string asText. allStyles := { 'Announcement' -> TextColor green. 'Collections' -> TextColor blue. 'File' -> TextColor blue. 'Metacello' -> TextColor red. 'Monticello' -> TextColor magenta. 'Morphic' -> TextColor orange. 'Mooose' -> TextColor green. 'FAMIX' -> TextColor green. 'Roassal' -> TextColor green. }. allStyles do: [ :assoc | ((assoc key asParser, #newline asParser negate star) matchingRangesIn: string) do: [:aRange | text addAttribute: (assoc value) from: aRange first to: aRange last] ]. text] timeToRun ==> 15 sec

2014-08-25 16:11 GMT+02:00 Usman Bhatti usman.bhatti@gmail.com:

It was just a illustrative example to show the slowness of PParser due to the recent modifications. What I want to achieve is to be able to express keyword or lines matching (may be some day complete code blocks as well). We found that doing it with the parser is quite easy.

If there is little context around (deep tree structures, backtracking) then a lexer may be enough. If you give me a bit of context (or a petit parser description), I can work out the equivalent SmaCC grammar.

It would be worth trying SmaCC to compare its performance with PP-based solution but I am not knowledgable about SmaCC. Any pointers to where should I start from?

I updated the SmaCC chapter in the PharoForTheEnterprise book; it is also available as help inside Pharo itself.

https://ci.inria.fr/pharo-contribution/job/PharoForTheEnterprise/ws/Smacc/Sm...

Thierry

regards,

Usman

...

Keyword [^\n] *

isn't it?

Thierry

[|string allStyles text| string := (PharoFilesOpener default changesFileOrNil contents copyFrom: 1 to: 60000). text := string asText. allStyles := { 'Announcement' -> TextColor green. 'Collections' -> TextColor blue. 'File' -> TextColor blue. 'Metacello' -> TextColor red. 'Monticello' -> TextColor magenta. 'Morphic' -> TextColor orange. 'Mooose' -> TextColor green. 'FAMIX' -> TextColor green. 'Roassal' -> TextColor green. }. allStyles do: [ :assoc | ((assoc key asParser, #newline asParser negate star) matchingRangesIn: string) do: [:aRange | text addAttribute: (assoc value) from: aRange first to: aRange last] ]. text] timeToRun ==> 15 sec

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Le 25/08/2014 18:36, Usman Bhatti a écrit :

Tx Thierry. I'd prefer to read and understand the basics of SmaCC first from the Pharo book for intelligent communication :). I'll let you know.

Well, I teach parsing with SmaCC, so I may help :)

Thierry

Usman

On Mon, Aug 25, 2014 at 4:48 PM, Thierry Goubier <thierry.goubier@gmail.com mailto:thierry.goubier@gmail.com> wrote:

2014-08-25 16:11 GMT+02:00 Usman Bhatti <usman.bhatti@gmail.com
<mailto:usman.bhatti@gmail.com>>:



    It was just a illustrative example to show the slowness of
    PParser due to the recent modifications. What I want to
    achieve is to be able to express keyword or lines matching
    (may be some day complete code blocks as well). We found that
    doing it with the parser is quite easy.


If there is little context around (deep tree structures,
backtracking) then a lexer may be enough. If you give me a bit of
context (or a petit parser description), I can work out the
equivalent SmaCC grammar.


    It would be worth trying SmaCC to compare its performance with
    PP-based solution but I am not knowledgable about SmaCC. Any
    pointers to where should I start from?


I updated the SmaCC chapter in the PharoForTheEnterprise book; it
is also available as help inside Pharo itself.

https://ci.inria.fr/pharo-contribution/job/PharoForTheEnterprise/ws/Smacc/Smacc.pier.pdf




Thierry


    regards,

    Usman


        Keyword [^\n] *

        isn't it?

        Thierry

        [|string allStyles  text|
        string := (PharoFilesOpener default changesFileOrNil
        contents copyFrom: 1 to: 60000).
        text := string asText.
        allStyles := {
        'Announcement' -> TextColor green.
        'Collections' -> TextColor blue.
        'File' -> TextColor blue.
         'Metacello' -> TextColor red.
        'Monticello' -> TextColor magenta.
        'Morphic' -> TextColor orange.
        'Mooose' -> TextColor green.
        'FAMIX' -> TextColor green.
        'Roassal' -> TextColor green.
        }.
        allStyles do: [ :assoc | ((assoc key asParser, #newline
        asParser negate star) matchingRangesIn: string) do:
        [:aRange | text addAttribute: (assoc value) from: aRange
        first to: aRange last] ].
        text] timeToRun ==> 15 sec



        _______________________________________________
        Moose-dev mailing list
        Moose-dev@iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch>
        https://www.iam.unibe.ch/mailman/listinfo/moose-dev



    _______________________________________________
    Moose-dev mailing list
    Moose-dev@iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch>
    https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
Moose-dev@iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch>
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Le 27/08/2014 23:12, stepharo a écrit :

On 25/8/14 19:02, Thierry Goubier wrote:

...

Le 25/08/2014 18:36, Usman Bhatti a écrit :

...

Tx Thierry. I'd prefer to read and understand the basics of SmaCC first from the Pharo book for intelligent communication :). I'll let you know.

Well, I teach parsing with SmaCC, so I may help :)

Thierry do you have slides and related material?

I should write some this year. At least if I manage to get some time between the current projects and the project proposals :(

Thierry

I updated the SmaCC chapter in the PharoForTheEnterprise book; it is also available as help inside Pharo itself.

I'm really happy that you did it ;)

https://ci.inria.fr/pharo-contribution/job/PharoForTheEnterprise/ws/Smacc/Sm...

Thierry

regards,

Usman


    Keyword [^\n] *

    isn't it?

    Thierry

    [|string allStyles  text|
    string := (PharoFilesOpener default changesFileOrNil contents
    copyFrom: 1 to: 60000).
    text := string asText.
    allStyles := {
    'Announcement' -> TextColor green.
    'Collections' -> TextColor blue.
    'File' -> TextColor blue.
     'Metacello' -> TextColor red.
    'Monticello' -> TextColor magenta.
    'Morphic' -> TextColor orange.
    'Mooose' -> TextColor green.
    'FAMIX' -> TextColor green.
    'Roassal' -> TextColor green.
    }.
    allStyles do: [ :assoc | ((assoc key asParser, #newline
    asParser negate star) matchingRangesIn: string) do: [:aRange |
    text addAttribute: (assoc value) from: aRange first to: aRange
    last] ].
    text] timeToRun ==> 15 sec



    _______________________________________________
    Moose-dev mailing list
    Moose-dev@iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch>
    https://www.iam.unibe.ch/mailman/listinfo/moose-dev



_______________________________________________
Moose-dev mailing list
Moose-dev@iam.unibe.ch <mailto:Moose-dev@iam.unibe.ch>
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

---

Moose-dev mailing list Moose-dev@iam.unibe.ch https://www.iam.unibe.ch/mailman/listinfo/moose-dev