New subject: Minnow WIKI Migration

23 Oct 2006

The process of porting minnow from swiki to pier, is likely to happen. 
It will just require a little bit of patience, as the needed components 
are road-tested and refined.

As you can imagine my own testing process has been somewhat hindered by 
image freezes. Now that I have been informed that Squeak vm 3.6.3 is 
actually stable. I have been able to create a pier-wiki with 6200 or 
more pages for testing purposes.

Initial Stats Using Pier with PRNullPersistency (i.e. everything in 
memory):

6200 pages  (generated one for each squeak class from squeak sources)
235352 internal links.
Adding a page 100-500ms
Removing a page 215 seconds!!
(many wikis dont support removing pages anyway)
Total Memory = 77Mb.

Compare this to minnow: Text data of its 5889 pages is about 30Mb.
Of course the swiki, has full page history, and uploaded files too.

Pier Magma should be able to handle this kind of load, but it remains to 
be explicitly tested.

Anticipated work to make things workable.
1. Some explicit caching of items that will slow pier-magma down with 
data on disk rather than in memory. Removal of pages may be extremely 
slow without this.
2. Explicit support for an indexed full text search which avoids the 
need to traverse the whole data tree for a simple search.
3. Some form of logging of user edits in addition to the default 
persistency strategy.

J J wrote:
...
 I would say go to Pier.  I think Kieth released some
software that you  can point >at the swiki and it will slurp it all up.  Am I
right Kieth?

I haven't written any proper data slurper for minnow. I believe there is 
already an importing tool. I have pointed wget at minnow to get the 
current set of pages as a test data set. Its about 30Mb or so.

Which leads me to a question. How would pier handle some random person 
running this script?

#!/usr/bin/ruby

for i in 1..5889
        print `wget --user=squeak --password=viewpoints 
http://minnow.cc.gatech.edu/squeak/#{i
}.edit`
end

this would probably create almost 5800 seaside sessions, in a matter of 
minutes?

Keith

Send instant messages to your online friends http://uk.messenger.yahoo.com 

Re: Minnow WIKI Migration