2 Jan
2017
2 Jan
'17
9:37 a.m.
On 01 Jan 2017, at 16:15, stephan
<stephan(a)stack.nl> wrote:
The more classical way to provide real-time data-analysis is using
gemstone technology http://www.gemstone.com/customers/DISA
<http://www.gemstone.com/customers/DISA>
I fully agree they should be be using
that. However most companies don’t (want to) know Smalltalk anymore. For those RDF was an
answer to their prayers and SPARQL the promise to be db-backend independent for eternity
:-)
You could, and there might not be much added value in
doing
that instead of just creating a domain model directly in smalltalk.
That’s a main
thing I have to research - which benefits do I get from using/implementing this with a
matching meta model.
Moose expects a dynamic model, but one that is not so
large.
With a 64-bit image the size is less of a problem than the
processing speed. You might want to distribute the processing
over many images, and create different mappings of data sources
to images to optimize your queries.
Distributed images … that’s something I didn’t
think about yet. This might really solve some concerns I have with the backend.
By default log files are extremely redundant.
Compressing them by
using a decent domain model, introducing value objects to reduce
data size can be useful.
True - if (only) log data would be the (only) problem.
Having done this for another for another project (REST API forensics) we easily achieved
1:1000+ ratios in compression. Especially modelling the IPv6 addresses as Singletons and
using a tree model for the paths/queries provided amazing compression. And once the paths
were objects and not mere dumb strings the final verdict came fast.
Most databases work badly with this kind of problem.
You might want to
expicitly model the mapping of data to multiple images. You are
basically creating a blackboard system
Never heard of if - but researching it
definitely matched the idea I have in mind!!!
This is a great pointer I need to investigate further. I hope to find more patterns there
helping me to do the system “right” with less iterations.
Although I did some research on the stuff I plan to build I found no matching paper on the
subject whatsoever. It seems I was missing the right terms for concepts and ideas :-(
This probably needs a double registration. From a
governance p.o.v.
you need the actual process steps, and you also need a log of how you
could have reached your conclusions with perfect hindsight
http://www.ics.uci.edu/~taylor/classes/121/IEEE86_Parnas_Clement.pdf
<http://www.ics.uci.edu/~taylor/classes/121/IEEE86_Parnas_Clement.pdf>
Another
gem!
Moose supports creating your own tools very well. We
did a data
migration with Moose, and created 60+ glamour browsers for it,
and daily visualisations of the process. We managed to keep things
in a single image, though we ran out of memory several times.
To me it sounds like you'd need to develop something to manage lots
of images, starting new ones on demand. That fits well with our
Pharo vision.
I reached the point where the decision for Moose/Pharo is already
taken (positively!). I’m just not decided yet on the backend(s) side.
Thank you very much for your comments! Very helpful and highly appreciated!
CU,
Udo