modred team mailing list archive

[David Tolnay <dtolnay@xxxxxxxxx>]

Before diving in to specifics about the implementation I think we need
to decide how we want modred to be different from (read: better than)
existing bootable cluster environments. Here is a short list to check
out:

Bootable Cluster CD (http://bccd.net/) - folks presented this at SC09
in portland, it was pretty neat stuff. Packed with education /
debugging / visualization features

Oscar (http://svn.oscar.openclustergroup.org/trac/oscar) - very
trivially simple way to transform an existing unix lab into a cluster
resource

Lnx-bbc (http://www.lnx-bbc.com/) - includes cowsay!

Perceus/warewulf (http://www.perceus.org/portal/) - a lot of other
sites made reference to this, haven't read too much about it

What specifically do you want to improve over any of these?


On 12/25/09, Frederic Koehler <fkfire@xxxxxxxxx> wrote:
>
>
> So, as far as I understand this project, the idea is to build
> both a client library and a program using the library to do clustering
> stuff, along with matching server/hub foo (the library might be the same or
> whatever, not important).
>
> So from this understanding, it seems that the system should provide some
> basic pseudo-operating system stuff and programs can build on that, just
> like they would normally build on their local libc/kernel and stuff.
>
> So (I sure like the word "so" today...) if we want the type of general
> os-like stuff it seems their needs to be support for:
>    * A simpe message passing model - abstract away all the TCP-foo,  maybe
> use existing foo here (obviously needs fleshing out)
>    * Permanent storage IO (clone the unix write(), read(), open() and sync()
> model,  or maybe just use one of the existing database-ish nosql things out
> there)
>            - Unix-ish model - you create your data hunk, say you want all
> this stuff in it, then after sync() we know it's actually somewhere written
> on a hard-drive, and other things can read it too
>            - Unless this isn't in fact needed (but I assume it is)
>            - Also need to figure out if it's filesystem-ish foo
> (hierarchial) we want or more relational database-ish stuff
>
>    * A task delegation model - some type of map/reduce-ish stuff
>           - Servers have a few built-in computations, and client utilizes
> them?
>           - Or more complex, servers run sandboxed computational code?
>    * A security system?
>         - Needs fleshing out
>         - Presumably what the "hub" manages - it's the trusted thing
>         - Obviously, not everybody is allowed to use the cluster for
> computation, not everybody can find out what everybody else is doing, etc.
>       - But also, is their a limit on storage, are some things prioritized
> over others, ?
>
> Theroretically, server's are written to provide the io backend and to allow
> for task delegation, clients use the api, although hub has it's work cut out
> delegating all the file io and figuring out what the state of that is.
>
> On top of some mixture of this, one could build a simple unix-ish
> pseudo-cli, theroretically, as well as real software.
>
> Anyway, before actually doing anything, people should read about PVM
> (Parallel Virtual Machine) and the like (maybe also Hadoop and other foo-ish
> stuff) so Modred isn't just a bad clone of it
>
> Anyway, (yes, twice in a row!), I figured _someone_ had to respond to Scott,
> otherwise he'd feel all lonely and sad :P Now he can have a warm fuzzy
> feeling of deep confusion and uncertainty instead :P
>
>
>
> On Fri, Dec 25, 2009 at 11:06 PM, Scott Lawrence <bytbox@xxxxxxxxx> wrote:
> > ---------- Forwarded message ----------
> > From: Scott Lawrence <bytbox@xxxxxxxxx>
> > Date: Fri, 25 Dec 2009 19:20:13 -0500
> > Subject: Design Overview
> > To: modred <modred@xxxxxxxxxxxxxxxxxxx>
> >
> > I'm going to assume that everyone understands the basic concepts for
> > modred: a set of networked computers (by 'networked' I mean, they're
> > all on the internet), divided for the sake of discussion into three
> > classes: the 'hub' (the dude in charge, who compupters who want to
> > join connect to), the 'servers' (dedicated computers that can be
> > pretty much relied on not to go down, although redundancy is always
> > nice), and the 'clients' (computers that send in requests and can be
> > used for spare CPU cycles.
> >
> > Ok, so much for assumptions... :-)
> >
> > Things *I* think any design should emphasize:
> >  * security.
> >  * relative ease of use, while retaining significant power.
> > Challenging.  In particular, it should be possible to set up a modred
> > network in under an hour, provided the computers are already set up.
> >  * along with the previous bullet point, having an interface that lets
> > one use the entire network like a single computer.  This is sort of
> > like the way google docs works, except the cloud is private
> >  * therefore, it should be a multi-user system with well-designed
> > privileges etc...
> >
> > I'm not going to discuss my implementation ideas, let's hear others first.
> >
> > --
> > Scott Lawrence
> >
> > Webmaster
> > The Blair Robot Project
> > Montgomery Blair High School
> >
> >
> >
> > --
> > Scott Lawrence
> >
> > Webmaster
> > The Blair Robot Project
> > Montgomery Blair High School
> >
>
>
>
> _______________________________________________
>  Mailing list: https://launchpad.net/~modred
>  Post to     : modred@xxxxxxxxxxxxxxxxxxx
>  Unsubscribe : https://launchpad.net/~modred
>  More help   : https://help.launchpad.net/ListHelp
>
>