modred team mailing list archive

[Frederic Koehler <fkfire@xxxxxxxxx>]

So, as far as I understand this project, the idea is to build
both a client library and a program using the library to do clustering
stuff, along with matching server/hub foo (the library might be the same or
whatever, not important).

So from this understanding, it seems that the system should provide some
basic pseudo-operating system stuff and programs can build on that, just
like they would normally build on their local libc/kernel and stuff.

So (I sure like the word "so" today...) if we want the type of general
os-like stuff it seems their needs to be support for:
   * A simpe message passing model - abstract away all the TCP-foo,  maybe
use existing foo here (obviously needs fleshing out)
   * Permanent storage IO (clone the unix write(), read(), open() and sync()
model,  or maybe just use one of the existing database-ish nosql things out
there)
           - Unix-ish model - you create your data hunk, say you want all
this stuff in it, then after sync() we know it's actually somewhere written
on a hard-drive, and other things can read it too
           - Unless this isn't in fact needed (but I assume it is)
           - Also need to figure out if it's filesystem-ish foo
(hierarchial) we want or more relational database-ish stuff

   * A task delegation model - some type of map/reduce-ish stuff
          - Servers have a few built-in computations, and client utilizes
them?
          - Or more complex, servers run sandboxed computational code?
   * A security system?
        - Needs fleshing out
        - Presumably what the "hub" manages - it's the trusted thing
        - Obviously, not everybody is allowed to use the cluster for
computation, not everybody can find out what everybody else is doing, etc.
      - But also, is their a limit on storage, are some things prioritized
over others, ?

Theroretically, server's are written to provide the io backend and to allow
for task delegation, clients use the api, although hub has it's work cut out
delegating all the file io and figuring out what the state of that is.

On top of some mixture of this, one could build a simple unix-ish
pseudo-cli, theroretically, as well as real software.

Anyway, before actually doing anything, people should read about PVM
(Parallel Virtual Machine) and the like (maybe also Hadoop and other foo-ish
stuff) so Modred isn't just a bad clone of it

Anyway, (yes, twice in a row!), I figured _someone_ had to respond to Scott,
otherwise he'd feel all lonely and sad :P Now he can have a warm fuzzy
feeling of deep confusion and uncertainty instead :P

On Fri, Dec 25, 2009 at 11:06 PM, Scott Lawrence <bytbox@xxxxxxxxx> wrote:

> ---------- Forwarded message ----------
> From: Scott Lawrence <bytbox@xxxxxxxxx>
> Date: Fri, 25 Dec 2009 19:20:13 -0500
> Subject: Design Overview
> To: modred <modred@xxxxxxxxxxxxxxxxxxx>
>
> I'm going to assume that everyone understands the basic concepts for
> modred: a set of networked computers (by 'networked' I mean, they're
> all on the internet), divided for the sake of discussion into three
> classes: the 'hub' (the dude in charge, who compupters who want to
> join connect to), the 'servers' (dedicated computers that can be
> pretty much relied on not to go down, although redundancy is always
> nice), and the 'clients' (computers that send in requests and can be
> used for spare CPU cycles.
>
> Ok, so much for assumptions... :-)
>
> Things *I* think any design should emphasize:
>  * security.
>  * relative ease of use, while retaining significant power.
> Challenging.  In particular, it should be possible to set up a modred
> network in under an hour, provided the computers are already set up.
>  * along with the previous bullet point, having an interface that lets
> one use the entire network like a single computer.  This is sort of
> like the way google docs works, except the cloud is private
>  * therefore, it should be a multi-user system with well-designed
> privileges etc...
>
> I'm not going to discuss my implementation ideas, let's hear others first.
>
> --
> Scott Lawrence
>
> Webmaster
> The Blair Robot Project
> Montgomery Blair High School
>
>
>
> --
> Scott Lawrence
>
> Webmaster
> The Blair Robot Project
> Montgomery Blair High School
>