modred team mailing list archive
-
modred team
-
Mailing list archive
-
Message #00053
Fwd: Fwd: Concept stuff
---------- Forwarded message ----------
From: Michael Cohen <gnurdux@xxxxxxxxx>
Date: Tue, 29 Dec 2009 00:10:08 -0500
Subject: Re: [Modred] Fwd: Concept stuff
To: Scott Lawrence <bytbox@xxxxxxxxx>
Incidentally, if we have tasks available from the server at a fixed
rate, and if the server will also perform arbitrary tasks for credits
actually proportional to CPU time (server is trusted so this is OK),
that could stabilize the "market" to prevent "inflation" or "deflation"
as well as giving people credits when the system starts up.
Michael Cohen
Scott Lawrence wrote:
> It's not complicated. Mikey just makes it seem like it is. I assure
> you - it's trivial! ;-)
>
> Modred should function well for 0-200 clients (3-250 computers total,
> say). We might extend the upper limit to like 500 if stuff goes really
> well, but we can hardly test that.
>
>
> On 12/28/09, Frederic Koehler <fkfire@xxxxxxxxx> wrote:
>> Man, that sounds pretty complicated. How big a cluster are we trying to
>> build, now?
>>
>> On Mon, Dec 28, 2009 at 10:54 PM, Scott Lawrence <bytbox@xxxxxxxxx> wrote:
>>
>>> I like this. And actually, the cluster could use the designated values
>>> to estimate how long tasks will take.
>>>
>>>
>>> Mikey: can you go to launchpad and make a blueprint for this?
>>>
>>> On 12/28/09, Michael Cohen <gnurdux@xxxxxxxxx> wrote:
>>>> The idea here is that if people are smart, stuff should basically
>>>> acquire a "market value." Credits would be basically like money. The
>>>> only issue is how credits originate, since someone needs to get them
>>>> before people spend them. But you could probably make some "official"
>>>> projects that you will automatically get credits at a certain rate for
>>>> participating in.
>>>>
>>>> Michael Cohen
>>>>
>>>> Scott Lawrence wrote:
>>>>> Oh, that's interesting. It might actually work... Other comments?
>>>>>
>>>>>
>>>>> On 12/28/09, Michael Cohen <gnurdux@xxxxxxxxx> wrote:
>>>>>> Wrong. You need to have credits to pay people with. And so you have
>>> to
>>>>>> spend more credits if you want to pay people more.
>>>>>>
>>>>>> Michael Cohen
>>>>>>
>>>>>> Scott Lawrence wrote:
>>>>>>> No. That leads to people trying to make their projects as valuable
>>>>>>> as
>>>>>>> possible, and within 3 days, the whole system will be worthless.
>>>>>>>
>>>>>>> Please remember that the mailing lists at launchpad don't perform
>>>>>>> reply-to mangling. Instruct your client accordingly.
>>>>>>>
>>>>>>> On 12/28/09, Michael Cohen <gnurdux@xxxxxxxxx> wrote:
>>>>>>>> I would actually award credits on a per-project basis. Each project
>>>>>>>> simply chooses how many credits to award for each task. Your
>>> computer
>>>>>>>> is offered so many credits for so much work; if it finishes it gets
>>> the
>>>>>>>> creds and otherwise not. Trying to do it based on time is dumb
>>> because
>>>>>>>> we don't care about time, we care about computational power
>>>>>>>> contributed.
>>>>>>>> If someone gives me 5000 hours, but in a VM limited to run at .1%
>>> of
>>>>>>>> their Pentium 3 CPU, then that isn't worth as much to me as 5 hours
>>> on
>>>>>>>> someone's brand new quad core powerhouse.
>>>>>>>>
>>>>>>>> Michael Cohen
>>>>>>>>
>>>>>>>> Scott Lawrence wrote:
>>>>>>>>> Some administrators will consider CPU time credits to be very
>>>>>>>>> important, though. Especially if they want them to be
>>>>>>>>> buyable/exchangable for that stink green stuff.
>>>>>>>>>
>>>>>>>>> Let's deal with this later - we could just validate the same way we
>>> do
>>>>>>>>> right answer/wrong answer validation.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 12/28/09, Frederic Koehler <fkfire@xxxxxxxxx> wrote:
>>>>>>>>>> ---------- Forwarded message ----------
>>>>>>>>>> From: Frederic Koehler <fkfire@xxxxxxxxx>
>>>>>>>>>> Date: Mon, Dec 28, 2009 at 10:17 PM
>>>>>>>>>> Subject: Re: [Modred] Concept stuff
>>>>>>>>>> To: Scott Lawrence <bytbox@xxxxxxxxx>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Network capacity can be protected by batching responses. Normal
>>>>>>>>>> clients
>>>>>>>>>> can
>>>>>>>>>> save several computations and send one big response (like when
>>>>>>>>>> exiting)
>>>>>>>>>> -
>>>>>>>>>> this will have similar bandwidth to big computations but avoid the
>>>>>>>>>> potential
>>>>>>>>>> for a really long computation wasting time on a bunch of computers
>>>>>>>>>> without
>>>>>>>>>> being solved.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mon, Dec 28, 2009 at 10:14 PM, Scott Lawrence <bytbox@xxxxxxxxx
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> * Maybe. But that still makes it too easy to gain false credits.
>>>>>>>>>>> * Yeah, I'm agreeing with this. The hub delegates to the
>>>>>>>>>>> servers,
>>>>>>>>>>> and
>>>>>>>>>>> the servers delegate to clients.
>>>>>>>>>>> * This will overload network capacity, and I don't like it. We
>>>>>>>>>>> should
>>>>>>>>>>> be able to trust clients to make long computations (long=over 30
>>>>>>>>>>> seconds). Maybe clients could give servers hints on how long
>>>>>>>>>>> they'll
>>>>>>>>>>> be on?
>>>>>>>>>>>
>>>>>>>>>>> On 12/28/09, Frederic Koehler <fkfire@xxxxxxxxx> wrote:
>>>>>>>>>>>> Hah, my email died, wow...Wonder what happened to it....
>>>>>>>>>>>> Anyway, this is something like what I wrote before:
>>>>>>>>>>>>
>>>>>>>>>>>> * CPU time credits can be very roughly estimated by averaging
>>>>>>>>>>>> response
>>>>>>>>>>> time.
>>>>>>>>>>>> It's not all _that_ important anyway if nothing is a behemoth
>>> task.
>>>>>>>>>>>> * The hub can immediately, upon establishing connection,
>>>>>>>>>>>> redirect
>>>>>>>>>>>> client
>>>>>>>>>>> to
>>>>>>>>>>>> a server. The server will still have to communicate with hub
>>>>>>>>>>>> somewhat,
>>>>>>>>>>> but
>>>>>>>>>>>> it can only send stuff necessarily pertaining to the hub.
>>>>>>>>>>>>
>>>>>>>>>>>> * Jobs should probably be many small tasks to avoid the risk of
>>>>>>>>>>>> losing
>>>>>>>>>>>> a
>>>>>>>>>>>> giant computation (since saving computation state is not
>>>>>>>>>>>> easy/generalizable). Beyond that, sending keep-alive packets is
>>>>>>>>>>>> enough
>>>>>>>>>>>> to
>>>>>>>>>>>> know when a client dies.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Dec 28, 2009 at 6:16 PM, Scott Lawrence <
>>> bytbox@xxxxxxxxx>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>> What?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 12/28/09, Frederic Koehler <fkfire@xxxxxxxxx> wrote:
>>>>>>>>>>>>>> On Mon, Dec 28, 2009 at 12:45 AM, Scott Lawrence
>>>>>>>>>>>>>> <bytbox@xxxxxxxxx>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> "What happens when a client disconnects with unfinished work?
>>> Is
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> work immeditately reassigned, or does the server wait for a
>>>>>>>>>>>>>>> specified
>>>>>>>>>>>>>>> period, etc. This could come up quite a lot because some
>>> clients
>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>> just disconnect as soon as work they submitted is completed."
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ouch. Good question. Here's one solution: small tasks
>>>>>>>>>>>>>>> (expected
>>>>>>>>>>> time
>>>>>>>>>>>>>>> <2 seconds) are always assigned to two or more
>>> clients/servers.
>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>> both disconnect, reassign, if one disconnects, use the other
>>>>>>>>>>>>>>> guy's
>>>>>>>>>>>>>>> answer. Large tasks, if a computer stops regularly checking
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> every
>>>>>>>>>>> 5
>>>>>>>>>>>>>>> or so seconds, give that computer's results to date to
>>>>>>>>>>>>>>> another
>>>>>>>>>>>>>>> computer. So yeah, I think a client should have to make
>>> regular
>>>>>>>>>>>>>>> reports to a server.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Here's another problem: how do we tell how many CPU time
>>> credits
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> grant a client? We can't always tell how long a problem
>>> should
>>>>>>>>>>>>>>> take
>>>>>>>>>>>>>>> beforehand.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Here's another problem: which computer should handle the
>>>>>>>>>>>>>>> clients?
>>>>>>>>>>>>>>> As
>>>>>>>>>>>>>>> I've been thinking about this, there are three types of
>>>>>>>>>>>>>>> computers,
>>>>>>>>>>> the
>>>>>>>>>>>>>>> single hub, the various dedicated servers (capable of storing
>>>>>>>>>>>>>>> permanent data), and the clients. (The hub is necessary -
>>>>>>>>>>>>>>> without
>>>>>>>>>>> it,
>>>>>>>>>>>>>>> the performance of the cluster drastically decreases.) So
>>>>>>>>>>>>>>> clients
>>>>>>>>>>>>>>> connect to the hub, and then the hub directs all computers.
>>> But
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> hub will get overloaded if 100 computers are checking in
>>>>>>>>>>>>>>> every
>>>>>>>>>>>>>>> 10
>>>>>>>>>>>>>>> seconds to give it more data (and then the hub has to pass
>>> this
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> other servers for storage, etc...). So at some point, the
>>>>>>>>>>>>>>> hub
>>>>>>>>>>>>>>> needs
>>>>>>>>>>>>>>> to tell the client to talk to the server. When?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Who wants to create that prototype?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 12/28/09, Scott Lawrence <bytbox@xxxxxxxxx> wrote:
>>>>>>>>>>>>>>>> This is where we start building prototypes. However, just to
>>>>>>>>>>>>>>>> keep
>>>>>>>>>>> the
>>>>>>>>>>>>>>>> theoretical side going: I disagree about the privacy issue.
>>>>>>>>>>>>>>>> Most
>>>>>>>>>>>>>>>> operations that would benefit from the CPU time of a cluster
>>>>>>>>>>> (notice
>>>>>>>>>>>>>>>> I'm not talking about the data storage and reliability
>>>>>>>>>>>>>>>> benefits,
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> aren't affected by the presence of clients) are not very
>>>>>>>>>>>>>>>> private.
>>>>>>>>>>>>>>>> Rendering nice screensavers ("Electric Sheep", I think that
>>>>>>>>>>>>>>>> one's
>>>>>>>>>>>>>>>> called), and hefty data sifting aren't private - who cares
>>>>>>>>>>>>>>>> about
>>>>>>>>>>> the
>>>>>>>>>>>>>>>> screensavers, and the data is generally public anyway (of
>>>>>>>>>>>>>>>> course
>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> wasn't, it would be marked so).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ray tracing and simulation could be more of an issue.
>>>>>>>>>>>>>>>> Hypothetical
>>>>>>>>>>>>>>>> situation: Alice is simulating how wind will affect her
>>>>>>>>>>>>>>>> proprietary
>>>>>>>>>>>>>>>> airplane design. Naturally, she can't hand off the whole
>>>>>>>>>>>>>>>> design,
>>>>>>>>>>> or
>>>>>>>>>>>>>>>> even parts of the design, to random client computers. This
>>> is
>>>>>>>>>>> where
>>>>>>>>>>>>>>>> the windows programmer says, "so the client computers can't
>>>>>>>>>>>>>>>> help
>>>>>>>>>>>>>>>> Alice." But that's not true - as a bad example, what if the
>>>>>>>>>>>>>>>> Modred
>>>>>>>>>>>>>>>> hub gave to a client computer 80 types of landing gear, and
>>>>>>>>>>>>>>>> told
>>>>>>>>>>> the
>>>>>>>>>>>>>>>> computer, not to simulate something, but to solve a general
>>>>>>>>>>>>>>>> formula
>>>>>>>>>>>>>>>> that could later be used in the computation in a trivial and
>>>>>>>>>>>>>>>> quick
>>>>>>>>>>>>>>>> way? If that client is evil, it will learn that Alice's
>>>>>>>>>>>>>>>> airplane
>>>>>>>>>>> has
>>>>>>>>>>>>>>>> some sort of landing gear.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Somebody needs to create a prototype of a server that can
>>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>>> arbitrary problems in some format, so we can all try to
>>>>>>>>>>>>>>>> trick
>>>>>>>>>>>>>>>> it.
>>>>>>>>>>> I
>>>>>>>>>>>>>>>> suggest lisp as the language, but it's up to the
>>>>>>>>>>>>>>>> implementer.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 12/28/09, Scott Lawrence <bytbox@xxxxxxxxx> wrote:
>>>>>>>>>>>>>>>>> ---------- Forwarded message ----------
>>>>>>>>>>>>>>>>> From: Frederic Koehler <fkfire@xxxxxxxxx>
>>>>>>>>>>>>>>>>> Date: Sun, 27 Dec 2009 23:21:28 -0500
>>>>>>>>>>>>>>>>> Subject: Re: [Modred] Concept stuff
>>>>>>>>>>>>>>>>> To: Scott Lawrence <bytbox@xxxxxxxxx>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * This is sort-of a solution (while obviously
>>>>>>>>>>>>>>>>> less-than-optimal
>>>>>>>>>>>>>>>>> security,
>>>>>>>>>>>>>>>>> some grid-computing stuff does this, like BOINC), however,
>>> it
>>>>>>>>>>> turns
>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> this may require custom validation methods - for example,
>>> it's
>>>>>>>>>>>>>>>>> normal
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> floating point values to be different on different
>>> computers,
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>> could apply for other computations.
>>>>>>>>>>>>>>>>> * A malicious client would only need to misbehave on
>>> certain
>>>>>>>>>>>>> problems
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> a malicious user could designate (or recognize obvious fake
>>>>>>>>>>>>> programs),
>>>>>>>>>>>>>>>>> allowing the fake program test to work.
>>>>>>>>>>>>>>>>> - A better idea would be to randomly reduplicate some
>>>>>>>>>>>>> computations
>>>>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>>>> times - the malicious client wouldn't notice anything, but
>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>> easily
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> singled out
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> * Thirdly is mostly the same thing I wrote before - only
>>>>>>>>>>>>> computations
>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>> are said to be totally unimportant privacy wise could
>>> benefit
>>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>> client-side computing.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So I really think that client-side computation is only a
>>> good
>>>>>>>>>>>>>>>>> idea
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> small subset of problems (like the type that there already
>>>>>>>>>>>>>>>>> exist
>>>>>>>>>>>>>>>>> massive
>>>>>>>>>>>>>>>>> grid computing solutions for, like SETI@HOME)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Sun, Dec 27, 2009 at 10:57 PM, Scott Lawrence <
>>>>>>>>>>> bytbox@xxxxxxxxx>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I want clients to be used for computation, and I want
>>> maximum
>>>>>>>>>>>>>>>>>> privacy+security given that restriction. Some ideas:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> With a large network, two computers can perform the same
>>>>>>>>>>>>> computation.
>>>>>>>>>>>>>>>>>> Furthermore, a smart modred hub can give fake problems to
>>>>>>>>>>> clients,
>>>>>>>>>>>>>>>>>> just to make sure that they're operating correctly. A
>>> client
>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> isn't operating correctly gets cut. (No second chances! A
>>>>>>>>>>>>>>>>>> program
>>>>>>>>>>>>>>>>>> could exploit that!)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If a user specifies a certain bit of data (SSN, for
>>> instance)
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> highly sensitive, modred should know not to hand off that
>>>>>>>>>>>>> computation
>>>>>>>>>>>>>>>>>> to a client. (If it does by accident, it certainly should
>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>> hand
>>>>>>>>>>>>>>>>>> off the data.) privacy++
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In all cases, computations should be anonymous. privacy++
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Other ideas?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 12/27/09, Frederic Koehler <fkfire@xxxxxxxxx> wrote:
>>>>>>>>>>>>>>>>>>> The idea for client-side computation implies that we have
>>>>>>>>>>>>>>>>>>> highly-trusted
>>>>>>>>>>>>>>>>>>> clients... (we know they won't provide invalid answers)
>>>>>>>>>>>>>>>>>>> Otherwise,
>>>>>>>>>>>>>>>>>>> client-side computation requires verifying answers and so
>>> is
>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>> useful
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> a few NP-ish problems. In addition (assuming trusted
>>>>>>>>>>>>>>>>>>> clients).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Also, it means that, since computations can contain
>>>>>>>>>>>>>>>>>>> sensitive
>>>>>>>>>>>>> data,
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> abillity to spread the computation is limited - unless we
>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> computation is not user-sensitive, it can only try to use
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>> user's
>>>>>>>>>>>>>>>>>>> client(s). This way we also know that the client has no
>>>>>>>>>>> interest
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> sabatoging answers to mess with other users (except to
>>>>>>>>>>>>>>>>>>> exploit
>>>>>>>>>>>>>>>>>> server-side
>>>>>>>>>>>>>>>>>>> weaknesses, which is inevitable).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Sat, Dec 26, 2009 at 10:28 PM, Scott Lawrence <
>>>>>>>>>>>>> bytbox@xxxxxxxxx>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> Here is what, as I envision it, will make modred unique
>>>>>>>>>>>>>>>>>>>> (and
>>>>>>>>>>>>> hard):
>>>>>>>>>>>>>>>>>>>> * Support for clients who can come and leave, lending
>>> CPU
>>>>>>>>>>> time
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>> using CPU time as they choose. There are some clusters
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>> support
>>>>>>>>>>>>>>>>>>>> this, but not very many.
>>>>>>>>>>>>>>>>>>>> * Support for computers participating across the
>>> internet.
>>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>> goes
>>>>>>>>>>>>>>>>>>>> along with the previous part, but remember we need
>>> security
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>> this worth anything. This also means that user data
>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>> potentially
>>>>>>>>>>>>>>>>>>>> be passed to untrusted computers - we need a way to
>>> prevent
>>>>>>>>>>>>>>>>>>>> this.
>>>>>>>>>>>>>>>>>>>> * The ability for clients to run on any OS, using perl,
>>>>>>>>>>> python,
>>>>>>>>>>>>>>>>>>>> java,
>>>>>>>>>>>>>>>>>>>> or (on unix systems) C and C++ (servers and the hub will
>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> run
>>>>>>>>>>>>>>>>>>>> on linux or at least another unix, or a dedicated OS
>>> which
>>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>> decide to write)
>>>>>>>>>>>>>>>>>>>> * Modred has great ease of use because it acts as a
>>> single
>>>>>>>>>>>>> unified
>>>>>>>>>>>>>>>>>>>> computer - a special client program exists that allows
>>> one
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> log
>>>>>>>>>>>>>>> in,
>>>>>>>>>>>>>>>>>>>> access and edit files, etc... This is very close to
>>> unique
>>>>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>> google
>>>>>>>>>>>>>>>>>>>> has it, though
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Because of that last point, many OS design issues should
>>>>>>>>>>>>>>>>>>>> come
>>>>>>>>>>> up
>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>>>> we code modred. (I think Freddy pointed this out?) Thus,
>>> we
>>>>>>>>>>> have
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>> chance to fix flaws in standard unix, incorporating plan
>>>>>>>>>>> 9-type
>>>>>>>>>>>>>>> stuff
>>>>>>>>>>>>>>>>>>>> (google it and read about it - Plan 9 from Bell Labs,
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> future of unix was) while also creating an actually
>>> usable
>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>>>> interface. (No offense, but to a newbie
>>> non-super-technical
>>>>>>>>>>>>>>>>>>>> user,
>>>>>>>>>>>>>>>>>>>> linux is a bit harsh...)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Some implementation questions and ideas:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> - how will updates be handled? Remember we've got 200
>>>>>>>>>>>>>>>>>>>> computers
>>>>>>>>>>>>>>>>>>>> potentially, some of which might be clients that want to
>>>>>>>>>>>>>>>>>>>> participate
>>>>>>>>>>>>>>>>>>>> in multiple clusters.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> - maybe we should have programs not include front ends.
>>>>>>>>>>>>> Instead,
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> modred software creates a front-end from the program's
>>> self
>>>>>>>>>>>>>>>>>>>> description. This would enforce a consistent user
>>>>>>>>>>>>>>>>>>>> interface
>>>>>>>>>>> if
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>> could implement it well
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> - how can we keep users from being able to snoop on
>>>>>>>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>>>>> others'
>>>>>>>>>>>>>>>>>>>> data?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> That's just a sample to get people thinking.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On 12/26/09, David Tolnay <dtolnay@xxxxxxxxx> wrote:
>>>>>>>>>>>>>>>>>>>>> Before diving in to specifics about the implementation
>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>> think
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> need
>>>>>>>>>>>>>>>>>>>>> to decide how we want modred to be different from
>>>>>>>>>>>>>>>>>>>>> (read:
>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>> than)
>>>>>>>>>>>>>>>>>>>>> existing bootable cluster environments. Here is a short
>>>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> check
>>>>>>>>>>>>>>>>>>>>> out:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Bootable Cluster CD (http://bccd.net/) - folks
>>> presented
>>>>>>>>>>> this
>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>> SC09
>>>>>>>>>>>>>>>>>>>>> in portland, it was pretty neat stuff. Packed with
>>>>>>>>>>>>>>>>>>>>> education
>>>>>>>>>>> /
>>>>>>>>>>>>>>>>>>>>> debugging / visualization features
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Oscar
>>>>>>>>>>>>>>>>>>>>> (http://svn.oscar.openclustergroup.org/trac/oscar)
>>> -
>>>>>>>>>>>>> very
>>>>>>>>>>>>>>>>>>>>> trivially simple way to transform an existing unix lab
>>>>>>>>>>>>>>>>>>>>> into
>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> cluster
>>>>>>>>>>>>>>>>>>>>> resource
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Lnx-bbc (http://www.lnx-bbc.com/) - includes cowsay!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Perceus/warewulf (http://www.perceus.org/portal/) - a
>>> lot
>>>>>>>>>>> of
>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>> sites made reference to this, haven't read too much
>>> about
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> What specifically do you want to improve over any of
>>>>>>>>>>>>>>>>>>>>> these?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On 12/25/09, Frederic Koehler <fkfire@xxxxxxxxx> wrote:
>>>>>>>>>>>>>>>>>>>>>> So, as far as I understand this project, the idea is
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>> build
>>>>>>>>>>>>>>>>>>>>>> both a client library and a program using the library
>>> to
>>>>>>>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>> clustering
>>>>>>>>>>>>>>>>>>>>>> stuff, along with matching server/hub foo (the library
>>>>>>>>>>> might
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>> whatever, not important).
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> So from this understanding, it seems that the system
>>>>>>>>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>> basic pseudo-operating system stuff and programs can
>>>>>>>>>>>>>>>>>>>>>> build
>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> that,
>>>>>>>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>>>>>> like they would normally build on their local
>>> libc/kernel
>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> stuff.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> So (I sure like the word "so" today...) if we want the
>>>>>>>>>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> general
>>>>>>>>>>>>>>>>>>>>>> os-like stuff it seems their needs to be support for:
>>>>>>>>>>>>>>>>>>>>>> * A simpe message passing model - abstract away all
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> TCP-foo,
>>>>>>>>>>>>>>>>>>>> maybe
>>>>>>>>>>>>>>>>>>>>>> use existing foo here (obviously needs fleshing out)
>>>>>>>>>>>>>>>>>>>>>> * Permanent storage IO (clone the unix write(),
>>>>>>>>>>>>>>>>>>>>>> read(),
>>>>>>>>>>>>>>>>>>>>>> open()
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> sync()
>>>>>>>>>>>>>>>>>>>>>> model, or maybe just use one of the existing
>>>>>>>>>>>>>>>>>>>>>> database-ish
>>>>>>>>>>>>> nosql
>>>>>>>>>>>>>>>>>> things
>>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>> there)
>>>>>>>>>>>>>>>>>>>>>> - Unix-ish model - you create your data
>>> hunk,
>>>>>>>>>>> say
>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>>>>>> this stuff in it, then after sync() we know it's
>>> actually
>>>>>>>>>>>>>>>>>>>>>> somewhere
>>>>>>>>>>>>>>>>>>>>>> written
>>>>>>>>>>>>>>>>>>>>>> on a hard-drive, and other things can read it too
>>>>>>>>>>>>>>>>>>>>>> - Unless this isn't in fact needed (but I
>>>>>>>>>>>>>>>>>>>>>> assume
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> is)
>>>>>>>>>>>>>>>>>>>>>> - Also need to figure out if it's
>>>>>>>>>>>>>>>>>>>>>> filesystem-ish
>>>>>>>>>>>>> foo
>>>>>>>>>>>>>>>>>>>>>> (hierarchial) we want or more relational database-ish
>>>>>>>>>>>>>>>>>>>>>> stuff
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> * A task delegation model - some type of
>>>>>>>>>>>>>>>>>>>>>> map/reduce-ish
>>>>>>>>>>>>> stuff
>>>>>>>>>>>>>>>>>>>>>> - Servers have a few built-in computations,
>>> and
>>>>>>>>>>>>> client
>>>>>>>>>>>>>>>>>>>> utilizes
>>>>>>>>>>>>>>>>>>>>>> them?
>>>>>>>>>>>>>>>>>>>>>> - Or more complex, servers run sandboxed
>>>>>>>>>>>>> computational
>>>>>>>>>>>>>>>>>> code?
>>>>>>>>>>>>>>>>>>>>>> * A security system?
>>>>>>>>>>>>>>>>>>>>>> - Needs fleshing out
>>>>>>>>>>>>>>>>>>>>>> - Presumably what the "hub" manages - it's the
>>>>>>>>>>>>>>>>>>>>>> trusted
>>>>>>>>>>>>>>>>>>>>>> thing
>>>>>>>>>>>>>>>>>>>>>> - Obviously, not everybody is allowed to use
>>> the
>>>>>>>>>>>>> cluster
>>>>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>> computation, not everybody can find out what everybody
>>>>>>>>>>>>>>>>>>>>>> else
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> doing,
>>>>>>>>>>>>>>>>>>>> etc.
>>>>>>>>>>>>>>>>>>>>>> - But also, is their a limit on storage, are
>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>> things
>>>>>>>>>>>>>>>>>>>> prioritized
>>>>>>>>>>>>>>>>>>>>>> over others, ?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Theroretically, server's are written to provide the io
>>>>>>>>>>>>>>>>>>>>>> backend
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> allow
>>>>>>>>>>>>>>>>>>>>>> for task delegation, clients use the api, although hub
>>>>>>>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>>> cut
>>>>>>>>>>>>>>>>>>>>>> out
>>>>>>>>>>>>>>>>>>>>>> delegating all the file io and figuring out what the
>>>>>>>>>>>>>>>>>>>>>> state
>>>>>>>>>>> of
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> is.
>>>>>>>>>>>>>>>>>>>>>> On top of some mixture of this, one could build a
>>> simple
>>>>>>>>>>>>>>>>>>>>>> unix-ish
>>>>>>>>>>>>>>>>>>>>>> pseudo-cli, theroretically, as well as real software.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Anyway, before actually doing anything, people should
>>>>>>>>>>>>>>>>>>>>>> read
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>>>>> PVM
>>>>>>>>>>>>>>>>>>>>>> (Parallel Virtual Machine) and the like (maybe also
>>>>>>>>>>>>>>>>>>>>>> Hadoop
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>> foo-ish
>>>>>>>>>>>>>>>>>>>>>> stuff) so Modred isn't just a bad clone of it
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Anyway, (yes, twice in a row!), I figured _someone_
>>>>>>>>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> respond
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> Scott,
>>>>>>>>>>>>>>>>>>>>>> otherwise he'd feel all lonely and sad :P Now he can
>>> have
>>>>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>> warm
>>>>>>>>>>>>>>>>>> fuzzy
>>>>>>>>>>>>>>>>>>>>>> feeling of deep confusion and uncertainty instead :P
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> On Fri, Dec 25, 2009 at 11:06 PM, Scott Lawrence
>>>>>>>>>>>>>>>>>>>>>> <bytbox@xxxxxxxxx>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>> ---------- Forwarded message ----------
>>>>>>>>>>>>>>>>>>>>>>> From: Scott Lawrence <bytbox@xxxxxxxxx>
>>>>>>>>>>>>>>>>>>>>>>> Date: Fri, 25 Dec 2009 19:20:13 -0500
>>>>>>>>>>>>>>>>>>>>>>> Subject: Design Overview
>>>>>>>>>>>>>>>>>>>>>>> To: modred <modred@xxxxxxxxxxxxxxxxxxx>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I'm going to assume that everyone understands the
>>> basic
>>>>>>>>>>>>>>> concepts
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>>>> modred: a set of networked computers (by 'networked'
>>>>>>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>>>>>>> mean,
>>>>>>>>>>>>>>>>>> they're
>>>>>>>>>>>>>>>>>>>>>>> all on the internet), divided for the sake of
>>> discussion
>>>>>>>>>>>>> into
>>>>>>>>>>>>>>>>>>>>>>> three
>>>>>>>>>>>>>>>>>>>>>>> classes: the 'hub' (the dude in charge, who
>>>>>>>>>>>>>>>>>>>>>>> compupters
>>>>>>>>>>> who
>>>>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>>> join connect to), the 'servers' (dedicated computers
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>> pretty much relied on not to go down, although
>>>>>>>>>>>>>>>>>>>>>>> redundancy
>>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>>> always
>>>>>>>>>>>>>>>>>>>>>>> nice), and the 'clients' (computers that send in
>>>>>>>>>>>>>>>>>>>>>>> requests
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>>> used for spare CPU cycles.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Ok, so much for assumptions... :-)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Things *I* think any design should emphasize:
>>>>>>>>>>>>>>>>>>>>>>> * security.
>>>>>>>>>>>>>>>>>>>>>>> * relative ease of use, while retaining significant
>>>>>>>>>>> power.
>>>>>>>>>>>>>>>>>>>>>>> Challenging. In particular, it should be possible to
>>>>>>>>>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> modred
>>>>>>>>>>>>>>>>>>>>>>> network in under an hour, provided the computers are
>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>>>>> up.
>>>>>>>>>>>>>>>>>>>>>>> * along with the previous bullet point, having an
>>>>>>>>>>>>>>>>>>>>>>> interface
>>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>>> lets
>>>>>>>>>>>>>>>>>>>>>>> one use the entire network like a single computer.
>>> This
>>>>>>>>>>> is
>>>>>>>>>>>>>>> sort
>>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>>> like the way google docs works, except the cloud is
>>>>>>>>>>> private
>>>>>>>>>>>>>>>>>>>>>>> * therefore, it should be a multi-user system with
>>>>>>>>>>>>>>>>>>>>>>> well-designed
>>>>>>>>>>>>>>>>>>>>>>> privileges etc...
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I'm not going to discuss my implementation ideas,
>>> let's
>>>>>>>>>>>>>>>>>>>>>>> hear
>>>>>>>>>>>>>>>>>>>>>>> others
>>>>>>>>>>>>>>>>>>>>>>> first.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> Scott Lawrence
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Webmaster
>>>>>>>>>>>>>>>>>>>>>>> The Blair Robot Project
>>>>>>>>>>>>>>>>>>>>>>> Montgomery Blair High School
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>> Scott Lawrence
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Webmaster
>>>>>>>>>>>>>>>>>>>>>>> The Blair Robot Project
>>>>>>>>>>>>>>>>>>>>>>> Montgomery Blair High School
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>>>> Mailing list: https://launchpad.net/~modred
>>>>>>>>>>>>>>>>>>>>>> Post to : modred@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~modred
>>>>>>>>>>>>>>>>>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>>> Mailing list: https://launchpad.net/~modred
>>>>>>>>>>>>>>>>>>>>> Post to : modred@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~modred
>>>>>>>>>>>>>>>>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>> Scott Lawrence
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Webmaster
>>>>>>>>>>>>>>>>>>>> The Blair Robot Project
>>>>>>>>>>>>>>>>>>>> Montgomery Blair High School
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>> Mailing list: https://launchpad.net/~modred
>>>>>>>>>>>>>>>>>>>> Post to : modred@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~modred
>>>>>>>>>>>>>>>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Scott Lawrence
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Webmaster
>>>>>>>>>>>>>>>>>> The Blair Robot Project
>>>>>>>>>>>>>>>>>> Montgomery Blair High School
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Scott Lawrence
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Webmaster
>>>>>>>>>>>>>>>>> The Blair Robot Project
>>>>>>>>>>>>>>>>> Montgomery Blair High School
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Scott Lawrence
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Webmaster
>>>>>>>>>>>>>>>> The Blair Robot Project
>>>>>>>>>>>>>>>> Montgomery Blair High School
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Scott Lawrence
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Webmaster
>>>>>>>>>>>>>>> The Blair Robot Project
>>>>>>>>>>>>>>> Montgomery Blair High School
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> Mailing list: https://launchpad.net/~modred
>>>>>>>>>>>>>>> Post to : modred@xxxxxxxxxxxxxxxxxxx
>>>>>>>>>>>>>>> Unsubscribe : https://launchpad.net/~modred
>>>>>>>>>>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Scott Lawrence
>>>>>>>>>>>>>
>>>>>>>>>>>>> Webmaster
>>>>>>>>>>>>> The Blair Robot Project
>>>>>>>>>>>>> Montgomery Blair High School
>>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Scott Lawrence
>>>>>>>>>>>
>>>>>>>>>>> Webmaster
>>>>>>>>>>> The Blair Robot Project
>>>>>>>>>>> Montgomery Blair High School
>>>>>>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list: https://launchpad.net/~modred
>>>>>> Post to : modred@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~modred
>>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~modred
>>>> Post to : modred@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~modred
>>>> More help : https://help.launchpad.net/ListHelp
>>>>
>>>
>>> --
>>> Scott Lawrence
>>>
>>> Webmaster
>>> The Blair Robot Project
>>> Montgomery Blair High School
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~modred
>>> Post to : modred@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~modred
>>> More help : https://help.launchpad.net/ListHelp
>>>
>
>
--
Scott Lawrence
Webmaster
The Blair Robot Project
Montgomery Blair High School
References
