modred team mailing list archive

[Frederic Koehler <fkfire@xxxxxxxxx>]

So a normal, single-process program would run on one server, right?
And then theroretically it's resources would be shared like on a normal
computer,
like there would be a distributed file-system, cross-computer IPC, and the
like?

This would be like a more user-spacish mosix (http://www.mosix.org, might be
interesting to read about), it's also somewhat like OpenVMS clustering, I
think. Of course, both of those involve custom kernels, and aren't totally
similar, but the same conceptual use, right?

As far as IPC, there is no obvious problem with traditional pipes/sockets,
but IPC through shared mmap is most likely impossible to support (even MOSIX
doesn't support it) (technically speaking, it _might_ be supportable with
absolutely horrible performance, but probably unworthwhile).

On Wed, Dec 30, 2009 at 1:19 PM, Scott Lawrence <bytbox@xxxxxxxxx> wrote:

> My idea here, as I explained to Mikey last night over chat, is that
> the transparent access allows us to run programs out-of-the-box. Most
> programs won't perform well like this, for various reasons, and that's
> fine, a few will perform ok, and a few might happen to perform
> excellently.  But these programs could be tweaked by telling the
> cluster that when the program forks in such-and-such place, it should
> be put on another computer, and the program should generally be
> packaged with the following data, etc... etc..., and stuff like that
> to minimize network usage.
>
> Programs written for the cluster will generally not want to use the
> transparent features.  Of course, if they do, it'll still work.
>
>
> On 12/30/09, Michael Cohen <gnurdux@xxxxxxxxx> wrote:
> > If you want networked memory access, you need to be using mmap()
> > somewhere.  I agree with you that it is probably not such a great idea;
> > Scott wants transparent stuff over the cluster but in reality that is
> > probably pretty much not a possibility, as there is a big difference
> > between .0000001 second latency and .1 second latency, so any code that
> > makes almost any use of any inter-process communication would run many
> > times slower on the cluster than on one computer.
> >
> > Michael Cohen
> >
> > Frederic Koehler wrote:
> >> Uh...what exactly is this supposed to be used for?
> >>
> >> Trying to abstract networked data as mmap over some type of network file
> >> system
> >> is an ok option, but it's not simple (for one, figuring out the
> >> synchronization and the like, and also having to build a file system or
> >> finding some obscure one which meets requirements).
> >>
> >> Also, it's not easily usable for clients, so if they need to access data
> >> they still probably need some more traditional interface.
> >>
> >> On the other hand, a more traditional interface (for ex. cloning read(),
> >> write() and the like in a user-space library, or some other approach)
> >> could
> >> be used for both.
> >>
> >> Anyway, though, doing lots of reading/writing over the network is really
> >> bad
> >> performance-wise (it clogs up the network and has horrible latency), so
> I
> >> don't know if whatever you're planning is actually a good idea
> (especially
> >> if "memory access often involves the cluster"...).
> >>
> >>
> >> On Wed, Dec 30, 2009 at 12:01 AM, Michael Cohen <gnurdux@xxxxxxxxx>
> wrote:
> >>
> >>> I just realized that we could actually do this by mmapping to stuff in
> >>> some
> >>> kind of network file system.  It might be a little tricky to ensure
> that
> >>> the
> >>> data is actually up to date, but it probably could be done.
> >>>
> >>> Michael Cohen
> >>>
> >>>
> >>> Scott Lawrence wrote:
> >>>
> >>>> I, the perennial linux newbie, was somehow under the impression that
> >>>> memory access was a system call.  Ouch.
> >>>>
> >>>> So the issue is that memory access often involves the cluster, and so
> >>>> we should have a fast way to interrupt it to talk over the network.
> >>>> (Obviously, we wouldn't re-fetch the same data repeatedly.)
> >>>>
> >>>> Mikey suggested mmap.  The issue is that this only maps to actual
> >>>> files, so it wouldn't work for network access.  Anyone know of a way
> >>>> to mount a kernel driver in user space?  Or is there a better way?
> >>>>
> >>>>
> >>> _______________________________________________
> >>> Mailing list: https://launchpad.net/~modred
> >>> Post to     : modred@xxxxxxxxxxxxxxxxxxx
> >>> Unsubscribe : https://launchpad.net/~modred
> >>> More help   : https://help.launchpad.net/ListHelp
> >>>
> >>
> >
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~modred
> > Post to     : modred@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~modred
> > More help   : https://help.launchpad.net/ListHelp
> >
>
>
> --
> Scott Lawrence
>
> Webmaster
> The Blair Robot Project
> Montgomery Blair High School
>
> _______________________________________________
> Mailing list: https://launchpad.net/~modred
> Post to     : modred@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~modred
> More help   : https://help.launchpad.net/ListHelp
>