← Back to team overview

modred team mailing list archive

Re: A new conundrum

 

If you want networked memory access, you need to be using mmap() somewhere. I agree with you that it is probably not such a great idea; Scott wants transparent stuff over the cluster but in reality that is probably pretty much not a possibility, as there is a big difference between .0000001 second latency and .1 second latency, so any code that makes almost any use of any inter-process communication would run many times slower on the cluster than on one computer.

Michael Cohen

Frederic Koehler wrote:
Uh...what exactly is this supposed to be used for?

Trying to abstract networked data as mmap over some type of network file
system
is an ok option, but it's not simple (for one, figuring out the
synchronization and the like, and also having to build a file system or
finding some obscure one which meets requirements).

Also, it's not easily usable for clients, so if they need to access data
they still probably need some more traditional interface.

On the other hand, a more traditional interface (for ex. cloning read(),
write() and the like in a user-space library, or some other approach) could
be used for both.

Anyway, though, doing lots of reading/writing over the network is really bad
performance-wise (it clogs up the network and has horrible latency), so I
don't know if whatever you're planning is actually a good idea (especially
if "memory access often involves the cluster"...).


On Wed, Dec 30, 2009 at 12:01 AM, Michael Cohen <gnurdux@xxxxxxxxx> wrote:

I just realized that we could actually do this by mmapping to stuff in some
kind of network file system.  It might be a little tricky to ensure that the
data is actually up to date, but it probably could be done.

Michael Cohen


Scott Lawrence wrote:

I, the perennial linux newbie, was somehow under the impression that
memory access was a system call.  Ouch.

So the issue is that memory access often involves the cluster, and so
we should have a fast way to interrupt it to talk over the network.
(Obviously, we wouldn't re-fetch the same data repeatedly.)

Mikey suggested mmap.  The issue is that this only maps to actual
files, so it wouldn't work for network access.  Anyone know of a way
to mount a kernel driver in user space?  Or is there a better way?


_______________________________________________
Mailing list: https://launchpad.net/~modred
Post to     : modred@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~modred
More help   : https://help.launchpad.net/ListHelp






Follow ups

References