Re: NEWB: How to get started on writing new client?

On Sun, Nov 11, 2007 at 11:20:13PM -0500, Nic Watson wrote:
> OK.  I think I need a seccomp primer.  Any pointers?  I'm puzzled by the

a very short doc can be found here http://www.win.tue.nl/~aeb/linux/lk/lk-14.html

> term "bytecode".  From my reading of the makefiles, buyer clients are
> regular x86 (or PPC) machine code, not requiring any run-time
> interpretation until a system call is made.  Am I mistaken?  I think the
> term bytecode can be unfair to seccomp:  the clients run full speed.

This is a terminology mistake I did... I thought I could say "x86
bytecode" too. The clients run full speed without interpretation and
without needing virtual machines.

> Can seccomp code be multi-threaded?  A corollary question:  if a seller

As far as seccomp is concerned it could, but not in the CPUShare
seccomp model. As far as CPUShare is concerned an additional cpu in
SMP is only an additional client running in the same machine. It
doesn't make much sense to optimize specifically inter-node for SMP
when the problem to solve must not require high performance
interconnects for it to run nicely on top of CPUShare.

> is selling time on his box, is he selling one core or all the cores
> (i.e. can he have multiple simultaneous buyers?)

The ./CPUShare script will launch one client for each CPU in the
system. Those will run independently, each client may be connected to
a different buyer.

> I think the usefulness of your platform will depend on how capable the
> seccomp jail is.  Otherwise, the full OS virtualization approach (ala
> EC2) I think will win.

The two main problems in an approach ala EC2 is that the CPUShare
virtual machines must run into a VPN and they can't be exposed to the
live internet, and this has to be enforced at the virtual machine
level. The second problem is that this storage will go away at any
time if the client disconnects. The only reliable possible storage is
on the buyer hardware, providing persistent storage in the seller
hardware is not feasible or at least not easily and even if feasible
at all with fault tolerance it still wouldn't be 100% reliable.

> For example, in a previous e-mail, you note that CPUShare doesn't allow
> any local storage at this point, but it can be simulated in RAM.  The
> easiest-to-use way to do this is to mount a RAM disk.  Can you do that
> is a seccomp jail?

You certainly can't mount anything, you can't even read/write to
/dev/tmpfs unless the files were already open.

One way to simulate it, is to override the file operations if glibc to
call into your own version of open/read/write/close. Those functions
can also be written in a way that they will actually
open/read/write/close files in the _buyer_ filesystem (to be
persistent), and not in the ram of the seller.

The other way would be to run UML under seccomp and to modify uml to
have a virtual uml filesystem that open/read/write/close files on the
buyer too (or in ram).

> It is tempting to think of adding callbacks to the sell client python
> script to perform certain operations you can't in the seccomp jail, like
> mounting a RAM disk.  However, if you add more than a handful of
> callbacks, you're essentially implementing your own OS again.

If the interprocess communication isn't too heavy, you very well can
write your own protocol to communicate with the buyer and have your
per-seller directory where open/read/write/close() run by the seccomp
binary in the seller cpu, will execute open/read/write/close in the OS
of the buyer. No server or client changes are required for this.

> So, a simple question would be, why not just use pure virtual appliances
> on VMWare/VirtualBox/KVM as buyer clients?  Performance is similar.
> Security is similar.  If seccomp really is a temporary measure (my first

Security is similar but not the same:

	 http://kerneltrap.org/OpenBSD/Virtualization_Security

Infact I think it's somewhat safer to run the CPUShareLiveCD on top of
the hardware booting with the cd or with an usb stick, instead of
running it under virtualization, but the point is that even if run
inside virtualization seccomp is an additional bonus even when run
inside the guest OS. Seccomp is so secure because it's simple,
virtualization is all but simple.

KVM will certainly be supported.

> batch of vPro boxes just arrived), why not use a more convenient
> temporary approach?

The problem is that I can't just fork a KVM/QEMU task with a
filesystem passed by the buyer and be done with it. In some ways a VPN
should be created, and ideally once I enable KVM/QEMU I want to enable
a full p2p model where each KVM/QEMU virtual machine will see the
virtual IPs of each other seller and the virtual IP of the buyer too,
so really anything will run and it will look like if KVM is running in
a local network and not on the internet.

Both approaches will be supported at the same time. Everyone is
required to run seccomp. KVM/QEMU will be optional depending on the
level of the security each seller requires (the risk with KVM/QEMU
will increase).

The reason I think seccomp will makes sense even after KVM/QEMU is
introduced, is that writing an app like John for seccomp may be
simpler than for KVM/QEMU. seccomp avoids you to create filesystems,
building kernels just to run some assembly code in the CPU of the
sellers etc... As I said I don't want a virtual machine with fixed
filesystem and kernel, the virtual machine could even run windows as
far as I'm concerned. All CPUShare will do is to offer a virtual
machine to the buyer only reachable through VPN, and the virtual qemu
disk image will be passed by the buyer with whatever data inside. This
isn't something will happen too soon, I want to really get a pure p2p
network protocol with seccomp before adding the KVM/QEMU model to
CPUShare.
-- 
cpushare-discuss_X_at_X_cpushare.com mailing list - http://www.cpushare.com/
To unsubscribe, send mail to cpushare-discuss-unsubscribe_X_at_X_cpushare.com