CPUShare reached a great milestone with its fully working implementation for over a month. Real money has already flown into the system and some seller already has several euros in their account. The system is fully autonomous and working really well.

I got significant feedback (you can find some relevant post on the mailing list), but every potential buyer I talked with isn't confortable with the seccomp model. The most skilled are willing to look into it, but it would still take time to port apps to seccomp, and I see a lot of opportunity being missed in the short-mid term.

Everyone practically asks for virtual machines (if they don't ask explicitly they'd still be ok with them). The major obstacle is that nobody wants to modify their existing software, some would even want to run non-linux proprietary software on top of CPUShare.

The plan has always been to integrate real virtualization some day along with seccomp but the buyer requests made this really a first priority now. It's quite certain that CPUShare can't be successful without looking like a real ethernet cluster from the point of view of the buyer.

It's also certain it must be a p2p kind of cluster where each seller is directly connected to the buyer through encrypted VPN based on UDP (with the seller optionally behind a firewall doing UDP-NAT), or CPUShare would never scale to the million nodes (that's the plan yes).

So the current development plan for CPUShare is to make a tag of the current version, and to migrate entirely on top of KVM using VDE cryptcab or some other similar method (using the tap-fd on the KVM side) to create a virtual network of KVM virtual machines for each buy order. Supporting seccomp and KVM at the same time was the original plan but my resources are still limited, and given I also planned to change the seccomp protocol, initially I'll drop seccomp out of CPUShare and potentially I will add it later.

I still think seccomp is more secure for the seller and ideal perfect model if one has to write a new CPUShare app from scratch. And the size of the JtR patch kind of confirms it in practice too. But my time is limited and I can't port all software to seccomp, switching to KVM+VDE is going to open CPUShare to practically anything that runs on a cluster.

I also decided the CPUShare livecd will include trusted computing and that's likely the only form of trusted computing that CPUShare will support in the short term. This way buyers can be sure the seller is really running the livecd and it has absolutely no "software" way to look or interfere with what the buyer is doing with its hardware (unless there are bugs in the livecd). But at the same time the seller booting the livecd knows the buyer can't do any damage to his hardware (unless there are bugs in KVM) thanks to the whole software virtualization stack running in the seller computer being open source.

From a buyer point of view enabling trusted computing will translate in clicking a button while creating (or editing...) the buyer order. The isofs image size will be subtracted from the available ram because it will have to be stored in tmpfs. I think this is a feature lots of buyers will need in order to switch to CPUShare.

From a seller point of view, enabling trusted computing will simply mean booting the livecd directly on top of the hardware with a CDR or a usb stick (the usb stick can be unplugged after the system booted).

With this new model the buyer will have only to provide a bootable isofs to the CPUShare buy client (this isofs is readonly and will be cached in the seller filesystem). Then while booting the isofs, each KVM virtual machine will try to connect to the buyer IP:port where the vde_cryptcab server is listening to (the cpushare sell client will take care of that). Clients will talk with each other through the vde_switch running on the buyer node (the buyer has to start manually vde_switch and the vde_cryptcab server). In the future a fully interconnected network will be possible too with clients that can have udp open ports (so with DNAT or exposed to the internet).

Once the isofs starts running it should run dhcpcd so that the dhcp server in the buyer node will assign the IP. This can be achieved by connecting the vde_switch of the server to a tap device in the buyer node and having the dhcpd server in the buyer node listening on the tap device. The isofs in the seller guest will then mount an nfs filesystem in the buyer node. The nfs would better be running in a KVM guest running in the buyer node also connected to the vde_switch of the buyer node, but the nfs server could be anywhere (also in a different hardware node in the buyer local ethernet network), the tap traffic of the cluster could even be routed to the internet with an iptables rule.

Overall this will work exactly like a real cluster, you can ssh into each machine, and all operative systems will run.

Development of the isofs image won't require any CPUShare usage or knowledge at all, just a KVM + VDE local installation in a single node will be enough to develop the isofs to boot later into CPUShare.

Unfortunately all hardware except x86-64 systems won't be able to join CPUShare anymore in the short term. The isofs to boot in KVM can be x86 or x86-64 (only the host has to be a 64bit x86-64 supported by KVM).

The only requirement left for the buyer is not to blindly trust the results of the computations (especially if run once or without trusted computing attestation), and to keep in mind those isofs booted images can disappear at any time from the cluster (like when the seller shutdown its computer). But any pre-existing grid software for real world clusters must already handle nodes going down or it would fail abruptly on a real cluster where nodes break from time to time.

In very short CPUShare will effectively become similar to a p2p implementation of EC2, without local storage (local storage of the isofs will mostly be tmpfs with readonly data in squashfs), with nodes that can go down, but with unlimited network bandwidth, unlimited scalability, a virtualized ethernet network that can be connected to the local intranet, and a price decided by market forces .

Hope this model will make the buyers really confortable in using or at least testing CPUShare.

If you've any ideas please post to cpushare-discuss(at)cpushare.com mailing list (if you're not subscribed I'll have to approve your message manually but that's ok).