Re: NEWB: How to get started on writing new client?

On Thu, Nov 08, 2007 at 12:07:25AM -0500, Nic Watson wrote:
> Cool project!

Thanks!

> Just a newb here.  I've managed to install the client and run the basic
> seccomp_test.py successfully.

So far so good.

> So, how does one write a client for this beast?  Is there any
> documentation for the buyer's program API?  I can grok Python, C, and
> C++, but prose is of course easier to read.

I added some doc in the wiki but it's very basic and it doesn't cover
the implementation details much.

You can watch the john patch in the patches/ directory as an example.

> I don't mind the management stuff done in Python, but for performance
> reasons, I have some C++ code to run.  Is this possible?

Ok, the bytecode crunching the numbers at 100% cpu load must be
machine bytecode at this point because of the seccomp restriction. C++
may also work as long as the statically linked binary object doesn't
grow too much and as long as the c++ libs don't call into forbidden
syscalls.

> I see projects/bench/api.py, but that doesn't seem like much.

The collection of the results coming from the bytecode running on the
seller CPU, is handled through the api.py. This api.py can also open a
local socket and you can interface your c++ code to talk with the
apy.py that way. The buy client will keep the connections open with
the sell clients and your c++ code may run and then quit, but you
don't want the buy client to quit if you've an intermittent burst of
data to compute. Your c++ code when talking with the api.py should
also be notified when one client disconnects, you will have to restart
from scratch that fragment of computation on a different client.

An example of this will be povray. Povray will likely talk through a
local tcp socket to the buy client. The buy client is the python
apy.py, while the povray sever will be C. The Povray client is C too
but under seccomp.

povray C server ->
       cpushare buy client python-twisted ->
       		internet ->
			 cpushare sell client python twisted
			 	  -> povray client machine bytecode under seccomp

The reason for this model is that the cpushare protocol is very
complex, soon I'll change the protocol to be totally peer to peer,
even if it provides lower guarantees in terms of anti-cheating.  Most
of the anti-cheating logic will be vote based. Currently because my
servers sits always in the middle (it's a p3p currently, not p2p) I
can reliably know who disconnected first. Later it won't be as easy
and reliable, but I figured out that I need to allow the buyer to get
as much bandwidth as it wants even when the project is so small as
now. Furthermore this way I will be really able to handle an enormous
amount of transactions per second regardless of the network transfer
and with minimal server resources, which in turn it means my margins
on the commissions will be higher and the pure p2p model will allow to
pay for more development resources in the project. The downside is
that the buyer (or the seller) will require a port open, right now
it's not required.

The idea is that the api.py spoken by your by client (either directly
coding the brainer buy client code in python inside the .py file or by
talking to it through a socket) will hopefully won't require changes
when the cpushare proper protocol changes. You definitely don't want
to hardcode the cpushare protocol inside your c++ code or you risk
rewriting a huge chunk of code when I will have to change it. I
welcome anybody to write a cpushare client, but this doesn't seem the
right time yet for such an investment of time and resources, given the
protocol is still very much in the development.

The cpushare protocol at some point will mostly freeze, or it older
versions will remain backwards compatible, it's not the time just yet
for that. But clearly I want the applications ported now to CPUShare
like John to mostly keep working later when the cpushare protocol will
change.

I had to reach a working point, and we are at a minimal but feature
complete working point today. So I couldn't look into doing the best
protocol possible, just the simplest secure one. Now it's the time to
reach the next milestones... both for the website and for the cpushare
internals too. In the meantime I welcome users to use the project with
the current working protocol. As long as the network transfers aren't
too heavy, the server will handle that just fine. But for raytracing
the new protocol will work much better and then the only bandwidth
bottleneck will become the internet link of the buyer.
-- 
cpushare-discuss_X_at_X_cpushare.com mailing list - http://www.cpushare.com/
To unsubscribe, send mail to cpushare-discuss-unsubscribe_X_at_X_cpushare.com