Re: more questions

[ cc'ed to cpushare-discuss with Alex permission ]

On Sat, Oct 20, 2007 at 04:17:13PM +0300, Alex Fihman wrote:
>  Hi, Andi,
> 
> There are my notes about CPUShare.
> I think of it as a future successful
> commercial project, so here are my notes.
> 
> 1. How CPUShare is compared with another
> already existing companies.
> 
> Availability:
> Amazon EC2 and Parabon have thousands
> readily available computers. CPUShare has
> only about 30 boxes online, buyer may
> consider renting out PCs to be more stable,
> faster, secure, etc.etc.

I don't know Parabon much, and I couldn't understand exactly how the
sellers are compensated by just reading the website. I also seem to
understand it can't run machine bytecode so comparing the cost of 1
hour at full cpu speed with 1 hour with a JIT virtual machine is like
comparing apples to oranges. Not having a JIT also saves memory for
all sellers.

What I am sure about is that the number of boxes currently online in
CPUShare is absolutely irrelevant, the number of accounts if something
is more relevant. Demand for cpushare will always be the limit, supply
is going to be "unlimited" from any practical matter. Suppose that a 1
million buy order showup on the bid, the ask will grow exponentially
with blogs talking about CPUShare growing and working for real,
providing enough supply until the demand is fully satisfied. If the
bid will be solid and too big for low income countries to satisfy it,
likely big companies will start recycling their idle cycles and
sending invoices to CPUShare for the money they withdraw. The only
trouble is to find this 1 million buy order. The reason there are only
a few dozen nodes connected is only a direct consequence of no money
on the bid most of the time. And rightfully so.

> Storage:
> Amazon has additional storage service.
> Parabon has high-level protocol available,
> that allows transfer of file chunks.
> 
> Amazon allows installing any Linux software,
> that means any existing high-level protocol
> can be used.

CPUShare doesn't allow any local storage at this point in time. Local
storage can be simulated in ram though.

In the long term I plan to use virtualization to allow regular apps to
run on top of CPUShare, but lots more network traffic will have to
happen then to transfer the filesystem. I never want to wire the
CPUShare protocol to any bytecode, filesystem layout or anything like
that. The buyer has to be totally in control of what runs on the
seller computer so he can run anything he wants. All restrictions
should be applied through jailing methods like seccomp or with
specially tweaked virtual machines and nothing else.

> 2. Language limitations. Currently cpushare
> allows very limited version of C compiler.
> Parabon allows programming in Java (high-level,
> but limited with 1 language), Amazon EC2 is
> basically limited by Linux platform, but any
> software will do.

Besides potentially allowing virtualization and filesystem delivery in
the future, more bytecodes could be added to cpushare in a
seccomp-like behavior, so for example you could use python bytecode
instead of machine bytecode and run it inside a jail in the client
but then it'll need to define a specific python version to run it. The
python binary running it may be different for every seller in this
model. While if you do it yourself with the virtual machine model you
will have to send the python binary to the seller through the root
filesystem delivery to boot the KVM.

> 3.Platform. Boinc allows making windows
> client with windows software. Windows with all
> it's problems, is a most popular
> desktop environment, that will allow many to
> join. Consider using some sandbox for windows?

You can find the ISO link in the sell order. That's directly bootable
with virtualpc/vmware/whatever. What's missing is a .exe that makes
sure virtualpc or vmware are restarted at boot time.

colinux will also work (if stabilized but that's not a cpushare
issue).

> Now about the cpushare itself.
> 
> 1. When software crashes for some reason, the
> buy client reconnects, and payment is made.
> Say if my software crashes why should I pay
> again, if 1 hour didn't pass?

Well you're the buyer so you will have to pay if you disconnect. You
won't pay only if the seller disconnect before the 1 hour is
finished. You will have to pay even if the seller behaves badly but
you will be able to mark the seller as erratic and the screening
parameters of the buy order will eventually reject the bad sellers out
of your screen. The voting will be automated by the buy and sell
clients of course.

> 2. How can I write optimized code for SSE2, SSE3,
> 3d now, 64_86?

That's easy by running cpuid. Here seccomp payoffs big compared to a
Java JIT... see when I said not to compare apple to oranges in terms
of bare-hardware CPU performance.

> 3. current model can easily cause the network
> congestion. Network bandwidth is not counted and
> not billed.

Correct, billing may be allowed. But this should be a concern for the
seller not for you as buyer. The buyer should size the order so its
internet bandwidth is maxed out. As soon as the protocol is pure p2p,
there will be no other bottleneck other than the buyer bandwidth. You
will be able to size the order dynamically with the website. If you
reduce the size, when some computation finishes it will not be
restarted in another node and the total number of simultaneous
transactions will reach the new order size after 1 hour. If you
instead increase the order size, you will immediately start
computing. So the idea is to start small and grow the size of the buy
order until your preferred network traffic analyzer shows either the
upload or download bits per second reaching your adsl/hdsl line max
upload or download bandwidth.

You will have to keep 1 port open in your firewall using destination
nat if your buyer computer is different, to be sure to reach all sellers.

> 4. Global benchmark interfere with job running,
> and reports incorrect results. Also sell machine reported
> online and reconnected to the job only after benchmark.

Hmm no, the global benchmark is completely transparent to all
transactions and all transactions are totally asynchronous with
respect of the global benchmark.

You can imagine the global benchmark as a freeze event. It's like if
it never run, but for a couple of minutes every X couple of minutes,
the whole CPUShare freezes time and all real transactions are
temporarily halted. But they restart totally transparently. This is
implemented simply through a sigstop/sigcont (well the server
implementation is not simple at all though ;-).

> 5. When hour of additional payment comes, buy client
> is disconnected, why?

Mostly an implementation detail. The seller may have to change if
somebody is selling something that matches your buy order, but for a
lower price. In theory I could keep the transaction open if I
atomically find that the best seller is still the same that was about
to be disconnected. I doubt I'll change this in the short term, there
are more urgent matters than try to recycle the last partial bit of
the hour. Furthermore when the sigquit signal is sent, the sell
bytecode should complete and quit somehow, so no real electric energy
is wasted even in the current implementation that always terminates
the transaction when the time is over.

> 6. Unused currencies very annoying. Why don't you
> wipe them from website?

Hmm ok, good point ;-). An #if 0 should do the trick.

> 7. Lack of WiFi and proxy support.

proxy will likely not be supported. Think an employee running cpushare
on its desktop computer and making money with the electric energy of
the company without the company knowing it. Now this can end in many
ways for this employee, and certainly it can't be my fault. But I feel
safer in giving a secure way to the corporate firewall administrators
to block CPUShare. If the company overreacts to such an event in a big
way, I want to be sure that before they could ever think at blaming me
for implementing CPUShare, they would certainly think at blaming the
firewall administrator for allowing this encrypted traffic to the
cpushare servers on a well defined and unusual port to pass through
the corporate firewall. If the company has such restrictive rules,
they should also have a restrictive firewall that blocks CPUShare by
default like bittorrent or any other non standard protocol. I'm being
paranoid here I know, but given how small CPUShare is right now, I
have to. Every legal risk I can avoid, I avoid it, be sure.

> 8. CPUShare server often crashes. After that need to
> wait benchmark to reconnect.

I know, sad but unfixable...

> 9. It would be good to implement some existing
> protocol for grid computing.

This will hopefully happen on top of the api.py yes. Lots of work to
do. All 100% LGPL open source work. It doesn't need to be me. I only
provide the link-level wire transport, everything on top of it is
totally open.

> 10. Book web page - would be nice to see more information
> about machines available.

Indeed... the website is still very basic right now, the book page
needs all screening options to see the multiple dimensions of the
marketplace.

> 11. It would be good to have some roadmap for future
> development.

Ok I'll add to the wiki, I kept it private so far because nobody ever
asked ;-).

> 12. Payment model. In boinc there's no real money payment
> but some point system. Their model is payment per piece of work.
> Sometimes it may be more fair, allowing the seller to change
> priority of the task, say CPUShare working only when machine
> is idle.

This is be defined as seller performance. If you run CPUShare on a
busy computer your seller performance will drop and the price you can
ask will eventually go down too. Seller performance will be a points
per hour measure, where you keep track of the max points per hour, and
you use that as 100%. Then if a client only did half points per hour
of the fastest ever you tell the server this client has a 50 seller
performance. Over time the ones that are cheating will get an huge
mflops/mips but low seller performance and they will go out of your
screen.
-- 
cpushare-discuss_X_at_X_cpushare.com mailing list - http://www.cpushare.com/
To unsubscribe, send mail to cpushare-discuss-unsubscribe_X_at_X_cpushare.com