Bernd, Bruce, Gary, Mike, a question for you!

Anonymous
Topic 13374

Back to the original question:

Roughly speaking anything you see in your cache (number of Tasks, movement etc.) would be the same in the database, multiplied by the number of users (or actually CPUs). If you cut the current WUs in a half, you have twice the number of results the database needs to keep track of. The database size is still our limiting factor, we're currently running a server with 24GB main memory and it's already tight.

BM

Bernd Machenschalk
Bernd Machenschalk
Joined: 15 Oct 04
Posts: 2,684
Credit: 25,950,161
RAC: 34,820

Bernd, Bruce, Gary, Mike, a question for you!

Quote:
Quote:
The database size is still our limiting factor, we're currently running a server with 24GB main memory and it's already tight.

But what has server memory got to do with database size? The server I work on during the day (a lot fewer users than E@H I admit) only has 8G RAM, but the database is stored on the 300G of hard disk space. Surely you never want to hold the entire database in RAM?


Unfortunately we found we need to (at least the largest tables), to keep the server responsive. Having not much experience with DBs of this size I'm inclined to push the blame to MySQL, which BOINC is currently bound to. It might get better with other DBS, but you would also need to change BOINC code for that (David Anderson expects some 50 lines). No one of us or another project I know of has the time to actually do this and test it, especially as it is everything else but sure that you gain anything from it. Not to speak about e.g. migrating the DBS in a running system.

Doubelling the deadline also roughy doubles the size of the DB - we did this once about a year ago.

We already modified the scheduler to give shorter Tasks to slower machines. However our modifications only shift proababilities, you can't completely avoid to give a long Task to a slow machine or vice versa.

We might be able to bind the datafile to the hosts download rate in a similar way, but I'm afraid that might also require to change e.g. the Workunit Generator. I'll take a look at that, but I doubt that I come to that in the next few weeks. However, in the longer term - volunteer computing is a great concept, and the number of BOINC projects is continously growing. It might be that with a dialup connection Einstein@Home is not the best way to contribute your computing power to bleeding-edge science. My crystal ball is not very clear, but at the end of the LSC S5 science run we'll hopefully have more data to analyze than the intermediate set we are using now, so the data files for the next run will rather become larger.

BM

BM

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.