 |
| HaloJones 2007-06-19 23:16:44 | Stephen, How important is this project to you? I don't mean the accelerator but this little DC project? I don't mean to be awkward but we've all seen how very little has changed over the last year or so. Basically from where I sit, the V5 client is long overdue, the new "stuff" we crunch doesn't seem to change much and we've had way too much instability on the stats. I don't pretend to be able to understand what the client does but I don't see anything that a casual DC'er could feel has been accomplished. Where's the feedback? Where's our contribution going? What are you doing to ensure that we can have confidence that we're putting our efforts towards genuinely useful science and that you're making efforts to ensure the reliability of your project from our perspective. I suspect that many DC'ers - particularly on the little scientific projects like yours - know rather a lot about IT. None of us would dream of designing a particle accelerator but many of us are experts at designing IT infrastructure and how to make it 100% resilient without spending oodles of cash. If you would take the time to document what the IT part of this project does and how you have it set up, you might be surprised at the constructive positive response you would get. Looking around the various teams' DPAD forums, there is a fairly serious degree of dissatisfaction at the moment and that can easily be seen from the current level of points returns. You *need* to do something to address that. Burying your head in your science is not going to help. Sorry to be blunt but I think you need to understand that your project took a very serious hit with the stats mess and it's not going to just go back to the way it was without you showing some openness. |
| [DPC]white_panther 2007-06-20 11:41:34 | speak for your self and not for me, i'm ok with the way it went atm. |
| [DPC]white_panther 2007-06-20 11:41:57 | or just quit |
| Stephen Brooks 2007-06-20 11:51:29 | The project is very useful not just because of the accelerator but because the whole setup can be applied potentially to a large range of problems. Progress on the client etc. occurs in fits and starts (and has done historically), because when I'm updating the stuff on the web, I'm not reworking the innards of the code, and vice versa. I released a new lattice (PhaseRotDD) not so long ago. The reason the lattice all look boring and the same at the moment is that the difficult optimisiation work is in those blue/orange annuli that represent acceleration units. They do stuff to the energy and time-spread of the beam, which isn't always very visible in the current output, though if you press H there's a longitudinal phase space plot that shows a bit more of what's being done to the distribution. The other thing is that these optimisations with the acceleration in them seem to take a LOT of crunching right now to climb up in yield. DecayRotB is doing a gradual climb but it isn't anything like as fast as some of the earlier solenoids-only runs! But then again the problem is more complicated when you vary the acceleration too. As for where it's going, it seems the UK neutrino factory work is continuing provisionally until 2010 at least. I actually have put the muon cooling components (which are something to do with "V5"  into the development code in the last month, currently trying to get a single run to go through and exhibit the behaviour I want. That could take a while, and you may also get a non-V5 release in the coming months just to check that the changes I've made in the code over the last year haven't broken something else in an obscure way.The reliability of the project from the science perspective is actually very good - since what matters are the best genomes, which get distributed to 100s of people via the samplefiles mechanism, the scientifically-important data is extremely well backed up! The stats run on a custom C script I wrote that counts long text files and has been modified a couple of times to add indexing (so recounting the whole thing every hour is no longer necessary). It's just a custom thing so mistakes get made when I try to "improve" it - it also handles all the downloading. It's thinkable to put it onto a more standard database like MySQL or something, but it wouldn't make it any smaller (and you'd have to be careful about indexing the right way so duplicate checking can also be done quickly). Storing everyone's every result isn't actually scientifically very useful, but I thought it would be a very concrete way to keep count of the work done and maybe I could further analyse the resulting store of data to see if I can pick out trends that might suggest how the optimiser could be improved. In fact that's why you've got those coloured graphs next to the yield/Mpts one. Not sure why people are complaining so much about this now, especially when the stats were down for 6 weeks last year and various crashes before then... This particular fault was annoying and I still have to figure out what to replace the faulty Thecus N5200 units with (the ones that keep disappearing from the network and returning corrupt files), since the group wants to use them for office backups too. Actually, that's one thing your IT people could be very useful for - has anyone found a *reliable* NAS box, that preferably takes 4+ drives and can cope with having several people's HDDs (one of which will be the Muon1 results) backed up onto it each night? Everyone who's reviewed the N5200 has said "it's great" but have apparently only put about 2 MP3s and a video on it to test... |
| K`Tetch 2007-06-20 15:52:18 | Let me just translate, or at least interprit, what HaloJones is saying, for those that don't quite understand. 1) Many teams are getting annoyed because all their hard work (ie executing the client, and leaving it) is going untabulated sometimes. 2) Many teams have people in them that have dabbled with a bit of c++ coding or play around with windows and linux a bit, and so think they know how to run a stable system. 3) They have no interest in the science of the project, nor do they wish to make any effort to learn. 4) Stephen, who is a scientist, is crap because as well as dong all this cutting edge stuff, he's also doing all the coding, and all the stats work, and all the research work for this and it sometimes gets on top of him 5) DPAD/muon1 needs reliability equal to other distributed projects, which a) use the simpler brute force method over b) a vastly smaller workspace (10^15 for rc5-72 compaired to around 10^700 for just decayrot  run by c) a team of people who do nothing but work on the project with the help of d) a project budget.I'm NOT a big team. you'll see me usually around the bottom of the stats. look on phaseRotDD right now, and you'll see i'm near the bottom, because i don't run samplefiles, but you'll see i'm not with the other non-samplefile people, because I actually took the time to do my own designs. i also live in the real world, and know that stephen's doing a hell of a lot better than I could in his position, and I'm pretty sure he's doing a damned sight better than you could too, halojones. This - What are you doing to ensure that we can have confidence that we're putting our efforts towards genuinely useful science and that you're making efforts to ensure the reliability of your project from our perspective - really gets me though. What he's done it put a hell of a lot of documentation online. If you'd read it, you'd understand that its doing usefull science. If you understood how science works, you'd also understand that the project is reliable. Whats being lost is the chaff that's scientifically not usefull. It does seem from your statements that you don't knwo what you want. You want new and exciting versions, but you want them vastly different. At the same time you want more and new and better clients, but you want stephen to also work on making all the results stored, including the ones that are not of any use to him. Sounds like you want to have your cake and eat it. He can't work on new versions, and spend his time keeping the stats 100% reliable all the time. He can't, apparantly, deploy new lattices which show an improved yield, because to those that don't bother to understand what the project is about, it doesn't seem significant enough. He can't work on new papers for potentially better funding because he's working on all the other things you want. What exactly is it you REALLY want, HaloJones? |
| HaloJones 2007-06-20 21:16:35 | Well, I wanted what he delivered in his response to be honest. And I thank you for your "interpretation" although I take issue with your version. Do you dispute the drop in returned results? Had it not been for third party backed up stats, the project contributions would be completely and horribly wrong. And do you really think that the contribution stats are unimportant? That the competitive aspect of them does not drive teams to actively recruit new crunchers? You know, I don't care about any of the responses in this thread other than Stephen's. Mine was, I thought, a constructive request for "explanation" which I got. If I didn't care about the project I would have bailed but I've been trying to keep at least one of the major teams from deserting this project entirely. Hopefully, Stephen's response will help. Yours, K'Tetch, won't. |
| Stephen Brooks 2007-06-20 22:16:22 | -- Note to self: implement a simple log of "running totals" so I can do this backing up of the raw numbers myself. -- The odd thing is that I've got other sorts of (more detailed) log for users, but not exactly what Bok[Free-DC.org] had. I had a transactional log but it only started a few months back, so I couldn't restore really old points. |
| [DPC] Eclipse~Lord Alderaan 2007-06-25 13:37:03 | Well from the DPC forum dedicated to this project I cannot sense much dissatisfaction (anymore) at what has happened with the stats. At the time there were some complaints and some questions but most of those were answered by threads on this forum right here and we are tufting along right now having our own fun internal little subteam conflicts. After all the sample file system makes sure the best results are spread out and they will be hard to lose. Stats are important yes, but are just for fun and competition and not the goal of the project. Feedback?? I've seen Stephen reply in pretty much every topic made here... what more do you want? Where does your contribution go? As K'Tech pointed out there quite some reading material about what the project really is about (although that still leaves me just grasping the concepts). By the looks from the overall stats it looks like DPC is going just like it was (hits and fits), Team Anandtech is doing very nicely and might try to overtake Free-DC who is either showing weakness or is saving up for a large Flush. I've read one or two topics on the free-dc forum and it seemed to confirm both some people leaving and some people saving up for better times which makes it very unclear what exactly is going on. Time will tell my friends. |
