stephenbrooks.orgForumMuon1GeneralHigh-res scan project
Username: Password:
Page 1 2 3 4 5 6
Search site:
Subscribe to thread via RSS
Stephen Brooks
2010-09-25 20:47:19
Although it will be a while before any client updates (see my post in another thread on that topic), I've realised there's something else I want to do first, which can be done with the current client.

I've never really taken a high-resolution scan of varying just one parameter over all its possible values - mainly because it requires 1000 simulations in a queue and that would take a long time on my workstation.  This means I don't really have a clear idea of what the optimisation landscape looks like on a detailed level, particularly in the newer lattices.  This sort of insight might let me get past some of the intermittent stumbling blocks that harder optimisations have.

Anyway, I thought we could tackle this with some old-style workunit allocation distributed computing.  I've made some queue.txt files which you can see here:

http://stephenbrooks.org/muon1/hidden/scan/1/

What I hopefully want some volunteers to do is:
  • Download one of the queuexxx-xxx.txt files and post on this thread to say you've claimed it.

  • In your Muon1 client, flush any remaining results in results.txt using manualsend.bat (or store them elsewhere) and make sure the Muon1 client is not going to upload during the test (use muon1 -cfg:nonet).  Delete any auto*.sav files.

  • Rename your downloaded file to queuecli.txt, put it in your Muon1 directory, then run muon1 -c -q. Use -b -q and the filename queue.txt if you don't want to see the text output, or just -q and queuegfx.txt if you want fullscreen graphics.

  • When it has finished, all the results should be in results.txt.  Send me this file either via e-mail or a web link in this thread.  Don't delete it!  When you put your client back to normal (-cfg:net), it will be sent as usual and you will be given Mpts credit for the results you produced.


Hopefully as the blocks come back in, I should be able to build up a detailed graph - a cross-section across the optimisation of Linac900Ext10d2, centred around one of the recent best yield results, in the direction of the ls5l parameter (a solenoid length for one of the middle blocks of the linac).  The choice of ls5l was somewhat arbitrary, mainly I chose it because the current value was 490 in the middle of the range and I expect changes in it to have a visible effect on beam transmission.
tasburrfoot
2010-09-25 21:23:27
I have queue975-999.txt

Chocobo Navy of TeAm AnandTech

Running using "muon1_background -q". Shouldn't affect the results, correct?

[Edited by tasburrfoot at 2010-09-25 21:27:36]
Stephen Brooks
2010-09-25 21:30:14
I'm pretty sure muon1_background -q will just run Muon1 as usual.  It'll probably do the queue but then it'll carry on making up its own results.  Running muon1 -b -q ought to do the trick. 
tasburrfoot
2010-09-25 21:30:45
How do I flush the results if there are less than 10?  Just delete results.dat?  (Sorry for causing problems)

I use background, because I'm ususally using my machine while I have it running...
[Edited by tasburrfoot at 2010-09-25 21:31:24]
tasburrfoot
2010-09-25 21:43:04
What isn't working?  I flushed the initial cache I had (uploaded 16 results), then started on the first download.  Well, I guess there is no results.txt generated, so nothing had really finished.  Is it safe to let it run and see what happens?  Or should I kill it completely and start over?

I normally just run muon1_background and let it go...
Stephen Brooks
2010-09-25 21:44:28
OK, panic over - I'd stupidly forgot that the queue file for commandline operation was called "queuecli.txt". If you're running background, then "queue.txt" should be fine.  I've modified the instructions accordingly...

Your setup with queue.txt and background running should be fine, though I'd suggest running "muon -b -q" rather than muon1_background if you want it to stop after it's finished the queue (muon1_background is a stub for "muon1 -b"; it doesn't accept switches).
tasburrfoot
2010-09-25 21:46:41
EDIT: Nevermind.  Re-read the instructions at the top.  Will adjust accordingly...
[Edited by tasburrfoot at 2010-09-25 21:59:32]
Stephen Brooks
2010-09-25 21:49:33
Ah yes, flushing less than 10 isn't possible - but don't delete results.txt, just store it somewhere and put it back after your test run.  That way you can still get credit for it.  Thanks for your help!

I've got queue950-974.txt running on an old laptop I use as a server.  It'll probably take a week to finish, but that's the only machine I have at home that runs 24/7!  I'm off at a conference the next week so I won't have the desktop on.
[TN]opyrt
2010-09-25 23:01:43
Shouldn't you also move your autocli.sav (or similar for gfx, autogfx.sav, is it?) out of the way? 
Stephen Brooks
2010-09-25 23:15:51
Um, yeah, that's also a good idea (maybe I should have recommended a clean install?!) Anyway I'll edit the instructions at the top.
RGtx
2010-09-26 02:55:15
I'll take queue0-99.txt.
eveiga
2010-09-26 03:25:56
I'll take queue100-199.txt.
[Edited by eveiga at 2010-09-26 03:28:41]
KWSN Grizzly
2010-09-26 08:47:40
Taken 200-299.txt

Regds Grizzly
[DPC]white_panther
2010-09-26 12:19:48
i'll take queue300-399.txt

0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499
500-549
550-599
600-649
650-699
700-749
750-799
800-824
825-849
850-874
875-899
900-924
925-949
950-974 Stephen Brooks
975-999 tasburrfoot
tasburrfoot
2010-09-26 18:33:15
Finished with 975-999.

Downloaded 925-949 and running it now.
tomaz
2010-09-26 18:44:26
0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499
500-549 TOMAZ
550-599
600-649
650-699
700-749
750-799
800-824
825-849
850-874
875-899
900-924
925-949 tasburrfoot
950-974 Stephen Brooks
975-999 tasburrfoot
[Edited by tomaz at 2010-09-26 18:53:17]
tasburrfoot
2010-09-26 19:12:07
Finished with 975-999.

Downloaded 925-949 and running it now.
K`Tetch
2010-09-26 19:26:18
I'll grab 600-649 and run it overnight.  What sort of mpts are typical for these results?  (I'm doing video editing/rendering on my main box, so I can only run overnight, don't want to leave it out of stuff)
waffleironhead
2010-09-26 21:21:28
0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499
500-549 TOMAZ
550-599
600-649 K'Tetch
650-699
700-749
750-799
800-824
825-849
850-874
875-899
900-924 waffleironhead
925-949 tasburrfoot
950-974 Stephen Brooks
975-999 tasburrfoot
[Edited by tomaz at 2010-09-26 18:53:17]
waffleironhead
2010-09-26 22:18:28
I just started up with a clean install.  Some things to note if anyone else does this. 
1) make sure you run the client once so it updates the lattice files before you set it to :nonet, or else muon1 will not know what to do with the queuecli.txt file you have.  Linac900Ext10d2 lattice does not come packaged with the v444d file.

2) If you dont want to be running each of these tests five times I suggest you download the 100 result sample file and change the file to results.dat - that way you already have cached the current best optimizations and your client wont be running rechecks on -1.777777 results and wasting a bit of time. 
[TN]opyrt
2010-09-26 23:29:47
A bit off-topic, but I'm wondering if number of threads each optimization is ran on has any relevance, as there will be particles in the optimization?
tomaz
2010-09-27 05:20:48
waffle, you can also download lattice file manualy and put it in your lattice directory.
Stephen, wouldn't be a better comparasion if all results are run 5 times ?  I thought -q would recheck every result regardles if it is the best or not.  But I see now that -q is just for qiut after queue file is done.
tasburrfoot
2010-09-27 05:45:20
925-924 is finished and uploaded.  Starting 875-899 now.

0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499
500-549 TOMAZ
550-599
600-649 K'Tetch
650-699
700-749
750-799
800-824
825-849
850-874
875-899 tasburrfoot
900-924 waffleironhead
925-949 tasburrfoot
950-974 Stephen Brooks
975-999 tasburrfoot
Stephen Brooks
2010-09-27 08:29:17
OK, just found the wi-fi in the meeting room.  Looks like these units are going well (making the table was a good idea!)

K`Tetch - you might have been better with a 25 result file, these are probably quite long units (2000Mpts per simulation) but see how it goes on your computer.

Tas - I've received your 975-999 via e-mail but not 925-949?  I'm not keen on people just uploading these because I'll have to go and fish them out of the rest of your results in the database.  It's possible, just a slight nuisance.

waffleironhead - Yes, that's right.  So adapting an existing install might not be so bad after all.  I'd had in mind that most of these results are done as single runs, though the 5-run averages could be interesting for comparison, so it doesn't matter too much if you've done rechecked ones (I'll know which they are).

[TN]opyrt - Number of threads shouldn't affect the simulation results, so I'd suggest using "auto" (the maximum).

Tomaz - I might do a scan with #runs=5 later.  This one I want mostly #runs=1 (and the occasional #runs=5 to see how far away it is).  Most results the optmisation sees are single-run.
RGtx
2010-09-27 10:48:00
I'm using a small app.  called "TheFolderSpy" to detect changes to queue.txt, then run a bat file to save and rename (appending time) a copy, permitting me to recover single run results for those run 5 times.
tomaz
2010-09-27 11:34:50
I grabbed 550-599 for my office computer
I am running it with best so far result in result.dat, so there will be no 5 runners I think.

0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499
500-549 TOMAZ
550-599 TOMAZ
600-649 K'Tetch
650-699
700-749
750-799
800-824
825-849
850-874
875-899 tasburrfoot
900-924 waffleironhead
925-949 tasburrfoot
950-974 Stephen Brooks
975-999 tasburrfoot
tasburrfoot
2010-09-27 12:45:07
@Brooks: I sent them last night via email.  When I get home tonight I will check and make sure it sent out correctly, and resend.
Hydrox
2010-09-27 12:59:47
I'll take 700-749
tomaz
2010-09-27 17:01:35
2.401777 (12782.0 Mpts) [v4.44d] <Linac900Ext10d2> {7D783F7DABB9D80E20BDC149}

2.333601 (2566.7 Mpts) [v4.44d] <Linac900Ext10d2> {E4A9204304076E42BB109D3D}

2.421967 (12782.0 Mpts) [v4.44d] <Linac900Ext10d2> {54D5950ED26250733969503C}

2.482482 (12781.0 Mpts) [v4.44d] <Linac900Ext10d2> {553C3CD7E899C310EEE93449}

2.381809 (2568.5 Mpts) [v4.44d] <Linac900Ext10d2> {7D9F54FEC0B19A96767D5433}

Stephen, above are neighbouring results (500-504).  It seems like a very tricky thing.  Very small change of only one parameter results in rather large change in whole machine yield.  It means problems not only in modelig but also when you want to built a real machine. 
What means ls5l=500 or 501 in real world ?  Is it 100 m of some wired magnetic stuff or is it a 1 cm of something ?
Off topic tought:
Maybe parameters which are optimized to extreme values (000, 999) has much less influence.  Can one asume that those with 000 might be unnecessary and those with 999 need to be expanded ?  Can you try kind of selfbuilding lattice which would expand parameters acording to that logic (say to 9999 instead of 999) ?
[Edited by tomaz at 2010-09-27 17:05:14]
tasburrfoot
2010-09-28 03:49:08
0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499
500-549 TOMAZ
550-599 TOMAZ
600-649 K'Tetch
650-699
700-749 Hydrox
750-799
800-824
825-849
850-874 tasburrfoot
875-899 tasburrfoot
900-924 waffleironhead
925-949 tasburrfoot
950-974 Stephen Brooks
975-999 tasburrfoot

I sent the two emails containing results.  Outlook apparently didn't send the results before I closed it, so that email was actually sent this evening.  I'm starting on 850-874 now.
K`Tetch
2010-09-28 05:02:59
Stephen, at 2000 per, thats 100,000 mpts, or about 66 hours work.  not a big problem, but thanks for caring
Stephen Brooks
2010-09-28 08:57:57
TOMAZ - yes, this is the sort of thing I was looking for.  It's most likely a features of the finite number of particles being simulated in Muon1, so you get statistical random effects when you change their paths.  The 001 difference is perhaps 1mm in the length of a solenoid, but in this lattice it changes all solenoids in a block, maybe 10 of them, meaning differences in the trajectories of the particles will build up.  For a very small difference, the real difference in yield might be small, but our simulations could get noise from some particles leaving and others entering the transmitted region.

I expect the real machine to be quite sensitive to solenoid length but not THAT sensitive, and not in a "random" looking way.

The parameters that have become 000 or 999 are running up against a physical or technological limitation (e.g. maximum feasible size to make solenoids for a reasonable cost).

I'll also get someone to redo the same range twice to make sure Muon1 always produces the same output on deterministic lattices (that your client hasn't reverted to random somehow).

Tas - I've got your results OK now!  (about 8 hours ago)
tasburrfoot
2010-09-29 04:52:34
last one uploaded.  I will take 400-499 now.

0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499 tasburrfoot
500-549 TOMAZ
550-599 TOMAZ
600-649 K'Tetch
650-699
700-749 Hydrox
750-799
800-824
825-849
850-874 tasburrfoot
875-899 tasburrfoot
900-924 waffleironhead
925-949 tasburrfoot
950-974 Stephen Brooks
975-999 tasburrfoot
Cameron
2010-09-29 06:06:02
Taking 750-799
Stephen Brooks
2010-09-29 09:21:43
Some of the results have been sent back already:

0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499 tasburrfoot
500-549 TOMAZ
550-599 TOMAZ
600-649 K'Tetch
650-699
700-749 Hydrox
750-799 Cameron
800-824
825-849
850-874 tasburrfoot ---Received
875-899 tasburrfoot ---Received
900-924 waffleironhead ---Received
925-949 tasburrfoot ---Received
950-974 Stephen Brooks
975-999 tasburrfoot ---Received
eveiga
2010-09-30 05:58:43
Perhaps a dumb question but does the client give some indication when it is finished?

I ran it as muon1 -c -q

Will it exit after running the queue or will it just continue until I interrupt it?
Stephen Brooks
2010-09-30 10:11:40
If you've used the -q switch, it should exit when the queue is used up.
tomaz
2010-09-30 18:31:24
I took 825-849.
[DPC]white_panther
2010-10-01 00:56:12
i took 800-824
and have send the 300-399 results back

0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499 tasburrfoot
500-549 TOMAZ
550-599 TOMAZ
600-649 K'Tetch
650-699
700-749 Hydrox
750-799 Cameron
800-824 [DPC]white_panther
825-849 tomaz
850-874 tasburrfoot ---Received
875-899 tasburrfoot ---Received
900-924 waffleironhead ---Received
925-949 tasburrfoot ---Received
950-974 Stephen Brooks
975-999 tasburrfoot ---Received
tasburrfoot
2010-10-01 02:11:34
400-499 uploaded, and I have 650-699...

0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther
400-499 tasburrfoot
500-549 TOMAZ
550-599 TOMAZ
600-649 K'Tetch
650-699 tasburrfoot
700-749 Hydrox
750-799 Cameron
800-824 [DPC]white_panther
825-849 tomaz
850-874 tasburrfoot ---Received
875-899 tasburrfoot ---Received
900-924 waffleironhead ---Received
925-949 tasburrfoot ---Received
950-974 Stephen Brooks
975-999 tasburrfoot ---Received
Stephen Brooks
2010-10-01 09:08:54
Some more have come in:

0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther ---Received
400-499 tasburrfoot ---Received
500-549 TOMAZ ---Received
550-599 TOMAZ ---Received
600-649 K'Tetch
650-699 tasburrfoot
700-749 Hydrox
750-799 Cameron
800-824 [DPC]white_panther
825-849 tomaz
850-874 tasburrfoot ---Received
875-899 tasburrfoot ---Received
900-924 waffleironhead ---Received
925-949 tasburrfoot ---Received
950-974 Stephen Brooks
975-999 tasburrfoot ---Received

I'm in the final summary session of the conference at the Paul-Scherrer Institut in Switzerland right now.  Might make a graph of these results on Saturday or Monday.

I'd like one volunteer (other than TOMAZ) to do the 500-549 results a second time, just to ensure the client is giving consistent results on different machines.  It should do, but it doesn't hurt to check!
K`Tetch
2010-10-01 18:51:40
I still have 12 of my 50 to go.

do you want all of the blocks redone, or just 500-549?
[DPC]white_panther
2010-10-01 19:20:08
is it possible that i only get negative results on mine 800-824 block?
K`Tetch
2010-10-01 19:33:14
sure, if the yield is either too low, or not of the right energy (iirc -1.x is not enough, and -0.x is not right energy)
Stephen Brooks
2010-10-02 00:59:46
It's only really worth redoing one of the blocks.

It's quite expected that some values will produce only negative results if they are too far from the optimum to work at all.

My laptop is about 3/4 of the way through my small block of 25!
[DPC]white_panther
2010-10-02 01:39:47
i have send mine 800-824 block

and start now on the recheck of block 500-549.
tasburrfoot
2010-10-02 02:53:15
650-699 finished and uploaded.

switched back into auto and running in the background. 
[Edited by tasburrfoot at 2010-10-02 02:54:52]
[TN]steinrar
2010-10-02 19:41:50
I'm also rechecking 500-549, hope it's not a waste of time
Started yesterday, but forgot to post about it here.  Not a big deal if i don't get credit for it.
Stephen Brooks
2010-10-02 20:06:30
You'll get Mpts credit for all of these so long as you remember to upload them.  A 3-way check isn't a bad thing.

0-99 RGtx
100-199 eveiga
200-299 KWSN Grizzly
300-399 [dpc]white_panther ---Received
400-499 tasburrfoot ---Received
500-549 TOMAZ ---Received (Being rechecked by [DPC]white_panther and [TN]steinrar)
550-599 TOMAZ ---Received
600-649 K'Tetch
650-699 tasburrfoot ---Received
700-749 Hydrox
750-799 Cameron
800-824 [DPC]white_panther ---Received
825-849 tomaz ---Received
850-874 tasburrfoot ---Received
875-899 tasburrfoot ---Received
900-924 waffleironhead ---Received
925-949 tasburrfoot ---Received
950-974 Stephen Brooks
975-999 tasburrfoot ---Received
Stephen Brooks
2010-10-03 01:28:07
I've now plotted the graph including the results received so far: (red points are #runs=5 results, the optimum Muon1 had found when I started the scan is in green)

: contact : - - -
E-mail: sbstrudel characterstephenbrooks.orgTwitter: stephenjbrooksMastodon: strudel charactersjbstrudel charactermstdn.io RSS feed

Site has had 24953446 accesses.