You are here: Foswiki>HAWC Web>JoshuaSchroeder (revision 27)EditAttach
-- JoshSchroder - 23 Sep 2014

Joshua Schroeder's Page

Tasks (2015-2016)

Log Entries (2015-2016)

October 13, 2015
  • Recently I have been working with the FDR scripts. (I have aerie on my computer, but this could also be done on the UMD cluster)
    • I started by logging in to the umd cluster and downloading the smoothed map files from /data/archive/hawcroot/maps/maps-20150710-aligned/SmoothedMaps, as well as /data/archive/hawcroot/maps/maps-20150710-aligned/cuts.txt. (scp -r your_username@pa-pub.umd.edu:/data/archive/hawcroot/maps/maps-20150710-aligned/SmoothedMaps /path/to/folder/on/local/machine/, and a similar command for the cuts file)
    • I acquired three scripts from Tolga: getHistogramData.py, drawHistogram.py, and processHistogramData.py
    • I combined the 10 bins in SmoothedMaps into one significance map using the command: aerie-apps-make-significance-map --input combined_bin0.fits.gz 0 combined_bin1.fits.gz 1 combined_bin2.fits.gz 2 combined_bin3.fits.gz 3 combined_bin4.fits.gz 4 combined_bin5.fits.gz 5 combined_bin6.fits.gz 6 combined_bin7.fits.gz 7 combined_bin8.fits.gz 8 combined_bin9.fits.gz 9 -o significance_combined_map.fits.gz --sensi cuts.txt -M 90 -m -89
    • When I plot the significance map using </path/to/aerie/src/map-maker/scripts/plotMollweide.py significance_combined_map.fits.gz> I get Sigmap.png
    • You can see the crab on the right and the galactic plane on the left as a strip of green. However, other sources are somewhat hard to see.
    • I now run Tolga's scripts over the significance map. So first I run <python getHistogramData.py --input significance_combined_map.fits.gz --output output.fits.gz>. this should output four files. I was concerned only with output.fits.gzoffplane.dat and output.fits.gzonplane.dat. I ran processHistogramData.py on both of these files (<python processHistogramData.py --input output.fits.gzoffplane.dat>), generating two more files, output.fits.gzoffplane_conservative_sigmas.dat and output.fits.gzonplane_conservative_sigmas.dat.
    • Taking a look at <less output.fits.gzonplane_conservative_sigmas.dat>, we see a list of values. The value corresponding to 0.000100 will give you the sigma cutoff after which the chance of seeing a random source at this sigma is .0001 (this is called the contamination). My value was 4.225825. This is the cut we want to place on minimum sigma for spotting sources. So to plot the significance map with this cut, I ran </path/to/aerie/src/map-maker/scripts/plotMollweide.py significance_combined_map.fits.gz -m 4.225825 -M 20> This sets the minimum sigma to the value we obtained from Tolga's scripts and the maximum to 20. This plot looks like this:sigcombinedmap2.png


      Edit: The file I should have been using to look at offplane sources is in file_name.offplane_conservative_sigmas.dat, and should have had a cutoff of about 5.49 sigma if I wanted a contamination of .0001

November 13, 2015
  • I have learned a lot more about FDR after reading a paper about it and playing around with the output. The getHistogramData.py file actually generates two outputs that are histograms of data. Each pixel on the map is determined to be on the galactic plane or off the galactic plane, and its sigma value is put into one of the bins of the histogram. This output is in the files filenameonplane.dat and filenameoffplane.dat. processHistogramData.py will give you analyze one of these histograms and give you a sigma cutoff for that region for various levels of contamination, which are organized in a table in the output file, filenameoffplane_conservative_sigmas.dat, etc. The onplane sigma values should only be used when looking at the galactic plane, and so to look for offplane sources, one should use the output of the offplane sigma file.
  • Once you have the sigma cutoff, you can use the file plotHotSpots.py to output all of the sources on the map with a sigma higher than the cutoff using the script /path/to/hawc_software/aerie/src/map-maker/scripts/plotHotSpots.py. When I ran this code with (the incorrect) 4.225 sigma cutoff, I found bizarrely bright pixels on the very edges of the map, looking like this:
    • significance_combined_map-hs1.pngsignificance_combined_map-hs3.pngsignificance_combined_map-hs7.png
    • As you can see, they are around -43 to -44 in Declination. In fact, all such suspicious sources occured within a 2 degree strip along the top or bottom of the equitorial map. Therefore, I advise removing everything above 80 deg and everything below -41 deg when making the significance map. The bright single pixel sources above do not seem to be present in the maps of the data and the background that i looked at when we zoomed in on these sources, so I suspect there is some error in the smoothing or significance map making code that is generating these.
    • All in all, I saw 15 hotspots with a sigma > 4.225 that have not been seen with other projects so far, however the cutoff I should have used is 5.49 sigma, so these candidate sources cannot be verified to 99.99% certainty with the methods I have used thus far. However, If anyone wants to examine these further, I will attach their RA and Dec coordinates here.
November 18, 2015
  • I have been installing SExtractor for some time now. It's been a journey, and I have decided to make a separate page describing how I installed SExtractor here.

Tasks(2014-2015)

New shift wizard list 2/3/15

  • water levels link
  • graphs (good)
    • TDC error rate
    • TDC error ratio
    • max PE
    • log NPE
    • Reconstructed Core
    • azimuth angle
    • zenith angle
    • Dec vs Ra
    • edge hits
  • TDC error rate
  • TDC trigger rate
  • Status of HVAC
  • input/output voltages and temp of UPS
  • Status of GTC
  • Weather/temperature
  • temp in counting house, main computer rack, and FEB1 crate: Analog Front + glance at others
  • TDC and scalar disk space

Event Display

  • Slideshow (done)
  • New poster
  • Boot up automatically (Fixed)
  • Sam's List

Fix HV/LV links

Things to fix
  • Translate data into more readable form (fixed)
  • Update HV editing page (fixed)
  • Set warning and alarm thresholds (fixed)

Fix up HAWC website

Things to fix
  • header image? (fixed)
  • Milagro links

Set up Shift Wizard

Things to fix
  • Can't find TDC DAQ on monitor (fixed)
  • HV Link doesn't seem to work (fixed)
  • LV link doesn't show status (fixed)
  • Disk space link doesn't seem to show status (fixed)
  • Weather/temperature link doesn't seem to work
  • Need data for status of GTC (doesn't exist yet)
  • Need data for input/output voltages and temperature of UPS (Doesn't exist yet)
  • Need data for status and temperature of HVAC (Doesn't exist yet)
  • logNPE plot link doesn't seem to work (fixed)
  • Main Dashboard link fix (fixed)
  • Reference plots

List of Log Entries (2014-2015)

September 23, 2014
  • Tried to figure out the wiki
  • Played with the Shift Wizard
September 24, 2014 September 29, 2014
  • Finished reading the HAWC Construction Proposal
  • Read how to to edit the shift wizard here: I need an Admin account before I can edit it.
  • Added a few items to the Shift Wizard problem list
September 30, 2014
  • Practiced coding on Codecademy
  • Read part of A Thin Cosmic Rain

October 6, 2014
  • Practiced coding
  • Read more of A Thin Cosmic Rain
  • Read how to edit the HAWC website here: did not have username/password that the server would accept when I tried to log in.
October 7, 2014
  • Resolved some issues with Shiftwizard, as noted under the shiftwizard heading
October 13, 2014
  • Did some background reading
October 15, 2014 October 21, 2014
  • Logged into HAWC public website server and tried to figure out how to edit it
October 22, 2014
  • Started entering HV tank names and set values
  • Downloaded web editor
October 27, 2014
  • Continued entering HV tank names and set values
October 28, 2014
  • Finished entering HV tank names and set values
  • Updated the page describing how to edit the HV Measurements
November 5, 2014
  • Fixed up the MSU HAWC public website
November 12, 2014
  • Made further changes to the MSU HAWC public website
January 14, 2015
  • got the event display to begin displaying events automatically on reboot. I used the crontab to change this by entering the command: sudo crontab -e. From there, you enter the time you want the command to execute followed by the command. I went ahead and entered the command to run on boot for both the superuser and the "hawc" user, just in case.
  • I have been working on a slideshow to go with the event display.
January 27, 2015
  • finished the powerpoint presentation.
Feb 3, 2015
  • Tolga showed me how to enter questions into the new shift wizard, and I have entered some questions
  • Creating new to do list for shiftwizard.
Apr 6, 2015
  • Edited the poster for the Event Display. The new poster should be in the HAWC dropbox
Topic attachments
I Attachment Action Size Date Who Comment
Sigmap.pngpng Sigmap.png manage 437.6 K 13 Oct 2015 - 15:13 JoshSchroder Significance map for 10 bins before cuts
goodsources.datdat goodsources.dat manage 0.9 K 16 Nov 2015 - 18:58 JoshSchroder  
sigcombinedmap2.pngpng sigcombinedmap2.png manage 55.3 K 13 Oct 2015 - 15:29 JoshSchroder Significance map for 10 bins after cuts
significance_combined_map-hs1.pngpng significance_combined_map-hs1.png manage 173.6 K 16 Nov 2015 - 18:46 JoshSchroder  
significance_combined_map-hs3.pngpng significance_combined_map-hs3.png manage 188.0 K 16 Nov 2015 - 18:46 JoshSchroder  
significance_combined_map-hs7.pngpng significance_combined_map-hs7.png manage 222.1 K 16 Nov 2015 - 18:46 JoshSchroder  
Edit | Attach | Print version | History: r32 | r28 < r27 < r26 < r25 | Backlinks | View wiki text | Edit WikiText | More topic actions...
Topic revision: r27 - 18 Nov 2015, JoshSchroder
 

This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback