Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for detailed ACT DR6 noise models in monte carlo #93

Merged
merged 16 commits into from
Feb 4, 2022

Conversation

xzackli
Copy link
Collaborator

@xzackli xzackli commented Jan 11, 2022

This PR adds support for the ACT DR6 noise models in the mnms package. Support for tiled and wavelet noise simulations requires specifying some additional noise parameters. I've added an example for just PA6 in the paramfiles/global_dr6_v3_4pass_pa6.dict. In this iteration, the param file for DR6 by default is essentially running the following (and then re-ordering the results for the pspipe scripts),

from mnms import noise_models as nm

tnm = nm.TiledNoiseModel('pa6a', 'pa6b',  # or WaveletNoiseModel
    downgrade=4, lmax=5400, 
    notes='20211203', 
    mask_name="dr6v3_default_union_mask", 
    union_sources="20210209_sncut_10_aggressive")
map1 = tnm.get_sim(0, 123)  # split 0, sim 123

This setup, along with proper .soapack.yml, uses the pre-generated 20211203 models.

These noise simulations have a covariance model that is pre-generated on NERSC, but the new realizations are created on the fly. The random seed is set by the simulation spectrum index (specified by iStart and iStop), which must be in [0, 9999]. This specific implementation assumes that noise correlations are dominated by those between arrays on the same wafer, and thus generates noise simulations in pairs.

The notion of pa6_f150 (pspipe array id) = pa6b (soapack qid) is implemented by hand in the param file.

Todos before merge:

  • .rst file with a tutorial and a minimal soapack configuration for NERSC and tiger
  • testing to ensure the mean noise power spectra of the simulations matches the inputs

@xzackli
Copy link
Collaborator Author

xzackli commented Feb 1, 2022

@thibautlouis this PR is now tested, I fixed a remaining bug that caused issues with the signal spectrum. I've also written some documentation in the montecarlo folder, and added an example SLURM script.

I'm a little scared to merge however, as it turns out the changes are very close to where I rewrote large parts of the sim script. I will merge master with this PR by hand tomorrow and run some tests.

@xzackli
Copy link
Collaborator Author

xzackli commented Feb 3, 2022

@thibautlouis this is ready for merge! Feel free to merge when you have reviewed it.

@xzackli
Copy link
Collaborator Author

xzackli commented Feb 3, 2022

Much of the added complexity comes from having to include the pixel window. The ACT DR6 mnms sims have the following property: in order to generate a map like the ones on disk, we run

map = pixwin(S) + mnms

This is different from the Gaussian realization, which is like

unpixwin(map) = S + n_gaussian

@thibautlouis
Copy link
Collaborator

thibautlouis commented Feb 4, 2022

Hi Zack, thanks for this ! I'm a bit puzzled by how mnms handle pixel window function, in any case I don't think it should stop us for merging. if that's ok for you, for now we should keep the old version of mc_get_spectra.py in the pipeline and have this new version as another script maybe mc_mnms_get_spectra.py ? (mostly because it increase the number of dependencies quiet a bit) long term I expect we will get rid of the former

@thibautlouis
Copy link
Collaborator

thibautlouis commented Feb 4, 2022

once it is merged I will rename the data_analysis_utils function "deconvolve_pixwin_CAR", since it is a confusing name in this script, I will give it a more general name (as it deserves)

@xzackli
Copy link
Collaborator Author

xzackli commented Feb 4, 2022

@thibautlouis ok, I will commit a quick name change!

@xzackli
Copy link
Collaborator Author

xzackli commented Feb 4, 2022

Ok, now the new file is mc_mnms_get_spectra.py and I have restored the existing mc_get_spectra.py.

@thibautlouis thibautlouis merged commit 25d5518 into master Feb 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants