I've been trying to make high resolution skymaps, with 9 arcmin pixels to match GLAST's expected 68% PSF. I've found that the public version of galprop (v50p) has an enormous memory footprint. For 2400x1200 pixel skymaps it wants 70 GB of RAM... (and this is a 2D calculation.)

I think I've tracked down the origin of its greediness. It appears that a number of large 3D and 4D arrays are being allocated in gen_{pi0_decay,bremss}_skymap() that are not being freed at the end of the routine. From a quick look, it does not appear that these arrays are used anywhere after the routine returns, so this appears to be a memory leak.

An example (gen_bremss_skymap.cc):



Are there any objections to adding

to the end of the routine?

Similarly for gen_pi0_decay_skymap().

_________________
Michael Kuhlen
Institute for Advanced Study, Princeton

Additionally I wonder whether it is necessary to allocate all the gamma-ray and synchrotron emissivity and skymap arrays in create_galaxy() at the beginning of the code. As far as I can tell, these arrays aren't used until after all the heavy lifting (CR transport) has been done. In fact it seems they're only needed in the generate and store routines (e.g. {gen,store}_pi0_decay_emiss()) at the end of the code.

For high resolution skymaps these arrays can be very large, and if I'm not mistaken this memory is not being used except in those routines, so why carry it along for the rest of the calculation? Perhaps there are good design reasons for doing this, but allocating these arrays right before, and deallocating them right after, the generating and storing routines would significantly reduce galprop's memory footprint.

_________________
Michael Kuhlen
Institute for Advanced Study, Princeton

you are right, and we did not have this problem since we never tried such high-resolution all-sky maps, and hence memory was not an issue (although we found generally the necessity for > 2GB machines, and normally we use 16GB or more).
We will take note of your suggestions in a next version.
Meanwhile of course you can make the changes in your copy,
and let us know your experiences.

_________________
Andy Strong, MPE

I implemented the changes and it seems to be working and using much less memory. Of course I also quickly realized that it doesn't make all that much sense to produce skymaps with a pixel size smaller than the HI and CO maps that are read in (rbands_hi7.fits and rbands_co4.fits), which have a resolution of 30 arcmin. Actually, the rebinning to the skymap resolution in read_HIR() and read_COR() doesn't work when there is more than one skymap pixel per HI/CO map pixel.

For now I'm sticking with 0.5 degree resolution skymaps.

_________________
Michael Kuhlen
Institute for Advanced Study, Princeton