Overcoming computational challenges to realize meter- to submeter-scale resolution in cloud simulations using the super-droplet method
<p>A particle-based cloud model was developed for meter- to submeter-scale-resolution simulations of warm clouds. Simplified cloud microphysics schemes have already made meter-scale-resolution simulations feasible; however, such schemes are based on empirical assumptions, and hence they contai...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2023-11-01
|
| Series: | Geoscientific Model Development |
| Online Access: | https://gmd.copernicus.org/articles/16/6211/2023/gmd-16-6211-2023.pdf |
| _version_ | 1797641791648825344 |
|---|---|
| author | T. Matsushima S. Nishizawa S. Shima S. Shima |
| author_facet | T. Matsushima S. Nishizawa S. Shima S. Shima |
| author_sort | T. Matsushima |
| collection | DOAJ |
| description | <p>A particle-based cloud model was developed for meter- to submeter-scale-resolution simulations of warm clouds. Simplified cloud microphysics schemes have already made meter-scale-resolution simulations feasible; however, such schemes are based on empirical assumptions, and hence they contain huge uncertainties. The super-droplet method (SDM) is a promising candidate for cloud microphysical process modeling and is a particle-based approach, making fewer assumptions for the droplet size distributions. However, meter-scale-resolution simulations using the SDM are not feasible even on existing high-end supercomputers because of high computational cost. In the present study, we overcame challenges to realize such simulations. The contributions of our work are as follows: (1) the uniform sampling method is not suitable when dealing with a large number of super-droplets (SDs). Hence, we developed a new initialization method for sampling SDs from a real droplet population. These SDs can be used for simulating spatial resolutions between meter and submeter scales. (2) We optimized the SDM algorithm to achieve high performance by reducing data movement and simplifying loop bodies using the concept of effective resolution. The optimized algorithms can be applied to a Fujitsu A64FX processor, and most of them are also effective on other many-core CPUs and possibly graphics processing units (GPUs). Warm-bubble experiments revealed that the throughput of particle calculations per second for the improved algorithms is 61.3 times faster than those for the original SDM. In the case of shallow cumulous, the simulation time when using the new SDM with 32–64 SDs per cell is shorter than that of a bin method with 32 bins and comparable to that of a two-moment bulk method. (3) Using the supercomputer Fugaku, we demonstrated that a numerical experiment with <span class="inline-formula">2</span> <span class="inline-formula">m</span> resolution and 128 SDs per cell covering <span class="inline-formula">13 824<sup>2</sup>×3072</span> <span class="inline-formula">m<sup>3</sup></span> domain is possible. The number of grid points and SDs are 104 and 442 times, respectively, those of the highest-resolution simulation performed so far. Our numerical model exhibited <span class="inline-formula">98</span> % weak scaling for 36 864 nodes, accounting for <span class="inline-formula">23</span> % of the total system. The simulation achieves <span class="inline-formula">7.97</span> PFLOPS, <span class="inline-formula">7.04</span> % of the peak ratio for overall performance, and a simulation time for SDM of <span class="inline-formula">2.86×10<sup>13</sup></span> particle <span class="inline-formula">⋅</span> steps per second. Several challenges, such as incorporating mixed-phase processes, inclusion of terrain, and long-time integrations, remain, and our study will also contribute to solving them. The developed model enables us to study turbulence and microphysics processes over a wide range of scales using combinations of direct numerical simulation (DNS), laboratory experiments, and field studies. We believe that our approach advances the scientific understanding of clouds and contributes to reducing the uncertainties of weather simulation and climate projection.</p> |
| first_indexed | 2024-03-11T13:50:47Z |
| format | Article |
| id | doaj.art-fbb9d77acf77475da7735cd00b1fce6a |
| institution | Directory Open Access Journal |
| issn | 1991-959X 1991-9603 |
| language | English |
| last_indexed | 2024-03-11T13:50:47Z |
| publishDate | 2023-11-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Geoscientific Model Development |
| spelling | doaj.art-fbb9d77acf77475da7735cd00b1fce6a2023-11-02T09:19:21ZengCopernicus PublicationsGeoscientific Model Development1991-959X1991-96032023-11-01166211624510.5194/gmd-16-6211-2023Overcoming computational challenges to realize meter- to submeter-scale resolution in cloud simulations using the super-droplet methodT. Matsushima0S. Nishizawa1S. Shima2S. Shima3Center for Planetary Science, Kobe University, Kobe, JapanRIKEN Center for Computational Science, Kobe, JapanRIKEN Center for Computational Science, Kobe, JapanGraduate School of Information Science, University of Hyogo, Kobe, Japan<p>A particle-based cloud model was developed for meter- to submeter-scale-resolution simulations of warm clouds. Simplified cloud microphysics schemes have already made meter-scale-resolution simulations feasible; however, such schemes are based on empirical assumptions, and hence they contain huge uncertainties. The super-droplet method (SDM) is a promising candidate for cloud microphysical process modeling and is a particle-based approach, making fewer assumptions for the droplet size distributions. However, meter-scale-resolution simulations using the SDM are not feasible even on existing high-end supercomputers because of high computational cost. In the present study, we overcame challenges to realize such simulations. The contributions of our work are as follows: (1) the uniform sampling method is not suitable when dealing with a large number of super-droplets (SDs). Hence, we developed a new initialization method for sampling SDs from a real droplet population. These SDs can be used for simulating spatial resolutions between meter and submeter scales. (2) We optimized the SDM algorithm to achieve high performance by reducing data movement and simplifying loop bodies using the concept of effective resolution. The optimized algorithms can be applied to a Fujitsu A64FX processor, and most of them are also effective on other many-core CPUs and possibly graphics processing units (GPUs). Warm-bubble experiments revealed that the throughput of particle calculations per second for the improved algorithms is 61.3 times faster than those for the original SDM. In the case of shallow cumulous, the simulation time when using the new SDM with 32–64 SDs per cell is shorter than that of a bin method with 32 bins and comparable to that of a two-moment bulk method. (3) Using the supercomputer Fugaku, we demonstrated that a numerical experiment with <span class="inline-formula">2</span> <span class="inline-formula">m</span> resolution and 128 SDs per cell covering <span class="inline-formula">13 824<sup>2</sup>×3072</span> <span class="inline-formula">m<sup>3</sup></span> domain is possible. The number of grid points and SDs are 104 and 442 times, respectively, those of the highest-resolution simulation performed so far. Our numerical model exhibited <span class="inline-formula">98</span> % weak scaling for 36 864 nodes, accounting for <span class="inline-formula">23</span> % of the total system. The simulation achieves <span class="inline-formula">7.97</span> PFLOPS, <span class="inline-formula">7.04</span> % of the peak ratio for overall performance, and a simulation time for SDM of <span class="inline-formula">2.86×10<sup>13</sup></span> particle <span class="inline-formula">⋅</span> steps per second. Several challenges, such as incorporating mixed-phase processes, inclusion of terrain, and long-time integrations, remain, and our study will also contribute to solving them. The developed model enables us to study turbulence and microphysics processes over a wide range of scales using combinations of direct numerical simulation (DNS), laboratory experiments, and field studies. We believe that our approach advances the scientific understanding of clouds and contributes to reducing the uncertainties of weather simulation and climate projection.</p>https://gmd.copernicus.org/articles/16/6211/2023/gmd-16-6211-2023.pdf |
| spellingShingle | T. Matsushima S. Nishizawa S. Shima S. Shima Overcoming computational challenges to realize meter- to submeter-scale resolution in cloud simulations using the super-droplet method Geoscientific Model Development |
| title | Overcoming computational challenges to realize meter- to submeter-scale resolution in cloud simulations using the super-droplet method |
| title_full | Overcoming computational challenges to realize meter- to submeter-scale resolution in cloud simulations using the super-droplet method |
| title_fullStr | Overcoming computational challenges to realize meter- to submeter-scale resolution in cloud simulations using the super-droplet method |
| title_full_unstemmed | Overcoming computational challenges to realize meter- to submeter-scale resolution in cloud simulations using the super-droplet method |
| title_short | Overcoming computational challenges to realize meter- to submeter-scale resolution in cloud simulations using the super-droplet method |
| title_sort | overcoming computational challenges to realize meter to submeter scale resolution in cloud simulations using the super droplet method |
| url | https://gmd.copernicus.org/articles/16/6211/2023/gmd-16-6211-2023.pdf |
| work_keys_str_mv | AT tmatsushima overcomingcomputationalchallengestorealizemetertosubmeterscaleresolutionincloudsimulationsusingthesuperdropletmethod AT snishizawa overcomingcomputationalchallengestorealizemetertosubmeterscaleresolutionincloudsimulationsusingthesuperdropletmethod AT sshima overcomingcomputationalchallengestorealizemetertosubmeterscaleresolutionincloudsimulationsusingthesuperdropletmethod AT sshima overcomingcomputationalchallengestorealizemetertosubmeterscaleresolutionincloudsimulationsusingthesuperdropletmethod |