Turbulent flow calculations are most efficient when codes are designed from the beginning to address specific applications. In this sense, the field is different to many other users of HPC, where only a handful of codes need to be parallelised and optimised. Principally there are three different code structures used in the consortium for large calculations:
- Finite Difference, compressible or incompressible, structured or body-fitted
- Finite Volume, compressible or incompressible, structured or unstructured
- Spectral hp element, incompressible, unstructured
Initially the consortium received support from the Southampton HPCI centre. At this time, some parts of the codes were optimised and a database project was initiated to prototype the exchange of data in common formats. When the Southampton HPCI Centre closed, support was transferred to Daresbury under the direction of Professor David Emerson. As coordinator of CCP12, Professor Emerson has been involved with all of the CFD research consortia since their formation in 1994. Support work has involved staff at Daresbury Laboratory, both through CCP12 and the HPCI Centre, in a range of activities that includes parallelisation of key application codes, porting, tuning and optimising codes, and evaluation of codes and algorithms on the UK's flagship computing facilities. With respect to the UKTC activities, Daresbury have worked on a range of codes and have contributed at various levels, from general advice to detailed optimisations. These include the parallelisation and optimisation of the SBLI code, the porting and evaluation of NEWT with Professor Savill at Cambridge, and the initial conversion of the vectorial code EBL to F90 with Professor Coleman at Southampton.
In the current grant, Daresbury are focusing on two main codes in the consortium, namely incompact3d and Code_Saturne. For incompact3d, which is a high-order finite difference code that can support grid stretching in one direction only, the work consists of adding a second stretched direction that will allow users to perform high-quality square cylinder simulations and also improve simulations around obstacles using the immersed boundary method. For Code_Saturne, the work is to test and give directions to optimise its performance on different platforms. Daresbury also work on determining the optimal number of processors for running on ARCHER by testing and profiling the UKTC codes. The profiling data is also used to identify where the greatest needs lie, i.e. where in the code the most AUs will be spent, investigate the MPI communication to avoid bottlenecks and barriers, explore where parallel I/O may be introduced, and test the codes for extreme scalability in preparation for future exascale platforms.