combustion
Transcription
combustion
HPC and combustion Thierry POINSOT IMFT, Université de Toulouse, CNRS and CERFACS Contributions from G. Staffelbach, L. Gicquel, B. Cuenot, O. Vermorel, F. Duchaine, E. Riber (CERFACS), L. Selle (IMFT), V. Moureau (CORIA), S. Mendez (UM2) 1 OUTLINE • COMBUSTION RESEARCH AND HPC • PERFORMING HPC COMBUSTION SIMULATION ON TIER0/ TIER1 SYSTEMS FOR INDUSTRY: ! HOW CAN THIS BE DONE ? WHAT DID THE FRENCH COMMUNITY DO ? ! WHAT ARE THE WEAKEST PARTS OF OUR ORGANIZATION TO DO THIS ? 2 COMBUSTION OVERVIEW Just remember two equations: ENERGY ON EARTH TODAY = COMBUSTION 3 ENERGY ON EARTH = COMBUSTION COMBUSTION IS PRODUCING MORE THAN 90 PERCENT OF THE ENERGY TODAY. THIS WILL DECREASE... BUT NOT TOMORROW 4 COMBUSTION OVERVIEW Just remember two equations: ENERGY ON EARTH TODAY = COMBUSTION ENERGY ON EARTH TOMORROW = COMBUSTION 5 CLIMATE CHANGE AND ENERGY MARKET: 2010/2030 •COMBUSTION SCIENCE MUST ALLOW THIS WITHOUT INCREASING EMISSIONS, WASTING FOSSIL FUELS OR MAKING CLIMATE CHANGE WORSE (!...) OPTIMIZATION AND SIMULATION NEEDED 6 SIMULATION OF COMBUSTION: MULTISCALE -MULTIPHYSICS Heat transfer, radiation, flow, turbulence, chemistry, fatigue, vibration, acoustics Engines! Contrails 7 COMBUSTION: MULTISCALE MULTIPHYSICS Within the combustion chamber: nanoseconds and nanometers 1 cm 1 cm Fields of density in a H2-O2 engine 8 COMBUSTION: MULTISCALE MULTIPHYSICS Outside the engine: miles and days ! 2 miles 9 WHICH EQUATIONS ? •The Navier Stokes equations (5 + N unknowns: density, velocities and energy, N species). Partial differential equations -> non local, intense communication required •Kinetics : N = 10 to 300 species reacting through 3000 reactions (everything local) •Heat transfer through the walls, radiation, noise, soot •All these flows are turbulent 10 WHAT DO WE COMPUTE ? •FINITE VOLUME CODES USING DOMAIN DECOMPOSITION AND MPI. •Typically 100 Mcells with 100 variables at each cell (3 velocities, density, energy + 5 to 90 species) over 1000000 time steps. -> 10^16 unknowns 11 WHAT ABOUT PARALLELISM ? •THE NAVIER STOKES EQUATIONS ARE ‘EMBARRASINGLY DIFFICULT’ TO PARALLELIZE -> THIS PROBLEM HAS BEEN IDENTIFIED AND IS REMODELING OUR COMMUNITY IN LARGER COLLABORATIVE TEAMS. •THIS IS NOT A ‘ONE-PROFESSOR ONECODE’ SHOW ANY MORE BECAUSE THE CODES ARE USED FOR INDUSTRY APPLICATIONS ON A DAILY BASIS 12 ok !"##$!!%% http://success.coria-cfd.fr A joint initiative of French labs for the promotion of SUper-Computing for the modeling of Combustion, mixing and complex fluids in rEal SyStems. The SUCCESS scientific group SUCCESS was created in 2012 to help the promotion of super-computing in the area of Computational Fluid Dynamics (CFD) for complex geometries. It is coordinated by the CORIA lab and is composed of 8 French public labs Our objectives Distribute in the labs research HPC codes for CFD in complex geometries Ensure the training of users Manage the development roadmap Share databases of high-resolution simulations Promote super-computing Some facts 8 French public labs Around 120 researchers and students 2 PRACE proposals accepted over the recent years Several prizes related to !"##$!! codes: Bull-Joseph Fourier prize, IBM faculty award, ... The codes A massively-parallel finite-volume and finiteelement 3D code for the simulation of compressible turbulent reactive and two-phase flows. A massively-parallel finite-volume 3D code for the simulation of turbulent reactive and twophase flows at low-Mach number. The labs CNRS labs: CORIA, EM2C, I3M, LEGI, IMFT, LMA EPIC labs: CERFACS, IFP-EN 13 AVBP Strong scaling examples 65536 (1) ANL INTREPID, Bluegene P PRACE/TGCC, CURIE, BullX 57344 (2) (3) GENCI/CINES, JADE, SGI Altix ICE (4) PRACE/JSC, JUQUEEN, Bluegene Q equivalent performance 49152 INCITE/ARNL, INTREPID, Bluegene P HLRS/PRACE, HERMIT, CRAY XE6 Ideal 40960 (5) (5) 1 billion cells (BG/Q) (1) (2) (3) (4) (5) 32768 93M Tetrahedra case - 1 step Chemistry - 2 tasks per node 200M Tetrahedra case - 2 step Chemistry 29M Tetrahedra case - 7 step Chemistry 75M Tetrahedra case - No chemistry - 64 tasks per nod 75M Tetrahedra case - No chemistry - 4 tasks per node 24576 16384 8192 0 0 8192 16384 24576 32768 cores 40960 49152 57344 65536 14 DNS analysis of a Re = 40,000 swirl burner V. Moureau, P. Domingo, L.Vervisch, D.Veynante YALES2 scale-up on Babel @ IDRIS (Blue Gene/P) Up to 12288 cores and 2.6 billion tetrahedrons linear YALES2 12288 12288 10240 10240 Scale-up 2.6B tets 8192 8192 2008 6144 6144 4096 2048 0 4096 329M tets 878M tets 2048 41M tets 14M tets 0 2048 4096 6144 8192 10240 12288 0 Number of cores Juelich Workshop 2010 YALES2 solver (CORIA Rouen) ORGANIZATION OF THE FRENCH NUMERICAL COMBUSTION COMMUNITY SNECMA SNECMA DMS TURBOMECA RHODIA ALSTOM AIR LIQUIDE HERAKLES AIRBUS SIEMENS ANSALDO 16 G6S'564'58+*Q !"#$%&$'()'(*+, -./!0%&/'(+1, 2"3-%&-+1456)+%7658*, !/792%&"65*+8))+, $:"%&"(18;<, $:%281=<'>+1 $?2@$2 "0A/!A .@2/0 :18>B%76( -'=+ "'=+)* CERFACS Institut Français du Pétrole !"#$%&'()*()+,#-./#0)-1(23'(4567(89&:;< @++=* -'=+ :*+5*Q C@2-"0%&D8))65';<+, $:/E."2-0%&E'5=+*, C@2-"0%&D+51'1, C7C%&E'5=+6(F, /2@0:G$ 7C0 0GC$." 0@C0GA. C!2"2@C #2//0/! 0!/%G!H:!A2 I0J%A2%#/0@-2 K C$0@#./A =>?=@! A4+,)--'B 17 -'LM(4+5%;'LM618+*%N -'LM(O1P%;+14+5*Q%-20R%E65;+)'16R%I2@-! =>@!C!D@=>E(@F(GFCH(G=@I(@I!(JD4(@FF1 GFCH=>E(6F@I(=>(7C4?!(4>K(=>?=@!(7CFL!?@D !"#$%&'()$*+,-.*/$01#2)&34$#5$67*6!+$85$9:;< MN THIS ORGANIZATION ALLOWS TO DO OTHER THINGS IN OTHER FIELDS: S. Mendez, F. Nicoud YALES 2 Université de Montpellier 19 TWO TYPICAL PRACE EXAMPLES • PRECCINSTA: A laboratory burner using a Turbomeca injection system. Developed in EC project PRECCINSTA and used as the classical benchmark for LES codes in the world. Most important problems in the GT industry can be reproduced on the PRECCINSTA burner. CH4 + Air at 1 bar. • MASCOTTE: a high pressure experiment corresponding to rocket combustion (60 to 100 bars). Very powerful flames: H2/O2. Very extreme conditions (flames of the order or 10 microns) 20 PRECCINTSA (2005): THE FIRST LES OF SWIRLED BURNERS 3 millions cells 21 Roux, Lartigue, Poinsot, Meier and Bérat Comb and Flame 141, 2005 COLD FLOW - PRECCINSTA 22 Comparison of mean velocity fields All LES are compared with measurements DLR (LDA): • Velocity profiles are compared at five stations along the burner. • Comparison for axial, tangential and radial velocities (mean and RMS) X= 1,5 mm W X= 5 mm X= 15 mm X= 25 mm X= 35 mm FROM SWIRLER 23 TO OUTLET U Distance from axis [mm] RMS velocity profiles: Stanford code (red), CERFACS code (black) Uxp Profiles: Red solid: CDP - Black solid: TTGC - Black dotted: TTGC_SSS - Circles: Exp. and DLR experiments (symbols) 40 40 40 40 40 20 20 20 20 20 0 0 0 0 0 -20 -20 -20 -20 -20 -40 -40 -40 -40 -40 0 5 10 15 x=1.5 mm 24 0 5 10 15 x=5 mm 0 5 10 15 x=15 mm 0 5 10 15 x=25 mm 0 5 10 15 x=35 mm PRECCINSTA: •First computed with LES in 2004: 1 Million cells (AVBP) •Repeated in 2007 with 10 Mcells (AVBP) •In 2009, repeated with 100 Mcells, 500 Mcells (YALES) •In 2010 and 2011, on PRACE machines: JUGENE: 2 billion and CURIE 12 billion cells (YALES) 25 =84>%?84%'#5$#@$(#1'&)4$>485A$3B)$C$&183)18#5$85$ D,+**67/!.$EF1$G#>1)%>H$*I,6.J 3T DLR PRECCINSTA BURNER, Experiments: W. Meier et al., Combust. Flame, 150(1/2):2–26, 2007 More details in: Roux et al, Combustion and Flame (2005) Moureau et al, Journal of Computational Physics (2007) (2 papers) Galpin et al, Combustion and Flame (2008) Moureau et al, Combustion and Flame (2011) Franzelli et al, Combustion and Flame (2011) 3U IS THIS ENOUGH ?: •At 20 billion cells, we are almost reaching what we need in terms of resolution for the large flow structures but near walls and within the flame front, this is not enough • This is a single burner at atmospheric pressure with gaseous fuel (no liquid phase). Real combustors have 16 to 24 burners, working at 20 to 100 bars, with liquid fuels. •We still miss a 10^6 factor in power 28 ROCKET COMBUSTION: • PRACE Project by EM2C and CERFACS • ROCKET COMBUSTION IS A VERY SPECIFIC AND NARROW FIELD: IT IS ALSO DIFFICULT SINCE FLUIDS IN THESE ENGINES ARE IN SUPERCRITICAL CONDITIONS (100 bars) • CERFACS AND EM2C HAVE DEVELOPED SUPERCRITICAL CAPABILITIES IN AVBP IN THE LAST FIVE YEARS • PRACE WORK PERFORMED IN 2012: ACOUSTIC EXCITATION OF SUPERCRITICAL FLAMES IN THE CONFIGURATION MASCOTTE OF ONERA 29 ! ! T. Schmitt, H. Layal, M. Boileau, S. Ducruix, S.Candel (EM2C), A. Ruiz, G. Staffelbach, B. Cuenot and T. Poinsot (CERFACS) Large-Eddy Simulation of high-frequency instabilities under transcritical conditions Objective: Observe and understand the flame behavior in transcritical flows of an experimental setup submitted to artificial acoustic transverse perturbation. 8.5M hours at TGCC ! CEA - CERFACS 30 WITH FORCING: CEA - CERFACS 31 WHAT IS GOING ON IN THE USA IN THE FUELD OF COMBUSTION AND HPC ? In the last 15 years: the ASCI projects. Most ASCI projects in the USA were actually combustion projects: gas turbines, scramjets, rockets, fires, galaxies... Now: the CODESIGN Centers. Example: Sandia CODESIGN Center. Main idea: «Compute combustors before you build them» 32 FOR THE FUTURE IN EUROPE • EESI and PRACE have opened the path for a fast HPC evolution in Europe. • Machines are already here (Tier1 and Tier0). • The main questions today are: (1) the codes: see GIS SUCCESS lead by CNRS (2) the money: from industry and the EC (3) the users ! 34 Money: the ERC (European Research Council) INTECOCIS advanced grant at Institut de Mécanique des Fluides and CERFACS 2013-2018 (intecocis.inp-toulouse.fr) • Five year, 2.5 Meuros project on HPC tools for combustion instabilities. Coordinator: IMFT • Ten researchers on numerical combustion, 5 years • Collaboration with GENCI, SAFRAN, ANSALDO, SIEMENS 35 USERS / FORMATION: ‘ignored’ problem ? The last ten years using AVBP at CERFACS and the last two years supporting an explosive development of AVBP and YALES at SAFRAN have shown that building an efficient HPC CFD team in combustion required multiple experts: CERFACS Combustion experts - know the physics - write the models - use the code but do not address HPC HPC experts - develop the code - port it on new machines - do the first demos 36 CERFACS Combustion experts - know the physics - write the models - use the code but do not address HPC Students: - know nothing Industry experts: - know combustors - know ‘old style’ CFD and learn HPC HPC experts - develop the code - port it on PRACE/ INCITE machines - do the first demos Combustion labs: - know theory and experiments - dont know the code details or HPC Sous traitants: - know a little bit on CFD and the combustors 37 CERFACS today Interface / formation - interfaces to access the code - formation to learn the basics Students: - know nothing Combustion experts - know the physics - write the models - use the code but do not address HPC Industry experts: - know the engine - know ‘old style’ CFD and learn HPC HPC experts - develop the code - port it on PRACE machines - do the first demos Combustion labs: - know theory and experiments - dont know the code details or HPC Sous traitants: - know a little bit on CFD and the engine 38 INTERFACES: DISTRIBUTING HPC CODES TO INDUSTRY Mmmh... C’est bientôt fini? La taille de ce tube est le vrai point critique aujourd’hui A . Dauptain Copyright 39 DISTRIBUTING LES CODES: 0 0 • Codes CANNOT be used in industry without additional interfaces (it works with labs but not with industry). • These interfaces must integrate a very large set of information coming from industrial needs • The time required to write these interfaces is VERY large: the codes MUST be modified and simplified to adjust to the interfaces 40 INTERFACES: THE C3S EXAMPLE • In the last 4 years, CERFACS has developed an interface called C3S for AVBP users in labs and industry 0 0 • C3S installed and working at CERFACS, IMFT, EM2C, IFPEN, SNECMA, Villaroche, SNECMA Vernon, TURBOMECA Bordes, etc 41 EXAMPLE : 42 BUT THIS IS NOT ENOUGH !CREATING ONE INTERFACE FOR ONE CODE IS NOT ENOUGH: • Keeping up with AVBP developments in C3S is difficult:-> must coordinate code and interface evolutions. The interface MUST evolve with the code (or vice versa ?) • If you write a piece of code, you should also, at the same time, write the interface !In 2011: a new initiative -> C3Sm (A. Dauptain) 43 C3SM • Is an engine... which writes interfaces: !developed by CERFACS !used by all french groups developing AVBP or any other code linked to AVBP, to automatically produce and update the interface and the documentation: YALES, PRISSMA (radiation), AVTP (heat transfer), N3S (RANS), Coolant (SAFRAN dedicated software for cooling)... 44 !"#$%&'&(&)*#+,-"+./'%-0)-/0'-&"1#2345 234 5 2*#$%&'&(&)*#%*"+#$&//-:)*#1 • #)0#(%;0'-&"#+,895#$0%#)*/#+;<*)&$$*.%/#+*#(&+* 5&'*.% 895 •2&"'*".#657 #)0#(%;0'-&"#+*##=&'*.%/#895#$*%>&%=0"'/#*'#(&=$0'-:)*/ ?<0"'#1#2&"'%-:.'-&"#=&"&)-'@-A.* B;<*)&$$*.%/#+*# (&+*/#/(-*"'-CA.*/ 2%;0'*.%/# +,-"'*%>0(*/ !"#$% &'% !'-)-/0'*.%/#+*#(&+*/# /(-*"'-CA.*/ 7*#(%;0'*.%#+,-"'*%>0(*/#*/'#0.#=-)-*.#+.#+-/$&/-'->D#-)#*/'# -"(&"'&.%"0:)*D#*'#(%;*#+*#),0''*"'*E ?$%F/#1#2&"'%-:.'-&"#%;$0%'-* 2%;0'*.%/# +,-"'*%>0(*/ &()*"&% B;<*)&$$*.%/#+*# (&+*/#/(-*"'-CA.*/ 234 5 !'-)-/0'*.%/#+*#(&+*/# /(-*"'-CA.*/ 7*/#+;<*)&$$*.%/#*'#.'-)-/0'*.%/#$*.<*"'#>0-%*#;<&).*%#)*.%# &.'-)#(@*G#*.HD#/0"/#$0//*%#$0%#),-"'*%=;+-0-%*#+.#(%;0'*.%# +,-"'*%>0(*E WHAT ABOUT FORMATION ? • PhDs are of course a good solution: !EM2C, CORIA, IMFT, IFP, CERFACS produce more than 20 PhD per year. They go to industry and they know HPC. !But this is not fast enough • Systematic courses on HPC/CFD/Combustion are required: in 2011, formation cycles at CERFACS 48 !"#$%&'()*#$+,-./*0-%*1(%2$*3..4*5&#")(6-7*"/&72* 89:;* <-.$*'-"=)&72*"/&72*>=$7;81?* !"#$%%&&&'()*+,(-'+*%.*,/0/01% %%%%(+.0606?@'*+<.';A<+% B6% ,"!!C% DEFGHDI% 060J.(*;% .% (+.0606?% /+&?+.=%<&+%'&=/7(*+%;'0*6'*%(.+?*J6?% ;(7K*6(;C% L>M;C% *6?06**+;% .6K% +*;*.+'>*+;C% 06% .'.K*=0'% .6K% .//10*K% +*;*.+'>A%% N>*% (+.0606?% ;*;;0&6;% .+*% (.7?>(% )8% DEFGHDI%+*;*.+'>*+;%.6K%*6?06**+;A%% L+.'J'.1% (+.0606?% ;*;;0&6;% K*K0'.(*K% (&% (>*% '&K*;% K*O*1&/*K% &+% '&#K*O*1&/*K% )8% DEFGHDI% .+*% /.+(% &<% (>*% /+&?+.=A% DEFGHDI% 0;% .7(>&+0P*K% (&% ?0O*% (+.0606?% ;*;;0&6;% Q0(>06% (>*% B6K0O0K7.1% +0?>(% (&% (+.0606?%RMBGSA% !"#!$%&'(&)*+%,"!!% ,#-%./+01%,"!,% %!2#!3%&'(&)*+%,"!!% !4#,"%5.67.+8%,"!,% !9#!:%576*%,"!,% 8.@(7'$.*#$/,*2$7$%(6-7*"/&72*<3!A8BC* ,-#,2%5.67.+8%,"!,% D(+(*(//&#&)(6-7*"/&72*>=$7;81?* 9"%5.67.+8#!;(%<*)+7.+8% ,"!,% ;%-2%(##&72*&7*E-%+%(7*FG*** !9#!$%=.+'>%,"!,% <-#="+$%*(%',&+$'+"%$/*(7.*-=6#&/(6-7* ,,%=.+'>%,"!,% ;(%())$)*=%-2%(##&72*+--)/* ,9#%,$%./+01%,"!,% D-H7/'()&72*')&#(+$*/'$7(%&-/*H&+,*+,$*H$(+,$%I+4=&72* J(/$.*#$+,-.-)-24*D5<1K?* ,:#,-%./+01%,"!,% 5-)"6-7*#$+,-./*0-%*-=6#&/(6-7*=%-J)$#/* 3#!!%=.8%,"!,% E"7.(#$7+()/*+-*"7.$%/+(7.*(7.*(7()4L$*,&2,*M.$)&+4* <ED*/&#")(6-7/* ,!#,:%=.8%,"!,% <-.$*'-"=)&72*"/&72*>85K5* %9"#9!%=.8#!;(%576*%,"!,% !"#$%&'()*#$+,-./*0-%*+,$*$)$'+%-#(27$6'*=%-J)$#* "/&72*<35<* -#4%%576*%,"!,% 5-)"6-7*#$+,-./*0-%*)&7$(%*/4/+$#/* !!#!,%576*%,"!,% *K7+%-."'6-7*+-*.(+(*(//&#&)(6-7* !4#!3%576*%,"!,% 5-)"6-7/*-0*=(%6()*.&N$%$76()*$O"(6-7/P*(==)&'(6-7/* +-*+,$*H(@$*=%-=(2(6-7*=%-J)$#* ,:%#%,-%%576*%,"!,% 8.@(7'$.*()2-%&+,#/*0-%*.(+(*(//&#&)(6-7* ,2#,3%576*%,"!,% 49 50 www.cerfacs.fr/elearning/combustion/ 51 CONCLUSIONS • DANS 10 ANS, LES GROUPES QUI AURONT CONTINUE A DEVELOPPER DES CODES DE PETITE TAILLE EN COMBUSTION, AURONT BEAUCOUP DE MAL A PUBLIER MAIS L’EFFORT POUR CONSTRUIRE DES GRANDS CODES EST ENORME • SUCCESS: REPONSE ‘LOGICIEL’ DES COMBUSTIONNISTES FRANCAIS. COUVRE LES LABOS ET LES INDUSTRIELS • L’ARRIVEE DE GENCI A PERMIS A LA COMMUNAUTE ‘COMBUSTION NUMERIQUE’ DE DISPOSER DES MOYENS NÉCESSAIRES POUR DEMONTRER SON SAVOIR FAIRE • PRACE ET INCITE COMPLETENT CE DISPOSITIF • LA COMBUSTION EST UN EXEMPLE OU CES OUTILS HPC SONT UTILISES AUJOURD’HUI PAR L’INDUSTRIE QUI SUPPORTE LEUR DEVELOPPEMENT DE FACON TRES FORTE 52