Microbenchmarking like (eg. with QuickBench) can be helpful, though for the content / effects system in FreeOrion, it's maybe not so useful, since the cases where we care about the performance are somewhat harder to test in isolation. For example, doing a full effects update on a big game a few hundred turns in, such as by changing a planet's focus in the UI. To test that, there needs to be a whole Universe and other gamestate set up (if not also a UI), and the only way to reproducibly and practically do that would be to have a save game that you load before starting the timing for each repetition of the benchmark. Just benchmarking small bits of the system in isolation might not give useful results, as it may not adequately replicate things like caching issues or the time it takes to look up objects to evaluate a condition, which could affect the results more in a "real world" test.
What I usually do to test performance effects of changes and to identify bottlenecks is run the game with the Visual Studio profiler attached, and cycle through systems in the UI by tapping the hotkey to do that, which causes an effects update to be run repeatedly. There are other profilers that can work similarly ("Tracy" seem popular recently).
Also, there are a bunch of lines in the logs about how long stuff takes, including some specific output of individual bits of content evalulation that took a long time, like:andThat said, microbenchmarking is fine with me if you want to set it up. I'm not sure how to do so for non-trivial cases for game mechanic internals like evaluating ValueRefs or Condition or executing effects, without having a populated Universe and other gamestate, though...
Specifically for CheckSum calculations, I don't really care if they are a bit slow, as they aren't run much or likely to substantially affect the player experience.
Also, to be clear, the point of the changes in the linked pull request is not just performance. It's also making the game mechanics classes testable at compile time, which also helps ensure they work correctly, not just quickly / less slowly.
What I usually do to test performance effects of changes and to identify bottlenecks is run the game with the Visual Studio profiler attached, and cycle through systems in the UI by tapping the hotkey to do that, which causes an effects update to be run repeatedly. There are other profilers that can work similarly ("Tracy" seem popular recently).
Also, there are a bunch of lines in the logs about how long stuff takes, including some specific output of individual bits of content evalulation that took a long time, like:
Code:
20:41:54.529167 {0x00008858} [debug] timer : ScopedTimer.cpp:67 : StoreTargetsAndCausesOfEffectsGroup < 347 > cause type: ECT_SPECIES specific cause: SP_FULVER sources: 13 scope: And [ Planet Not VisibleToEmpire empire = Source.Owner Not Source Not OwnedBy empire = Source.Owner WithinStarlaneJumps jumps = 2 condition = Source] time: 9326 µs20:41:54.555173 {0x0000235c} [debug] timer : ScopedTimer.cpp:67 : StoreTargetsAndCausesOfEffectsGroup < 981 > cause type: ECT_SPECIES specific cause: SP_FULVER sources: 48 scope: And [ Planet Not VisibleToEmpire empire = Source.Owner Not Source Not OwnedBy empire = Source.Owner WithinStarlaneJumps jumps = 2 condition = Source] time: 19.4 msCode:
20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - planet species time: 24 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - ship species time: 6 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - specials time: 1 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - techs time: 5 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - policies time: 2 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - buildings time: 4 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - ship hull/parts time: 4 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - fields time: 0 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - eval waiting time: 1 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:285 : Effect TargetSets Evaluation - reordering time: 5 ms20:41:54.566175 {0x0000ab04} [debug] timer : ScopedTimer.cpp:295 : Effect TargetSets Evaluation time: 59.2 msSpecifically for CheckSum calculations, I don't really care if they are a bit slow, as they aren't run much or likely to substantially affect the player experience.
Also, to be clear, the point of the changes in the linked pull request is not just performance. It's also making the game mechanics classes testable at compile time, which also helps ensure they work correctly, not just quickly / less slowly.
Statistics: Posted by Geoff the Medio — Wed May 29, 2024 7:03 pm