diff options
Diffstat (limited to '2023/talks/gc.md')
| -rw-r--r-- | 2023/talks/gc.md | 134 |
1 files changed, 134 insertions, 0 deletions
diff --git a/2023/talks/gc.md b/2023/talks/gc.md new file mode 100644 index 00000000..dac7add0 --- /dev/null +++ b/2023/talks/gc.md @@ -0,0 +1,134 @@ +[[!meta title="emacs-gc-stats: Does garbage collection actually slow down Emacs?"]] +[[!meta copyright="Copyright © 2023 Ihor Radchenko"]] +[[!inline pages="internal(2023/info/gc-nav)" raw="yes"]] + +<!-- Initially generated with emacsconf-publish-talk-page and then left alone for manual editing --> +<!-- You can manually edit this file to update the abstract, add links, etc. ---> + + +# emacs-gc-stats: Does garbage collection actually slow down Emacs? +Ihor Radchenko (he) - Mastodon: <https://emacs.ch/@yantar92>, <mailto:yantar92@posteo.net> + +[[!inline pages="internal(2023/info/gc-before)" raw="yes"]] + +Talk sources, PDF, raw data, and analysis are published at <https://dx.doi.org/10.5281/zenodo.10213384> . + +Is Emacs responsiveness really affected by slow garbage collector? +Should `gc-cons-threshold' be increased during startup? +Or maybe during the whole Emacs session? + +I will try to answer these questions using the real data collected from +Emacs users who installed +<https://elpa.gnu.org/packages/emacs-gc-stats.html> package and submitted +their results to <https://lists.gnu.org/archive/html/emacs-gc-stats/>. + +About the speaker: + +Materials science researcher, Org mode users since many years ago, Org +mode (unofficial) co-maintainer :) + +The talk is an excuse to sum up emacs-gc-stats data for later discussion +of changing Emacs GC defaults: +https://yhetil.org/emacs-devel/87v8j6t3i9.fsf@localhost/ + +# Discussion + +## Questions and answers + +- Q: Are the GC duration statistics correlated with users? I mean: + does the same user experience GCs of various durations, or do some + users experience GCs of >0.2 s exclusively while others never + experience GCs of >0.2 s? + - A: Some users have <0.1 GC time, while others struggle with + near 1 sec. Really varies. But the number of people + with >0.2sec is significant enough to make GC a big deal. You + can check it yourself - there are GC stats plots for each + individual user in <https://zenodo.org/records/10213384>. +- Q:Having recently been working on a high-performance smooth + scrolling mode, which needs to respond to scroll events + arriving >50-60 times per second, a 100ms delay is *very* + noticeable in this scenario. For normal buffer interation and + commands 0.1s a reasonable dividing line, but I'd estimate you can + easily feel a 20ms delay during varoius "fast" interactions. Do + you think there is hope to "spread out" GC latency to keep it + below say 15ms, even if more frequent (without just repeating many + short GC's in a row)? + - A: The only reasonable "spread out" is deferring GC to + _after_ that scrolling. Like (let ((gc-cons-threshold <large + enough number to avoid multiple GCs>)) (do the scrolling)). + This is also what recommended by Emacs devs (AFAIR). +- Q:Opinions about gcmh-mode? + - A: (Not Ihor): Ironically it uses too many timers, creating + garbage of its own. It should use `timer-set-time` instead of + creating and throwing away timers after each command (via + `post-command-hook`) Interesting! + - A: (from Ihor): the problem is it ends up consuming a ton of + memory, increasing GC time, and that most GCs occur when Emacs + is being used intensively and there is no chance for Emacs to go + on idle and perform the GC. Since GC cons threshold is raised to + ~1G (gcmh-high-cons-threshold) while Emacs is used - you will + face a really bad hang (seconds to tens of seconds regularly). + Ends up not helping much, recommend increasing + gc-cons-percentage=0.2 or so instead. +- Q: + - A: +- Q: Is there some way to free up memory (such as via + `unload-feature`) in Emacs? Often I only need a package loaded for + a single task/short period but it persists in memory afterwards. + - A: <https://elpa.gnu.org/packages/memory-usage.html>, and + built-in M-x memory-report - most of the time, it is some + history/cache variables of large buffers that are occupying + memory. The library code itself is rarely affecting GC. (The + other question is when libraries add timers/heavy mode-line + constructs/post-command-hooks/etc - that's indeed a problem, + but solved by disabling or not using a package; no need to + unload) +- Q: Very nice presentation! I just experimented with the threshold + and lowered my gc-elapsed from 1.1 to 0.06 seconds (during startup). + Interestingly, going to 10MB increased the time, 4MB was the + sweet-spot for my system. What is the recommended way to lower the + value back to the default value after startup is complete? + - A: after-init-hook +- Q:what were you using to flip through the PNGs? (thanks for the + answer. look-mode on melpa does that too ;) + - A: []{.underline} + [[https://feh.finalrewind.org/]{.underline}](https://feh.finalrewind.org/) +- Q: What was the final point you were making regarding Emacs 30? You + got cut off... + - A: M-x malloc-trim +- Q: With 16-32G RAMs a minimal OS swapping, how about systematically doing this temporary deferral @yantar92 suggested and leave it down for a longer GC at night and whatnot? Or would cons/allocation also degrade too noticeably? + - Not the speaker: That would cause Emacs to use a lot more total memory + - Indeed. Essentially the question is at what point all my daily mostly-textual Emacs usage doesn't come close to using all the available memory on a 32G sys? (but my mind went more to being concerned about new cons/alloca and fragmentation for the intra-day use) I'll have to look into it more before being cogent. One more onto the todo list then :) + - A: for increasing thresholds up to RAM limits, do remember that individual GC time will increase - with 32Gb RAM you will likely make individual GC prohibitedly slow sooner than later. I'd say that it only makes sense to increase the thresholds when you have multiple agglomerated GCs. Going beyond this is of little use. (I am thinking about adding some kind of summary statistics command to emacs-gc-stats, so that one can look into GC duration, frequency, init time, and agglomeration and then adjust the settings according to the results) + +## Notes + +- <https://elpa.gnu.org/packages/emacs-gc-stats.html> +- Data, presentation, and analysis: + <https://dx.doi.org/10.5281/zenodo.10213384> +- This presentation is a direct continuation of emacs-devel thread: +- <https://yhetil.org/emacs-devel/20230310110747.4hytasakomvdyf7i@Ergus/> + - At some point, Eli asked to collect GC statistics - + <https://yhetil.org/emacs-devel/83y1n2n11e.fsf@gnu.org/> + - <https://elpa.gnu.org/packages/emacs-gc-stats.html> and my talk + summarizing the results are the answer to that request. + - Now, we can continue the discussion on emacs-devel with real + data at hand :) + - I hope to push for a temporary bump of `gc-cons-threshold' + during Emacs init and possibly for increasing + `gc-cons-percentage'. +- Came for clear-cut magic bullet answers, left with nuanced analysis - and that, surprise, Eli was overall right? Now what to do with that viral gc init snippet that I've never taken time to measure myself but keep anyway... + - A: I do believe that temporarily raising thresholds is ok for init time. that's the only clear-cut conclusion, unortunately +- Thanks yantar92, both for the detailed investigation and exposition. I've been deferring to much-smarter-than-me Henrik for my default position (Doom has it in it's init), for lack for doing any measurements myself. +- Thanks for your work on this project. Very thorough. +- Definitely a huge extra thanks for the tireless Org-mode work yantar92! +- A: Do not take things Doom does blindly. I am still horrified by let-binding major-mode + - Good advice, thanks. I don't personally (more of a vanilla/DIY type myself), but I'd be remiss to leverage Henrik's insights nonetheless :) +- A: (fun fact: memory-info tries to get memory information on remote system when connected via TRAMP) ... not a problem (anymore; after that very surpising bug report) for emacs-gc-stats + + +[[!inline pages="internal(2023/info/gc-after)" raw="yes"]] + +[[!inline pages="internal(2023/info/gc-nav)" raw="yes"]] + + |