|
|
[[!meta title="emacs-gc-stats: Does garbage collection actually slow down Emacs?"]]
[[!meta copyright="Copyright © 2023 Ihor Radchenko"]]
[[!inline pages="internal(2023/info/gc-nav)" raw="yes"]]
<!-- Initially generated with emacsconf-publish-talk-page and then left alone for manual editing -->
<!-- You can manually edit this file to update the abstract, add links, etc. --->
# emacs-gc-stats: Does garbage collection actually slow down Emacs?
Ihor Radchenko (he) - Mastodon: <https://emacs.ch/@yantar92>, <mailto:yantar92@posteo.net>
[[!inline pages="internal(2023/info/gc-before)" raw="yes"]]
Talk sources, PDF, raw data, and analysis are published at <https://dx.doi.org/10.5281/zenodo.10213384> .
Is Emacs responsiveness really affected by slow garbage collector?
Should `gc-cons-threshold' be increased during startup?
Or maybe during the whole Emacs session?
I will try to answer these questions using the real data collected from
Emacs users who installed
<https://elpa.gnu.org/packages/emacs-gc-stats.html> package and submitted
their results to <https://lists.gnu.org/archive/html/emacs-gc-stats/>.
About the speaker:
Materials science researcher, Org mode users since many years ago, Org
mode (unofficial) co-maintainer :)
The talk is an excuse to sum up emacs-gc-stats data for later discussion
of changing Emacs GC defaults:
https://yhetil.org/emacs-devel/87v8j6t3i9.fsf@localhost/
# Discussion
## Questions and answers
- Q: Are the GC duration statistics correlated with users? I mean:
does the same user experience GCs of various durations, or do some
users experience GCs of >0.2 s exclusively while others never
experience GCs of >0.2 s?
- A: Some users have <0.1 GC time, while others struggle with
near 1 sec. Really varies. But the number of people
with >0.2sec is significant enough to make GC a big deal. You
can check it yourself - there are GC stats plots for each
individual user in <https://zenodo.org/records/10213384>.
- Q:Having recently been working on a high-performance smooth
scrolling mode, which needs to respond to scroll events
arriving >50-60 times per second, a 100ms delay is *very*
noticeable in this scenario. For normal buffer interation and
commands 0.1s a reasonable dividing line, but I'd estimate you can
easily feel a 20ms delay during varoius "fast" interactions. Do
you think there is hope to "spread out" GC latency to keep it
below say 15ms, even if more frequent (without just repeating many
short GC's in a row)?
- A: The only reasonable "spread out" is deferring GC to
_after_ that scrolling. Like (let ((gc-cons-threshold <large
enough number to avoid multiple GCs>)) (do the scrolling)).
This is also what recommended by Emacs devs (AFAIR).
- Q:Opinions about gcmh-mode?
- A: (Not Ihor): Ironically it uses too many timers, creating
garbage of its own. It should use `timer-set-time` instead of
creating and throwing away timers after each command (via
`post-command-hook`) Interesting!
- A: (from Ihor): the problem is it ends up consuming a ton of
memory, increasing GC time, and that most GCs occur when Emacs
is being used intensively and there is no chance for Emacs to go
on idle and perform the GC. Since GC cons threshold is raised to
~1G (gcmh-high-cons-threshold) while Emacs is used - you will
face a really bad hang (seconds to tens of seconds regularly).
Ends up not helping much, recommend increasing
gc-cons-percentage=0.2 or so instead.
- Q:
- A:
- Q: Is there some way to free up memory (such as via
`unload-feature`) in Emacs? Often I only need a package loaded for
a single task/short period but it persists in memory afterwards.
- A: <https://elpa.gnu.org/packages/memory-usage.html>, and
built-in M-x memory-report - most of the time, it is some
history/cache variables of large buffers that are occupying
memory. The library code itself is rarely affecting GC. (The
other question is when libraries add timers/heavy mode-line
constructs/post-command-hooks/etc - that's indeed a problem,
but solved by disabling or not using a package; no need to
unload)
- Q: Very nice presentation! I just experimented with the threshold
and lowered my gc-elapsed from 1.1 to 0.06 seconds (during startup).
Interestingly, going to 10MB increased the time, 4MB was the
sweet-spot for my system. What is the recommended way to lower the
value back to the default value after startup is complete?
- A: after-init-hook
- Q:what were you using to flip through the PNGs? (thanks for the
answer. look-mode on melpa does that too ;)
- A: []{.underline}
[[https://feh.finalrewind.org/]{.underline}](https://feh.finalrewind.org/)
- Q: What was the final point you were making regarding Emacs 30? You
got cut off...
- A: M-x malloc-trim
- Q: With 16-32G RAMs a minimal OS swapping, how about systematically doing this temporary deferral @yantar92 suggested and leave it down for a longer GC at night and whatnot? Or would cons/allocation also degrade too noticeably?
- Not the speaker: That would cause Emacs to use a lot more total memory
- Indeed. Essentially the question is at what point all my daily mostly-textual Emacs usage doesn't come close to using all the available memory on a 32G sys? (but my mind went more to being concerned about new cons/alloca and fragmentation for the intra-day use) I'll have to look into it more before being cogent. One more onto the todo list then :)
- A: for increasing thresholds up to RAM limits, do remember that individual GC time will increase - with 32Gb RAM you will likely make individual GC prohibitedly slow sooner than later. I'd say that it only makes sense to increase the thresholds when you have multiple agglomerated GCs. Going beyond this is of little use. (I am thinking about adding some kind of summary statistics command to emacs-gc-stats, so that one can look into GC duration, frequency, init time, and agglomeration and then adjust the settings according to the results)
## Notes
- <https://elpa.gnu.org/packages/emacs-gc-stats.html>
- Data, presentation, and analysis:
<https://dx.doi.org/10.5281/zenodo.10213384>
- This presentation is a direct continuation of emacs-devel thread:
- <https://yhetil.org/emacs-devel/20230310110747.4hytasakomvdyf7i@Ergus/>
- At some point, Eli asked to collect GC statistics -
<https://yhetil.org/emacs-devel/83y1n2n11e.fsf@gnu.org/>
- <https://elpa.gnu.org/packages/emacs-gc-stats.html> and my talk
summarizing the results are the answer to that request.
- Now, we can continue the discussion on emacs-devel with real
data at hand :)
- I hope to push for a temporary bump of `gc-cons-threshold'
during Emacs init and possibly for increasing
`gc-cons-percentage'.
- Came for clear-cut magic bullet answers, left with nuanced analysis - and that, surprise, Eli was overall right? Now what to do with that viral gc init snippet that I've never taken time to measure myself but keep anyway...
- A: I do believe that temporarily raising thresholds is ok for init time. that's the only clear-cut conclusion, unortunately
- Thanks yantar92, both for the detailed investigation and exposition. I've been deferring to much-smarter-than-me Henrik for my default position (Doom has it in it's init), for lack for doing any measurements myself.
- Thanks for your work on this project. Very thorough.
- Definitely a huge extra thanks for the tireless Org-mode work yantar92!
- A: Do not take things Doom does blindly. I am still horrified by let-binding major-mode
- Good advice, thanks. I don't personally (more of a vanilla/DIY type myself), but I'd be remiss to leverage Henrik's insights nonetheless :)
- A: (fun fact: memory-info tries to get memory information on remote system when connected via TRAMP) ... not a problem (anymore; after that very surpising bug report) for emacs-gc-stats
[[!inline pages="internal(2023/info/gc-after)" raw="yes"]]
[[!inline pages="internal(2023/info/gc-nav)" raw="yes"]]
|
