<a name="transcript"></a>
# Transcript
[[!template text="Hi everybody, my name is Andrea Corallo," start="00:00:00.200" video="mainVideo" id=subtitle]]
[[!template text="and this presentation is about" start="00:00:02.440" video="mainVideo" id=subtitle]]
[[!template text="the Emacs Lisp Native Compiler --" start="00:00:03.382" video="mainVideo" id=subtitle]]
[[!template text="having GNU Emacs able to" start="00:00:05.904" video="mainVideo" id=subtitle]]
[[!template text="compile and run Emacs Lisp as native code." start="00:00:07.184" video="mainVideo" id=subtitle]]
[[!template text="This project has been my hobby project" start="00:00:08.907" video="mainVideo" id=subtitle]]
[[!template text="for the last about two years and a half" start="00:00:11.667" video="mainVideo" id=subtitle]]
[[!template text="And we will see a little bit" start="00:00:14.111" video="mainVideo" id=subtitle]]
[[!template text="where this project is coming from," start="00:00:15.509" video="mainVideo" id=subtitle]]
[[!template text="where we are, and where we want to go." start="00:00:17.070" video="mainVideo" id=subtitle]]
[[!template text="So essentially everything you need to know" start="00:00:19.015" video="mainVideo" id=subtitle]]
[[!template text="about the Emacs Lisp Native Compiler," start="00:00:20.672" video="mainVideo" id=subtitle]]
[[!template text="and probably a little more." start="00:00:22.232" video="mainVideo" id=subtitle]]
[[!template text="Just a little bit of context on Emacs Lisp." start="00:00:23.521" video="mainVideo" id=subtitle]]
[[!template text="Well, Emacs Lisp is a programming language," start="00:00:26.635" video="mainVideo" id=subtitle]]
[[!template text="it's indeed a Lisp," start="00:00:30.278" video="mainVideo" id=subtitle]]
[[!template text="and one year ago I looked for some statistics," start="00:00:33.647" video="mainVideo" id=subtitle]]
[[!template text="and I was kind of pleased to see" start="00:00:36.722" video="mainVideo" id=subtitle]]
[[!template text="-- surprised to see actually --" start="00:00:38.842" video="mainVideo" id=subtitle]]
[[!template text="that it's still kind of popular" start="00:00:40.132" video="mainVideo" id=subtitle]]
[[!template text="as a programming language." start="00:00:41.073" video="mainVideo" id=subtitle]]
[[!template text="It doesn't rank that bad" start="00:00:42.944" video="mainVideo" id=subtitle]]
[[!template text="against other programming languages." start="00:00:43.785" video="mainVideo" id=subtitle]]
[[!template text="Also, the other important fact about Emacs Lisp" start="00:00:45.658" video="mainVideo" id=subtitle]]
[[!template text="is that there is a lot of Emacs Lisp out there," start="00:00:48.487" video="mainVideo" id=subtitle]]
[[!template text="and this will have an impact on this project." start="00:00:51.622" video="mainVideo" id=subtitle]]
[[!template text="It's a programming language that is capable" start="00:00:56.025" video="mainVideo" id=subtitle]]
[[!template text="of almost any task, so it's" start="00:00:57.734" video="mainVideo" id=subtitle]]
[[!template text="almost a general-purpose programming language," start="00:00:59.420" video="mainVideo" id=subtitle]]
[[!template text="and this reflects on Emacs itself," start="00:01:01.831" video="mainVideo" id=subtitle]]
[[!template text="that it's capable of almost any task." start="00:01:04.217" video="mainVideo" id=subtitle]]
[[!template text="Indeed this " ;almost" ; is something we want to fix," start="00:01:07.335" video="mainVideo" id=subtitle]]
[[!template text="because we want to do everything," start="00:01:09.781" video="mainVideo" id=subtitle]]
[[!template text="we want to do all of our computing with Emacs." start="00:01:10.820" video="mainVideo" id=subtitle]]
[[!template text="Also, an interesting aspect for me" start="00:01:14.321" video="mainVideo" id=subtitle]]
[[!template text="is that it's not specified by any standard." start="00:01:16.625" video="mainVideo" id=subtitle]]
[[!template text="This implies it can evolve in a more agile way" start="00:01:19.005" video="mainVideo" id=subtitle]]
[[!template text="without having to change the standard, etc." start="00:01:22.609" video="mainVideo" id=subtitle]]
[[!template text="And, in fact, it's kind of improving," start="00:01:25.770" video="mainVideo" id=subtitle]]
[[!template text="I believe relatively fast." start="00:01:27.970" video="mainVideo" id=subtitle]]
[[!template text="A little bit about Lisp in general." start="00:01:30.473" video="mainVideo" id=subtitle]]
[[!template text="First, it's the best programming language ever," start="00:01:32.892" video="mainVideo" id=subtitle]]
[[!template text="we all know it." start="00:01:35.052" video="mainVideo" id=subtitle]]
[[!template text="It has a lot of very nice properties," start="00:01:35.813" video="mainVideo" id=subtitle]]
[[!template text="like it's dynamic, it's homoiconic, etc." start="00:01:37.934" video="mainVideo" id=subtitle]]
[[!template text="But the interesting thing for implementors," start="00:01:40.462" video="mainVideo" id=subtitle]]
[[!template text="is that, in theory," start="00:01:42.496" video="mainVideo" id=subtitle]]
[[!template text="it can be implemented with very few primitives." start="00:01:43.576" video="mainVideo" id=subtitle]]
[[!template text="You build very few primitives that are like magic," start="00:01:46.817" video="mainVideo" id=subtitle]]
[[!template text="and on top of this," start="00:01:49.590" video="mainVideo" id=subtitle]]
[[!template text="you can implement the whole language." start="00:01:51.938" video="mainVideo" id=subtitle]]
[[!template text="This sounds very nice," start="00:01:53.873" video="mainVideo" id=subtitle]]
[[!template text="and very appealing for implementors" start="00:01:55.860" video="mainVideo" id=subtitle]]
[[!template text="meaning to implement a new Lisp implementation," start="00:01:57.381" video="mainVideo" id=subtitle]]
[[!template text="or improving or modifying an existing one." start="00:02:00.279" video="mainVideo" id=subtitle]]
[[!template text="So the question is:" start="00:02:02.641" video="mainVideo" id=subtitle]]
[[!template text="how many primitives do we have to implement" start="00:02:04.263" video="mainVideo" id=subtitle]]
[[!template text="if we want to change" start="00:02:07.764" video="mainVideo" id=subtitle]]
[[!template text="the GNU Emacs Lisp implementation?" start="00:02:09.485" video="mainVideo" id=subtitle]]
[[!template text="Unfortunately, not really as few as we would like." start="00:02:13.308" video="mainVideo" id=subtitle]]
[[!template text="In GNU Emacs, we have about 1500 primitives," start="00:02:20.234" video="mainVideo" id=subtitle]]
[[!template text="and the main reason for that is performance." start="00:02:25.400" video="mainVideo" id=subtitle]]
[[!template text="Actually, GNU Emacs was written" start="00:02:28.071" video="mainVideo" id=subtitle]]
[[!template text="when performance was a big issue," start="00:02:31.064" video="mainVideo" id=subtitle]]
[[!template text="and nowadays certain parts" start="00:02:34.393" video="mainVideo" id=subtitle]]
[[!template text="are still performance-critical." start="00:02:36.594" video="mainVideo" id=subtitle]]
[[!template text="We have a lot of C code;" start="00:02:38.591" video="mainVideo" id=subtitle]]
[[!template text="30% of the GNU Emacs codebase is C code," start="00:02:40.395" video="mainVideo" id=subtitle]]
[[!template text="and we have to maintain this." start="00:02:46.435" video="mainVideo" id=subtitle]]
[[!template text="But not only that, this is the main barrier" start="00:02:49.279" video="mainVideo" id=subtitle]]
[[!template text="for people that tried in the past" start="00:02:52.959" video="mainVideo" id=subtitle]]
[[!template text="to change the Emacs Lisp implementation." start="00:02:55.681" video="mainVideo" id=subtitle]]
[[!template text="Because not only do you have to" start="00:02:57.765" video="mainVideo" id=subtitle]]
[[!template text="replace these primitives," start="00:02:59.202" video="mainVideo" id=subtitle]]
[[!template text="all of them or part of them," start="00:03:01.683" video="mainVideo" id=subtitle]]
[[!template text="but sometimes they also share (these primitives)," start="00:03:04.012" video="mainVideo" id=subtitle]]
[[!template text="internal data structures." start="00:03:06.484" video="mainVideo" id=subtitle]]
[[!template text="For instance, it's very difficult to say:" start="00:03:08.055" video="mainVideo" id=subtitle]]
[[!template text="Now I want to go from C" start="00:03:09.846" video="mainVideo" id=subtitle]]
[[!template text="to a different programming language" start="00:03:12.926" video="mainVideo" id=subtitle]]
[[!template text="for implementing these primitives." start="00:03:14.006" video="mainVideo" id=subtitle]]
[[!template text="So this has been, effectively," start="00:03:15.740" video="mainVideo" id=subtitle]]
[[!template text="the main barrier for doing this work." start="00:03:17.128" video="mainVideo" id=subtitle]]
[[!template text="Another interesting aspect" start="00:03:20.486" video="mainVideo" id=subtitle]]
[[!template text="about the GNU Emacs implementation" start="00:03:22.190" video="mainVideo" id=subtitle]]
[[!template text="is that Lisp can run interpreted" start="00:03:23.970" video="mainVideo" id=subtitle]]
[[!template text="or byte-compiled for performance," start="00:03:26.291" video="mainVideo" id=subtitle]]
[[!template text="and the byte compiler itself" start="00:03:28.752" video="mainVideo" id=subtitle]]
[[!template text="is written in Emacs Lisp." start="00:03:30.733" video="mainVideo" id=subtitle]]
[[!template text="This implies that GNU Emacs has to go through" start="00:03:32.773" video="mainVideo" id=subtitle]]
[[!template text="a bootstrap procedure in order to be built." start="00:03:35.134" video="mainVideo" id=subtitle]]
[[!template text="But it's kind of interesting" start="00:03:37.319" video="mainVideo" id=subtitle]]
[[!template text="for something that started as a text editor," start="00:03:38.815" video="mainVideo" id=subtitle]]
[[!template text="or something like it." start="00:03:41.937" video="mainVideo" id=subtitle]]
[[!template text="The byte-code that is Emacs Lisp" start="00:03:43.203" video="mainVideo" id=subtitle]]
[[!template text="when it's been byte compiled," start="00:03:47.979" video="mainVideo" id=subtitle]]
[[!template text="it's running on a stack-based virtual machine" start="00:03:50.379" video="mainVideo" id=subtitle]]
[[!template text="that is implemented in C." start="00:03:53.180" video="mainVideo" id=subtitle]]
[[!template text="OK, so I've listed a bunch of areas" start="00:03:55.253" video="mainVideo" id=subtitle]]
[[!template text="where Emacs Lisp could improve:" start="00:03:59.842" video="mainVideo" id=subtitle]]
[[!template text="Namespace, Extensibility, Performance," start="00:04:02.178" video="mainVideo" id=subtitle]]
[[!template text="and Debuggability, and Diagnostic, we could say." start="00:04:07.025" video="mainVideo" id=subtitle]]
[[!template text="This activity, this project in particular," start="00:04:11.346" video="mainVideo" id=subtitle]]
[[!template text="is affecting primarily Performance" start="00:04:14.428" video="mainVideo" id=subtitle]]
[[!template text="and the performance area the Execution Engine." start="00:04:17.189" video="mainVideo" id=subtitle]]
[[!template text="That said," start="00:04:21.414" video="mainVideo" id=subtitle]]
[[!template text="I think it has an impact also on Extensibility," start="00:04:22.671" video="mainVideo" id=subtitle]]
[[!template text="and I hope it will have an impact also" start="00:04:25.957" video="mainVideo" id=subtitle]]
[[!template text="on programming diagnostics," start="00:04:27.913" video="mainVideo" id=subtitle]]
[[!template text="so giving better warnings, unknown." start="00:04:29.881" video="mainVideo" id=subtitle]]
[[!template text="So which are the benefits" start="00:04:32.604" video="mainVideo" id=subtitle]]
[[!template text="of increasing the Emacs Lisp performance?" start="00:04:36.795" video="mainVideo" id=subtitle]]
[[!template text="Indeed, we will have, if we do that," start="00:04:39.316" video="mainVideo" id=subtitle]]
[[!template text="programs that run faster." start="00:04:43.078" video="mainVideo" id=subtitle]]
[[!template text="But the main implication of that" start="00:04:45.774" video="mainVideo" id=subtitle]]
[[!template text="is that we could write less C;" start="00:04:48.840" video="mainVideo" id=subtitle]]
[[!template text="we could maintain and debug less C." start="00:04:50.860" video="mainVideo" id=subtitle]]
[[!template text="That is kind of a time-consuming task." start="00:04:53.041" video="mainVideo" id=subtitle]]
[[!template text="And we could also allow for" start="00:04:56.542" video="mainVideo" id=subtitle]]
[[!template text="writing performance-critical extensions" start="00:04:59.603" video="mainVideo" id=subtitle]]
[[!template text="directly in Lisp" start="00:05:01.824" video="mainVideo" id=subtitle]]
[[!template text="without having to use systems like" start="00:05:03.909" video="mainVideo" id=subtitle]]
[[!template text="I think there's a bunch." start="00:05:06.406" video="mainVideo" id=subtitle]]
[[!template text="These are very consistent benefits." start="00:05:09.927" video="mainVideo" id=subtitle]]
[[!template text="OK, Project Goals," start="00:05:14.899" video="mainVideo" id=subtitle]]
[[!template text="but I think the title of this slide" start="00:05:16.769" video="mainVideo" id=subtitle]]
[[!template text="maybe should be Project Requirements." start="00:05:18.930" video="mainVideo" id=subtitle]]
[[!template text="So when I started this activity," start="00:05:21.623" video="mainVideo" id=subtitle]]
[[!template text="I set some requirements for the project," start="00:05:23.331" video="mainVideo" id=subtitle]]
[[!template text="with the main goal in mind: to go upstream." start="00:05:27.049" video="mainVideo" id=subtitle]]
[[!template text="So I wanted to create something," start="00:05:29.570" video="mainVideo" id=subtitle]]
[[!template text="a modified implementation of GNU Emacs," start="00:05:31.594" video="mainVideo" id=subtitle]]
[[!template text="that was compatible" start="00:05:34.353" video="mainVideo" id=subtitle]]
[[!template text="as close to 100% as possible" start="00:05:36.776" video="mainVideo" id=subtitle]]
[[!template text="to the current implementation." start="00:05:38.696" video="mainVideo" id=subtitle]]
[[!template text="And when I say " ;current implementation," ;" start="00:05:40.620" video="mainVideo" id=subtitle]]
[[!template text="I don't refer to what" start="00:05:42.820" video="mainVideo" id=subtitle]]
[[!template text="the Emacs Lisp Programming Manual specifies" start="00:05:44.859" video="mainVideo" id=subtitle]]
[[!template text="as expected behavior of the implementation," start="00:05:46.980" video="mainVideo" id=subtitle]]
[[!template text="but really the implementation itself." start="00:05:49.907" video="mainVideo" id=subtitle]]
[[!template text="This is because there are a lot of corner cases" start="00:05:52.309" video="mainVideo" id=subtitle]]
[[!template text="that are not specified by the manual," start="00:05:53.942" video="mainVideo" id=subtitle]]
[[!template text="but programs do rely on that," start="00:05:56.493" video="mainVideo" id=subtitle]]
[[!template text="and given there is a ton of Emacs Lisp" start="00:05:58.193" video="mainVideo" id=subtitle]]
[[!template text="already around," start="00:06:00.625" video="mainVideo" id=subtitle]]
[[!template text="compatibility was definitely a major requirement." start="00:06:02.037" video="mainVideo" id=subtitle]]
[[!template text="I wanted to produce something that had" start="00:06:05.541" video="mainVideo" id=subtitle]]
[[!template text="reduced impact on the Emacs codebase," start="00:06:07.827" video="mainVideo" id=subtitle]]
[[!template text="at least as much as possible." start="00:06:09.687" video="mainVideo" id=subtitle]]
[[!template text="So I didn't want to rewrite all of GNU Emacs." start="00:06:11.644" video="mainVideo" id=subtitle]]
[[!template text="Indeed, because it would have been" start="00:06:14.827" video="mainVideo" id=subtitle]]
[[!template text="too much work for one single person," start="00:06:17.710" video="mainVideo" id=subtitle]]
[[!template text="but also still thinking" start="00:06:21.171" video="mainVideo" id=subtitle]]
[[!template text="to an upstream outcome all of this time." start="00:06:25.473" video="mainVideo" id=subtitle]]
[[!template text="Another requirement was to have" start="00:06:29.258" video="mainVideo" id=subtitle]]
[[!template text="no, or very reduced, impact on the user," start="00:06:32.076" video="mainVideo" id=subtitle]]
[[!template text="so I didn't want to change" start="00:06:34.804" video="mainVideo" id=subtitle]]
[[!template text="the way Emacs is used by you." start="00:06:36.556" video="mainVideo" id=subtitle]]
[[!template text="And last but not least," start="00:06:38.708" video="mainVideo" id=subtitle]]
[[!template text="introducing new dependencies," start="00:06:40.718" video="mainVideo" id=subtitle]]
[[!template text="those dependencies had to be Free Software," start="00:06:42.951" video="mainVideo" id=subtitle]]
[[!template text="possibly GPL," start="00:06:45.479" video="mainVideo" id=subtitle]]
[[!template text="and also an important requirement" start="00:06:47.295" video="mainVideo" id=subtitle]]
[[!template text="is that these dependencies had to be" start="00:06:50.097" video="mainVideo" id=subtitle]]
[[!template text="some kind of trusted software that we know" start="00:06:53.043" video="mainVideo" id=subtitle]]
[[!template text="is going to be maintained in the future." start="00:06:55.243" video="mainVideo" id=subtitle]]
[[!template text="Given Emacs has been around since forever," start="00:06:56.781" video="mainVideo" id=subtitle]]
[[!template text="and will be around forever and ever," start="00:06:59.363" video="mainVideo" id=subtitle]]
[[!template text="this was another very important point." start="00:07:01.285" video="mainVideo" id=subtitle]]
[[!template text="A little bit of history of this project/" start="00:07:05.367" video="mainVideo" id=subtitle]]
[[!template text="a quick timeline." start="00:07:08.808" video="mainVideo" id=subtitle]]
[[!template text="2019, in May, I did my first commit," start="00:07:10.748" video="mainVideo" id=subtitle]]
[[!template text="I think it was when I tried to write" start="00:07:14.490" video="mainVideo" id=subtitle]]
[[!template text="my first primitive function ever in C," start="00:07:17.210" video="mainVideo" id=subtitle]]
[[!template text="in GNU Emacs." start="00:07:20.332" video="mainVideo" id=subtitle]]
[[!template text="And this was an attempt to try to compile" start="00:07:21.852" video="mainVideo" id=subtitle]]
[[!template text="a function that, once executed, returning ?" start="00:07:24.934" video="mainVideo" id=subtitle]]
[[!template text="That was it. Six months after (about)," start="00:07:30.415" video="mainVideo" id=subtitle]]
[[!template text="I had something that was kind of working," start="00:07:33.857" video="mainVideo" id=subtitle]]
[[!template text="So I was able to start up a semi-standard Emacs" start="00:07:37.157" video="mainVideo" id=subtitle]]
[[!template text="and then to compile and load," start="00:07:42.899" video="mainVideo" id=subtitle]]
[[!template text="and replacing most of the functions" start="00:07:44.981" video="mainVideo" id=subtitle]]
[[!template text="that I had defined floating in my Lisp universe." start="00:07:47.321" video="mainVideo" id=subtitle]]
[[!template text="Those functions are the functions" start="00:07:51.022" video="mainVideo" id=subtitle]]
[[!template text="that are essentially composing Emacs," start="00:07:54.304" video="mainVideo" id=subtitle]]
[[!template text="at least the Lisp side of Emacs." start="00:07:56.044" video="mainVideo" id=subtitle]]
[[!template text="A lot of features were missing," start="00:07:57.984" video="mainVideo" id=subtitle]]
[[!template text="like I had no Garbage Collector support," start="00:08:01.186" video="mainVideo" id=subtitle]]
[[!template text="no bootstrap, I was not optimizing these functions" start="00:08:03.526" video="mainVideo" id=subtitle]]
[[!template text="Because optimization would be broken." start="00:08:07.108" video="mainVideo" id=subtitle]]
[[!template text="No image dump support, etc." start="00:08:10.169" video="mainVideo" id=subtitle]]
[[!template text="But I think this proved the design could work." start="00:08:12.850" video="mainVideo" id=subtitle]]
[[!template text="So I sent to email to emacs-devel. I said" start="00:08:16.431" video="mainVideo" id=subtitle]]
[[!template text="I have this stuff I'm working on," start="00:08:19.272" video="mainVideo" id=subtitle]]
[[!template text="and I wanted some feedback from the upstream" start="00:08:20.953" video="mainVideo" id=subtitle]]
[[!template text="to see if there was some interest." start="00:08:24.714" video="mainVideo" id=subtitle]]
[[!template text="I believe the outcome of this was positive" start="00:08:27.635" video="mainVideo" id=subtitle]]
[[!template text="because about one month after," start="00:08:30.557" video="mainVideo" id=subtitle]]
[[!template text="I pushed my branch within the Emacs git" start="00:08:32.937" video="mainVideo" id=subtitle]]
[[!template text="as a feature branch, and shortly after," start="00:08:35.798" video="mainVideo" id=subtitle]]
[[!template text="we started to use the bug tracker to track bugs." start="00:08:38.539" video="mainVideo" id=subtitle]]
[[!template text="So essentially we moved the development" start="00:08:42.779" video="mainVideo" id=subtitle]]
[[!template text="on the upstream infrastructure." start="00:08:45.580" video="mainVideo" id=subtitle]]
[[!template text="I believe two years after the first commit," start="00:08:50.140" video="mainVideo" id=subtitle]]
[[!template text="the project was merged" start="00:08:55.721" video="mainVideo" id=subtitle]]
[[!template text="after literally hundreds of bugs solved," start="00:08:57.882" video="mainVideo" id=subtitle]]
[[!template text="and improvements, suggestions unknown" start="00:09:01.022" video="mainVideo" id=subtitle]]
[[!template text="and this was about six months ago." start="00:09:03.943" video="mainVideo" id=subtitle]]
[[!template text="Before discussing how the native compiler works," start="00:09:08.723" video="mainVideo" id=subtitle]]
[[!template text="I think it's worth looking at" start="00:09:12.464" video="mainVideo" id=subtitle]]
[[!template text="how Lisp is implemented in GNU Emacs." start="00:09:14.324" video="mainVideo" id=subtitle]]
[[!template text="We have Lisp_Objects" start="00:09:17.745" video="mainVideo" id=subtitle]]
[[!template text="floating around our Lisp universe," start="00:09:19.405" video="mainVideo" id=subtitle]]
[[!template text="and they are internally represented in this way." start="00:09:22.045" video="mainVideo" id=subtitle]]
[[!template text="We have what is called a tagged pointer," start="00:09:24.006" video="mainVideo" id=subtitle]]
[[!template text="that is just a regular pointer" start="00:09:25.706" video="mainVideo" id=subtitle]]
[[!template text="that is pointing to the area of memory" start="00:09:27.506" video="mainVideo" id=subtitle]]
[[!template text="where we hold the real data of the object." start="00:09:29.386" video="mainVideo" id=subtitle]]
[[!template text="But within this tagged pointer," start="00:09:32.027" video="mainVideo" id=subtitle]]
[[!template text="we reserve a few bits" start="00:09:34.287" video="mainVideo" id=subtitle]]
[[!template text="to indicate the type of object we are pointing to." start="00:09:36.848" video="mainVideo" id=subtitle]]
[[!template text="This is important because" start="00:09:39.068" video="mainVideo" id=subtitle]]
[[!template text="each time we access an object," start="00:09:40.628" video="mainVideo" id=subtitle]]
[[!template text="we have to typically check those bits" start="00:09:42.388" video="mainVideo" id=subtitle]]
[[!template text="to check that the object we are manipulating" start="00:09:46.269" video="mainVideo" id=subtitle]]
[[!template text="is of the right kind," start="00:09:49.129" video="mainVideo" id=subtitle]]
[[!template text="remove those bits, and, if we are happy," start="00:09:50.630" video="mainVideo" id=subtitle]]
[[!template text="access the object, otherwise unknown." start="00:09:52.630" video="mainVideo" id=subtitle]]
[[!template text="All the objects are like this," start="00:09:55.910" video="mainVideo" id=subtitle]]
[[!template text="except for typically Fixnums," start="00:09:57.791" video="mainVideo" id=subtitle]]
[[!template text="that are small integers" start="00:09:59.831" video="mainVideo" id=subtitle]]
[[!template text="that we manage to fit directly within the pointer." start="00:10:01.131" video="mainVideo" id=subtitle]]
[[!template text="Also for manipulating Fixnums," start="00:10:04.992" video="mainVideo" id=subtitle]]
[[!template text="we have to check the tag bits each time." start="00:10:07.392" video="mainVideo" id=subtitle]]
[[!template text="Whenever we are not sure of the type of object" start="00:10:09.513" video="mainVideo" id=subtitle]]
[[!template text="we are manipulating (read: almost every time)," start="00:10:13.273" video="mainVideo" id=subtitle]]
[[!template text="we have to check those bits and remove those bits" start="00:10:16.594" video="mainVideo" id=subtitle]]
[[!template text="before doing any manipulation on the Fixnum." start="00:10:19.014" video="mainVideo" id=subtitle]]
[[!template text="How Emacs Lisp is byte-compiled and executed" start="00:10:21.854" video="mainVideo" id=subtitle]]
[[!template text="in, let's call it, " ;Vanilla" ;." start="00:10:26.115" video="mainVideo" id=subtitle]]
[[!template text="If we have a Lisp expression of this kind:" start="00:10:27.875" video="mainVideo" id=subtitle]]
[[!template text="We take the variable 'a' we do plus 2," start="00:10:30.156" video="mainVideo" id=subtitle]]
[[!template text="and then we multiply the result by 3," start="00:10:32.876" video="mainVideo" id=subtitle]]
[[!template text="the byte compiler will produce this LAP code." start="00:10:34.976" video="mainVideo" id=subtitle]]
[[!template text="LAP code is essentially" start="00:10:37.477" video="mainVideo" id=subtitle]]
[[!template text="the assembly for the byte-code," start="00:10:38.597" video="mainVideo" id=subtitle]]
[[!template text="so it's the " ;intermediate representation" ;" start="00:10:40.117" video="mainVideo" id=subtitle]]
[[!template text="that will assembled into byte-code. (.elc files)" start="00:10:43.798" video="mainVideo" id=subtitle]]
[[!template text="How is this program executed?" start="00:10:48.558" video="mainVideo" id=subtitle]]
[[!template text="As I mentioned, it's executed in a virtual machine" start="00:10:50.839" video="mainVideo" id=subtitle]]
[[!template text="that is stack-based," start="00:10:53.739" video="mainVideo" id=subtitle]]
[[!template text="but we start with an execution stack that's empty," start="00:10:55.800" video="mainVideo" id=subtitle]]
[[!template text="and a stack pointer pointing to its bottom." start="00:10:58.640" video="mainVideo" id=subtitle]]
[[!template text="And we execute the first instruction," start="00:11:01.780" video="mainVideo" id=subtitle]]
[[!template text="that is pushing in the stack the value of 'a'," start="00:11:04.380" video="mainVideo" id=subtitle]]
[[!template text="in this case, 100. Then we push the constant 2." start="00:11:07.220" video="mainVideo" id=subtitle]]
[[!template text="Then we do the summation," start="00:11:10.460" video="mainVideo" id=subtitle]]
[[!template text="and we have the result in the stack." start="00:11:12.500" video="mainVideo" id=subtitle]]
[[!template text="Same: we push the constant 3," start="00:11:14.140" video="mainVideo" id=subtitle]]
[[!template text="we do the multiplication," start="00:11:16.960" video="mainVideo" id=subtitle]]
[[!template text="and we will be able to return." start="00:11:17.720" video="mainVideo" id=subtitle]]
[[!template text="Now, what's good and what's bad about this?" start="00:11:19.360" video="mainVideo" id=subtitle]]
[[!template text="A good thing is that it's very simple" start="00:11:22.800" video="mainVideo" id=subtitle]]
[[!template text="to start from Lisp" start="00:11:25.900" video="mainVideo" id=subtitle]]
[[!template text="and compile this kind of LAP output." start="00:11:27.560" video="mainVideo" id=subtitle]]
[[!template text="At least it's reasonably simple." start="00:11:31.420" video="mainVideo" id=subtitle]]
[[!template text="The compiler is not that complex." start="00:11:34.340" video="mainVideo" id=subtitle]]
[[!template text="The bad thing is that all this machinery" start="00:11:36.340" video="mainVideo" id=subtitle]]
[[!template text="-- push and pop, etc. --" start="00:11:39.100" video="mainVideo" id=subtitle]]
[[!template text="it's very different from how a modern CPU works." start="00:11:40.760" video="mainVideo" id=subtitle]]
[[!template text="Because modern CPUs," start="00:11:44.420" video="mainVideo" id=subtitle]]
[[!template text="they are not stack-based anymore" start="00:11:45.820" video="mainVideo" id=subtitle]]
[[!template text="but they have instead a fixed number of registers," start="00:11:47.760" video="mainVideo" id=subtitle]]
[[!template text="and they work with assignment and operation" start="00:11:51.000" video="mainVideo" id=subtitle]]
[[!template text="within these registers" start="00:11:54.120" video="mainVideo" id=subtitle]]
[[!template text="that are generally called " ;general-purpose." ;" start="00:11:55.140" video="mainVideo" id=subtitle]]
[[!template text="So to execute this LAP program," start="00:11:57.140" video="mainVideo" id=subtitle]]
[[!template text="there is another program," start="00:11:59.360" video="mainVideo" id=subtitle]]
[[!template text="that is the implementation of the VM itself" start="00:12:00.860" video="mainVideo" id=subtitle]]
[[!template text="that is doing conversion during runtime." start="00:12:02.700" video="mainVideo" id=subtitle]]
[[!template text="So it's interpreting the LAP program" start="00:12:06.500" video="mainVideo" id=subtitle]]
[[!template text="and it's converting it into instructions" start="00:12:08.420" video="mainVideo" id=subtitle]]
[[!template text="that we can execute on the CPU." start="00:12:11.460" video="mainVideo" id=subtitle]]
[[!template text="This conversion is done each time" start="00:12:13.280" video="mainVideo" id=subtitle]]
[[!template text="we will run some byte-code." start="00:12:15.040" video="mainVideo" id=subtitle]]
[[!template text="And it's something that we want to avoid." start="00:12:17.200" video="mainVideo" id=subtitle]]
[[!template text="Instead of this live conversion," start="00:12:19.860" video="mainVideo" id=subtitle]]
[[!template text="we want to convert once:" start="00:12:21.660" video="mainVideo" id=subtitle]]
[[!template text="our Lisp program into native code," start="00:12:26.320" video="mainVideo" id=subtitle]]
[[!template text="that is, a binary program that can be executed" start="00:12:29.020" video="mainVideo" id=subtitle]]
[[!template text="directly by our CPU." start="00:12:32.480" video="mainVideo" id=subtitle]]
[[!template text="We want to save all this unnecessary conversion" start="00:12:34.500" video="mainVideo" id=subtitle]]
[[!template text="that we do each time we are running a program" start="00:12:36.780" video="mainVideo" id=subtitle]]
[[!template text="while we are running it." start="00:12:39.100" video="mainVideo" id=subtitle]]
[[!template text="And we want to do this process just once," start="00:12:39.860" video="mainVideo" id=subtitle]]
[[!template text="when we are compiling." start="00:12:42.160" video="mainVideo" id=subtitle]]
[[!template text="That's the main goal of this activity." start="00:12:43.840" video="mainVideo" id=subtitle]]
[[!template text="How is the byte compiler implemented?" start="00:12:46.340" video="mainVideo" id=subtitle]]
[[!template text="As any compiler it's a pipeline of transformations." start="00:12:50.300" video="mainVideo" id=subtitle]]
[[!template text="We go through macro expansion, closure conversion," start="00:12:54.040" video="mainVideo" id=subtitle]]
[[!template text="we have a bunch of source level optimization." start="00:12:58.660" video="mainVideo" id=subtitle]]
[[!template text="Then we go into LAP," start="00:13:02.220" video="mainVideo" id=subtitle]]
[[!template text="that's the transformation we are interested in," start="00:13:04.040" video="mainVideo" id=subtitle]]
[[!template text="and after a few optimizations on LAP," start="00:13:06.900" video="mainVideo" id=subtitle]]
[[!template text="LAP is assembled into byte-code." start="00:13:10.700" video="mainVideo" id=subtitle]]
[[!template text="So if we list it" start="00:13:14.240" video="mainVideo" id=subtitle]]
[[!template text="in terms of intermediate representations," start="00:13:16.980" video="mainVideo" id=subtitle]]
[[!template text="we can simplify this pipeline like this." start="00:13:19.420" video="mainVideo" id=subtitle]]
[[!template text="We start with Lisp, and at a certain point" start="00:13:23.700" video="mainVideo" id=subtitle]]
[[!template text="we are manipulating the program in LAP form," start="00:13:26.200" video="mainVideo" id=subtitle]]
[[!template text="and then at the end we produce the byte-code" start="00:13:29.620" video="mainVideo" id=subtitle]]
[[!template text="that is the .elc file that you run" start="00:13:32.080" video="mainVideo" id=subtitle]]
[[!template text="What I wanted to realize was something like this." start="00:13:34.660" video="mainVideo" id=subtitle]]
[[!template text="I wanted to start from LAP, do something," start="00:13:37.760" video="mainVideo" id=subtitle]]
[[!template text="and jump into GCC using libgccjit" start="00:13:41.300" video="mainVideo" id=subtitle]]
[[!template text="and in particular" start="00:13:44.700" video="mainVideo" id=subtitle]]
[[!template text="the libgccjit Intermediate Representation" start="00:13:45.660" video="mainVideo" id=subtitle]]
[[!template text="that we will discuss." start="00:13:48.100" video="mainVideo" id=subtitle]]
[[!template text="Now, why I wanted to do something like this?" start="00:13:50.440" video="mainVideo" id=subtitle]]
[[!template text="Essentially, writing a compiler from scratch" start="00:13:53.220" video="mainVideo" id=subtitle]]
[[!template text="for Emacs Lisp would have been a very big task." start="00:13:57.400" video="mainVideo" id=subtitle]]
[[!template text="So I wanted to rely on, as much as I could," start="00:14:01.620" video="mainVideo" id=subtitle]]
[[!template text="the Emacs Lisp byte compiler," start="00:14:05.220" video="mainVideo" id=subtitle]]
[[!template text="because I had to produce something" start="00:14:07.280" video="mainVideo" id=subtitle]]
[[!template text="that was as compatible as possible" start="00:14:10.380" video="mainVideo" id=subtitle]]
[[!template text="to the current implementation." start="00:14:13.020" video="mainVideo" id=subtitle]]
[[!template text="So this was (I believe) a very good idea" start="00:14:14.340" video="mainVideo" id=subtitle]]
[[!template text="to save an enormous quantity of work" start="00:14:18.440" video="mainVideo" id=subtitle]]
[[!template text="and to produce something" start="00:14:20.620" video="mainVideo" id=subtitle]]
[[!template text="that was compatible in terms of semantics" start="00:14:22.440" video="mainVideo" id=subtitle]]
[[!template text="with the original implementation." start="00:14:25.040" video="mainVideo" id=subtitle]]
[[!template text="Also, I didn't want to implement a code generator" start="00:14:26.920" video="mainVideo" id=subtitle]]
[[!template text="for each architecture we were targeting," start="00:14:30.760" video="mainVideo" id=subtitle]]
[[!template text="nor wanted to implement all these optimizations" start="00:14:32.960" video="mainVideo" id=subtitle]]
[[!template text="that are already in GCC," start="00:14:36.000" video="mainVideo" id=subtitle]]
[[!template text="so I thought it was a good idea" start="00:14:37.940" video="mainVideo" id=subtitle]]
[[!template text="to rely on an existing compiler like GCC ?." start="00:14:40.220" video="mainVideo" id=subtitle]]
[[!template text="Let's talk about libgccjit." start="00:14:44.820" video="mainVideo" id=subtitle]]
[[!template text="It was added by David Malcolm in GCC 5" start="00:14:47.340" video="mainVideo" id=subtitle]]
[[!template text="It allows you to describe a C-ish semantic to GCC" start="00:14:50.280" video="mainVideo" id=subtitle]]
[[!template text="and have it compile." start="00:14:55.780" video="mainVideo" id=subtitle]]
[[!template text="It's good for reading Jitters or AoT compilers." start="00:14:57.480" video="mainVideo" id=subtitle]]
[[!template text="And if we talk about GCC:" start="00:15:01.500" video="mainVideo" id=subtitle]]
[[!template text="it's a compiler, it's a very good one," start="00:15:04.500" video="mainVideo" id=subtitle]]
[[!template text="it has support for a remarkable number" start="00:15:07.280" video="mainVideo" id=subtitle]]
[[!template text="of target architectures." start="00:15:09.440" video="mainVideo" id=subtitle]]
[[!template text="It's also very good at generating fast code," start="00:15:11.740" video="mainVideo" id=subtitle]]
[[!template text="and it's been around for a long time;" start="00:15:15.180" video="mainVideo" id=subtitle]]
[[!template text="I believe it's like 1 year younger than Emacs." start="00:15:18.080" video="mainVideo" id=subtitle]]
[[!template text="It's still very well maintained," start="00:15:21.300" video="mainVideo" id=subtitle]]
[[!template text="and we can assume it will be maintained" start="00:15:23.320" video="mainVideo" id=subtitle]]
[[!template text="for quite a while. And, as I mentioned," start="00:15:25.940" video="mainVideo" id=subtitle]]
[[!template text="this was a very important point." start="00:15:29.020" video="mainVideo" id=subtitle]]
[[!template text="Also, it's GPL; it's Free Software," start="00:15:31.220" video="mainVideo" id=subtitle]]
[[!template text="and it's developed under the GNU umbrella," start="00:15:33.960" video="mainVideo" id=subtitle]]
[[!template text="so I thought it was a very good option." start="00:15:36.220" video="mainVideo" id=subtitle]]
[[!template text="So we can imagine a simple translation" start="00:15:40.460" video="mainVideo" id=subtitle]]
[[!template text="that goes from LAP to this subset of C" start="00:15:43.400" video="mainVideo" id=subtitle]]
[[!template text="that we can describe to libgccjit." start="00:15:46.500" video="mainVideo" id=subtitle]]
[[!template text="This simple translation we can see here," start="00:15:48.980" video="mainVideo" id=subtitle]]
[[!template text="it's actually pretty trivial." start="00:15:52.980" video="mainVideo" id=subtitle]]
[[!template text="Instead of doing operations" start="00:15:55.780" video="mainVideo" id=subtitle]]
[[!template text="within the execution stack we have seen before," start="00:15:58.600" video="mainVideo" id=subtitle]]
[[!template text="we have just an array that is replacing it," start="00:16:02.160" video="mainVideo" id=subtitle]]
[[!template text="and it's called 'local' in this case," start="00:16:05.180" video="mainVideo" id=subtitle]]
[[!template text="and we have assignments within this array," start="00:16:07.480" video="mainVideo" id=subtitle]]
[[!template text="so that they are done in place of the original" start="00:16:12.460" video="mainVideo" id=subtitle]]
[[!template text="push and pop activity of the stack." start="00:16:15.240" video="mainVideo" id=subtitle]]
[[!template text="The nice thing is that," start="00:16:17.200" video="mainVideo" id=subtitle]]
[[!template text="when you have done this translation," start="00:16:18.380" video="mainVideo" id=subtitle]]
[[!template text="GCC will be able to optimize this," start="00:16:20.360" video="mainVideo" id=subtitle]]
[[!template text="and remove all the unnecessary operations," start="00:16:23.260" video="mainVideo" id=subtitle]]
[[!template text="and generate code" start="00:16:25.940" video="mainVideo" id=subtitle]]
[[!template text="for the specific CPU you are targeting," start="00:16:27.280" video="mainVideo" id=subtitle]]
[[!template text="which will be running your code." start="00:16:29.760" video="mainVideo" id=subtitle]]
[[!template text="This sounds great; it sounds like" start="00:16:32.120" video="mainVideo" id=subtitle]]
[[!template text="a very simple and effective translation," start="00:16:35.020" video="mainVideo" id=subtitle]]
[[!template text="and in fact the first iteration of my compiler" start="00:16:37.540" video="mainVideo" id=subtitle]]
[[!template text="was doing just this." start="00:16:40.020" video="mainVideo" id=subtitle]]
[[!template text="It was essentially a big C function" start="00:16:41.220" video="mainVideo" id=subtitle]]
[[!template text="that was taking LAP and doing this conversion" start="00:16:45.420" video="mainVideo" id=subtitle]]
[[!template text="describing the output to libgccjit." start="00:16:48.340" video="mainVideo" id=subtitle]]
[[!template text="Unfortunately, if you do this," start="00:16:50.220" video="mainVideo" id=subtitle]]
[[!template text="you will discover that you have" start="00:16:53.820" video="mainVideo" id=subtitle]]
[[!template text="a performance upper bound limit of about 3x." start="00:16:55.840" video="mainVideo" id=subtitle]]
[[!template text="So it was an option," start="00:17:00.180" video="mainVideo" id=subtitle]]
[[!template text="but I thought it was a good occasion" start="00:17:04.460" video="mainVideo" id=subtitle]]
[[!template text="for trying to do something more." start="00:17:06.660" video="mainVideo" id=subtitle]]
[[!template text="And doing something more means" start="00:17:09.080" video="mainVideo" id=subtitle]]
[[!template text="implementing a smarter compiler" start="00:17:11.740" video="mainVideo" id=subtitle]]
[[!template text="that is doing some advanced analysis on the code," start="00:17:13.780" video="mainVideo" id=subtitle]]
[[!template text="and will be able to perform" start="00:17:17.280" video="mainVideo" id=subtitle]]
[[!template text="Lisp-specific optimizations" start="00:17:18.740" video="mainVideo" id=subtitle]]
[[!template text="-- optimizations that take advantage of" start="00:17:20.800" video="mainVideo" id=subtitle]]
[[!template text="the specific Lisp semantics," start="00:17:22.580" video="mainVideo" id=subtitle]]
[[!template text="something that GCC is not aware of." start="00:17:25.060" video="mainVideo" id=subtitle]]
[[!template text="And while I was thinking about that," start="00:17:27.860" video="mainVideo" id=subtitle]]
[[!template text="I thought that having a smarter compiler" start="00:17:31.340" video="mainVideo" id=subtitle]]
[[!template text="had also other advantages, like a smarter compiler" start="00:17:34.100" video="mainVideo" id=subtitle]]
[[!template text="that understands the semantics" start="00:17:38.220" video="mainVideo" id=subtitle]]
[[!template text="of the programming language being compiled" start="00:17:40.220" video="mainVideo" id=subtitle]]
[[!template text="would be also capable of" start="00:17:41.920" video="mainVideo" id=subtitle]]
[[!template text="giving feedback to the programmers," start="00:17:43.780" video="mainVideo" id=subtitle]]
[[!template text="like better warnings and errors." start="00:17:45.720" video="mainVideo" id=subtitle]]
[[!template text="So I was really fascinated about this idea," start="00:17:48.020" video="mainVideo" id=subtitle]]
[[!template text="and I wanted to change my implementation" start="00:17:51.460" video="mainVideo" id=subtitle]]
[[!template text="because I was not really happy about it." start="00:17:53.340" video="mainVideo" id=subtitle]]
[[!template text="I had a lot of C code in terms of" start="00:17:56.020" video="mainVideo" id=subtitle]]
[[!template text="lines that were not doing any smart job." start="00:17:58.420" video="mainVideo" id=subtitle]]
[[!template text="And I wanted to write all the interesting logic" start="00:18:02.200" video="mainVideo" id=subtitle]]
[[!template text="in Lisp." start="00:18:07.480" video="mainVideo" id=subtitle]]
[[!template text="So optimizing outside GCC" start="00:18:10.040" video="mainVideo" id=subtitle]]
[[!template text="before jumping into GCC," start="00:18:12.660" video="mainVideo" id=subtitle]]
[[!template text="as I mentioned, has two main targets:" start="00:18:15.940" video="mainVideo" id=subtitle]]
[[!template text="Either optimize the code before going into GCC," start="00:18:20.600" video="mainVideo" id=subtitle]]
[[!template text="or present to GCC some code" start="00:18:23.160" video="mainVideo" id=subtitle]]
[[!template text="that we know GCC can optimize effectively." start="00:18:25.480" video="mainVideo" id=subtitle]]
[[!template text="And also, this will give, as I mentioned," start="00:18:27.840" video="mainVideo" id=subtitle]]
[[!template text="better options for the compiler" start="00:18:30.900" video="mainVideo" id=subtitle]]
[[!template text="to provide warnings, errors" start="00:18:32.760" video="mainVideo" id=subtitle]]
[[!template text="-- better diagnostics." start="00:18:34.520" video="mainVideo" id=subtitle]]
[[!template text="So this is pretty much" start="00:18:36.440" video="mainVideo" id=subtitle]]
[[!template text="what the native compiler looks like nowadays," start="00:18:38.300" video="mainVideo" id=subtitle]]
[[!template text="in terms of passes." start="00:18:40.740" video="mainVideo" id=subtitle]]
[[!template text="We have a list of passes," start="00:18:42.820" video="mainVideo" id=subtitle]]
[[!template text="each of which is taking an input" start="00:18:44.660" video="mainVideo" id=subtitle]]
[[!template text="and producing an output." start="00:18:46.720" video="mainVideo" id=subtitle]]
[[!template text="So it's doing either analysis on the program" start="00:18:48.220" video="mainVideo" id=subtitle]]
[[!template text="that's being passed," start="00:18:51.140" video="mainVideo" id=subtitle]]
[[!template text="or it's performing a transformation." start="00:18:52.740" video="mainVideo" id=subtitle]]
[[!template text="All of these passes are implemented in Lisp," start="00:18:54.860" video="mainVideo" id=subtitle]]
[[!template text="and only the last pass is implemented in C." start="00:18:58.000" video="mainVideo" id=subtitle]]
[[!template text="That is the one that is talking to libgccjit." start="00:19:00.540" video="mainVideo" id=subtitle]]
[[!template text="To do that, I have introduced" start="00:19:05.160" video="mainVideo" id=subtitle]]
[[!template text="a new intermediate representation" start="00:19:07.860" video="mainVideo" id=subtitle]]
[[!template text="that I call LIMPLE, as a tribute to GCC GIMPLE," start="00:19:10.740" video="mainVideo" id=subtitle]]
[[!template text="that is the main internal representation of GCC," start="00:19:14.060" video="mainVideo" id=subtitle]]
[[!template text="at least one of the main ones." start="00:19:17.320" video="mainVideo" id=subtitle]]
[[!template text="Introducing a new intermediate representation" start="00:19:20.700" video="mainVideo" id=subtitle]]
[[!template text="-- a new way of representing my program --" start="00:19:25.180" video="mainVideo" id=subtitle]]
[[!template text="solved a bunch of problems." start="00:19:27.820" video="mainVideo" id=subtitle]]
[[!template text="First, it allowed me to implement" start="00:19:29.640" video="mainVideo" id=subtitle]]
[[!template text="non-trivial analysis and transformations," start="00:19:33.300" video="mainVideo" id=subtitle]]
[[!template text="the ones I needed in my compiler pipeline." start="00:19:37.380" video="mainVideo" id=subtitle]]
[[!template text="But also, it solved the problem of" start="00:19:40.100" video="mainVideo" id=subtitle]]
[[!template text="what was the boundary between" start="00:19:42.260" video="mainVideo" id=subtitle]]
[[!template text="what I had to implement in Lisp," start="00:19:43.940" video="mainVideo" id=subtitle]]
[[!template text="and what in C." start="00:19:46.220" video="mainVideo" id=subtitle]]
[[!template text="Because once I had" start="00:19:48.140" video="mainVideo" id=subtitle]]
[[!template text="my intermediate representation defined," start="00:19:49.140" video="mainVideo" id=subtitle]]
[[!template text="essentially the boundary between Lisp and C" start="00:19:51.960" video="mainVideo" id=subtitle]]
[[!template text="is just a function, that is," start="00:19:53.880" video="mainVideo" id=subtitle]]
[[!template text="the one that is implementing the final pass." start="00:19:55.740" video="mainVideo" id=subtitle]]
[[!template text="That is taking, as an input," start="00:19:57.880" video="mainVideo" id=subtitle]]
[[!template text="all of my programs in LIMPLE representation" start="00:19:59.320" video="mainVideo" id=subtitle]]
[[!template text="and it's doing his bit." start="00:20:01.720" video="mainVideo" id=subtitle]]
[[!template text="So I was convinced this design at least had sense." start="00:20:03.980" video="mainVideo" id=subtitle]]
[[!template text="When we go through some of these passes," start="00:20:08.080" video="mainVideo" id=subtitle]]
[[!template text="just to give you an idea of what these are doing:" start="00:20:10.220" video="mainVideo" id=subtitle]]
[[!template text="the first pass is just responsible for" start="00:20:12.700" video="mainVideo" id=subtitle]]
[[!template text="spilling the LAP from the byte compiler" start="00:20:14.660" video="mainVideo" id=subtitle]]
[[!template text="that effectively here we are using as a front end" start="00:20:18.200" video="mainVideo" id=subtitle]]
[[!template text="for our compiler pipeline." start="00:20:20.260" video="mainVideo" id=subtitle]]
[[!template text="The second pass, called 'limplify'," start="00:20:21.880" video="mainVideo" id=subtitle]]
[[!template text="will be in charge of converting LAP into LIMPLE." start="00:20:24.140" video="mainVideo" id=subtitle]]
[[!template text="LIMPLE is an intermediate representation" start="00:20:28.020" video="mainVideo" id=subtitle]]
[[!template text="that is Control Flow Graph based," start="00:20:31.360" video="mainVideo" id=subtitle]]
[[!template text="and it's capable of SSA." start="00:20:32.960" video="mainVideo" id=subtitle]]
[[!template text="So we can have a look to what this means." start="00:20:34.800" video="mainVideo" id=subtitle]]
[[!template text="Let's assume we have our LAP program," start="00:20:38.740" video="mainVideo" id=subtitle]]
[[!template text="as any program," start="00:20:41.400" video="mainVideo" id=subtitle]]
[[!template text="that's a simple list of instructions" start="00:20:42.620" video="mainVideo" id=subtitle]]
[[!template text="that we will execute one after the other." start="00:20:44.060" video="mainVideo" id=subtitle]]
[[!template text="Some of these instructions" start="00:20:45.740" video="mainVideo" id=subtitle]]
[[!template text="are special instructions" start="00:20:47.460" video="mainVideo" id=subtitle]]
[[!template text="that we call conditional branches," start="00:20:49.300" video="mainVideo" id=subtitle]]
[[!template text="where we check for a condition," start="00:20:51.340" video="mainVideo" id=subtitle]]
[[!template text="and if this is verified," start="00:20:52.900" video="mainVideo" id=subtitle]]
[[!template text="we jump to a different address within the program." start="00:20:54.420" video="mainVideo" id=subtitle]]
[[!template text="(Addresses that here we are calling 'labels'.)" start="00:20:57.040" video="mainVideo" id=subtitle]]
[[!template text="So we can split our program in chunks," start="00:20:59.680" video="mainVideo" id=subtitle]]
[[!template text="and those chunks we execute without interruption," start="00:21:03.540" video="mainVideo" id=subtitle]]
[[!template text="so we always enter from the top of those," start="00:21:08.300" video="mainVideo" id=subtitle]]
[[!template text="and we exit from the bottom." start="00:21:10.560" video="mainVideo" id=subtitle]]
[[!template text="We can name those, and split them apart," start="00:21:12.700" video="mainVideo" id=subtitle]]
[[!template text="and these are what we call basic blocks." start="00:21:16.080" video="mainVideo" id=subtitle]]
[[!template text="And now we have a bunch of these basic blocks" start="00:21:19.080" video="mainVideo" id=subtitle]]
[[!template text="that are floating," start="00:21:22.460" video="mainVideo" id=subtitle]]
[[!template text="and they are not any more sorted." start="00:21:23.480" video="mainVideo" id=subtitle]]
[[!template text="This is what is called" start="00:21:25.120" video="mainVideo" id=subtitle]]
[[!template text="a Control Flow Graph based representation." start="00:21:26.020" video="mainVideo" id=subtitle]]
[[!template text="Now we can get into the SSA topic." start="00:21:28.780" video="mainVideo" id=subtitle]]
[[!template text="That stands for Static Single Assignment." start="00:21:31.500" video="mainVideo" id=subtitle]]
[[!template text="I don't want to get into the details," start="00:21:34.000" video="mainVideo" id=subtitle]]
[[!template text="but just give you a feeling." start="00:21:35.960" video="mainVideo" id=subtitle]]
[[!template text="I added into our basic blocks" start="00:21:36.820" video="mainVideo" id=subtitle]]
[[!template text="in our Control Flow Graph a few assignments." start="00:21:38.580" video="mainVideo" id=subtitle]]
[[!template text="We will transform this into SSA" start="00:21:41.500" video="mainVideo" id=subtitle]]
[[!template text="just for the variable 'x'," start="00:21:43.940" video="mainVideo" id=subtitle]]
[[!template text="just for the sake of demonstrating it." start="00:21:45.140" video="mainVideo" id=subtitle]]
[[!template text="This is done through a number of phases" start="00:21:47.460" video="mainVideo" id=subtitle]]
[[!template text="that are essentially some analysis," start="00:21:49.860" video="mainVideo" id=subtitle]]
[[!template text="mainly renaming." start="00:21:51.860" video="mainVideo" id=subtitle]]
[[!template text="But the outcome, the one we see here," start="00:21:52.700" video="mainVideo" id=subtitle]]
[[!template text="looks quite similar to the original one," start="00:21:55.660" video="mainVideo" id=subtitle]]
[[!template text="but we can see that the variable 'x'" start="00:21:59.220" video="mainVideo" id=subtitle]]
[[!template text="has been renamed." start="00:22:01.220" video="mainVideo" id=subtitle]]
[[!template text="And now we don't have anymore just one," start="00:22:02.060" video="mainVideo" id=subtitle]]
[[!template text="but a number of these variables." start="00:22:03.500" video="mainVideo" id=subtitle]]
[[!template text="The interesting property is that" start="00:22:06.240" video="mainVideo" id=subtitle]]
[[!template text="each of these variables is assigned just once." start="00:22:08.100" video="mainVideo" id=subtitle]]
[[!template text="And this allows for the compiler" start="00:22:10.980" video="mainVideo" id=subtitle]]
[[!template text="to do prediction of the value of that variable," start="00:22:13.340" video="mainVideo" id=subtitle]]
[[!template text="depending on the position" start="00:22:16.860" video="mainVideo" id=subtitle]]
[[!template text="within the Control Flow Graph." start="00:22:19.140" video="mainVideo" id=subtitle]]
[[!template text="This is very important. For instance," start="00:22:19.940" video="mainVideo" id=subtitle]]
[[!template text="a very simple case is 'x1'" start="00:22:22.080" video="mainVideo" id=subtitle]]
[[!template text="that we see is assigned once by definition," start="00:22:23.540" video="mainVideo" id=subtitle]]
[[!template text="in particular here at the beginning." start="00:22:27.540" video="mainVideo" id=subtitle]]
[[!template text="Here it's very simple to understand" start="00:22:29.400" video="mainVideo" id=subtitle]]
[[!template text="that x1 will have the value 3." start="00:22:31.140" video="mainVideo" id=subtitle]]
[[!template text="While, for instance, it's more difficult to prove" start="00:22:33.180" video="mainVideo" id=subtitle]]
[[!template text="what is going to be the value of x5," start="00:22:35.480" video="mainVideo" id=subtitle]]
[[!template text="because it's calling a function," start="00:22:37.160" video="mainVideo" id=subtitle]]
[[!template text="or we don't know at the moment what x4 is." start="00:22:38.720" video="mainVideo" id=subtitle]]
[[!template text="So the compiler will gain the capability" start="00:22:42.040" video="mainVideo" id=subtitle]]
[[!template text="to do prediction on all the variables," start="00:22:46.340" video="mainVideo" id=subtitle]]
[[!template text="and the more we get information on one variable," start="00:22:48.560" video="mainVideo" id=subtitle]]
[[!template text="the more we can prove about the others." start="00:22:50.720" video="mainVideo" id=subtitle]]
[[!template text="Coming back to our passes, the next one" start="00:22:55.460" video="mainVideo" id=subtitle]]
[[!template text="is forward propagation." start="00:22:57.380" video="mainVideo" id=subtitle]]
[[!template text="This pass is responsible for" start="00:22:59.420" video="mainVideo" id=subtitle]]
[[!template text="doing what I briefly mentioned just before:" start="00:23:00.700" video="mainVideo" id=subtitle]]
[[!template text="doing proof over all the different variables" start="00:23:03.340" video="mainVideo" id=subtitle]]
[[!template text="in different positions of the Control Flow Graph," start="00:23:07.260" video="mainVideo" id=subtitle]]
[[!template text="about the values, types, or ranges." start="00:23:09.720" video="mainVideo" id=subtitle]]
[[!template text="This pass is also responsible for" start="00:23:12.800" video="mainVideo" id=subtitle]]
[[!template text="executing functions" start="00:23:15.180" video="mainVideo" id=subtitle]]
[[!template text="when we know that the function has no side effect" start="00:23:17.020" video="mainVideo" id=subtitle]]
[[!template text="and the pass managed to" start="00:23:18.740" video="mainVideo" id=subtitle]]
[[!template text="prove all the values of its argument." start="00:23:20.620" video="mainVideo" id=subtitle]]
[[!template text="So the function is then executed at compile time" start="00:23:22.540" video="mainVideo" id=subtitle]]
[[!template text="and it doesn't even exist anymore" start="00:23:24.900" video="mainVideo" id=subtitle]]
[[!template text="in the produced code." start="00:23:26.800" video="mainVideo" id=subtitle]]
[[!template text="Then we have another pass, this is" start="00:23:27.980" video="mainVideo" id=subtitle]]
[[!template text="an example of a pass that is very specific:" start="00:23:30.400" video="mainVideo" id=subtitle]]
[[!template text="it's trying to remove the call to funcall" start="00:23:33.520" video="mainVideo" id=subtitle]]
[[!template text="when those are not necessary." start="00:23:36.220" video="mainVideo" id=subtitle]]
[[!template text="There are a couple situations" start="00:23:38.140" video="mainVideo" id=subtitle]]
[[!template text="where this is very useful." start="00:23:39.860" video="mainVideo" id=subtitle]]
[[!template text="And not only is this beneficial" start="00:23:42.300" video="mainVideo" id=subtitle]]
[[!template text="because we are generating better code," start="00:23:45.340" video="mainVideo" id=subtitle]]
[[!template text="but when we manage to do that," start="00:23:47.660" video="mainVideo" id=subtitle]]
[[!template text="we allow GCC better analysis over the code," start="00:23:49.345" video="mainVideo" id=subtitle]]
[[!template text="because GCC knows nothing about funcall." start="00:23:52.100" video="mainVideo" id=subtitle]]
[[!template text="So if we are calling, from 'foo', directly, 'bar'," start="00:23:54.540" video="mainVideo" id=subtitle]]
[[!template text="for GCC it's way easier to do its analysis" start="00:23:57.500" video="mainVideo" id=subtitle]]
[[!template text="on top of this code." start="00:24:01.380" video="mainVideo" id=subtitle]]
[[!template text="Another interesting pass we can mention is 'tco'." start="00:24:03.460" video="mainVideo" id=subtitle]]
[[!template text="This is performing Tail Recursion Elimination." start="00:24:06.340" video="mainVideo" id=subtitle]]
[[!template text="It allows a more functional programming style," start="00:24:08.900" video="mainVideo" id=subtitle]]
[[!template text="if you want." start="00:24:11.980" video="mainVideo" id=subtitle]]
[[!template text="We can jump to the last pass" start="00:24:13.320" video="mainVideo" id=subtitle]]
[[!template text="that is called 'final', and as I mentioned," start="00:24:14.380" video="mainVideo" id=subtitle]]
[[!template text="this one is responsible for" start="00:24:16.300" video="mainVideo" id=subtitle]]
[[!template text="taking our program in LIMPLE representation" start="00:24:17.620" video="mainVideo" id=subtitle]]
[[!template text="and describing it to libgccjit in the gccjit IR." start="00:24:19.980" video="mainVideo" id=subtitle]]
[[!template text="That's the main task. It's also" start="00:24:25.000" video="mainVideo" id=subtitle]]
[[!template text="defining inline functions" start="00:24:27.580" video="mainVideo" id=subtitle]]
[[!template text="for accessing fundamental data types, and so on." start="00:24:29.620" video="mainVideo" id=subtitle]]
[[!template text="This pass is also responsible for" start="00:24:32.500" video="mainVideo" id=subtitle]]
[[!template text="using some of the predictions" start="00:24:34.560" video="mainVideo" id=subtitle]]
[[!template text="done by previous passes to generate better code." start="00:24:36.380" video="mainVideo" id=subtitle]]
[[!template text="Things we had to add" start="00:24:39.660" video="mainVideo" id=subtitle]]
[[!template text="to have all of this machinery work" start="00:24:41.420" video="mainVideo" id=subtitle]]
[[!template text="and to be controllable:" start="00:24:43.980" video="mainVideo" id=subtitle]]
[[!template text="The first one is an opt called 'native-comp-speed'" start="00:24:45.340" video="mainVideo" id=subtitle]]
[[!template text="and it's equivalent to Common Lisp's 'speed'." start="00:24:47.580" video="mainVideo" id=subtitle]]
[[!template text="It represents the optimization level." start="00:24:50.020" video="mainVideo" id=subtitle]]
[[!template text="The default is 2 and is" start="00:24:52.020" video="mainVideo" id=subtitle]]
[[!template text="the maximum optimization level" start="00:24:53.500" video="mainVideo" id=subtitle]]
[[!template text="that is meant to reflect" start="00:24:55.500" video="mainVideo" id=subtitle]]
[[!template text="all the original semantics of Emacs Lisp." start="00:24:58.700" video="mainVideo" id=subtitle]]
[[!template text="So it's the one that should be used by default." start="00:25:00.960" video="mainVideo" id=subtitle]]
[[!template text="The second one is 'compilation unit'" start="00:25:02.580" video="mainVideo" id=subtitle]]
[[!template text="and it's a kind of new object" start="00:25:04.740" video="mainVideo" id=subtitle]]
[[!template text="that has been added to Emacs." start="00:25:06.060" video="mainVideo" id=subtitle]]
[[!template text="Let's have a look to" start="00:25:11.180" video="mainVideo" id=subtitle]]
[[!template text="how the Garbage Collector works in this case." start="00:25:13.060" video="mainVideo" id=subtitle]]
[[!template text="The GNU Emacs Garbage Collector" start="00:25:14.460" video="mainVideo" id=subtitle]]
[[!template text="is a simple mark-and-sweep garbage collector." start="00:25:15.940" video="mainVideo" id=subtitle]]
[[!template text="It does a tree walk through all the objects" start="00:25:18.760" video="mainVideo" id=subtitle]]
[[!template text="and follows references from one object to another." start="00:25:21.520" video="mainVideo" id=subtitle]]
[[!template text="All the objects reachable during the mark phase" start="00:25:25.200" video="mainVideo" id=subtitle]]
[[!template text="will be kept in our Lisp universe." start="00:25:28.060" video="mainVideo" id=subtitle]]
[[!template text="All the other ones will be freed." start="00:25:31.340" video="mainVideo" id=subtitle]]
[[!template text="In this case we have a bunch of functions," start="00:25:33.780" video="mainVideo" id=subtitle]]
[[!template text="'foo1', 'foo2', 'bar1', etc., that are defined." start="00:25:35.180" video="mainVideo" id=subtitle]]
[[!template text="When a function is defined," start="00:25:38.860" video="mainVideo" id=subtitle]]
[[!template text="it's accessible through its symbol," start="00:25:40.420" video="mainVideo" id=subtitle]]
[[!template text="so we have the symbol referring to the function." start="00:25:42.500" video="mainVideo" id=subtitle]]
[[!template text="The function, in this case a native-compiled one," start="00:25:44.460" video="mainVideo" id=subtitle]]
[[!template text="is referring to the compilation unit." start="00:25:47.820" video="mainVideo" id=subtitle]]
[[!template text="The compilation unit is essentially" start="00:25:50.140" video="mainVideo" id=subtitle]]
[[!template text="the ELF file that has been compiled," start="00:25:53.120" video="mainVideo" id=subtitle]]
[[!template text="and contains all those functions" start="00:25:58.700" video="mainVideo" id=subtitle]]
[[!template text="that came from the original .el file," start="00:26:01.260" video="mainVideo" id=subtitle]]
[[!template text="and that we have loaded into memory." start="00:26:03.340" video="mainVideo" id=subtitle]]
[[!template text="If, for instance, 'bar1 and 'bar2 are undefined," start="00:26:05.200" video="mainVideo" id=subtitle]]
[[!template text="functions 3 and 4 will be no longer reachable," start="00:26:10.100" video="mainVideo" id=subtitle]]
[[!template text="and we will be able to free them" start="00:26:14.300" video="mainVideo" id=subtitle]]
[[!template text="and unload the compilation unit." start="00:26:16.140" video="mainVideo" id=subtitle]]
[[!template text="We discussed quite a lot about Control Flow Graph," start="00:26:18.260" video="mainVideo" id=subtitle]]
[[!template text="SSA, and a lot of boring stuff," start="00:26:21.300" video="mainVideo" id=subtitle]]
[[!template text="and I promised you that we are doing" start="00:26:23.660" video="mainVideo" id=subtitle]]
[[!template text="a lot of interesting proofs over variables," start="00:26:25.500" video="mainVideo" id=subtitle]]
[[!template text="So let's have some examples of them." start="00:26:27.420" video="mainVideo" id=subtitle]]
[[!template text="Let's jump into a quick demo" start="00:26:30.320" video="mainVideo" id=subtitle]]
[[!template text="to see what all of this abstract theory" start="00:26:31.940" video="mainVideo" id=subtitle]]
[[!template text="and this esoteric propagation engine can do for us" start="00:26:34.580" video="mainVideo" id=subtitle]]
[[!template text="and how the user can interact with it." start="00:26:37.780" video="mainVideo" id=subtitle]]
[[!template text="I've defined a bunch of functions," start="00:26:39.340" video="mainVideo" id=subtitle]]
[[!template text="and I will native-compile and load it." start="00:26:42.200" video="mainVideo" id=subtitle]]
[[!template text="Alright, Emacs Lisp native compiled and loaded." start="00:26:47.500" video="mainVideo" id=subtitle]]
[[!template text="At this point, I can disassemble 'foo1'" start="00:26:48.940" video="mainVideo" id=subtitle]]
[[!template text="to make sure it's native code and I'm not lying." start="00:26:52.420" video="mainVideo" id=subtitle]]
[[!template text="These are the instructions" start="00:26:56.420" video="mainVideo" id=subtitle]]
[[!template text="that will be executed directly by my CPU" start="00:26:58.600" video="mainVideo" id=subtitle]]
[[!template text="when I call this function." start="00:27:01.420" video="mainVideo" id=subtitle]]
[[!template text="Alright, very cool." start="00:27:03.720" video="mainVideo" id=subtitle]]
[[!template text="Now, Lisp: (symbol-function #'foo1)" start="00:27:07.620" video="mainVideo" id=subtitle]]
[[!template text="Interestingly, this is returning a subroutine," start="00:27:16.180" video="mainVideo" id=subtitle]]
[[!template text="as it would be a primitive function." start="00:27:19.820" video="mainVideo" id=subtitle]]
[[!template text="Because this is native code," start="00:27:21.920" video="mainVideo" id=subtitle]]
[[!template text="even if it's written in Lisp," start="00:27:23.800" video="mainVideo" id=subtitle]]
[[!template text="has been converted to native code" start="00:27:24.940" video="mainVideo" id=subtitle]]
[[!template text="as if it's a primitive function." start="00:27:26.440" video="mainVideo" id=subtitle]]
[[!template text="But we can do also a new thing:" start="00:27:29.300" video="mainVideo" id=subtitle]]
[[!template text="asking for the type of the subroutine." start="00:27:31.660" video="mainVideo" id=subtitle]]
[[!template text="Alright, very cool. It says this is a function," start="00:27:34.540" video="mainVideo" id=subtitle]]
[[!template text="it's taking one argument of type 't'" start="00:27:38.460" video="mainVideo" id=subtitle]]
[[!template text="(that means anything" start="00:27:40.380" video="mainVideo" id=subtitle]]
[[!template text="because we don't have any information)," start="00:27:41.660" video="mainVideo" id=subtitle]]
[[!template text="and is returning a type 't'," start="00:27:43.060" video="mainVideo" id=subtitle]]
[[!template text="so also there we don't have much information." start="00:27:45.300" video="mainVideo" id=subtitle]]
[[!template text="OK, very cool, but not very useful." start="00:27:47.480" video="mainVideo" id=subtitle]]
[[!template text="Let's see #'foo2. #'foo2 is slightly different," start="00:27:49.800" video="mainVideo" id=subtitle]]
[[!template text="it doesn't take any argument, but it's returning" start="00:27:53.780" video="mainVideo" id=subtitle]]
[[!template text="an integer included between 3 and 3." start="00:27:55.940" video="mainVideo" id=subtitle]]
[[!template text="Wow, amazing!" start="00:27:58.140" video="mainVideo" id=subtitle]]
[[!template text="Let's get into something a little more complex:" start="00:28:01.460" video="mainVideo" id=subtitle]]
[[!template text="#'foo3 takes one argument we know nothing about," start="00:28:04.140" video="mainVideo" id=subtitle]]
[[!template text="but it's returning a number." start="00:28:09.540" video="mainVideo" id=subtitle]]
[[!template text="And why it's returning a number?" start="00:28:11.840" video="mainVideo" id=subtitle]]
[[!template text="Essentially because 1+ is returning a number," start="00:28:13.380" video="mainVideo" id=subtitle]]
[[!template text="and in all the other cases," start="00:28:16.420" video="mainVideo" id=subtitle]]
[[!template text="it would signal an error" start="00:28:18.980" video="mainVideo" id=subtitle]]
[[!template text="if it's not happy about its input argument." start="00:28:20.740" video="mainVideo" id=subtitle]]
[[!template text="Let's have a look to #'foo4." start="00:28:23.480" video="mainVideo" id=subtitle]]
[[!template text="#'foo4 is a little bit more complex." start="00:28:27.860" video="mainVideo" id=subtitle]]
[[!template text="It will return nil" start="00:28:33.020" video="mainVideo" id=subtitle]]
[[!template text="if the 'when' condition is not satisfied," start="00:28:34.780" video="mainVideo" id=subtitle]]
[[!template text="so it's type 'null' here." start="00:28:37.500" video="mainVideo" id=subtitle]]
[[!template text="It can return a floating point;" start="00:28:39.820" video="mainVideo" id=subtitle]]
[[!template text="we don't do propagation of floating point so far," start="00:28:41.300" video="mainVideo" id=subtitle]]
[[!template text="or it can return any integer between 4 and 9." start="00:28:43.700" video="mainVideo" id=subtitle]]
[[!template text="Wow. Let's go on with #'foo5." start="00:28:47.180" video="mainVideo" id=subtitle]]
[[!template text="#'foo5 is even more complex" start="00:28:52.940" video="mainVideo" id=subtitle]]
[[!template text="because other than" start="00:28:55.860" video="mainVideo" id=subtitle]]
[[!template text="having to satisfy this condition," start="00:28:57.300" video="mainVideo" id=subtitle]]
[[!template text="we can see that the result of the propagation" start="00:28:59.280" video="mainVideo" id=subtitle]]
[[!template text="of this complex condition" start="00:29:02.380" video="mainVideo" id=subtitle]]
[[!template text="is propagated also across the 'plus'." start="00:29:03.900" video="mainVideo" id=subtitle]]
[[!template text="So this foo5 can return nil," start="00:29:05.560" video="mainVideo" id=subtitle]]
[[!template text="a floating point we know nothing about," start="00:29:08.340" video="mainVideo" id=subtitle]]
[[!template text="or an integer included between 12 and 24." start="00:29:09.820" video="mainVideo" id=subtitle]]
[[!template text="Let's go on with #'foo6." start="00:29:13.660" video="mainVideo" id=subtitle]]
[[!template text="#'foo6 is returning anything but an integer." start="00:29:18.180" video="mainVideo" id=subtitle]]
[[!template text="I think it should be pretty obvious why," start="00:29:23.420" video="mainVideo" id=subtitle]]
[[!template text="because if it's not an integer we return it," start="00:29:26.620" video="mainVideo" id=subtitle]]
[[!template text="otherwise we signal an error." start="00:29:28.220" video="mainVideo" id=subtitle]]
[[!template text="Let's finish with #'foo7 very quickly." start="00:29:30.100" video="mainVideo" id=subtitle]]
[[!template text="#'foo7 has another very complex condition," start="00:29:32.980" video="mainVideo" id=subtitle]]
[[!template text="at least for me, but it's also interesting to see" start="00:29:38.020" video="mainVideo" id=subtitle]]
[[!template text="that we are also propagating values for symbols." start="00:29:40.420" video="mainVideo" id=subtitle]]
[[!template text="So we can return the symbol 'big," start="00:29:42.300" video="mainVideo" id=subtitle]]
[[!template text="the symbol 'small," start="00:29:45.260" video="mainVideo" id=subtitle]]
[[!template text="or an integer included between -100 and 100." start="00:29:47.080" video="mainVideo" id=subtitle]]
[[!template text="Now, the question is: why all of this is useful" start="00:29:51.300" video="mainVideo" id=subtitle]]
[[!template text="other than having Andrea very happy" start="00:29:54.540" video="mainVideo" id=subtitle]]
[[!template text="when he's playing with this all day?" start="00:29:56.900" video="mainVideo" id=subtitle]]
[[!template text="Well, we have to come back one second" start="00:29:59.660" video="mainVideo" id=subtitle]]
[[!template text="to how Lisp_Objects are represented within Emacs." start="00:30:01.540" video="mainVideo" id=subtitle]]
[[!template text="Lisp_Objects are represented as machine words," start="00:30:05.020" video="mainVideo" id=subtitle]]
[[!template text="where we reserve a few bits to indicate the type." start="00:30:09.580" video="mainVideo" id=subtitle]]
[[!template text="And every time we access the object," start="00:30:12.680" video="mainVideo" id=subtitle]]
[[!template text="when this is a Fixnum," start="00:30:15.660" video="mainVideo" id=subtitle]]
[[!template text="or a regular object where this is a pointer," start="00:30:17.220" video="mainVideo" id=subtitle]]
[[!template text="we always have to extract these bits," start="00:30:20.020" video="mainVideo" id=subtitle]]
[[!template text="make sure that they satisfy a condition," start="00:30:21.700" video="mainVideo" id=subtitle]]
[[!template text="so make sure that we are going to manipulate" start="00:30:24.620" video="mainVideo" id=subtitle]]
[[!template text="the object of the type we expect," start="00:30:27.220" video="mainVideo" id=subtitle]]
[[!template text="and then we can extract the object" start="00:30:28.680" video="mainVideo" id=subtitle]]
[[!template text="and do the manipulation." start="00:30:31.140" video="mainVideo" id=subtitle]]
[[!template text="If the compiler managed to prove" start="00:30:32.640" video="mainVideo" id=subtitle]]
[[!template text="that the contained object is of the right type," start="00:30:34.620" video="mainVideo" id=subtitle]]
[[!template text="it will be able to not emit the type check," start="00:30:37.500" video="mainVideo" id=subtitle]]
[[!template text="and save a lot of instructions." start="00:30:42.540" video="mainVideo" id=subtitle]]
[[!template text="This is a very powerful optimization." start="00:30:44.600" video="mainVideo" id=subtitle]]
[[!template text="Let's discuss some potential future development" start="00:30:48.660" video="mainVideo" id=subtitle]]
[[!template text="in this area." start="00:30:50.860" video="mainVideo" id=subtitle]]
[[!template text="First I think it would be extremely nice" start="00:30:52.100" video="mainVideo" id=subtitle]]
[[!template text="to extend the propagation" start="00:30:53.460" video="mainVideo" id=subtitle]]
[[!template text="to types that are not built in." start="00:30:55.020" video="mainVideo" id=subtitle]]
[[!template text="There are a lot of cases" start="00:30:57.020" video="mainVideo" id=subtitle]]
[[!template text="where we could optimize effectively." start="00:30:58.260" video="mainVideo" id=subtitle]]
[[!template text="For instance when we do" start="00:31:00.700" video="mainVideo" id=subtitle]]
[[!template text="a lot of accesses to structures" start="00:31:02.620" video="mainVideo" id=subtitle]]
[[!template text="-- lots of stuff like that --" start="00:31:05.820" video="mainVideo" id=subtitle]]
[[!template text="where we keep on checking and checking" start="00:31:07.020" video="mainVideo" id=subtitle]]
[[!template text="the same object for the type" start="00:31:08.980" video="mainVideo" id=subtitle]]
[[!template text="where it's obvious, where it should be trivial" start="00:31:10.740" video="mainVideo" id=subtitle]]
[[!template text="to prove that it's the right type" start="00:31:13.180" video="mainVideo" id=subtitle]]
[[!template text="after the first access or things like that." start="00:31:14.820" video="mainVideo" id=subtitle]]
[[!template text="So I believe this is a low-hanging fruit" start="00:31:16.460" video="mainVideo" id=subtitle]]
[[!template text="in terms of performance." start="00:31:19.020" video="mainVideo" id=subtitle]]
[[!template text="Also I think it would be really nice" start="00:31:21.280" video="mainVideo" id=subtitle]]
[[!template text="to extend the declare mechanism" start="00:31:23.180" video="mainVideo" id=subtitle]]
[[!template text="to allow the user to declare argument types." start="00:31:24.820" video="mainVideo" id=subtitle]]
[[!template text="(Optionally. Indeed, optionally.)" start="00:31:27.700" video="mainVideo" id=subtitle]]
[[!template text="Doing that would give the compiler" start="00:31:30.580" video="mainVideo" id=subtitle]]
[[!template text="a lot of information to propagate value types" start="00:31:32.980" video="mainVideo" id=subtitle]]
[[!template text="within the function." start="00:31:35.220" video="mainVideo" id=subtitle]]
[[!template text="But those will allow the compiler" start="00:31:36.460" video="mainVideo" id=subtitle]]
[[!template text="to give the user really good diagnostics" start="00:31:38.940" video="mainVideo" id=subtitle]]
[[!template text="during compile time. Like the compiler could say:" start="00:31:43.140" video="mainVideo" id=subtitle]]
[[!template text="Hey, here you are calling a function" start="00:31:45.500" video="mainVideo" id=subtitle]]
[[!template text="that is expecting this argument of this type," start="00:31:47.100" video="mainVideo" id=subtitle]]
[[!template text="but I'm proving that you are calling it" start="00:31:49.300" video="mainVideo" id=subtitle]]
[[!template text="with an argument that is of THIS type," start="00:31:52.100" video="mainVideo" id=subtitle]]
[[!template text="and is not a subtype of the expected one." start="00:31:55.300" video="mainVideo" id=subtitle]]
[[!template text="So you are doing something not coherent." start="00:31:57.940" video="mainVideo" id=subtitle]]
[[!template text="This kind of interprocedural logic." start="00:32:00.340" video="mainVideo" id=subtitle]]
[[!template text="And I think the compiler should also take advantage" start="00:32:02.900" video="mainVideo" id=subtitle]]
[[!template text="(under certain circumstances)" start="00:32:05.100" video="mainVideo" id=subtitle]]
[[!template text="of this interprocedural analysis" start="00:32:06.740" video="mainVideo" id=subtitle]]
[[!template text="in order to optimize even more, when possible." start="00:32:08.860" video="mainVideo" id=subtitle]]
[[!template text="Also I think we should" start="00:32:12.620" video="mainVideo" id=subtitle]]
[[!template text="work on our ? to improve the code generation" start="00:32:13.820" video="mainVideo" id=subtitle]]
[[!template text="depending on the prediction" start="00:32:15.580" video="mainVideo" id=subtitle]]
[[!template text="that the compiler is doing." start="00:32:17.200" video="mainVideo" id=subtitle]]
[[!template text="We already take advantage of those predictions," start="00:32:18.260" video="mainVideo" id=subtitle]]
[[!template text="but I think we could do better." start="00:32:20.220" video="mainVideo" id=subtitle]]
[[!template text="A quick look at some performance results." start="00:32:22.400" video="mainVideo" id=subtitle]]
[[!template text="These are from the elisp-benchmarks package" start="00:32:25.220" video="mainVideo" id=subtitle]]
[[!template text="within GNU ELPA." start="00:32:28.820" video="mainVideo" id=subtitle]]
[[!template text="This is the performance uplift," start="00:32:30.860" video="mainVideo" id=subtitle]]
[[!template text="and we can identify about 4 " ;classes" ; of results." start="00:32:32.620" video="mainVideo" id=subtitle]]
[[!template text="The first one there is no performance uplift," start="00:32:38.580" video="mainVideo" id=subtitle]]
[[!template text="because there is not much we can do," start="00:32:41.540" video="mainVideo" id=subtitle]]
[[!template text="and the time is probably not spent" start="00:32:42.980" video="mainVideo" id=subtitle]]
[[!template text="within the execution engine." start="00:32:44.820" video="mainVideo" id=subtitle]]
[[!template text="And the ones around 3x are the ones" start="00:32:46.380" video="mainVideo" id=subtitle]]
[[!template text="Where probably we are not triggering" start="00:32:49.100" video="mainVideo" id=subtitle]]
[[!template text="manually specific optimizations," start="00:32:50.780" video="mainVideo" id=subtitle]]
[[!template text="but just the fact" start="00:32:52.740" video="mainVideo" id=subtitle]]
[[!template text="that we are converting into native code" start="00:32:53.700" video="mainVideo" id=subtitle]]
[[!template text="is giving us this performance uplift." start="00:32:57.200" video="mainVideo" id=subtitle]]
[[!template text="Then there is a bunch of other benchmarks" start="00:33:00.600" video="mainVideo" id=subtitle]]
[[!template text="where the Lisp optimizations are triggering," start="00:33:03.380" video="mainVideo" id=subtitle]]
[[!template text="and the uplift is way bigger," start="00:33:05.780" video="mainVideo" id=subtitle]]
[[!template text="and then we have 3 benchmarks that at the time" start="00:33:09.720" video="mainVideo" id=subtitle]]
[[!template text="are completely optimized out." start="00:33:12.000" video="mainVideo" id=subtitle]]
[[!template text="That means the compiler became " ;so smart" ;" start="00:33:13.520" video="mainVideo" id=subtitle]]
[[!template text="that it was able to compute the result" start="00:33:15.680" video="mainVideo" id=subtitle]]
[[!template text="in the compile time and just put the result" start="00:33:18.260" video="mainVideo" id=subtitle]]
[[!template text="in the generated binary." start="00:33:20.260" video="mainVideo" id=subtitle]]
[[!template text="Let's discuss a little bit the compilation model." start="00:33:21.500" video="mainVideo" id=subtitle]]
[[!template text="This is an Hybrid one;" start="00:33:23.980" video="mainVideo" id=subtitle]]
[[!template text="it's both JIT-like and Ahead-of-Time-like." start="00:33:26.180" video="mainVideo" id=subtitle]]
[[!template text="Emacs is composed of what we call an Emacs image," start="00:33:29.520" video="mainVideo" id=subtitle]]
[[!template text="essentially the Emacs binary that we start." start="00:33:33.500" video="mainVideo" id=subtitle]]
[[!template text="It's including all the C code," start="00:33:36.280" video="mainVideo" id=subtitle]]
[[!template text="plus all the Lisp code that we preload." start="00:33:37.820" video="mainVideo" id=subtitle]]
[[!template text="Then we have the rest of the Emacs Lisp codebase" start="00:33:41.860" video="mainVideo" id=subtitle]]
[[!template text="that can be loaded just if it's required." start="00:33:46.580" video="mainVideo" id=subtitle]]
[[!template text="Same for the external packages, if we have any." start="00:33:49.540" video="mainVideo" id=subtitle]]
[[!template text="If we build an Emacs Lisp" start="00:33:52.860" video="mainVideo" id=subtitle]]
[[!template text="with native compilation enabled, by default," start="00:33:55.220" video="mainVideo" id=subtitle]]
[[!template text="only the Emacs image will be native compiled." start="00:33:58.040" video="mainVideo" id=subtitle]]
[[!template text="All the other code will be compiled" start="00:34:01.300" video="mainVideo" id=subtitle]]
[[!template text="on the fly when it's loaded and executed" start="00:34:04.020" video="mainVideo" id=subtitle]]
[[!template text="the first time, if it's necessary." start="00:34:06.820" video="mainVideo" id=subtitle]]
[[!template text="Same for the packages, in a transparent way" start="00:34:08.900" video="mainVideo" id=subtitle]]
[[!template text="and asynchronous way." start="00:34:10.980" video="mainVideo" id=subtitle]]
[[!template text="Also worth noting" start="00:34:12.740" video="mainVideo" id=subtitle]]
[[!template text="that the result of this compilation" start="00:34:13.580" video="mainVideo" id=subtitle]]
[[!template text="will be stored into a cache directory" start="00:34:15.500" video="mainVideo" id=subtitle]]
[[!template text="within the home directory of the user." start="00:34:17.340" video="mainVideo" id=subtitle]]
[[!template text="So it will be reused in the following sessions" start="00:34:20.000" video="mainVideo" id=subtitle]]
[[!template text="if the same file is loaded," start="00:34:23.700" video="mainVideo" id=subtitle]]
[[!template text="without having to recompile multiple times" start="00:34:25.540" video="mainVideo" id=subtitle]]
[[!template text="the same file in different sessions." start="00:34:27.620" video="mainVideo" id=subtitle]]
[[!template text="It works a little bit like this:" start="00:34:29.420" video="mainVideo" id=subtitle]]
[[!template text="When we load the byte-code for the first time," start="00:34:31.620" video="mainVideo" id=subtitle]]
[[!template text="we spawn a native compilation process." start="00:34:33.900" video="mainVideo" id=subtitle]]
[[!template text="Meanwhile we keep using the byte-code available." start="00:34:36.000" video="mainVideo" id=subtitle]]
[[!template text="When the native compilation is finished," start="00:34:39.420" video="mainVideo" id=subtitle]]
[[!template text="we hot-swap the definition of the functions" start="00:34:41.280" video="mainVideo" id=subtitle]]
[[!template text="that are contained in the file" start="00:34:44.060" video="mainVideo" id=subtitle]]
[[!template text="and start using the native code transparently." start="00:34:46.140" video="mainVideo" id=subtitle]]
[[!template text="We do this asynchronously," start="00:34:48.060" video="mainVideo" id=subtitle]]
[[!template text="and for more compilation units at the same time," start="00:34:49.980" video="mainVideo" id=subtitle]]
[[!template text="so it looks a little bit like this." start="00:34:53.240" video="mainVideo" id=subtitle]]
[[!template text="Let's try a quick demo of all of this machinery." start="00:34:56.380" video="mainVideo" id=subtitle]]
[[!template text="I've started a fresh Emacs" start="00:34:58.660" video="mainVideo" id=subtitle]]
[[!template text="with native compilation support," start="00:35:00.980" video="mainVideo" id=subtitle]]
[[!template text="and at this moment nothing is going on." start="00:35:03.380" video="mainVideo" id=subtitle]]
[[!template text="We will test the machinery with Tetris," start="00:35:05.160" video="mainVideo" id=subtitle]]
[[!template text="because I can't imagine anything better" start="00:35:07.540" video="mainVideo" id=subtitle]]
[[!template text="to test this." start="00:35:10.560" video="mainVideo" id=subtitle]]
[[!template text="What I do expect is that when I launch Tetris" start="00:35:12.940" video="mainVideo" id=subtitle]]
[[!template text="it will be loaded, it will immediately" start="00:35:15.180" video="mainVideo" id=subtitle]]
[[!template text="start execution of the byte-compiled version," start="00:35:18.020" video="mainVideo" id=subtitle]]
[[!template text="so we won't see any delay in the user experience," start="00:35:20.560" video="mainVideo" id=subtitle]]
[[!template text="and in the meanwhile, a parallel process" start="00:35:23.100" video="mainVideo" id=subtitle]]
[[!template text="will start to native-compile Tetris itself." start="00:35:25.260" video="mainVideo" id=subtitle]]
[[!template text="When the native compilation will be finished," start="00:35:28.260" video="mainVideo" id=subtitle]]
[[!template text="the functions of all Tetris will be hot-swapped." start="00:35:30.300" video="mainVideo" id=subtitle]]
[[!template text="So we will not see any interruption." start="00:35:33.340" video="mainVideo" id=subtitle]]
[[!template text="So Tetris started, and it's running," start="00:35:36.380" video="mainVideo" id=subtitle]]
[[!template text="we have seen no delay, and in the meanwhile," start="00:35:39.780" video="mainVideo" id=subtitle]]
[[!template text="the native compilation probably already finished," start="00:35:41.700" video="mainVideo" id=subtitle]]
[[!template text="we can have a look." start="00:35:43.620" video="mainVideo" id=subtitle]]
[[!template text="In this I see the native compilation log buffer." start="00:35:45.060" video="mainVideo" id=subtitle]]
[[!template text="So we see that Tetris has been native compiled," start="00:35:47.260" video="mainVideo" id=subtitle]]
[[!template text="and all of its dependencies." start="00:35:49.820" video="mainVideo" id=subtitle]]
[[!template text="Now Tetris is still running," start="00:35:51.860" video="mainVideo" id=subtitle]]
[[!template text="but I can do " ;C-h f tetris" ;" start="00:35:53.940" video="mainVideo" id=subtitle]]
[[!template text="and we can see that 'tetris'" start="00:36:00.580" video="mainVideo" id=subtitle]]
[[!template text="is an interactive native compiled" start="00:36:02.640" video="mainVideo" id=subtitle]]
[[!template text="Lisp function, so it has been native-compiled." start="00:36:04.980" video="mainVideo" id=subtitle]]
[[!template text="I can even disassemble if I want." start="00:36:08.040" video="mainVideo" id=subtitle]]
[[!template text="OK, so very cool." start="00:36:13.600" video="mainVideo" id=subtitle]]
[[!template text="I guess we can say this mechanism is working." start="00:36:14.980" video="mainVideo" id=subtitle]]
[[!template text="Also worth noting that if I go back" start="00:36:18.120" video="mainVideo" id=subtitle]]
[[!template text="to the *Async-native-compile-log* buffer," start="00:36:20.760" video="mainVideo" id=subtitle]]
[[!template text="we see we have compiled another bunch of files." start="00:36:24.220" video="mainVideo" id=subtitle]]
[[!template text="I think these are because of my 'C-h f'," start="00:36:28.060" video="mainVideo" id=subtitle]]
[[!template text="this help function command and disassemble," start="00:36:31.700" video="mainVideo" id=subtitle]]
[[!template text="and so on." start="00:36:33.900" video="mainVideo" id=subtitle]]
[[!template text="The first time you run Emacs, you will have," start="00:36:35.020" video="mainVideo" id=subtitle]]
[[!template text="from time to time, these processes spawned." start="00:36:37.820" video="mainVideo" id=subtitle]]
[[!template text="Emacs is " ;compiling itself" ;," start="00:36:41.380" video="mainVideo" id=subtitle]]
[[!template text="and it's replacing the byte-code definition" start="00:36:43.200" video="mainVideo" id=subtitle]]
[[!template text="with the native one. But after a few sessions," start="00:36:45.620" video="mainVideo" id=subtitle]]
[[!template text="you will not see this anymore," start="00:36:47.740" video="mainVideo" id=subtitle]]
[[!template text="because the output of this compilation," start="00:36:49.860" video="mainVideo" id=subtitle]]
[[!template text="as I mentioned, are stored in the user directory." start="00:36:51.460" video="mainVideo" id=subtitle]]
[[!template text="To conclude: Emacs with native compilation support" start="00:36:55.000" video="mainVideo" id=subtitle]]
[[!template text="is coming up in Emacs 28," start="00:36:57.660" video="mainVideo" id=subtitle]]
[[!template text="that is gonna be the next major stable release" start="00:36:59.180" video="mainVideo" id=subtitle]]
[[!template text="that will be released." start="00:37:01.140" video="mainVideo" id=subtitle]]
[[!template text="So we ought to celebrate with a big party," start="00:37:02.180" video="mainVideo" id=subtitle]]
[[!template text="I believe. But before going to the party," start="00:37:04.940" video="mainVideo" id=subtitle]]
[[!template text="I'd like to list a few points" start="00:37:07.420" video="mainVideo" id=subtitle]]
[[!template text="that I think have been success factors" start="00:37:09.180" video="mainVideo" id=subtitle]]
[[!template text="in upstreaming this work." start="00:37:11.440" video="mainVideo" id=subtitle]]
[[!template text="It has been extremely important" start="00:37:13.520" video="mainVideo" id=subtitle]]
[[!template text="to get in touch with upstream as soon as possible," start="00:37:15.520" video="mainVideo" id=subtitle]]
[[!template text="as soon as I had a proof of concept." start="00:37:18.240" video="mainVideo" id=subtitle]]
[[!template text="It's been extremely important" start="00:37:20.780" video="mainVideo" id=subtitle]]
[[!template text="to involve the community as much as possible," start="00:37:22.700" video="mainVideo" id=subtitle]]
[[!template text="and this included keeping a development blog," start="00:37:24.760" video="mainVideo" id=subtitle]]
[[!template text="and posts about that on emacs-devel," start="00:37:28.580" video="mainVideo" id=subtitle]]
[[!template text="and also producing material," start="00:37:31.700" video="mainVideo" id=subtitle]]
[[!template text="participating in conferences," start="00:37:33.820" video="mainVideo" id=subtitle]]
[[!template text="and giving presentations like the one I'm doing," start="00:37:36.120" video="mainVideo" id=subtitle]]
[[!template text="to explain what I was doing and how it works." start="00:37:38.420" video="mainVideo" id=subtitle]]
[[!template text="It has been extremely important, also," start="00:37:40.780" video="mainVideo" id=subtitle]]
[[!template text="to be able to rely on the upstream infrastructure." start="00:37:43.140" video="mainVideo" id=subtitle]]
[[!template text="So, to develop the software" start="00:37:45.960" video="mainVideo" id=subtitle]]
[[!template text="as a feature branch in the official git," start="00:37:47.640" video="mainVideo" id=subtitle]]
[[!template text="but even more, I would say," start="00:37:49.180" video="mainVideo" id=subtitle]]
[[!template text="to use the official bug tracker" start="00:37:50.060" video="mainVideo" id=subtitle]]
[[!template text="for solving bugs of this branch." start="00:37:51.760" video="mainVideo" id=subtitle]]
[[!template text="This gave the opportunity" start="00:37:52.980" video="mainVideo" id=subtitle]]
[[!template text="to stay really in close touch with maintainers," start="00:37:54.780" video="mainVideo" id=subtitle]]
[[!template text="and senior developers of Emacs." start="00:37:58.260" video="mainVideo" id=subtitle]]
[[!template text="That helped me a lot. And at the same time" start="00:38:00.020" video="mainVideo" id=subtitle]]
[[!template text="they were informed about what I was doing" start="00:38:03.080" video="mainVideo" id=subtitle]]
[[!template text="and what was the status of this feature branch." start="00:38:04.920" video="mainVideo" id=subtitle]]
[[!template text="Extremely important." start="00:38:07.460" video="mainVideo" id=subtitle]]
[[!template text="And also I think it played a major role" start="00:38:08.900" video="mainVideo" id=subtitle]]
[[!template text="to try to design this enormous patch" start="00:38:11.260" video="mainVideo" id=subtitle]]
[[!template text="in a way that the impact on the current codebase" start="00:38:14.220" video="mainVideo" id=subtitle]]
[[!template text="was minimized (at least as much as possible)." start="00:38:18.220" video="mainVideo" id=subtitle]]
[[!template text="And also minimizing" start="00:38:21.220" video="mainVideo" id=subtitle]]
[[!template text="the impact on the user operation of the software," start="00:38:23.760" video="mainVideo" id=subtitle]]
[[!template text="in this case Emacs." start="00:38:26.340" video="mainVideo" id=subtitle]]
[[!template text="So yes, mandatory Special Thanks:" start="00:38:28.140" video="mainVideo" id=subtitle]]
[[!template text="Emacs developers, and especially maintainers" start="00:38:29.740" video="mainVideo" id=subtitle]]
[[!template text="and senior developers like Stefan Monnier," start="00:38:33.460" video="mainVideo" id=subtitle]]
[[!template text="that helped me a lot across this long journey." start="00:38:36.780" video="mainVideo" id=subtitle]]
[[!template text="And, well, all the community" start="00:38:40.540" video="mainVideo" id=subtitle]]
[[!template text="that really got so excited about this project" start="00:38:42.900" video="mainVideo" id=subtitle]]
[[!template text="and gave me the energy" start="00:38:45.260" video="mainVideo" id=subtitle]]
[[!template text="to go through all of this time and development" start="00:38:46.340" video="mainVideo" id=subtitle]]
[[!template text="and bugs and solving, etc. etc." start="00:38:49.220" video="mainVideo" id=subtitle]]
[[!template text="So yes, it was a really exciting time," start="00:38:52.080" video="mainVideo" id=subtitle]]
[[!template text="and I think we have to look forward" start="00:38:55.020" video="mainVideo" id=subtitle]]
[[!template text="and start thinking about how to improve all this" start="00:38:58.100" video="mainVideo" id=subtitle]]
[[!template text="for the following years." start="00:39:01.500" video="mainVideo" id=subtitle]]
[[!template text="And that's it." start="00:39:03.020" video="mainVideo" id=subtitle]]
[[!template text="I think I should be online for questions." start="00:39:04.400" video="mainVideo" id=subtitle]]
[[!template text="Thank you very much." start="00:39:06.140" video="mainVideo" id=subtitle]]
[[!template text="captions by John Cummings" start="00:39:07.580" video="mainVideo" id=subtitle]]
