-hello everyone my name is toniang +Hello, everyone! My name is Tuấn-Anh.
-I've been using amax for about 10 years +I've been using Emacs for about 10 years.
-today I'm going to talk about 360 +Today, I'm going to talk about tree-sitter,
-a new imax package that allows ems to +00:00:09.280 --> 00:00:11.351 +a new Emacs package that allows Emacs
-pass multiple programming languages -
-in real time +00:00:11.351 --> 00:00:17.840 +to parse multiple programming languages +in real-time.
-so what is the problem statement -
-in order to support programming +So what is the problem statement?
-functionalities for a particular +00:00:21.840 --> 00:00:24.131 +In order to support programming +functionalities
-language +00:00:24.131 --> 00:00:25.760 +for a particular language,
a text editor needs to have some degree
-of language understanding +of language understanding.
-traditionally text editors have relied +Traditionally, text editors have relied
+00:00:31.840 --> 00:00:34.960 very heavily on regular expressions for +this.
-this -
-e-max is no different most language -
-major modes use -
-regular expressions for syntax -
-highlighting code navigation -
-folding indexing and so on regular -
-expressions are +00:00:34.960 --> 00:00:37.013 +Emacs is no different.
-problematic for a couple of reasons +00:00:37.013 --> 00:00:40.170 +Most language major modes use regular +expressions
-they're slow and inaccurate they also +00:00:40.170 --> 00:00:42.960 +for syntax-highlighting, code navigation,
-make +00:00:42.960 --> 00:00:46.618 +folding, indexing, and so on.
-the code hard to read and write +00:00:46.618 --> 00:00:50.559 +Regular expressions are problematic for +a couple of reasons.
-sometimes +00:00:50.559 --> 00:00:53.778 +They're slow and inaccurate.
-it's because the regular expressions +00:00:53.778 --> 00:00:56.800 +They also make the code hard to read and +write.
-themselves are very hairy +00:00:56.800 --> 00:01:01.199 +Sometimes it's because the regular +expressions themselves are very hairy,
+00:01:01.199 --> 00:01:05.199 and sometimes because they are just not +powerful enough.
-powerful enough -
-some helper code is usually needed to +00:01:05.199 --> 00:01:08.625 +Some helper code is usually needed
-pass more intricate language features +00:01:08.625 --> 00:01:11.200 +to parse more intricate language +features.
-that also illustrates the core problem +00:01:11.200 --> 00:01:16.159 +That also illustrates the core problem +with regular expressions,
-with regular expressions -
+00:01:16.159 --> 00:01:21.119 in that they are not powerful enough to +parse programming languages.
-pass programming languages -
-an example feature that regular -
+00:01:21.119 --> 00:01:25.040 +An example feature that regular expressions cannot handle very well
-is string interpolation which is a very -
+00:01:25.040 --> 00:01:28.320 +is string interpolation, which is a very common feature
-in many modern programming languages +in many modern programming languages.
-it would be much nicer if image somehow +It would be much nicer if Emacs somehow
+00:01:34.079 --> 00:01:39.520 had structural understanding of source +code, like IDEs do.
-code -
-like ides do -
-there have been multiple efforts to +00:01:39.520 --> 00:01:41.981 +There have been multiple efforts
-bring this kind of programming language -
-understanding into Emacs +00:01:41.981 --> 00:01:45.280 +to bring this kind of programming +language understanding into Emacs.
-there are language specific persons +There are language-specific parsers
-written in elise -
-they can be thought of as the next +written in Elisp
-logical step of the glue code on top +00:01:48.640 --> 00:01:50.675 +that can be thought of
-of tribal expressions moving from +00:01:50.675 --> 00:01:51.989 +as the next logical step +of the glue code
-partial local +00:01:51.989 --> 00:01:53.856 +on top of regular expressions,
-pattern recognition into a full-fledged +00:01:53.856 --> 00:01:57.356 +moving from partial local pattern +recognition
-parser +00:01:57.356 --> 00:01:59.840 +into a full-fledged parser.
-the most prominent example of this +00:01:59.840 --> 00:02:02.023 +The most prominent example of this +approach
-approach is probably the famous -
-js2 mode +00:02:02.023 --> 00:02:06.479 +is probably the famous js2-mode.
-however this approach has several issues -
-parsing is computationally expensive and +However, this approach has several issues.
-imagine +00:02:10.080 --> 00:02:12.606 +Parsing is computationally expensive,
-is not good at that kind of stuff +00:02:12.606 --> 00:02:16.800 +and Emacs Lisp is not good at that kind +of stuff.
-furthermore maintenance is very +00:02:16.800 --> 00:02:19.156 +Furthermore, maintenance is very +troublesome.
-troublesome in order to work on these +00:02:19.156 --> 00:02:22.160 +In order to work on these parsers,
-process +00:02:22.160 --> 00:02:24.239 +first, you have to know Elisp +well enough,
-first you have to know at least well +00:02:24.239 --> 00:02:26.606 +and then you have to be comfortable with
-enough and then you have to be +00:02:26.606 --> 00:02:29.739 +writing a recursive descending parser,
-comfortable with writing a +00:02:29.739 --> 00:02:34.000 +while constantly keeping up with changes +to the language itself,
-recursive ascendant parser while +00:02:34.000 --> 00:02:36.356 +which can be evolving very quickly,
-constantly keeping up with changes to +00:02:36.356 --> 00:02:39.360 +like Javascript, for example.
-the language itself +00:02:39.360 --> 00:02:42.373 +Together, these constraints +significantly reduce
-which can be evolving very quickly like -
-javascript for example -
-together these constraints significantly -
-reduce the pull of potential maintenance +00:02:42.373 --> 00:02:45.680 +the pool of potential maintainers.
-the biggest issue though in my opinion -
-is lack of the set of generic +The biggest issue, though, in my opinion,
-and reusable apis this makes them very +00:02:47.760 --> 00:02:52.139 +is lack of the set of generic and +reusable APIs.
-hard to use +00:02:52.139 --> 00:02:54.319 +This makes them very hard to use
for minor modes that want to deal with
+00:02:55.920 --> 00:02:59.920 cross-cutting concerns across multiple -
-languages +languages.
-the other approach which has been -
-gaining a lot of momentum in recent +The other approach which has been
-years +00:03:01.760 --> 00:03:04.319 +gaining a lot of momentum +in recent years
is externalizing language understanding
-to another process +to another process,
-also known as language server protocol +also known as language server protocol.
-this second approach is actually a very -
-interesting one +00:03:12.239 --> 00:03:16.560 +This second approach is actually a very +interesting one.
-my decoupling language understanding +By decoupling language understanding
-from the editing facility itself -
-the usb servers can attract a lot more +from the editing facility itself,
-contributors +00:03:21.280 --> 00:03:25.120 +the LSP servers can attract a lot more +contributors,
-which makes maintenance easier however +00:03:25.120 --> 00:03:27.189 +which makes maintenance easier.
-they also have several issues available +00:03:27.189 --> 00:03:32.400 +However, they also have several issues +of their own.
-being a separate process they are +00:03:32.400 --> 00:03:34.089 +Being a separate process,
-usually more resource +00:03:34.089 --> 00:03:37.073 +they are usually more +resource-intensive,
-intensive and depending on the language +00:03:37.073 --> 00:03:39.920 +and depending on the language,
-the usb server itself can bring with it +the LSP server itself can bring with it
a host of additional dependencies
-external to Emacs which may message to +00:03:44.640 --> 00:03:50.640 +external to Emacs, which may be messy to +install and manage.
-install and manage -
-furthermore json over rpc has pretty -
-high latency +00:03:50.640 --> 00:03:55.120 +Furthermore, JSON over RPC has pretty +high latency.
-for one-off tasks like jumping to source +For one-off tasks like jumping to source
-or on-demand completion is great +or on-demand completion, it's great.
-but for things like code highlighting +But for things like code highlighting,
-the latency is just too much +the latency is just too much.
-I was using rust and I was following the +I was using Rust and I was following the
-community effort to improve its id +00:04:08.319 --> 00:04:11.760 +community effort to improve its +IDE support,
-support -
+00:04:11.760 --> 00:04:15.760 hoping to integrate some of that into +Emacs itself.
-Emacs itself -
-then I heard someone from community -
-mention tree sitter +00:04:15.760 --> 00:04:19.759 +Then I heard someone from the community +mention tree-sitter,
-and I decided to check it out -
-basically trisita is an incremental +and I decided to check it out.
-parsing library and a parser generator +00:04:23.360 --> 00:04:28.720 +Basically, tree-sitter is an incremental +parsing library and a parser generator.
-it was introduced by the item editor in +00:04:28.720 --> 00:04:33.040 +It was introduced by the Atom editor in +2018.
-2018 +00:04:33.040 --> 00:04:35.923 +Besides Atom, it is also being +integrated
-besides item is also being integrated +00:04:35.923 --> 00:04:37.623 +into the NeoVim editor,
-into the neo-vim +00:04:37.623 --> 00:04:41.040 +and Github is using it to power
-editor and github is using it to power +00:04:41.040 --> 00:04:42.423 +their source code analysis
-their source code analysis and -
-navigation features +00:04:42.423 --> 00:04:45.840 +and navigation features.
-it is written in c and can be compiled -
-for all +It is written in C and can be compiled
-major platforms it can even be compiled +00:04:48.639 --> 00:04:50.623 +for all major platforms.
-to web assembly to run on the web that's +00:04:50.623 --> 00:04:53.120 +It can even be compiled
-how github is using it +00:04:53.120 --> 00:04:55.323 +to web assembly to run on the web.
-on their website +00:04:55.323 --> 00:05:00.800 +That's how Github is using it +on their website.
-so why is trisita an interesting +00:05:00.800 --> 00:05:05.840 +So why is tree-sitter an interesting +solution to this problem?
-solution to this problem +00:05:05.840 --> 00:05:10.000 +There are multiple features that make it +an attractive option.
-there are multiple features that make it +00:05:10.000 --> 00:05:11.839 +It is designed to be fast.
-an attractive option -
-it is designed to be fast by being -
-incremental +00:05:11.839 --> 00:05:13.680 +By being incremental,
-the initial parts of a typical big fight +the initial parse of a typical big file
-can take tens of milliseconds +can take tens of milliseconds,
while subsequent incremental processes
-are sub milliseconds +are sub-millisecond.
-it achieves this by using structural -
-sharing +00:05:22.560 --> 00:05:26.240 +It achieves this by using +structural sharing,
meaning replacing only affected nodes
-in the old tree when it needs to -
-also unlike lsp being in the same +in the old tree when it needs to.
-process +00:05:32.960 --> 00:05:37.120 +Also, unlike LSP, being in +the same process,
-it has much lower latency +it has much lower latency.
-secondly it provides a uniform -
-programming interface +00:05:40.639 --> 00:05:44.960 +Secondly, it provides a uniform +programming interface.
-the same data structures and functions +The same data structures and functions
+00:05:47.039 --> 00:05:50.400 work on parse trees of different -
-languages +languages.
-syntax knows of different languages +Syntax nodes of different languages
differ only by their types
-and their possible child nodes this -
-is a big advantage over language -
-specific parcels +00:05:54.160 --> 00:05:55.723 +and their possible child nodes.
-thirdly it's written in self-contained +00:05:55.723 --> 00:06:02.240 +This is a big advantage over +language-specific parsers.
-embeddable c +00:06:02.240 --> 00:06:06.880 +Thirdly, it's written in self-contained +embeddable C.
-as I mentioned previously it can even be +00:06:06.880 --> 00:06:11.723 +As I mentioned previously, it can even +be compiled to webassembly.
-compiled +00:06:11.723 --> 00:06:16.106 +This makes integrating it into various +editors quite easy
-to webassembly this makes integrating it +00:06:16.106 --> 00:06:22.880 +without having to install any external +dependencies.
-into various editors +00:06:22.880 --> 00:06:25.503 +One thing that is not mentioned here
-quite easy without having to install -
-any external dependencies -
-one thing that is not mentioned here is -
-that being a parcel generator +00:06:25.503 --> 00:06:28.000 +is that being a parser generator,
-scrummers are declarative +its grammars are declarative.
-together with being editor independent +Together with being editor-independent,
+00:06:34.880 --> 00:06:39.139 this makes the pool of potential +contributors much larger.
-contributors -
-much larger so I was convinced -
-that trisito is a good fit for Emacs +00:06:39.139 --> 00:06:45.520 +So I was convinced that tree-sitter is a +good fit for Emacs.
-last year I started writing the bindings -
-using -
-dynamic model support introduced in imax -
-25. +Last year, I started writing the bindings
-dynamic module means there is platform +00:06:48.000 --> 00:06:53.280 +using dynamic module support introduced +in Emacs 25.
-specific native code involved +00:06:53.280 --> 00:06:58.479 +Dynamic module means there is +platform-specific native code involved,
-but since they are pre-compiled binaries +but since there are pre-compiled binaries
-for the three major platforms +for the three major platforms,
-it should work in most places currently +00:07:02.880 --> 00:07:04.706 +it should work in most places.
-the core functionalities are in a pretty -
-good shape +00:07:04.706 --> 00:07:09.440 +Currently, the core functionalities are +in a pretty good shape.
-syntax highlighting is working nicely -
-the whole thing is split into three -
-packages +Syntax highlighting is working nicely.
-tree sitter is the main package that +00:07:12.560 --> 00:07:16.080 +The whole thing is split into three +packages.
-other packages should depend on +00:07:16.080 --> 00:07:20.319 +tree-sitter is the main package that +other packages should depend on.
-tree system lens is the language bundle +tree-sitter-langs is the language bundle
that includes support
-for most common languages +for most common languages.
-and finally the core apis are in the -
-package tsc +00:07:27.199 --> 00:07:32.160 +And finally, the core APIs are in the +package tsc,
-which stands for trees the core +which stands for tree-sitter-core.
-it is the implicit dependency of the +It is the implicit dependency of the
-three-seater package -
-the main package includes the miner mode -
-3-seater mode -
-this provides the base for other major +tree-sitter package.
-or minor modes to build on +00:07:43.520 --> 00:07:47.520 +The main package includes the minor mode +tree-sitter-mode.
-using image change tracking hooks it +00:07:47.520 --> 00:07:52.560 +This provides the base for other major +or minor modes to build on.
-enables +00:07:52.560 --> 00:07:54.839 +Using Emacs's change tracking hooks,
-incremental parsing and provides a +00:07:54.839 --> 00:07:57.073 +it enables incremental parsing
-syntax tree that is always up to date +00:07:57.073 --> 00:08:00.800 +and provides a syntax tree that is +always up to date
-after any edits in a buffer +after any edits in a buffer.
-there is also a basic debug mode that +00:08:04.080 --> 00:08:06.223 +There is also a basic debug mode
-shows the parse tree in another buffer +00:08:06.223 --> 00:08:10.080 +that shows the parse tree in +another buffer.
-here is a quick demo +Here is a quick demo.
-here I mean an empty python buffer with +00:08:13.360 --> 00:08:15.673 +Here I'm in an empty Python buffer
-three seater enabled +00:08:15.673 --> 00:08:17.520 +with tree-sitter enabled.
I'm going to turn on the debug mode to
-see the parse tree +see the parse tree.
-since the buffer is empty there is only +00:08:26.560 --> 00:08:28.106 +Since the buffer is empty,
-one node in the syntax tree the top +00:08:28.106 --> 00:08:30.423 +there is only one node in the +syntax tree:
-level module node +00:08:30.423 --> 00:08:33.279 +the top-level module node.
-let's try typing some code +Let's try typing some code.
-as you can see as I type into the python -
-buffer +00:09:11.040 --> 00:09:14.640 +As you can see, as I type into the +Python buffer,
-the syntax tree updates in real time -
-the other minor mode included in the +the syntax tree updates in real time.
-main package is 3-seater +00:09:19.120 --> 00:09:22.039 +The other minor mode included in the +main package
-hl mode it overrides font-lock mode and +00:09:22.039 --> 00:09:24.389 +is tree-sitter-hl-mode.
-provides its own set of phases +00:09:24.389 --> 00:09:26.349 +It overrides font-lock mode
-and customization options it is query +00:09:26.349 --> 00:09:28.480 +and provides its own set of phases
-driven +00:09:28.480 --> 00:09:30.139 +and customization options
-that means instead of regular +00:09:30.139 --> 00:09:32.800 +It is query-driven.
-expressions +00:09:32.800 --> 00:09:36.240 +That means instead of regular +expressions,
-it uses a list like query language to +00:09:36.240 --> 00:09:39.518 +it uses a Lisp-like query language
-map syntax notes +00:09:39.518 --> 00:09:40.320 +to map syntax nodes
-to highlighting phrases I'm going to +00:09:40.320 --> 00:09:41.923 +to highlighting phrases.
-open a python file with small snippets +00:09:41.923 --> 00:09:45.760 +I'm going to open a python file with +small snippets
-that showcase syntax highlighting +that showcase syntax highlighting.
-so this is the default highlighting +So this is the default highlighting
-provided by python mode -
-this is the highlighting enabled by tree +provided by python-mode.
-sitter +00:10:00.880 --> 00:10:04.640 +This is the highlighting enabled +by tree-sitter.
as you can see string interpolation -- cgit v1.2.3