summaryrefslogblamecommitdiffstats
path: root/2023/captions/emacsconf-2023-overlay--improving-compiler-diagnostics-with-overlays--jeff-trull--main.vtt
blob: 9c948e2ba6f69a5a9744623b1881dfe9c7bf6640 (plain) (tree)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319






































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































                                                                                      
WEBVTT captioned by sachac, checked by sachac

NOTE Introduction

00:00.000 --> 00:00:04.897
Hi, I'm Jeff Trull, and today I'm going to talk to you

00:00:04.898 --> 00:00:08.460
about improving C++ compiler diagnostics

00:08.460 --> 00:13.600
using overlays and other features from Emacs.

00:13.600 --> 00:15.840
First an overview of my talk.

00:15.840 --> 00:00:17.656
I'm going to cover what overlays are

00:00:17.657 --> 00:00:19.325
and how you can use them in code,

00:00:19.326 --> 00:00:21.478
then I'm going to talk about C++

00:00:21.479 --> 00:00:24.480
and why its compiler errors can be so onerous.

00:24.480 --> 00:00:26.750
Finally, we'll take that information

00:00:26.751 --> 00:00:28.447
and build a new minor mode

00:00:28.448 --> 00:00:33.560
using overlays and other Emacs features.

NOTE Overlays and what they can do

00:33.560 --> 00:35.520
First of all, overlays.

00:35.520 --> 00:36.680
What are they?

00:36.680 --> 00:00:39.124
They are objects consisting of a buffer range

00:00:39.125 --> 00:00:40.400
and a set of properties.

00:40.400 --> 00:43.120
That means that they cover a region in a buffer.

00:43.120 --> 00:00:45.533
The properties can be a certain set

00:00:45.534 --> 00:00:47.344
of special property names,

00:00:47.345 --> 00:00:50.288
in which case they can be used to cause

00:00:50.289 --> 00:00:52.569
special effects in the buffer,

00:00:52.570 --> 00:00:55.660
but they never change the underlying text.

00:55.660 --> 00:59.900
You can use them for things like hiding things.

00:59.900 --> 00:01:02.886
So, for example, overlays are working right now

00:01:02.887 --> 00:01:04.660
in this window. `org-present`,

00:01:04.661 --> 00:01:07.595
the technology I'm using for this presentation,

00:01:07.596 --> 00:01:10.031
is hiding the asterisk before every headline,

00:01:10.032 --> 00:01:12.520
as well as the things called emphasis markers;

00:01:12.521 --> 00:01:16.269
that is, those things that make things look

00:01:16.270 --> 00:01:20.700
monospaced for verbatim, or italic, or bold.

01:20.700 --> 00:01:24.421
The special characters we use to mark off those sections

00:01:24.422 --> 00:01:28.940
are also hidden by `org-present` using overlays.

01:28.940 --> 00:01:30.601
But those things are still in the buffer

00:01:30.602 --> 00:01:31.980
and they're still visible to code.

01:31.980 --> 00:01:34.921
So if I run this little snippet of code down here,

00:01:34.922 --> 00:01:37.403
it's going to go up to the headline "Overlays

00:01:37.404 --> 00:01:40.051
and what they can do," and it's going to tell us

00:01:40.052 --> 00:01:41.540
what's there in the buffer.

01:41.540 --> 01:45.100
Let's go down and run this.

01:45.100 --> 00:01:48.957
So according to this code, the contents of the buffer

00:01:48.958 --> 00:01:51.990
to the left of the headline is a star in a space,

00:01:51.991 --> 00:01:55.204
which means that even though we can't see that star,

00:01:55.205 --> 00:01:58.220
it's still there, because it's hidden by an overlay.

01:58.220 --> 02:02.500
And that's kind of the essence of what overlays are.

NOTE Simple overlay example - creating an overlay

02:02.500 --> 02:04.780
Let's do a simple overlay example.

02:04.780 --> 00:02:06.719
We have some text on the right here,

00:02:06.720 --> 00:02:09.340
which is a famous poem by William Carlos Williams,

02:09.340 --> 02:12.180
which has been the subject of many memes.

02:12.180 --> 02:17.860
Let's create an overlay that covers it.

02:17.860 --> 02:20.700
I'll go down here and use this snippet of code here.

02:20.700 --> 00:02:25.918
We'll go up to the top, and we'll mark everything

00:02:25.919 --> 00:02:29.540
between `#+BEGIN_VERSE` and `#+END_VERSE`.

02:29.540 --> 00:02:33.276
You can see we've created an overlay

00:02:33.277 --> 00:02:35.700
from position 74 to 224.

NOTE Adding properties

02:35.700 --> 00:02:38.063
Now we can take that overlay that we already created

00:02:38.064 --> 00:02:41.211
and add a property, in this case a `face` property,

00:02:41.212 --> 00:02:43.540
to change the appearance of the text.

02:43.540 --> 00:02:46.279
This is a poem, and it's currently using

00:02:46.280 --> 00:02:48.083
a face that is monospaced,

00:02:48.084 --> 00:02:50.491
and so it looks like a computer program,

00:02:50.492 --> 00:02:51.900
even though it's a poem.

02:51.900 --> 00:02:54.585
I think it would be nicer to use something

00:02:54.586 --> 00:02:57.980
with variable-width font, maybe with some serifs.

02:57.980 --> 03:01.140
So let's give that a try.

03:01.140 --> 03:03.700
Now you can see that the poem looks quite a bit different.

03:03.700 --> 03:10.940
It looks more like what we'd see in a book.

NOTE Deleting an overlay

03:10.940 --> 03:13.100
We can also delete overlays.

03:13.100 --> 03:15.140
So I've named this one.

03:15.140 --> 00:03:17.765
So we can just go down and run `delete-overlay`

00:03:17.766 --> 00:03:20.048
and get rid of it, and it'll go back to

00:03:20.049 --> 00:03:22.660
the appearance it had before.

03:22.660 --> 03:23.660
And there it is.

03:23.660 --> 03:24.660
It's back to normal.

NOTE Setting fonts the right way

03:24.660 --> 00:03:28.473
Now, if you're interested in changing all of the verses

00:03:28.474 --> 00:03:31.108
inside an Org Mode file to a different face

00:03:31.109 --> 00:03:32.785
or a different font family,

00:03:32.786 --> 00:03:35.060
this isn't the way you'd really do it.

03:35.060 --> 03:37.520
I'll just show you that real quick.

03:37.520 --> 00:03:43.471
The right way is probably to change the `org-verse` face,

00:03:43.472 --> 00:03:48.868
which is the face used for all of the verse blocks

00:03:48.869 --> 00:03:51.620
inside your Org Mode file.

03:51.620 --> 03:55.100
And so this is how you do it here:

03:55.100 --> 03:56.100
`face-remap-add-relative`.

03:56.100 --> 03:58.340
Let's give it a try.

03:58.340 --> 03:59.340
It worked!

NOTE More properties

03:59.540 --> 00:04:01.805
There are more advanced things that you can do

00:04:01.806 --> 00:04:03.300
other than just changing fonts.

04:03.300 --> 00:04:05.543
There's a whole long list of them in the manual,

00:04:05.544 --> 00:04:12.580
but let's talk about the ones we're going to use today.

NOTE Visibility

04:12.580 --> 04:17.380
You can make text invisible, just like `org-present` did.

04:17.380 --> 04:21.820
The simplest way is to set the `invisible` property to true,

04:21.820 --> 04:24.500
so here's a code snippet that will do that.

04:24.500 --> 00:04:26.159
What we're going to do is

00:04:26.160 --> 00:04:28.966
go and find the word "plums" inside the poem,

00:04:28.967 --> 00:04:31.284
and then we're going to make it invisible

00:04:31.285 --> 00:04:33.436
by creating an overlay that covers it,

00:04:33.437 --> 00:04:36.820
and then setting the invisible property to true.

04:36.820 --> 04:37.940
Boom!

04:37.940 --> 04:38.940
It's gone.

04:38.940 --> 04:39.940
We've eaten the plums.

04:39.940 --> 04:42.180
Visibility is a huge topic and very complicated.

04:42.180 --> 04:44.220
There are powerful mechanisms for using it.

04:44.220 --> 00:04:46.626
I suggest reading the manual

00:04:46.627 --> 00:04:49.780
if you'd like to know more about that.

NOTE Adding text

04:49.780 --> 00:04:52.117
Another thing we can do with properties

00:04:52.118 --> 00:04:54.980
is to add text either before or after an overlay.

04:54.980 --> 00:04:57.347
Since we've made the word "plums" invisible,

00:04:57.348 --> 00:05:00.574
or anything that you make invisible in the buffer,

00:05:00.575 --> 00:05:02.662
if you add text then afterwards,

00:05:02.663 --> 00:05:05.700
it looks like you've replaced the original words

05:05.700 --> 05:08.220
with new words.

05:08.220 --> 00:05:12.046
So let's add a property, a `before-string` property,

00:05:12.047 --> 00:05:14.193
to the overlay that we used before

00:05:14.194 --> 00:05:17.137
to make it seem as though we're eating cherries

00:05:17.138 --> 00:05:18.180
instead of plums.

05:18.180 --> 05:19.180
Boom!

05:19.580 --> 05:22.020
There it is.

05:22.020 --> 05:27.820
So that's how you can replace words using overlays.

NOTE Custom properties

05:27.820 --> 00:05:29.760
You can also have custom properties

00:05:29.761 --> 00:05:31.700
that you name and then use yourself.

05:31.700 --> 05:35.320
For example, you can use it to mark regions in the buffer.

05:35.320 --> 00:05:38.008
You can also use it to add information

00:05:38.009 --> 00:05:41.180
to regions in the buffer for your own tracking

05:41.180 --> 05:45.380
in a minor mode or something like that, which we will use.

NOTE Notes on properties

05:45.380 --> 05:49.620
Finally, two notes on properties.

05:49.620 --> 00:05:51.950
We've been talking about overlay properties,

00:05:51.951 --> 00:05:54.540
but there's also something called text properties.

05:54.540 --> 05:57.460
Text properties are attached to text in a buffer.

05:57.460 --> 06:00.900
When you copy that text, the properties come along with it.

06:00.900 --> 00:06:03.056
If you modify the properties,

00:06:03.057 --> 00:06:05.500
the buffer is considered modified.

06:05.500 --> 06:08.460
Org Mode makes heavy use of text properties,

06:08.460 --> 00:06:11.677
as we can see by running this little code snippet here,

00:06:11.678 --> 00:06:14.060
which is going to tell us the properties

06:14.060 --> 00:06:16.565
and the string attached

00:06:16.566 --> 00:06:20.740
to the "Some poetry" headline on the right.

06:20.740 --> 06:23.660
There's also some controversy regarding performance.

06:23.660 --> 00:06:25.520
It may be that text properties

00:06:25.521 --> 00:06:27.860
perform better than overlay properties,

06:27.860 --> 00:06:28.892
so do some research

00:06:28.893 --> 00:06:31.060
if you're going to make heavy use of them.

06:31.060 --> 06:36.100
I prefer overlays because they're just easier to use.

NOTE Improving C++ compiler output

06:36.100 --> 06:37.540
C++ compiler output.

06:37.540 --> 00:06:41.170
So my day job is C++ programmer,

00:06:41.171 --> 00:06:46.560
and although I've been an Emacser for many years,

00:06:46.561 --> 00:06:52.860
it can be a little bit of a chore dealing with errors.

06:52.860 --> 00:06:55.680
The error messages that come out of the compiler

00:06:55.681 --> 00:06:57.580
can be pretty hard to understand.

06:57.580 --> 00:07:00.537
This has often been a barrier,

00:07:00.538 --> 00:07:04.640
particularly for people who are new to C++.

07:04.640 --> 07:09.040
So let's see what that's like.

07:09.040 --> 00:07:10.559
I have an example

00:07:10.560 --> 00:07:14.780
which is generously supplied by Ben Deane of Intel.

07:14.780 --> 00:07:17.082
So let's see what it looks like

00:07:17.083 --> 00:07:19.313
when you compile a C++ program

00:07:19.314 --> 00:07:24.400
that has a difficult error in it.

07:24.400 --> 07:27.400
Okay.

07:28.400 --> 07:31.400
Okay.

07:31.400 --> 07:35.680
So you see we have a lot of fairly verbose messages.

07:35.680 --> 07:39.400
The most verbose one I think is probably here.

07:39.400 --> 07:41.000
This one here.

07:41.000 --> 07:42.000
These are pretty bad.

07:42.000 --> 07:43.000
I think there might be bigger ones.

07:43.000 --> 00:07:43.720
Oh, yeah. Here we go.

00:07:43.721 --> 00:07:44.960
Here's my favorite one.

00:07:44.961 --> 00:07:51.063
You can see... Let's look for specialization... Basically,

00:07:51.064 --> 00:07:55.178
this whole section of the buffer here,

00:07:55.179 --> 00:07:58.228
that is specifying the specific types

00:07:58.229 --> 00:08:02.000
that a function template was instantiated with.

08:02.000 --> 08:04.000
And it's a lot there.

08:04.000 --> 00:08:05.473
So if you're trying to figure out

00:08:05.474 --> 00:08:06.817
what's wrong with your program

00:08:06.818 --> 00:08:08.884
and you're looking at something like this,

00:08:08.885 --> 00:08:11.000
it can be really, really hard to understand.

08:11.000 --> 08:12.000
Okay.

08:12.000 --> 08:17.680
Back to our presentation.

NOTE The problem with C++ error messages

08:17.680 --> 00:08:20.063
So it's often this way in C++

00:08:20.064 --> 00:08:23.400
because we compose types from other types.

08:23.400 --> 00:08:26.216
They can be long to begin with,

00:08:26.217 --> 00:08:30.240
but then a couple of other factors come into play.

NOTE Many standard class templates have defalut arguments

08:30.240 --> 08:33.280
First of all, we can have default template arguments.

08:33.280 --> 00:08:35.363
These are arguments you didn't write,

00:08:35.364 --> 00:08:37.008
but that are implicitly there

00:08:37.009 --> 00:08:38.325
and can sometimes refer

00:08:38.326 --> 00:08:40.300
to the arguments that you did write,

00:08:40.301 --> 00:08:42.440
which causes them to get a bit bigger,

00:08:42.441 --> 00:08:47.520
such as these allocator arguments here and here.

NOTE Some types are aliases for longer things, too

08:47.520 --> 08:49.360
Then there are type aliases.

08:49.360 --> 00:08:54.014
For example, `std::string` here expands to

00:08:54.015 --> 00:08:58.320
a type with three template arguments.

08:58.320 --> 00:09:01.940
So you can imagine, when we combine

00:09:01.941 --> 00:09:04.733
those two things together,

00:09:04.734 --> 00:09:09.763
our simple vector of maps from strings to ints

00:09:09.764 --> 00:09:14.257
becomes this humongous thing here, which...

00:09:14.258 --> 00:09:17.360
Let's run the comparison.

09:18.360 --> 09:20.960
Yeah.

NOTE Reporting type information accurately means long lines

09:20.960 --> 00:09:24.924
So in summary, to properly understand an error

00:09:24.925 --> 00:09:27.370
when you're a C++ programmer

00:09:27.371 --> 00:09:29.718
requires knowing the exact types

00:09:29.719 --> 00:09:32.280
that were supplied to your function.

09:32.280 --> 00:09:34.430
And types are built recursively,

00:09:34.431 --> 00:09:36.646
and therefore the types can--

00:09:36.647 --> 00:09:40.513
the correct exact name for the type

00:09:40.514 --> 00:09:42.776
can just be really huge

00:09:42.777 --> 00:09:46.360
and have many levels and layers to it.

09:46.360 --> 00:09:48.113
So when I was trying to understand

00:09:48.114 --> 00:09:49.466
the things I'd done wrong,

00:09:49.467 --> 00:09:52.401
especially when I was a newer C++ programmer,

00:09:52.402 --> 00:09:54.570
but honestly still even recently,

00:09:54.571 --> 00:09:57.440
if I was having a really intractable problem,

09:57.440 --> 00:10:00.123
I would just copy the entire error message out,

00:10:00.124 --> 00:10:01.735
stick it in the scratch buffer,

00:10:01.736 --> 00:10:03.649
and then manually reformat it

00:10:03.650 --> 00:10:05.563
so I could see what it was telling me

00:10:05.564 --> 00:10:07.261
I'd actually called the function

00:10:07.262 --> 00:10:09.320
or whatever it was with, the exact type.

10:09.320 --> 00:10:11.311
I had to sit there

00:10:11.312 --> 00:10:13.240
and go through the whole thing.

10:13.240 --> 10:15.240
But there's a better way.

10:15.240 --> 10:18.240
Now, anyway.

NOTE Emacs can help - Treat C++ type names as just another kind of balanced expression

10:18.240 --> 10:23.960
So what can Emacs do to help us with this problem?

10:23.960 --> 00:10:28.870
First of all, if you think about a type name,

00:10:28.871 --> 00:10:33.080
it's a lot like what we call S-expressions

10:33.080 --> 10:35.480
or balanced expressions.

10:35.480 --> 10:38.400
Lisp code itself is an S-expression.

10:38.400 --> 00:10:41.464
It's basically things with parentheses

00:10:41.465 --> 00:10:44.214
and little atoms or symbols in it,

00:10:44.215 --> 00:10:46.520
or strings or numbers.

10:46.520 --> 00:10:50.231
But parenthesized balanced expressions

00:10:50.232 --> 00:10:55.800
are things that Emacs was actually built to deal with.

10:55.800 --> 00:10:58.944
They were... I found an old manual from 1981,

00:10:58.945 --> 00:11:02.160
and the two major modes that they recommended

11:02.160 --> 00:11:05.765
or that they actually documented in the manual were

00:11:05.766 --> 00:11:08.400
one, assembly language, and two, Lisp.

11:08.400 --> 00:11:10.652
They mentioned that there were other modes,

00:11:10.653 --> 00:11:12.700
but they didn't say anything about them.

11:12.700 --> 00:11:14.625
So Lisp is something

00:11:14.626 --> 00:11:17.440
with a really long history with Emacs.

11:17.440 --> 00:11:19.976
Balanced expressions and manipulating them

00:11:19.977 --> 00:11:21.434
and doing them efficiently

00:11:21.435 --> 00:11:24.155
is just a thing that Emacs knows how to do,

00:11:24.156 --> 00:11:25.640
and Emacs is good at it.

11:25.640 --> 00:11:27.705
There's just a legacy

00:11:27.706 --> 00:11:31.320
of algorithms and functions for doing it.

11:31.320 --> 00:11:33.182
So we take types,

00:11:33.183 --> 00:11:37.839
and we take the angle brackets in the types,

00:11:37.840 --> 00:11:40.840
and we get the symbols right.

11:40.840 --> 00:11:41.814
Then we can treat them

00:11:41.815 --> 00:11:44.312
as though they were balanced expressions or S-expressions,

00:11:44.313 --> 00:11:49.320
the same kind that Emacs is really good at handling.

NOTE Add overlays to improve readability

11:49.320 --> 00:11:51.979
Secondly, we can use overlays

00:11:51.980 --> 00:11:55.260
to improve the readability of errors.

11:55.260 --> 00:11:58.012
We can take long lines and break and indent them

00:11:58.013 --> 00:12:00.160
using `before-string`s, so the same thing

12:00.200 --> 12:03.440
I used to add "cherries" into the poem.

12:03.440 --> 00:12:06.611
We can use that to insert new lines

00:12:06.612 --> 00:12:08.725
followed by indentation

00:12:08.726 --> 00:12:15.160
and produce a much nicer-looking listing of a type.

12:15.160 --> 00:12:19.641
We can also use the `invisible` property

00:12:19.642 --> 00:12:22.400
to hide unwanted detail.

NOTE Create a minor mode that runs during compilation

12:22.400 --> 12:24.960
Last of all, we can create a minor mode.

12:24.960 --> 00:12:27.854
When we're compiling things in Emacs,

00:12:27.855 --> 00:12:30.140
we often use `compilation-mode`.

12:30.140 --> 00:12:32.097
`compilation-mode` allows you to install

00:12:32.098 --> 00:12:33.553
compilation filters that run

00:12:33.554 --> 00:12:36.434
when the compiler is producing output,

00:12:36.435 --> 00:12:39.980
and at that time, then, we can add our overlays.

12:39.980 --> 00:12:42.868
We can also add in minor-mode commands

00:12:42.869 --> 00:12:45.757
that do whatever we want to the keymap.

00:12:45.758 --> 00:12:48.321
In this case, we're going to show and hide

00:12:48.322 --> 00:12:50.176
lower-level details interactively

00:12:50.177 --> 00:12:53.906
so that we can see a simplified version

00:12:53.907 --> 00:12:59.500
or a more detailed version of a type, depending on our needs.

NOTE Parsing types as balanced expressions

12:59.500 --> 13:03.980
First of all, parsing types as balanced expressions.

13:03.980 --> 00:13:05.686
We need to be able to quickly locate

00:13:05.687 --> 00:13:07.162
the boundaries and the contents

00:13:07.163 --> 00:13:08.500
of parenthesized expressions,

13:08.500 --> 13:12.100
or in this case, expressions in angle brackets.

13:12.100 --> 00:13:14.995
We use a syntax table inside Emacs

00:13:14.996 --> 00:13:18.800
to allow movement functions like `forward-list`

00:13:18.801 --> 00:13:21.100
to jump between matching angle brackets.

13:21.100 --> 13:23.460
By default, they're just parentheses.

13:23.460 --> 13:25.900
First of all, let's look at our syntax table.

13:25.900 --> 00:13:29.189
We're going to add here syntax entries

00:13:29.190 --> 00:13:33.900
to handle angle brackets as though they were parentheses.

13:33.900 --> 00:13:37.247
Then we have a lot of types

00:13:37.248 --> 00:13:42.980
that have colons in them, and those are namespaces in C++.

13:42.980 --> 00:13:45.766
By default, Emacs does not recognize them

00:13:45.767 --> 00:13:49.134
as parts of symbols, so we're going to tell Emacs

00:13:49.135 --> 00:13:52.839
that a colon is something called a symbol constituent,

00:13:52.840 --> 00:13:54.860
that it can be part of a name.

13:54.860 --> 00:13:57.613
Once we do that, then we can use our functions

00:13:57.614 --> 00:13:59.442
like `forward-list`, `backward-word`,

00:13:59.443 --> 00:14:03.288
all of the navigation and movement functions that we have

00:14:03.289 --> 00:14:06.623
that do things, that do more complicated things

00:14:06.624 --> 00:14:08.707
like S-expressions and so on,

00:14:08.708 --> 00:14:11.485
can be used now with our angle brackets

00:14:11.486 --> 00:14:16.100
and inside of our types.

NOTE Indent and fill with overlays - Use ancient "pretty printing" algorithms"

14:16.100 --> 00:14:18.462
The next thing we can do is

00:14:18.463 --> 00:14:21.540
perform indent and fill with overlays.

14:21.540 --> 00:14:23.735
We're going to use `before-string` properties

00:14:23.736 --> 00:14:25.630
to break lines and create indentation

00:14:25.631 --> 00:14:28.900
to make the output look a little better.

14:28.900 --> 14:35.320
Today, we fill mostly text and we indent mostly code.

14:35.320 --> 00:14:37.307
We fill text in order to prevent it

00:14:37.308 --> 00:14:39.902
from running off the side of the right margin,

00:14:39.903 --> 00:14:43.940
and we indent code to line up syntactic elements.

14:43.940 --> 14:47.080
Back in the day, they had algorithms that could do both.

14:47.080 --> 14:52.260
Those are what we're going to leverage.

NOTE Overlays can mimic line breaks and indentation

14:52.260 --> 00:14:54.582
We can use the `before-string` property

00:14:54.583 --> 00:14:57.760
to insert a new line in the correct number of spaces

14:57.760 --> 15:00.240
to emulate indentation.

15:00.240 --> 00:15:03.525
As a simplified example, here's some code

00:15:03.526 --> 00:15:07.280
that will indent 4 upon each open angle bracket.

15:07.280 --> 15:14.520
Let's give it a try.

NOTE Hiding details - Marking depths with overlays

15:14.520 --> 15:18.280
The next thing we're going to need to do is hide details.

15:18.280 --> 00:15:22.688
So we have nested types, and the user is going to want to

00:15:22.689 --> 00:15:27.371
be able to reveal lower-level or hide lower-level parts

00:15:27.372 --> 00:15:30.131
of the nested type interactively

00:15:30.132 --> 00:15:35.480
once we've already reformatted the error messages.

15:35.480 --> 15:40.440
Let's see how we can do that using invisible properties.

15:40.440 --> 00:15:43.992
The first thing we're going to do is

00:15:43.993 --> 00:15:46.680
mark depths within the type.

15:46.680 --> 00:15:49.328
When we're originally analyzing and formatting

00:15:49.329 --> 00:15:51.920
and doing the indentation and the line breaks,

15:51.920 --> 00:15:55.071
at the same time, we're going to go through

00:15:55.072 --> 00:15:58.817
and mark the nested levels inside the type names,

00:15:58.818 --> 00:16:00.840
just as this diagram shows.

16:00.840 --> 00:16:03.573
So depth 1, for example, will be everything

00:16:03.574 --> 00:16:06.120
inside the first level of angle brackets.

16:06.120 --> 00:16:09.038
Depth 2 will be everything inside the second level,

00:16:09.039 --> 00:16:09.600
and so on.

16:09.760 --> 00:16:12.070
And then later on, when the users request it,

00:16:12.071 --> 00:16:16.303
we can go and look at the depth that they've selected

00:16:16.304 --> 00:16:19.360
and then mark those sections invisible.

16:19.360 --> 16:20.520
Let's see how that might work.

16:20.520 --> 00:16:24.022
First of all, let's delete the overlays

00:16:24.023 --> 00:16:28.400
that we already have that created the indentation.

16:28.400 --> 00:16:32.419
Now we're going to go and do that marking

00:16:32.420 --> 00:16:35.740
with the custom depth properties here.

16:35.740 --> 00:16:38.760
To prove that I didn't pull a fast one,

00:16:38.761 --> 00:16:42.082
let's go and see what `describe-char` tells us

00:16:42.083 --> 00:16:44.660
about the depths inside here.

16:44.660 --> 16:46.460
Let's start here.

16:46.460 --> 16:52.820
Okay, so inside this part here, `std::string`,

16:52.820 --> 16:54.980
There are two overlays.

16:54.980 --> 00:16:57.780
One of them is of depth 1, and the other is of depth 2,

00:16:57.781 --> 00:17:00.601
which makes sense, because depth 1 is going to be

00:17:00.602 --> 00:17:02.011
from about here to here,

00:17:02.012 --> 00:17:07.660
and depth 2 is going to be from about here to this area.

17:07.660 --> 00:17:10.829
So it's reasonable that there should be two,

00:17:10.830 --> 00:17:12.660
and that's what we expect.

NOTE Hiding to a target depth

17:12.660 --> 00:17:17.353
Now that we've marked the nested types with their depths,

00:17:17.354 --> 00:17:21.380
let's experiment with hiding details.

17:21.380 --> 00:17:26.773
This fragment of code takes a user-supplied depth,

00:17:26.774 --> 00:17:29.085
in this case 2, and will hide,

00:17:29.086 --> 00:17:30.875
based on those markings

00:17:30.876 --> 00:17:33.932
that we've already made on the overlays,

00:17:33.933 --> 00:17:36.020
the custom depth properties.

17:36.020 --> 17:40.020
We'll take those and apply your requested level of detail.

17:40.020 --> 17:42.020
So let's try it out.

17:42.020 --> 17:43.020
Depth 2.

17:43.020 --> 00:17:46.005
All right, that hid everything under the `std::map`,

00:17:46.006 --> 00:17:47.260
so the deepest level.

17:47.260 --> 17:52.140
If we make it 1, we should get a level higher than that.

17:52.140 --> 17:54.540
So now level 1 and below are hidden.

17:54.540 --> 17:59.660
Now if we put it back to 3, it should reveal everything.

17:59.660 --> 18:04.900
So that's what we're going to use in our minor mode.

NOTE Demo

18:04.900 --> 18:05.900
Let's have a demo.

18:05.900 --> 00:18:08.538
We're going to revisit the initial example

00:18:08.539 --> 00:18:10.380
with the minor mode installed.

18:10.380 --> 00:18:12.101
Now we're going to have a compilation filter

00:18:12.102 --> 00:18:13.593
that will run on every chunk of output

00:18:13.594 --> 00:18:15.780
produced by the compiler.

18:15.780 --> 00:18:17.849
It's going to add those overlays

00:18:17.850 --> 00:18:20.420
with the line breaks and the indentation.

18:20.420 --> 00:18:22.206
It's also going to add overlays

00:18:22.207 --> 00:18:23.880
that mark up the nested types

00:18:23.881 --> 00:18:26.220
with the depths for each region.

18:26.220 --> 18:31.580
Let's add the hook for `tspew-mode`.

18:31.580 --> 18:37.220
And now we can compile again.

18:38.220 --> 00:18:41.503
All right, we can already see

00:18:41.504 --> 00:18:47.195
that these things are formatted a little bit better

00:18:47.196 --> 00:18:49.180
than they were before.

18:49.180 --> 18:50.180
They're not all on one line.

18:50.180 --> 18:53.580
Things are getting kind of lined up here.

18:53.580 --> 19:05.620
Here's a good example.

19:05.620 --> 00:19:08.637
And here's our big ugly one from before

00:19:08.638 --> 00:19:10.900
with all the characters in it.

19:10.900 --> 19:14.500
Let's try hiding some of this information.

19:14.500 --> 00:19:17.431
We'll just slowly decrease the level of detail

00:19:17.432 --> 00:19:19.740
and you can see how it works.

19:19.740 --> 00:19:22.333
Over here, where there's these ellipses

00:19:22.334 --> 00:19:25.460
next to string constant, the "..." there,

19:25.460 --> 00:19:30.386
that's where we are starting to hide information

00:19:30.387 --> 00:19:32.900
and go to the next level.

19:32.900 --> 19:36.460
Hiding more, hiding more, hiding more.

19:36.460 --> 19:38.220
Now we can go back and start adding it back.

19:38.220 --> 00:19:42.736
You can see here now we just have about four layers,

00:19:42.737 --> 00:19:45.540
which is a lot easier to understand.

19:45.540 --> 00:19:47.733
And if we start understanding what it is

00:19:47.734 --> 00:19:52.180
and we need more detail, we can just increase detail again.

19:52.180 --> 00:19:55.402
And every time we increase or decrease detail,

00:19:55.403 --> 00:19:58.900
it reformats so it still stays kind of consolidated

19:58.900 --> 19:59.900
and nice looking.

19:59.900 --> 20:01.980
Let's increase it a little bit more.

20:02.060 --> 20:04.540
Okay, so you can see how that worked.

20:04.540 --> 20:08.340
Let's go back to our presentation.

20:08.340 --> 20:10.220
All right.

NOTE Conclusion

20:10.220 --> 00:20:12.996
In conclusion, we saw how we could solve

00:20:12.997 --> 00:20:15.367
a real problem for C++ programmers

00:20:15.368 --> 00:20:18.534
by combining several Emacs features: overlays,

00:20:18.535 --> 00:20:20.489
compilation mode extensions,

00:20:20.490 --> 00:20:25.700
and balanced expression navigation using syntax tables.

20:25.700 --> 00:20:27.978
Emacs is often compared unfavorably

00:20:27.979 --> 00:20:31.460
to newer IDEs and editors with slicker user interfaces.

20:32.220 --> 00:20:36.386
What Emacs has that they don't is powerful abstractions,

00:20:36.387 --> 00:20:38.862
tons of libraries, and decades of work

00:20:38.863 --> 00:20:42.100
by some of the luminaries in the field of software.

20:42.100 --> 00:20:45.343
I think that this project would have been much harder to do

00:20:45.344 --> 00:20:48.020
in a prettier but less powerful environment.

20:48.020 --> 20:50.860
In short, there's plenty of hope for Emacs.

20:50.860 --> 20:51.220
Thank you.