summaryrefslogblamecommitdiffstats
path: root/2023/captions/emacsconf-2023-parallel--parallel-text-replacement--lovro-valentino-picotti--main.vtt
blob: ba813a8115802318a14593479be70070de799d63 (plain) (tree)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































                                                             
WEBVTT captioned by Lovro

NOTE Introduction

00:00:00.000 --> 00:00:01.540
Hi everyone!

00:00:01.640 --> 00:00:04.540
Welcome to our talk on Parallel Text Replacement.

00:00:04.640 --> 00:00:06.940
My name is Lovro, and I'll be telling you about an

00:00:07.040 --> 00:00:09.260
interesting problem that my friend Valentino and I

00:00:09.360 --> 00:00:11.660
set out to solve one afternoon.

00:00:11.760 --> 00:00:13.580
We will describe the problem, take a look at some

00:00:13.680 --> 00:00:16.780
of the existing work and then present our solution.

00:00:16.880 --> 00:00:18.980
Afterwards, we will show some demos and conclude

00:00:19.080 --> 00:00:21.420
with a quick overview of the implementation.

00:00:21.520 --> 00:00:23.340
Let's get straight into it!

NOTE Problem: Goal

00:00:23.440 --> 00:00:25.700
Here is a problem that most of us have dealt with

00:00:25.800 --> 00:00:26.940
at some point.

00:00:27.040 --> 00:00:29.780
Assume we have a piece of code such as the following.

00:00:29.880 --> 00:00:32.420
We use a code example here, but in general what we're

00:00:32.520 --> 00:00:35.500
about to discuss can be applied to any piece of text.

00:00:35.600 --> 00:00:37.540
After a bit of thinking, we decide that the names of

00:00:37.640 --> 00:00:39.860
the two variables, "foo" and "bar", should actually be

00:00:39.960 --> 00:00:40.780
swapped.

00:00:40.880 --> 00:00:43.460
That is, "foo" should be replaced with "bar", and "bar"

00:00:43.560 --> 00:00:44.940
should be replaced with "foo".

00:00:45.040 --> 00:00:48.980
The question is: what is a good way to achieve this?

00:00:49.080 --> 00:00:51.660
We could perform the edits manually if the code is

00:00:51.760 --> 00:00:53.780
small enough, and we might even be done reasonably

00:00:53.880 --> 00:00:54.620
quickly.

00:00:54.720 --> 00:00:56.620
However, consider two things.

00:00:56.720 --> 00:00:58.860
Imagine the usual case where there's just too much

00:00:58.960 --> 00:01:00.660
code to edit by hand.

00:01:00.760 --> 00:01:03.580
We have no other option than to automate the task.

00:01:03.680 --> 00:01:06.020
More importantly though, we have a whole programmable

00:01:06.120 --> 00:01:08.180
text editor right at our fingertips.

00:01:08.280 --> 00:01:10.180
We should object to doing things that the computer

00:01:10.280 --> 00:01:12.260
can do for us.

NOTE Problem: Naive Multi-pass

00:01:12.360 --> 00:01:15.460
So, one way to automate it is by using our old friend

00:01:15.560 --> 00:01:18.940
query-replace (M-%) multiple times in a sequence.

00:01:19.040 --> 00:01:22.140
We first do a pass where we replace "foo" with "bar",

00:01:22.240 --> 00:01:25.540
then we do another pass where we replace "bar" with "foo".

00:01:25.640 --> 00:01:26.860
But that's clearly not right.

00:01:26.960 --> 00:01:29.060
We all know that this naive multi-pass approach

00:01:29.160 --> 00:01:31.460
doesn't work because it results in interference

00:01:31.560 --> 00:01:34.100
between the two replacements.

NOTE Problem: Clever Multi-pass

00:01:34.200 --> 00:01:36.700
Instead, we have to be a bit more clever.

00:01:36.800 --> 00:01:39.740
We should first replace "foo" with a temporary string,

00:01:39.840 --> 00:01:42.020
in this case "oof", that we will call a "token".

00:01:42.120 --> 00:01:45.380
To avoid interference, we must be careful to ensure

00:01:45.480 --> 00:01:48.020
that the token does not contain whatever we're about

00:01:48.120 --> 00:01:49.500
to replace next.

00:01:49.600 --> 00:01:52.620
Then we do a second pass to replace "bar" with "foo",

00:01:52.720 --> 00:01:55.980
and finally a third pass to replace the token with "bar".

00:01:56.080 --> 00:01:57.620
This gives us the result we want.

NOTE Problem: Terminology

00:01:57.720 --> 00:02:01.820
Putting the implementation aside for a moment, this style

00:02:01.920 --> 00:02:05.500
of text replacement, where we replace multiple sources

00:02:05.600 --> 00:02:08.940
with their targets, without running into interference

00:02:09.040 --> 00:02:11.660
issues between replacement pairs, is what we call

00:02:11.760 --> 00:02:12.740
a "parallel replacement".

00:02:12.840 --> 00:02:16.260
This is the essence of the problem we're trying to solve.

00:02:16.360 --> 00:02:18.580
The examples with swapping that we've shown so far

00:02:18.680 --> 00:02:21.220
are really just one of the many use cases that are

00:02:21.320 --> 00:02:23.740
supported by a general parallel replacement utility.

00:02:25.040 --> 00:02:28.660
To avoid confusion, let us clarify that the word "parallel"

00:02:28.760 --> 00:02:31.660
is not in reference to hardware parallelization, but

00:02:31.760 --> 00:02:34.780
rather comes from analogy with the Lisp let operator,

00:02:34.880 --> 00:02:38.060
where the bindings of variables are performed in parallel,

00:02:38.160 --> 00:02:40.100
rather than sequentially as in let*.

00:02:40.200 --> 00:02:43.580
Parallel in this context means that none of the bindings

00:02:43.680 --> 00:02:46.780
are in scope within any of the initial value forms.

00:02:46.880 --> 00:02:50.100
In other words, just like a let's initialization form

00:02:50.200 --> 00:02:53.620
cannot refer to any of the earlier bindings, a

00:02:53.720 --> 00:02:56.660
replacement pair's source should not be able to replace

00:02:56.760 --> 00:03:00.100
the previously substituted targets of any other pair.

00:03:00.200 --> 00:03:03.340
This is what we mean by "no interference".

NOTE Problem: Scaling Multi-pass

00:03:04.440 --> 00:03:07.900
However, manually invoking multiple carefully chosen

00:03:08.000 --> 00:03:11.420
query-replace commands gets old very quickly.

00:03:11.520 --> 00:03:14.100
Say we scaled up the problem and wanted to perform n

00:03:14.200 --> 00:03:18.220
swaps instead of just two, e.g. to swap, or rather,

00:03:18.320 --> 00:03:22.060
rotate, "foo" to "bar", "bar" to "baz", "baz" to "quux"

00:03:22.160 --> 00:03:23.700
and "quux" to "foo".

00:03:23.800 --> 00:03:26.260
We would first have to perform n - 1 additional

00:03:26.360 --> 00:03:29.140
replacements to introduce the necessary tokens,

00:03:29.240 --> 00:03:32.140
effectively doubling the number of steps.

00:03:32.240 --> 00:03:34.700
Even if we tried to automate this, think about what

00:03:34.800 --> 00:03:37.580
tokens the code would have to generate if we had no

00:03:37.680 --> 00:03:40.420
prior knowledge of the replacement pairs given by the

00:03:40.520 --> 00:03:41.460
user.

00:03:41.560 --> 00:03:44.060
We would have to program defensively and use long

00:03:44.160 --> 00:03:47.460
randomly-generated strings that, one, hopefully do

00:03:47.560 --> 00:03:50.180
not interfere with any of the replacement pairs,

00:03:50.280 --> 00:03:53.380
and two, might slow down the search if they're overly long.

00:03:53.480 --> 00:03:54.820
Can we do better?

NOTE Solution: Single-pass

00:03:55.920 --> 00:03:56.740
Yes we can!

00:03:56.840 --> 00:03:59.580
We can actually perform just a single pass.

00:03:59.680 --> 00:04:02.180
The trick is to alternate between the replacement

00:04:02.280 --> 00:04:05.900
pairs, replacing whichever source occurs the earliest,

00:04:06.000 --> 00:04:08.340
and making sure to continue scanning after the end

00:04:08.440 --> 00:04:12.180
of the substituted target in order to avoid interference.

00:04:12.280 --> 00:04:14.420
This interleaving of replacements is not something

00:04:14.520 --> 00:04:17.140
that's easy to do by hand with query-replace.

NOTE Solution: Existing

00:04:18.240 --> 00:04:20.860
Since this is Emacs we're talking about, of course

00:04:20.960 --> 00:04:23.460
there already exist solutions that implement this idea.

00:04:23.560 --> 00:04:25.860
Here are few that we could find.

00:04:25.960 --> 00:04:28.700
The EmacsWiki has a page dedicated to this problem.

00:04:28.800 --> 00:04:31.460
Stack Overflow has an old post where a couple of

00:04:31.560 --> 00:04:33.860
users provided their solutions.

00:04:33.960 --> 00:04:36.820
Mastering Emacs also gives a method along with other

00:04:36.920 --> 00:04:38.900
interesting query-replace-regexp (C-M-%) patterns.

00:04:39.000 --> 00:04:42.260
More recently, Tony Zorman made a blogpost providing

00:04:42.360 --> 00:04:44.980
a solution with an interface based on query-replace.

00:04:45.080 --> 00:04:47.540
I encourage you to take a look at these solutions if

00:04:47.640 --> 00:04:48.940
you're interested in the details.

00:04:50.040 --> 00:04:52.940
But while a step in the right direction, these solutions

00:04:53.040 --> 00:04:55.340
are not satisfactory because they all lack one or

00:04:55.440 --> 00:04:56.820
more of the following.

00:04:56.920 --> 00:04:59.900
One, they are not completely automated and require

00:05:00.000 --> 00:05:02.500
the user to come up with a relatively complicated

00:05:02.600 --> 00:05:05.380
and verbose query-replace-regexp invocation.

00:05:06.080 --> 00:05:08.940
Two, they are restricted to performing only 2-element

00:05:09.040 --> 00:05:11.780
swaps rather than general parallel replacements.

00:05:12.680 --> 00:05:15.060
Three, they don't provide any sort of interactivity

00:05:15.160 --> 00:05:17.820
during replacement and instead perform it in one shot.

00:05:18.620 --> 00:05:21.300
Four, they don't attempt to integrate with the familiar

00:05:21.400 --> 00:05:24.900
query-replace interface, which supports skipping, undo,

00:05:25.000 --> 00:05:28.340
history and more advanced features like Lisp expressions

00:05:28.440 --> 00:05:29.900
and recursive query edits.

00:05:30.700 --> 00:05:33.700
Most importantly however, five, none of them were

00:05:33.800 --> 00:05:36.380
designed with regular expressions in mind and instead

00:05:36.480 --> 00:05:38.460
only ever consider literal strings.

00:05:39.560 --> 00:05:43.060
In fact, the only one that comes close is the

00:05:43.160 --> 00:05:46.420
half-automated solution that invokes query-replace-regexp

00:05:46.520 --> 00:05:48.100
with a specially crafted replacement.

00:05:48.800 --> 00:05:51.660
As an example, here's how you would use this technique

00:05:51.760 --> 00:05:54.340
to perform a 3-element parallel regex replacement.

00:05:54.440 --> 00:05:57.740
It uses the backslash-comma Lisp expression feature

00:05:57.840 --> 00:06:01.180
in order to choose the appropriate target to substitute.

00:06:01.280 --> 00:06:03.700
Aside from being very clumsy and tedious to write out,

00:06:03.800 --> 00:06:06.860
this approach makes it really hard to use more complex

00:06:06.960 --> 00:06:09.260
regular expressions that make use of capture groups

00:06:09.360 --> 00:06:10.600
themselves.

00:06:10.800 --> 00:06:12.200
This was the biggest limitation that we wanted

00:06:12.200 --> 00:06:15.020
to get rid of and the main motivation for our work.

00:06:15.720 --> 00:06:19.820
So, as an alternative to the existing zoo of 80% solutions,

00:06:19.920 --> 00:06:24.140
we aim to provide a 100% solution, one that handles

00:06:24.240 --> 00:06:27.020
regexes and consolidates all of the existing ideas

00:06:27.120 --> 00:06:28.180
into a single package.

NOTE Solution: query-replace-parallel

00:06:29.080 --> 00:06:31.260
We call it query-replace-parallel.

00:06:31.360 --> 00:06:34.060
The package is free and open-source and can currently

00:06:34.160 --> 00:06:37.300
be found on GitHub under hokomo/query-replace-parallel.

00:06:37.400 --> 00:06:40.140
The name is not yet finalized and we're open to any

00:06:40.240 --> 00:06:41.170
suggestions.

00:06:41.503 --> 00:06:43.180
We hope to get it published on an Elisp

00:06:43.280 --> 00:06:45.780
package archive in the near future, but for now you

00:06:45.880 --> 00:06:48.300
can just download and load the main Elisp file manually.

00:06:48.900 --> 00:06:51.300
With all of that said, let's go through a few demos

00:06:51.400 --> 00:06:54.140
to illustrate some use cases and see how to use the package.

NOTE Demonstration: Swap

00:06:55.240 --> 00:06:57.460
Our first demo is a simple swap, like the one we

00:06:57.560 --> 00:06:59.140
showed at the beginning of the presentation.

00:06:59.240 --> 00:07:02.060
This chunk of text is actually one of the tests

00:07:02.160 --> 00:07:03.140
from our package's code.

00:07:03.840 --> 00:07:06.420
Assuming we have loaded  the package, we can execute

00:07:06.520 --> 00:07:09.580
the query-replace-parallel command, a parallel version

00:07:09.680 --> 00:07:11.220
of the standard query-replace.

00:07:11.320 --> 00:07:13.940
This command works with literal strings and will

00:07:14.040 --> 00:07:15.900
ask for each source and target in turn.

00:07:16.000 --> 00:07:21.260
Our goal is to replace "foo" with "bar"

00:07:21.360 --> 00:07:22.180
and "bar" with "foo".

00:07:24.680 --> 00:07:26.940
After inputting our replacements, we terminate the

00:07:27.040 --> 00:07:29.060
prompt by pressing enter with empty input.

00:07:29.860 --> 00:07:32.500
At this point, everything functions the same as in

00:07:32.600 --> 00:07:34.380
a standard query-replace invocation.

00:07:35.280 --> 00:07:37.300
The echo area shows the match and the replacement

00:07:37.400 --> 00:07:38.603
we're about to make.

00:07:38.703 --> 00:07:40.220
We can perform replacements,

00:07:43.920 --> 00:07:46.403
undo them,

00:07:46.503 --> 00:07:49.103
skip them,

00:07:49.203 --> 00:07:50.140
execute them until the end,

00:07:50.240 --> 00:07:51.020
and so on.

NOTE Demonstration: LaTeX

00:07:53.970 --> 00:07:56.180
The second demo shows our first regex use case.

00:07:56.280 --> 00:07:58.620
Imagine we have the following LaTeX code.

00:07:58.720 --> 00:08:01.380
We realize that we haven't been completely consistent

00:08:01.480 --> 00:08:03.940
in our use and naming of macros, so we decide to

00:08:04.040 --> 00:08:04.660
fix the problem.

00:08:05.536 --> 00:08:08.300
This time we execute query-replace-parallel-regexp

00:08:08.400 --> 00:08:10.900
because we want to work with regex instead of literal

00:08:11.000 --> 00:08:11.500
strings.

00:08:12.000 --> 00:08:13.420
We want to achieve two things.

00:08:13.520 --> 00:08:16.860
First, we want to wrap all usages of the variable n

00:08:16.960 --> 00:08:17.980
with the natvar macro.

00:08:18.080 --> 00:08:20.940
Using the backslash-less-than and blackslash-greater-than

00:08:21.040 --> 00:08:23.740
constructs allows us to only match letters n not

00:08:23.840 --> 00:08:25.260
appearing as part of a larger word.

00:08:25.360 --> 00:08:29.460
Second, we want to rename natvar to intvar because

00:08:29.560 --> 00:08:32.180
the variables a, b and c are integers and not natural

00:08:32.280 --> 00:08:32.700
numbers.

00:08:33.300 --> 00:08:35.660
We enter empty input to terminate the prompt and can

00:08:35.760 --> 00:08:37.180
now perform the replacements.

00:08:42.280 --> 00:08:44.380
There we go, the fixes are done and we didn't have

00:08:44.480 --> 00:08:46.100
to think about in which order to apply them.

NOTE Demonstration: Regex

00:08:48.700 --> 00:08:50.900
We now take a look at a more complicated regex

00:08:51.000 --> 00:08:53.580
example to demonstrate that even advanced query-replace

00:08:53.680 --> 00:08:54.300
features are supported.

00:08:55.100 --> 00:08:57.340
Each "foo" and "bar" in this example is followed by

00:08:57.440 --> 00:08:57.740
a number.

00:08:58.440 --> 00:09:01.280
The goal is to not only swap "foo" and "bar", but

00:09:01.380 --> 00:09:03.620
also increase or decrease the corresponding number.

00:09:03.720 --> 00:09:06.500
We first match "foo" and capture the number that

00:09:06.600 --> 00:09:07.100
follows it.

00:09:07.200 --> 00:09:09.900
For the target, we make use of the backslash-comma

00:09:10.000 --> 00:09:12.500
Lisp expression feature in order to replace the

00:09:12.600 --> 00:09:14.940
match with "bar" followed by the number's successor.

00:09:15.540 --> 00:09:17.540
We do the same thing for "bar", except that we

00:09:17.640 --> 00:09:19.140
replace the number with its predecessor.

00:09:27.040 --> 00:09:29.020
Performing the replacements, we can see how each

00:09:29.120 --> 00:09:31.220
number is incremented or decremented appropriately.

NOTE Demonstration: Order

00:09:36.320 --> 00:09:38.660
We haven't covered it explicitly so some of you may

00:09:38.760 --> 00:09:41.260
be wondering how parallel replacement deals with

00:09:41.360 --> 00:09:43.740
overlapping matches and whether the order of the

00:09:43.840 --> 00:09:45.380
replacement pairs is significant.

00:09:45.480 --> 00:09:47.860
This demo will clarify the exact behavior.

00:09:48.960 --> 00:09:51.700
The first example has the sources "watch" and "stopwatch".

00:09:57.500 --> 00:10:00.500
Conceptually, the matches overlap, but the rule is

00:10:00.600 --> 00:10:02.900
that matches are always processed earliest first,

00:10:03.000 --> 00:10:05.940
regardless of their length or the ordering of the pairs.

00:10:06.040 --> 00:10:08.980
Therefore it is "stopwatch" that gets replaced,

00:10:09.080 --> 00:10:10.940
and not its substring "watch".

00:10:16.040 --> 00:10:19.540
The second example uses the sources "watch" and "watchword".

00:10:19.640 --> 00:10:22.540
Both of the matches now conceptually start at the same

00:10:22.640 --> 00:10:23.020
position.

00:10:23.720 --> 00:10:26.300
In situations like these the order of the pairs does

00:10:26.400 --> 00:10:29.460
matter, and ties are broken by prefering the pair that

00:10:29.560 --> 00:10:32.180
was entered first, which is behavior that is inherited

00:10:32.280 --> 00:10:33.460
from the Elisp regex engine.

00:10:34.460 --> 00:10:37.380
So, the substring "watch" in "watchword" is what gets

00:10:37.480 --> 00:10:38.460
replaced in this case.

00:10:39.460 --> 00:10:41.740
Situations where the order of the pairs is significant

00:10:41.840 --> 00:10:44.740
are not very common however, so the user generally

00:10:44.840 --> 00:10:46.660
doesn't have to worry about this edge case.

00:10:46.760 --> 00:10:49.860
The order only matters when two or more sources

00:10:49.960 --> 00:10:52.340
share the same prefix, as in this example.

NOTE Demonstration: Fun

00:10:54.440 --> 00:10:56.860
The final demo tests the limits of the package and

00:10:56.960 --> 00:10:59.660
shows that it fully integrates with query-replace.

00:10:59.760 --> 00:11:02.940
It is really just for fun and can even serve as a

00:11:03.040 --> 00:11:04.140
small Emacs brainteaser.

00:11:04.240 --> 00:11:05.460
See if you can keep up!

00:11:06.360 --> 00:11:09.060
We open a directory and enter Writable Dired mode

00:11:09.160 --> 00:11:11.780
in order to rename the directories "foo" and "bar".

00:11:11.880 --> 00:11:14.660
Instead of doing it quickly by hand, we decide to

00:11:14.760 --> 00:11:17.260
show off and use query-replace-parallel-regexp.

00:11:17.360 --> 00:11:19.900
We enter our pairs and make use of the

00:11:20.000 --> 00:11:22.380
backslash-question-mark query edit feature.

00:11:25.080 --> 00:11:27.820
Now whenever we perform a replacement, the query

00:11:27.920 --> 00:11:30.740
edit makes Emacs stop and prompt us for additional

00:11:30.840 --> 00:11:32.180
input to use as the target.

00:11:36.680 --> 00:11:39.140
We confirm the renames and now enter the "bar-lib"

00:11:39.240 --> 00:11:41.900
directory in order to perform the same kind of

00:11:42.000 --> 00:11:43.900
replacement on "baz" and "quux".

00:11:44.500 --> 00:11:47.820
Rather than save time, we decide to be extra lazy

00:11:47.920 --> 00:11:48.820
and take the long route.

00:11:48.920 --> 00:11:52.220
We recall the first pair and initiate a recursive

00:11:52.320 --> 00:11:54.460
invocation of query-replace-parallel-regexp.

00:11:54.560 --> 00:11:57.020
We are now replacing the replacement.

00:12:01.020 --> 00:12:04.540
We apply our fixes and then do the same thing again

00:12:04.640 --> 00:12:05.870
with the second pair.

00:12:05.970 --> 00:12:07.500
Recall and recurse.

00:12:16.300 --> 00:12:19.860
We confirm the prompt and finally rename our directories.

00:12:25.360 --> 00:12:26.620
Wow, that really paid off.

NOTE Implementation

00:12:29.120 --> 00:12:31.380
Before we finish, a few quick words about the

00:12:31.480 --> 00:12:32.900
implementation for the curious.

00:12:33.300 --> 00:12:36.380
Both query-replace-parallel and query-replace-parallel-regexp

00:12:36.480 --> 00:12:39.140
delegate to the complex perform-replace function

00:12:39.240 --> 00:12:41.780
which is the workhorse of query-replace's interactive

00:12:41.880 --> 00:12:42.420
mechanism.

00:12:43.120 --> 00:12:45.420
The way we achieve multiple interleaved replacements

00:12:45.520 --> 00:12:49.020
is by providing perform-replace with a big "matcher regex"

00:12:49.120 --> 00:12:50.380
and a special replacement function.

00:12:50.480 --> 00:12:54.300
Essentially, a complex parallel replacement like this

00:12:54.400 --> 00:12:57.420
is transformed into a standard replacement like this.

00:12:57.520 --> 00:13:00.100
This is similar to the trick shown earlier in the

00:13:00.200 --> 00:13:00.780
presentation.

00:13:00.880 --> 00:13:03.820
Each source is put in its own capture group to allow

00:13:03.920 --> 00:13:06.340
the replacement function to determine which one matched

00:13:06.440 --> 00:13:08.380
and return the appropriate target.

00:13:08.980 --> 00:13:11.580
However, we now take care to support arbitrary

00:13:11.680 --> 00:13:13.380
regular expressions as sources.

00:13:13.480 --> 00:13:16.980
We achieve this by converting each source regex into

00:13:17.080 --> 00:13:19.820
an equivalent one for which we can guarantee that its

00:13:19.920 --> 00:13:22.820
capture groups will not clash with our matcher regex.

00:13:22.920 --> 00:13:25.900
Information about this conversion is stored, and

00:13:26.000 --> 00:13:28.220
once the replacement function is called it has

00:13:28.320 --> 00:13:30.260
enough data to apply the replacement from the

00:13:30.360 --> 00:13:32.020
viewpoint of the original regex.

00:13:32.720 --> 00:13:34.900
The regex transformation is reliable because it

00:13:35.000 --> 00:13:38.420
uses the rx library, allowing us to treat regexes

00:13:38.520 --> 00:13:41.940
as s-expressions and avoid any nasty manual parsing.

00:13:42.640 --> 00:13:46.540
In fact, rx itself is based on one of Olin Shivers'

00:13:46.640 --> 00:13:48.336
100% solutions:

00:13:48.436 --> 00:13:51.220
SRE, or the S-expression regex notation.

00:13:51.320 --> 00:13:54.340
We all stand on the shoulders of many giants, so

00:13:54.440 --> 00:13:56.500
let's strive to design good solutions that we can

00:13:56.600 --> 00:13:59.140
all benefit from, many years into the future!

00:13:59.240 --> 00:14:02.900
Finally, because query-replace's core is not completely

00:14:03.000 --> 00:14:06.060
customizable, we did have to sprinkle in some advice

00:14:06.160 --> 00:14:07.500
to get certain things working.

00:14:07.600 --> 00:14:11.060
This concerns only minor cosmetic fixes and not the

00:14:11.160 --> 00:14:13.940
core replacement functionality, but we have nontheless

00:14:14.040 --> 00:14:16.580
tried to do it in the simplest and least intrusive way

00:14:16.680 --> 00:14:17.140
possible.

NOTE End

00:14:18.740 --> 00:14:21.580
In conclusion, go download and play with the package.

00:14:21.680 --> 00:14:24.460
Even if you're not performing overlapping replacements,

00:14:24.560 --> 00:14:26.780
you can still use query-replace-parallel for the

00:14:26.880 --> 00:14:29.620
peace of mind knowing that things won't go wrong if

00:14:29.720 --> 00:14:31.860
you perform more than one replacement at a time.

00:14:32.460 --> 00:14:34.540
Feel free to let us know about any interesting or

00:14:34.640 --> 00:14:37.460
crazy use cases you might come up with, as well as

00:14:37.560 --> 00:14:40.540
improvements or bugs that make it only a 99% solution.

00:14:40.640 --> 00:14:45.560
Thanks for listening and have a great EmacsConf!