diff options
Diffstat (limited to '')
| -rw-r--r-- | 2024/talks/p-search.md | 176 |
1 files changed, 176 insertions, 0 deletions
diff --git a/2024/talks/p-search.md b/2024/talks/p-search.md index 01e8aed7..59d16379 100644 --- a/2024/talks/p-search.md +++ b/2024/talks/p-search.md @@ -53,6 +53,182 @@ tools. Code: <https://github.com/zkry/p-search> +# Discussion + +## Questions and answers + +- Q: Do you think a reduced version of this functionality could be + integrated into isearch? Right now you can turn on various flags + when using isearch with M-s \<key\>, like M-s SPC to match spaces + literally. Is it possible to add a flag to \"search the buffer + semantically\"? (Ditto with M-x occur, which is more similar to your + buffer-oriented results interface) + - A: it\'s essencially a framwork so you would create a generator; + but it does not exist yet. +- Q: Any idea how this would work with personal information like + Zettlekastens? + - A: Useable as is, because all the files are in directory. So + only have to set the files to search in only. You can then add + information to ignore some files (like daily notes). + Documentation is coming. +- Q: How good does the search work for synonyms especially if you use + different languages? + - A: There is an entire field of search to translate the word that + is inputted to normalize it (like plural -\> singular + transformation). Currently p-search does not address this. + - A: for different languages it gets complicated (vector search + possible, but might be too slow in Elisp). +- Q: When searching by author I know authors may setup a new machine + and not put the exact same information. Is this doing anything to + combine those into one author? + - A: Currently using the git command. So if you know the emails + the author have used, you can add different priors. +- Q: A cool more powerful grep \"Rak\" to use and maybe has some good + ideas in increasing the value of searches, for example using Raku + code while searching. is Rak written in Raku. Have you seen it? + - [https://github.com/lizmat/App-Rak](https://github.com/lizmat/App-Rak){rel="noreferrer noopener"} + - [https://www.youtube.com/watch?v=YkjGNV4dVio&t=167s&pp=ygURYXBwIHJhayByYWt1IGdyZXA%3D](https://www.youtube.com/watch?v=YkjGNV4dVio&t=167s&pp=ygURYXBwIHJhayByYWt1IGdyZXA%3D){rel="noreferrer noopener"} + - A: I have to look into that. Tree-sitter AST would also be cool + to include to have a better search. +- Q: Have you thought about integrating results from using cosine + similarity with a deep-learning based vector embedding? This will + let us search for \"fruit\" and get back results that have \"apple\" + or \"grapes\" in them \-- that kind of thing. It will probably also + handle the case of terms that could be abbreviated/formatted + differently like in your initial example. + - A: Goes back to semantic search. Probably can be implemented, + but also probably too slow. And it is hard to get the embeddings + and the system running on the machine. +- Q: I missed the start of the talk, so apologies if this has been + covered - is it possible to save/bookmark searches or search + templates so they can be used again and again? + - A: Exactly. I just recently added bookmarking capabilities, so + we can bookmark and rerun our searches from where we left off. + I tried to create a one-to-one mapping from the search object to + the search object - there is a command to do this- to get a data + representation of the search, to get a custom plist and resume + the search where we left off, which can be used to create + command to trigger a prior search. +- Q: You mentioned about candidate generators. Could you explain about + to what the score is assigned to. Is it to a line or whatever the + candidate generates? How does it work with rg in your demo? + + FOLLOW-UP: How does the git scoring thingy hook into this?\ + +- - A: Candidate generator produces documents. Documents have + properties (like an id and a path). From that you get + subproperties like the content of the document. Each candidate + generator know how to search in the files (emails, buffers, + files, urls, \...). There is only the notion of score + + document. + - Then another method is used to extract the lines that matches in + the document (to show precisely the lines that matches). + +- Q: Hearing about this makes me think about how nice the emergent + workflow with denote using easy filtering with orderless. It is + really easy searching for file tags, titles etc. and do things with + them. Did this or something like this help or infulce the design of + psearch? + - A: You can search for whatever you want. No hardcoding is + possible for anything (file, directories, tags, titlese\...). + +- Q: \[comments from IRC\] \<NullNix\> git covers the \"multiple + names\" thing itself: see .mailmap 10:51:19 + - \<NullNix\> thiis is a git feature, p-search shouldn\'t need to + implement it 10:51:34 + - \<NullNix\> To me this seems to have similarities to notmuch \-- + honestly I want notmuch with the p-search UI :) (of course, + notmuch uses a xapian index, because repeatedly grepping all + traffic on huge mailing lists would be insane.) 10:55:30 + - \<NullNix\> (notmuch also has bookmark-like things as a core + feature, but no real weighting like p-search does.) 10:56:07 + - A: I have not used notmuch, but many extensions are + possible. mu4e is using a full index for the search. This + could be adapted here to with the SQL database as source. + +- Q: You can search a buffer using ripgrep by feeding it in as stdin + to the ripgrep process, can\'t you? + - A: Yes you can. But the aim is to search many different things + in elisp. So there is a mechanism in psearch anyway to be able + to represent anything including buffers. This is working pretty + well. + +- Q: Thanks for making this lovely thing, I\'m looking forward to + trying it out. Seems modular and well thought out. Questions about + integreation and about the interface + - A: project.el is used to search only in the local files of the + project (as done by default) + +- Q: how happy are you with the interface? + - A: psearch is going over the entire files trying to find the + best. Many features can be added, e.g., to improve debuggability + (is this highly ranked due to a bug? due to a high weight? many + matching documents?) + - A: hopefully will be on ELPA at some point with proper + documentation. + +- Q: Remembering searches is not available everywhere (rg.el? but AI + package like gptel already have it). Also useful for using the + document in the future. + - A: Retrievel augmented generation: p-search could be used for + the search, combining it with an AI to fine-tune the search with + a Q-A workflow. Although currently no API. + - (gptel author here: I\'m looking forward to seeing if I can use + gptel with p-search) + - A: as the results are surprisingly good, why is that not used + anywhere else? But there is a lot of setup to get it right. You + need to something like emacs with many configuration (transient + is helping to do that) without scaring the users. + - Everyone uses emacs differently, so unclear how people will + really use it. (PlasmaStrike) For example consult-omni + (elfeed-tube, \...) searching multiple webpages at the same + time, with orderless. However, no webpage offers this option. + Somehow those tools stay in emacs only. (Corwin Brust) This is + the strength of emacs: people invest a lot of time to improve + their workflow from tomorrow. \[see xkcd on emacs learning curve + vs nano vs vim\] + - [https://github.com/armindarvish/consult-omni](https://github.com/armindarvish/consult-omni){rel="noreferrer noopener"} + - [https://github.com/karthink/elfeed-tube](https://github.com/karthink/elfeed-tube){rel="noreferrer noopener"} + - [https://www.reddit.com/r/ProgrammerHumor/comments/9d6f19/text_editor_learning_curves_fixed/](https://www.reddit.com/r/ProgrammerHumor/comments/9d6f19/text_editor_learning_curves_fixed/){rel="noreferrer noopener"} + - A: emacs is not the most beginner friendly, but the solution + space is very large + - (Corwin Brust) Emacs supports all approaches and is extensible. + (PlasmaStrike) Youtube much larger, but somehow does not have + this nice sane interface. + +- Q: Do you think the Emacs being kinda slow will get in the way of + being able to run a lot of scoring algorithms? + - A: The code currently is dumb in a lot of places (like going of + all files to calculate a score), but that is not that slow + surprisingly. Elisp enumerating all files and multiplying + numbers in the emacs repo isn\'t really slow. But if you have to + search in files, this will be slow without relying on ripgrep on + a faster tool. Take for example the search in info files / elisp + info files, the search in elisp is almost instant. For + human-size documents, probably fast enough \-- and if not, there + is room for optimizations. For coompany-size documents (like + repos), could be too small. + +- Q: When do you have to make something more complicated to scale + better? + - A: I do not know yet really. I try to automate tasks as much as + possible, like in the emacs configuration meme \"not doing work + I have to do the configuration\". Usually I do not add web-based + things into emacs. + +## Notes + +- I like the dedicated-buffer interface (I\'m assuming using + magit-section and transient). +- \<meain\> Very interesting ideas. I was very happy when I was able + to do simple +- filters with orderless, but this is great \[11:46\] +- \<NullNix\> I dunno about you, but I want to start using p-search + yesterday. +- (possibly integrating lsp-based tokens + somehow\...) \[11:44\] +- \<codeasone\> Awesome job Ryota, thank you for sharing! + [[!inline pages="internal(2024/info/p-search-after)" raw="yes"]] |