diff options
Diffstat (limited to '2023/talks/hyperdrive.md')
| -rw-r--r-- | 2023/talks/hyperdrive.md | 194 |
1 files changed, 194 insertions, 0 deletions
diff --git a/2023/talks/hyperdrive.md b/2023/talks/hyperdrive.md new file mode 100644 index 00000000..9c70a791 --- /dev/null +++ b/2023/talks/hyperdrive.md @@ -0,0 +1,194 @@ +[[!meta title="hyperdrive.el: Peer-to-peer filesystem in Emacs"]] +[[!meta copyright="Copyright © 2023 Joseph Turner and Protesilaos Stavrou"]] +[[!inline pages="internal(2023/info/hyperdrive-nav)" raw="yes"]] + +<!-- Initially generated with emacsconf-publish-talk-page and then left alone for manual editing --> +<!-- You can manually edit this file to update the abstract, add links, etc. ---> + + +# hyperdrive.el: Peer-to-peer filesystem in Emacs +Joseph Turner and Protesilaos Stavrou, <https://ushin.org> xmpp:discuss@conference.ushin.org (XMPP MUC for USHIN discussion), <mailto:joseph@ushin.org> + +[[!inline pages="internal(2023/info/hyperdrive-before)" raw="yes"]] + +This talk explores [hyperdrive.el](https://ushin.org/hyperdrive/hyperdrive-manual.html), an Emacs interface to [hyperdrive](https://docs.holepunch.to/building-blocks/hyperdrive), a +peer-to-peer shared filesystem. + +Peer-to-peer networks give you the freedom to choose your sources of +information and to communicate directly with them. Emacs is a +[freedom-respecting](https://www.gnu.org/philosophy/free-software-even-more-important.html) text editor/operating system that puts you in the +driver's seat. By bringing these two worlds together, hyperdrive.el +aims to inspire deliberation about digital communication freedom. + +Hyperdrives are peer-to-peer shared folders, well-suited for data that +changes over time, like personal blogs. It complements Bittorrent and +IPFS, which work best with unchanging data, like journal archives. + +Here's a scenario: Alice creates a new hyperdrive and adds some files. +Her computer returns a public key URL that uniquely identifies the +hyperdrive. Alice shares that URL with Bob, who can then download +Alice's files directly from her computer. No third-party servers are +required to route the connection. + +Data is distributed among peers; once Bob has loaded Alice's files, +Carol can get them from Bob (or anyone else who has a copy) even when +Alice is offline. Drives are mutable; when Alice adds/removes/changes +files in the drive, Bob can refresh her drive on his machine to get +the latest changes. Drives are versioned; anyone with the URL can +"check out" prior versions of Alice's drive to see what her files used +to look like. + +Currently supported features in hyperdrive.el include: + +- [directly edit hyperdrive files](https://ushin.org/hyperdrive/hyperdrive-manual.html#Write-to-a-hyperdrive) +- [dired-like directory view](https://ushin.org/hyperdrive/hyperdrive-manual.html#Directory-view) +- [org-mode link support](https://ushin.org/hyperdrive/hyperdrive-manual.html#Org-mode-links) +- [version history navigation/diffing](https://ushin.org/hyperdrive/hyperdrive-manual.html#View-the-hyperdrive-version-history) +- [built-in bookmark.el integration](https://ushin.org/hyperdrive/hyperdrive-manual.html#Bookmark-a-hyperdrive) +- [local directory mirroring](https://ushin.org/hyperdrive/hyperdrive-manual.html#Mirror-a-whole-directory) +- [audio/video streaming](https://ushin.org/hyperdrive/hyperdrive-manual.html#Stream-audio-and-video) + +Planned features include: + +- peer discovery (swarming) +- diffing directories between versions + +Check out [the manual](https://ushin.org/hyperdrive/hyperdrive-manual.html#Installation) for installation instructions! + +You're welcome to join our public XMPP chat room! + +- xmpp:discuss@conference.ushin.org ([Join anonymously from your browser](https://anonymous.cheogram.com/discuss@conference.ushin.org)) +- #_bifrost_discuss_conference.ushin.org:aria-net.org (Matrix bridge) + +Bugs can be submitted to the [ushin issue tracker](https://todo.sr.ht/~ushin/ushin). Patches, comments or +questions can be submitted to the [ushin public inbox](https://lists.sr.ht/~ushin/ushin). + +About the speaker: + +I'm Joseph Turner. I enjoy fiddle, Aikido, peer-to-peer networks, +Emacs, and swimming in cold water. I work with [USHIN](https://ushin.org/), a tiny +educational US nonprofit whose mission is to promote personal, +community, and global health through free and open universal shared +information for everybody. This year, we're focusing on the +hyperdrive.el project, with the goal of bringing Emacs and +peer-to-peer together. + +I am Protesilaos. Friends call me "Prot" and you are welcome to do the +same. I have been an Emacs user for ~4 years. I use Emacs full-time for +practically every aspect of my computing. I am the author and maintainer +of several packages for Emacs and am enthusiastic about its potential +for user freedom. + +# Discussion + +## Questions and answers + +- Q: It's not clear how hyperdrive (not hyperdrive.el) works. Do I + need to install something on my computer to use it? Can I use it + from my phone? + - A: Like the emacs transmission client connects with the + transmission-daemon, hyperdrive.el connects with the + hyper-gateway daemon. + - Accessing hyperdrives on mobile: + <https://github.com/AgregoreWeb/agregore-mobile> +- Q: What lessons have you learned while developing hyperdrive.el? + - A: Great support and guidance from talented folks like Adam, + Mauve, Jonas, Prot!! +- Q: I use multiple computers and my partner also would like acess to + my notes. so 2 questions. How well would this work with using this + to editing my zettelkasten hyperdrive using multiple computers + - A: Hyperdrives are single-writer, so you'd be better off + linking between drives. In the future, we plan to add support + for <https://github.com/nobiot/org-transclusion> to + hyperdrive.el. + - Q2: How well would it work if my and my partner worked on the + same hyperdrive zettelkasten + - A: If you linked between drives, it could work quite well! +- Q: What would be a good way of getting Hyperdrives if you don't + want to install NPM and hava a binary. Could you compile it with + deno or the "rust or zig or go?" cli alternative tool? I would + prefer to download a single binary. + - A: Jonas has been using hyper-gateway installed with a `guix + shell` command. Thank you, Jonas!!! + - Quick gist: + <https://gist.github.com/tarsius/509e9c65c9df1bc243d77cd968d60daa> + - Q: <https://github.com/datrs/hypercore> rust hyperdrive? + - A: I'm not familiar with this rust port yet. +- Q: If you had your druthers, what would make your work on + hyperdrive.el easier? + - A: User feedback!! Please try it out :) +- Q: Have you tried putting a git repo in hyperdrive? Does it work + well? + - A: If you use a bare repository, take care to gc and create pack + files before mirroring to the drive, and only publish periodic + updates, then that might be okay. (And make sure not to repack + old pack files.) +- Q: Is data transferred between nodes in the clear or encrypted? + - A: Encrypted in transit. +- Q: Is there a searchable catalogue of hyperdrives? + - A: Not yet, but we have plans for a distributed "trust" + network that could be used with hyperdrive: + <https://git.sr.ht/~ushin/trust.el> +- Q: Any plans for FUSE or posix semantics? + - A: Not yet. There was + <https://github.com/andrewosh/hyperdrive-fuse> , but it's not + maintained currently. +- Q: Any plans for a TRAMP interface? + - A: Good idea! A TRAMP interface may make it easier to offer + live hyperdrive filename completions. +- Q: How does this comprare to syncthing? + - A: Syncthing is useful for sharing files among a small group of + trusted peers, like an F2F network + (<<https://en.wikipedia.org/wiki/Friend-to-friend>>). + Hyperdrives are useful for publicly sharing a set of files which + you can updated going forward and which others can link to. +- Q: If you edit a file on the hyperdrive, then edit the same file on + the local mirror. How is the conflict handled when you sync the + mirror again? + - A: If I understand correctly, you're asking about what happens + when you write to the same hyperdrive from multiple machines. + The short answer is, "Please don't do that." However, it + appears that the Holepunch team is making progress on + <https://docs.holepunch.to/building-blocks/autobase> for + "autohmatically rebasing" hyperdrive history, effectively + allowing for multi-writer hyperdrives. +- Q: wouldn't user be able to collaborate asynchronously by viewing diffs on a serially "shared" file, in other words, a user would copy another peer's text file, edit and upload their changes, share the link to their updated file so that then others can see diffs and in that way co-create an evolving file? + - A: You could try this. Another idea that we have in the works is integration with <https://github.com/nobiot/org-transclusion> + +## Notes + +- damn, I never even knew about dired-jump (C-x C-j). Main + differences with (C-x d) from the file are one fewer keystroke and + having the point on the file you came from. +- Btw, hyperdrive looks like another one of those things that would be amazing if I collaborated with anyone using emacs +- Also incidentally hyperdrive is amazing +- I installed hyperdrive.el and tried to run it and immediately got: transient-setup: Symbol’s function definition is void: transient-prefix-object + - Need to upgrade transient.el + - Gah, you are right. I had an obsolete version just hanging out causing pain. + - Okay, I got hyperdrive.el working and it was super easy. hyper://fwsn55wnznts5mpkee16j89ja38nfz6zne4wijzap1z9ka4jsxio/Dird_a333f1_4884540.jpg +- I think hyperdrive is a pretty easy way to share files. Easier than Dropbox. But maybe for delivery? I'm not sure. + - A: What do you mean by "delivery"? + - I was thinking of asset delivery. I write professionally, so I have things like PDFs and other documents to deliver. Sometimes I have big files to deliver to clients. Using something like hyperdrive would be cooler than Dropbox. ... Hmmmm, is there a hyperdrive web gateway? + - A: A good non-Emacs option is <https://github.com/AgregoreWeb/agregore-browser> Agregore is Chromium with built-in support for more protocols besides HTTP, like Hyperdrive, Bittorrent, IPFS, Gemini, and Gun. <https://github.com/RangerMauve/hyper-gateway> , the program that hyperdrive.el connects to as a client is, I think, exactly what you asking about. I don't know of a public HTTP gateway, but you could easily run hyper-gateway on a VPS. + - That would be fun. Then I could control the domain name so it would look cool and professional and like I was extra high-tech. + - A: hyper-gateway lets you serve multiple drives at various subdomains. For example, you could serve your primary drive at <https://reverik.tld>, and then folks could access, e.g., the USHIN drive at <https://aaj45d88g4eenu76rpmwzjiabsof1w8u6fufq6oogyhjk1ubygxy.reverik.tld> + + +- Great work, everyone. Thank you. +- Prot's presentations are so clear. Perfectly model pedagogy. +- Just like his code: his .el buffers SQUEAK as you C-v through them. +- Prot is indeed a model of paying attention to detail. +- I have learned so much from his videos, and from his code. +- Prot switching from vim/etc to Emacs seems to correspond to an inflexion point in the current Emacs renaissance does it not? Coincidence, or... I wonder what way the causal arrow goes :) + - I think the current "Emacs renaissance" probably dates back to around the time that package.el was developed and then when MELPA came online. momentum seems to have been building since then + - We must also consider Org-mode's inception as a key moment in recent Emacs history +- My Emacs environment would be much poorer without Prot's contributions, that's for sure. +- My hypothesis is that Diogenes himself made his way to Cyprus and spent the past immortal couple thousand years as a mountain hermit waiting to return to mortal life when the Emacs community needed him. Humor aside, there are a few key personalities---many of them here this weekend---who have been disproportionately catalytic to the community recently. Prot is certainly one. Thanks to all of you! + + + +[[!inline pages="internal(2023/info/hyperdrive-after)" raw="yes"]] + +[[!inline pages="internal(2023/info/hyperdrive-nav)" raw="yes"]] + + |