[Speaker 0]: I guess we are now live.
So Joseph, thanks for being here.
Thanks for talking to the hyperdrive.
We already had some, or we already have a lot
of questions here. And I guess I would start
with, let's call it the difficult,
the most difficult 1. So when you were
developing hyperdrive for your colleague,
what do you, or what have you learned the
most?
[Speaker 1]: I have learned how much faster and more
enjoyable the development of this project can
be with talented people working by my side,
like Jonas and Adam and Prat and Mo,
it's been really a pleasure to work with
these folks.
[Speaker 0]: So you have started at first on your own and
then probably pushed it somewhere in open
source or how did it develop,
your development experience?
[Speaker 1]: A few years ago, we started looking into
using peer-to-peer technology for sharing all
kinds of information. And we came across Move
SignWeaver, who was recommended to us by a
mutual friend. And we started working with
Move, and then about a year ago,
we started looking into using Emacs,
the peer-to-peer software,
so that we could make use of all of the
powerful things that Emacs already does with
org mode and other packages.
And then we started working with Adam and
Pratt and Jonas.
[Speaker 2]: Yes.
[Speaker 0]: So we are skipping to the next question.
So to read it out, I use multiple computers
and my partner also would like access to my
notes. So, 2 questions at first.
First 1, how well would this work with using
this to edit my Zettelkasten hyperdrive using
multiple computers?
[Speaker 1]: Hyperdrive is single writer currently.
So what that means is that if you have a
hyperdrive that you've created,
you're the only 1 who can make changes to it.
And that's limited right now to editing 1
hyperdrive from 1 machine.
In theory, you could use the same private key
and write to it from multiple machines,
but you would have to make sure that you sync
it on both machines and didn't make
concurrent writes because then you would fork
the history of your hyperdrive,
and that would be bad.
But we've spent a lot of time making links to
hyperdrives work well,
relative links within hyperdrives to other
files inside of your drive.
So you should be able to,
with some exceptions, just take your personal
information management set of org files or
whatever it is that you have,
and upload them into a hyperdrive if all of
that is publicly available or would be good
to share publicly. And you can make that
available for other people to link to.
So you can have multiple different
hyperdrives that link to 1 another.
[Speaker 0]: So it's like a huge network of hyperdrives
connected to each other in some way.
[Speaker 1]: Yeah.
[Speaker 0]: So that's kind of neat and kind of cool.
There was a follow-up question or the second
part of the question. Okay,
then using the same hyperdrive is probably
not possible, but interlinking would be the
best way to do it. There was a question
concerning how they should install it.
So What would be a good way of getting
hyperdrives if you do not want to install npm
and have a binary? Could you compile it with
denner or rusk or zig or go?
CLI alternative tool, I would prefer to
download a single binary.
[Speaker 1]: There's something that Jonas was playing
around with using Geeks to install Hyper
Gateway. So the way that HyperDrive.el,
the Emacs package, works right now is similar
to the way that the transmission Emacs client
for BitTorrent works, where you have a client
in Emacs that connects to a daemon that is a
separate process that's running on your
machine, the transmission daemon.
But in this case, we have HyperGateway,
which is running as a daemon on your machine.
And then hyperdrive.el
connects to that daemon and sends requests,
and all of the hyperdrive stuff under the
hood happens with her gateway.
But so that package can,
or hypergateway, the program can be
installed, The easiest way is to just
download it from the GitHub releases.
You could also use NPM to install it.
And then the third option that we've been
playing around with is Jonas was writing a
little script to install it using Geeks,
since Geeks now comes with Node 18.
And so you should be able to install it using
Geeks.
[Speaker 0]: Right, thank you. We have 2 people here
joined with microphone.
Do we have now any question to Joseph or just
here for chilling out.
I guess it's a no. Plasma,
yeah.
[Speaker 3]: What about using, having some of the
information being private in the hyperdrives.
[Speaker 1]: That's not what we have been focusing on at
this point. At this point,
what we've been working on is mainly using
hyperdrives for a public forum type tool.
But you could encrypt those files if you
wanted to. You can also just,
a poor man's security would just be to share
your HyperDrive link only with those people
that you want to have access to your drive.
But the way that it works right now is anyone
who has the link to a hyperdrive can access
its content. So long as there are peers
available on the network who can serve it to
you.
[Speaker 0]: Any follow up question from your side,
Plasma?
[Speaker 3]: I had 1, I'll just have to re-remember it.
[Speaker 0]: If you remember it, just feel free to
interrupt me.
[Speaker 3]: What about working? I've looked at this
before. What about, if I remember correctly,
it doesn't do as well with large files,
so if you're going to store 200 gigs of video
files, stuff like IPFS works a lot better,
or BitTorrent. This is,
are you, were you using the,
any way of using multiple protocols for stuff
like that? Or what were you doing with,
or were you just doing the small files with
the same protocol? Or
[Speaker 1]: I would love to see an IPFS client in Emacs
as well that could interface with Kubo or
some other IPFS daemon and I think that those
could work really well together.
We mostly have been playing around with
sharing relatively small files,
up to hundreds of megabytes or maybe a
gigabyte. We haven't played around yet with
hyperdrive.el, the Emacs client,
testing that with HyperGateway.
But there may be other experiments that have
been done that show that that works well.
The main thing is that IPFS uses content
addressability to reduce duplication of the
content. Whereas in HyperDrive,
if you upload the same file with the same
contents twice, now you have double the
content being stored in your HyperDrive.
It's not deduplicated.
You can always clear out part of the history
of your hyperdrive But IPFS has really good
built-in deduplication whereas hyperdrive
does not
[Speaker 4]: I have a question.
[Speaker 3]: What about like commenting on other like if
you have a couple of different Hypercore
blogs, what about like commenting between
them? Like you have some people who have a
commenting form on Reddit for their blog
posts.
[Speaker 1]: So Move SignWeaver has been doing a lot of
work recently with the distributed press API
to integrate ActivityPub with these
peer-to-peer technologies.
Move can give you more information about
that. But there is another feature that we'd
like to add to hyperdrive.el,
which is peer discovery using the swarming
feature that HyperCore,
HyperSWARM offers, where you'd be able to say
that my node, my peer-to-peer node is
interested in Emacs and free software as
topics. And those would be 2 different
topics. I would advertise on the network that
I'm interested in those topics.
And I would be able to discover other peers
on the network who have also advertised that
they're interested in those same topics.
And then they would tell me,
hey, here's the public key of my hyperdrive.
Come check it out. I have posted information
about those topics. And so in that way,
you'd be able to, in a distributed fashion,
discover other peers on the network who are
interested in topics that you're interested
in.
[Speaker 3]: Something that would be useful in addition to
that idea is like if you had your emacs
Zettelkasten Publish like let's say you have
some private data You make sure that that's
scrubbed out before it goes to your hyper
core and then you have another part of it
that gets turned into a website for it's also
given to other hyper core clients but you'd
rather get the emacs users the org documents
then you also publish some of them on a
website so everybody as much people can get
it as possible. And then a way of figuring
out who you'd want to do,
or if you're an Emacs user,
maybe figure out that they're all related to
each other, but you want to get the art mode
documents because you're using EMAX.
Yeah.
[Speaker 0]: Maybe a side note, we have 4 minutes here on
before we switch into the next track,
just to let you know.
[Speaker 1]: Thank you. So the hyper drive mirror feature
that we added, would allow you to selectively
choose which files you want to share in a
hyperdrive. So, with Prot's denote file
naming scheme or Carl Voigt's file tags
naming scheme, you could just specify a
regular expression. And you could say,
I want to share out of my directory of org
files, I want to share only those files that
have been tagged as public,
or only those files that have been tagged as
emacs and then only those ones would get
uploaded into your hyperdrive
[Speaker 3]: or exclude all in any of the ones that say
private
[Speaker 0]: yep mike had a question
[Speaker 4]: yeah I have a question for the hyperdrive.
So I just maybe I missed it and you haven't
put a link.
[Speaker 1]: Oh, Mikhail, we can't hear you.
[Speaker 3]: Heard you for a second.
[Speaker 1]: Yes?
[Speaker 4]: Can someone hear me? Okay,
I have no idea what happened to my
microphone, but now it's back.
[Speaker 3]: Now we can. You can see the microphone on the
top of the screen. So
[Speaker 4]: yes, thank you. Okay. I have a question to
hyperdrive. Is the hyperdrive a find on the
hole punch point, point T O hole Or is it
just another hyperdrive?
[Speaker 1]: That's exactly the project that we're using.
So the HolePunch team has released hyperdrive
and other hyper core libraries as free
software libraries that you can use.
And so MoV SignWeaver,
the project that MoV is working on,
HyperGateway, depends on those libraries and
it makes it easy for you to build other
clients like hyperdrive.el
which connect to the hyperdrive network.
I hope that answers your question.
[Speaker 4]: Yes it does, thank you.
And what did make you choose hyperdrive for
this Emacs project?
[Speaker 1]: Mainly the fact that the drives are mutable,
which makes it distinct from IPFS or
BitTorrent, where when you share some piece
of content, you're stuck with that static
piece of content, which works well for some
cases, but if you say you have a Zettelkasten
or you have a set of org files that you want
to share with people, you want to be able to
update those files and have other people pull
those updates from you.
And so HyperDrive allows you to have these
mutable sets of files that you can share and
use the same link for other peers to pull the
latest changes from you.
Also, it's versioned, as we showed in the
video, which is really helpful for having
community deliberations and community
discussions where you want to be able to
reference some something that somebody said
in the past and not have it get deleted or
changed or something.
[Speaker 0]: We are now switching to talk So just for
letting you know if you want to say something
now. Too late. The BB room is still open,
so you can still discuss.
There's also a lot going on on the pad.
But you can also discuss here inside and
answer the pet questions maybe later.
[Speaker 1]: Yeah, there are good questions.
I'll go ahead, please.
[Speaker 3]: continuing here on the pad?
[Speaker 1]: Are we I can hear you.
[Speaker 5]: Yeah, so the question I had on the pad was,
would it make sense in any sense to put a
FUSE interface or put the POSIX semantics in
front of this at some point?
[Speaker 1]: Yeah, that would be cool.
It's kind of a similar question to any plans
for a Tramp interface.
There was a project that the HyperCore
HolePunch team was working on a year or more
ago that provided a FUSE interface.
And I think it didn't pan out.
But it's a good idea. Same with the Tramp
interface. It seems like a good idea that
would make it possible to more easily hook
into the built-in Emacs functionality for,
for example, like incremental file name
completion, which we don't currently support
in Hyperdrive.el. So I'd love to have
feedback and design ideas for those projects.
[Speaker 5]: Yeah, there's just Everything in Emacs just
sort of assumes the file system is there and
usable in that way. That's all.
[Speaker 1]: Yeah, it's a good idea.
[Speaker 3]: An idea for the privacy type thing is
Syncthing links. Because I think you can set
up Syncthing in such a way that you have the
private networks that other people can't
actually get access to.
[Speaker 1]: I did not know that that was possible with
Syncthing. I'll have to look into that.
[Speaker 3]: At least I think it is anyway,
because yeah, there's ways you can explicitly
authorize devices. Yeah,
right. I think you could actually set it up
in such a way that you can have private stuff
and links, and then that might be a way that
you can get a completely distributed
Zettelcast and with private notes.
[Speaker 1]: Yeah. Good idea. There's a question in the
pad about DATRS, a Rust version of
HyperDrive. I had not heard of that,
so I'll have to look into that.
If you had your druthers,
what would make your work on hyperdrive.dl
easier? It's been a lot of fun.
I would love to have more user feedback.
That would be my wish.
I tried putting a git repo in HyperDrive.
Does it work well? I don't think that would
work well because, as I mentioned a moment a
few moments ago, the data that you put into a
hyperdrive is duplicated.
So if you had the whole work tree in
hyperdrive every time you made a change and
saved it, it would be duplicated.
If you had just a bare repository,
I don't know, try it.
[Speaker 3]: They're trying to solve the same problem,
but 1 of the optimizations they have for
being able to view a whole bunch of people's
data is they made shallow clones a lot
[Speaker 1]: Would you phrase that again,
please?
[Speaker 3]: easier. Right? So like Git and Hypercore,
1 of the things they do is they allow you to
have a whole history of every single change
for a dataset Zettelkasten project.
But 1 of the optimizations Hypercore did to
make it more network web friendly is they
made the shallow clones work a lot better and
a lot... Yeah, they made that work a lot
better so you don't have to download every
single thing for every single project.
And because they both are implementing the
delta upgrades, I don't see how they could
work really well together.
At least from what it looked like to me.
It can't hurt to experiment.
[Speaker 1]: But yeah, I would agree with you.
Is data transferred between nodes in the
clear or encrypted? That's a good question.
I don't know how it's encrypted.
I don't, I wouldn't recommend sharing
sensitive data with hyperdrive right now?
I would recommend if you want to play with
it, have it be something where you're
expecting the data to be shared.
Is there a searchable catalog?
[Speaker 3]: It's also the data in transport versus data
at rest. I'm pretty sure the data at rest
would not be encrypted.
Right. You can separate that into those 2
questions.
[Speaker 1]: Right. Right. Is there a searchable catalog
of hyper drives? So that's a thing,
an idea that we've been a distributed trust
network for discovering peers that are
trusted for a particular topic.
And we actually made a demo video of a
previous prototype that's available on the
Ashen hyperdrive that you can watch that
shows the basic idea. But the idea is just
that you would have a list of peers that you
think are worth listening to or worth reading
for a particular topic.
And those peers would have peers that they
think are worth listening to for that same
topic. And so you would say,
if I'm interested in Emacs,
I want to see all the peers that I trust for
the topic Emacs. And if,
say, Adam Porter shows up in my list and Adam
Porter trusts Jonas and Jonas trusts Pratt,
I would be able to read hyperdrive
information from all of those people by
looking at the indirect relationships that I
have by following the chain of relationships,
kind of like a web of trust.
And so it would also allow you to have a
network of peers that you trust to block
other people on your behalf.
So it would be useful for subjective
moderation where you can remove spam and bad
actors from the people that you follow
without having to delegate that powerful
responsibility to some third party in a
permanent way where that third party might
abuse that power. So it allows you to share
your list of trusted peers and your list of
blocked peers with other people in a
peer-to-peer way.
[Speaker 3]: Have you ever looked at GNUnet?
It kind of does some...
It's trying to do something weird with the
internet where it redesigns it from the
ground up to be peer-to-peer,
local first, or something like that.
[Speaker 1]: I would like to know more about GNUnet.
Yes. I have heard of it,
but I haven't really researched it.
If you edit a file on the hyperdrive,
then edit the same file on the local mirror,
how is the conflict handled when you sync the
mirror again? So I think if I understand the
question, the answer is that you can't edit
the file in 2 different places,
I think is the answer to the question.
If you were to manually copy the private key
from 1 machine onto another machine,
then you could cause a conflict,
like a merge conflict,
but you would have to go out of your way to
do that. And It's not handled.
I think the Hypercore Hole Punch team has
another project that they're working on that
would, it's called AutoBase,
that would merge those conflicts.
But we're not using that right now.
And I think it's in early development still.
So there might be a solution in the future.
[Speaker 3]: What's a surprising change of thoughts or
what's the most interesting thing you weren't
expecting to discover while developing this?
Like change of thoughts on how you write or I
don't know.
[Speaker 1]: Well, I'm relatively new to Emacs and to Lisp
and really to programming in general.
And so it's been a fantastic learning
experience. Adam, Alpha Papa,
Adam and I have been doing a lot of pair
programming sessions where we work together
and I get to learn from him.
And we've had meetings with Jonas and Prat
and meetings with Mauve where it's a
fantastic learning experience for me to
discover how to build software in an
efficient and intelligent way.
It's a huge pleasure. If there are no more
questions, I just wanted to encourage
everyone to try it out and to let us know
what you think. It would be really helpful to
have some feedback from people who are using
it in new and creative ways that we haven't
anticipated.
[Speaker 6]: Hi, I'd just like to say that I tried this
new thing called hyperdrive.el
today, and I think it's pretty cool.
[Speaker 2]: Sorry, that was somebody else.
Hey Joseph, how's it going?
Oh, talk today.
[Speaker 3]: Oh, thanks. Wonderful.
[Speaker 1]: Who's that? Oh, hey. Well,
I'm going to say goodbye.
Thank you. And thank you for your questions,
[Speaker 3]: I know that
[Speaker 1]: PlasmaStrike. I've met you before.
Appreciate your questions,
your thoughts.
[Speaker 2]: Oh, by the way, Joseph,
we have our, our first,
I don't know if our first new user,
but we have the first link being shared,
to hyperdrive file in the chat and I loaded
it and it works. And it's funny too.
It's worth looking at.
So. Oh, I think it's frozen.
I don't know if anybody can hear me.
[Speaker 3]: I can.
[Speaker 2]: Okay, cool. The browser is frozen.
It's it's not, okay. Just unfroze.
Anyway. All right. Well,
By the way, I enjoyed your talks about
hyperbole. I'm going to rewatch those later
when I get a chance. It was nice to meet you,
too. Bob is a really great guy to work with.
[Speaker 3]: Definitely a lot of interesting people.
[Speaker 2]: I owe him 1. Yes, sir.
All right, you have a good day,
[Speaker 3]: Will do, I like the insistence on local
first. Feels like it's a good dovetail with
the hyper core
[Speaker 2]: enjoy the conference. Yeah,
yeah, I think there's a lot of a lot of
interesting possibilities to build on this we
have some plans that we Will get to you later
this well in the coming year And we'll see
where the hyperdrive people,
you know, upstream how they develop it as
well and yeah, so exciting times.