VF-1, AV-98 and Agena

This post is mostly a response to Tomasino's

public daydreaming about seamless browsing of

Gopherspace and Geminispace with the one

client[1]. But it's also a little bit of an

update about various software projects of

mine.

First of all, a bit about my VF-1 gopher

client: there hasn't been a formal release

since July, but there has been quite a bit

of sporadic development in the Git repo since

then. I have been promising various people

that 1.0.0 is "just around the corner" for

quite a few months now and I really do mean

it! The real hold up is writing good and

complete documentation, which I really just

need to push through and do. Code-wise, I

think we are good to go, but what I'd like to

do is ask interested people to please

consider cloning the repo at

https://github.com/solderpunk/VF-1.git and

surfing Gopherspace with the development

version for a while. If anything breaks or

otherwise seems wrong, please let me know!

Lots of the development work has been tidying

the code up, removing unecessary duplication,

etc., but there is some exciting new stuff

visible to the end-user that you should play

with if possible:

• Maybe most importantly, VF-1 will now

connect to servers via IPv6 if they have an

AAAA record and the server is listening on

IPv6. If there's nothing listening on

IPv6, or you don't have an IPv6 connection

or your system doesn't even have IPv6

support enabled, it will automatically fall

back to IPv4. In my experience this

doesn't result in any annoying slow timeout

problems like IPv6 often did in the old

days, but it would be reassuring to hear

from more people that this still works

nicely. If this does cause you problems or

you have an irrational hatred of IPv6 you

can put set ipv6 false in your RC file

and VF-1 won't ever even try to use it.

• Content handlers can now be shell

pipelines, so you should now be able to

Make VF-1 Sexy[2] Tomasino-style without

having to resort to writing little helper

scripts.

• There's now a debug mode, which you you can

turn on from the prompt with `set debug

trueor when starting VF-1 with--debug`.

This will print various bits of info, in

ANSI-escape-coded green text, as VF-1 does

things so you can see when VF-1 is trying

to use IPv4 or IPv6, what kind of MIME type

it's inferring for content, etc. Probably

not much use for normal people, but it was

handy for me when adding the IPv6 stuff.

• If debug doesn't satisfy your desire to

peek under the hood enough, there's also

now a (largely gratuitious, I admit)

blackbox flight recorder you can access

with the blackbox (or bb) command which

will tell you how long VF-1 has been

running for and, in that time, how many

requests you've sent, how many unique IPv4

and IPv6 hosts you've hit, how many bytes

you've downloaded, how many of each

different item type you've visisted, how

many errors of various kinds you've

encountered, etc. Again, mostly written

to aid me as a developer and not too

interesting to normal users.

Okay, that's it for VF-1. Now, onto AV-98.

I guess some readers may be in the dark about

AV-98 if they didn't start following my

Gemini posts[3] after I promised[4] to stop

spamming my own phlog with lunatic rants

about new internet protocols. To update

quickly, Gemini has certainly become "a

thing" and there are now about 10 public

servers running the protocol, whose core has

more-or-less solidified, modulo a

long-running and surprisingly complicated

discussion about how to handle text wrapping.

There are multiple server and client

implementations appearing from multiple

authors. On the client side, I have written

a Gemini client derived from VF-1, and in my

own grand tradition of giving nice software

terrible names referencing vintage anime I

called it AV-98[5]. It's how I surf

Geminispace. It has precisely the same look

and feel as VF-1. As the TLS-handling

becomes a bit more sophisticated this may

stop being true, but after the initial

Gopher-to-Gemini conversion, it turned out

that I had to remove more code than I had

to add, in order to offer more or less

exactly the same user experience. I took

this as a sign that the design decisions made

for Gemini were all more or less Good Things

and am pretty happy about it!

Right, responding at last to poor Tomasino,

if he's still awake. Tomasino pondered:

I was thinking on AV-98 and VF-1 and how

they're so wonderfully similar. It would be

so neat to have a single client that could

browse gopher & gemini protocols seemlessly

I've certainly had the same thought! In the

earliest days I was worried that this was

going to mean maintaining three separate

programs which sounded like No Fun, but

happily it turns out that, IMHO, this is no

longer necessary after I wrote another piece

of software, called Agena[6].

One way that Gemini differs from Gopher is

that, whereas Gopher clients only send a

"selector" to servers as part of their

request (Gopher selectors are roughly

equivalent to the path component of a HTTP

URL), Gemini clients send a full-blown URL

including the protocol and hostname (although

you can leave the protocol out and it will,

naturally enough, default to gemini://). The

main motivation for this was to permit

hostname-based virtual hosting, but it also

allows proxy servers - even

protocol-translating proxy servers! And

Agena is just such a thing, namely a

Gopher-to-Gemini proxy.

When you use AV-98 to follow a link to a

resource hosted via Gopher, it will try to

connect to a Gemini server running on

localhost (although you can use `set

gopher_proxy` to try anywhere else instead,

and more or on that later), and will send it

a request for the Gopher resource. If you

have Agena running, it will request that

gopher resource and then send it back to

AV-98 as a Gemini response. If the Gopher

resource is a menu, it will be translated

into a Geminimap. Agena is aware of the h

itemtype URL: hack, and will transform any

such items in a Gopher menu into ordinary

links. If the Gopher resource is not a menu,

Agena will try to figure out a MIME type for

it, in much the same way as VF-1 does, as

this is required in the Gemini response

header.

Since I've also been hacking on VF-1 I

haven't done it for a while, but after I

first wrote Agena for about a week or so I

surfed Gopherspace from home exclusively

using the AV-98-Agena combo and it worked

very, very nicely indeed. I'd even call it

seamless. And the great thing is that any

Gemini client can easily add support for a

Gopher proxy and then use Agena. The

Gopher-to-Gemini translation logic only needs

to be improved and maintained in one project,

and nobody ever needs to bother to write a

client which actually speaks both protocols

(of course, if you really want to, I won't

try to stop you!). This is a really

beautiful arrangement, IMHO.

But Tomasino goes on to note:

Oh, it would be terrible to start your

journey thinking you're on Gemini and

safely using TLS only to move into Gopher

without realizing it and be unprotected

Good point! And I like some of Tomasino's

ideas about using symbols or colours to

indicate which protocol a link is over. But

here's another thought: when you are running

Agena locally, the fact that Gemini mandates

TLS doesn't do you much good, because you're

only securing a connection over your loopback

interface, and the Gopher request goes out in

the plain so your ISP can log and analyse the

response (well, unless you're some kind of

non-VPN-using werido). But there's no reason

that your Gopher proxy has to be local! Some

brave soul could run a public Agena instance,

and AV-98 users could browser Gopherspace

through it. Then your ISP wouldn't be able

to view the content, and the original Gopher

host wouldn't learn who you were, either!

The person running the Agena instance, of

course, would know everything, although

because Gemini very deliberately doesn't have

anything analogous to cookies or superfluous

request headers which can be fingerprinted,

the admin of the public proxy has quite

limited ability to track your Gopher browsing

habits over time.

This solution isn't perfect, of course, it

would also slow Gopher browsing down a little

bit. But it's an interesting idea that's

possible with extant software. I need

another project like I need a hole in the

head right now, but if somebody else wanted

to set such a public proxy up, I'd help

spread the word.

[1] gopher://gopher.black:70/1/phlog/20190918-av-98-and-vf-1-daydream

[2] gopher://gopher.black:70/1/phlog/20190806-making-vf-1-sexy

Gopher-Gemini

[3] gopher://zaibatsu.circumlunar.space:70/1/~solderpunk/gemini

[4] gopher://zaibatsu.circumlunar.space:70/0/~solderpunk/phlog/project-gemini.txt

[5] https://tildegit.org/solderpunk/AV-98

[6] gopher://zaibatsu.circumlunar.space:70/0/~solderpunk/gemini/announcing-agena-a-gemini-to-gopher-proxy.txt