2017-12-30 Gopher Wiki

The problem with the Gopher protocol as defined in RFC 1436 is that it doesn’t allow you to send anything to the server except for a query string. That’s not good enough for wiki editing. At the same time, we don’t want to make the interaction between Gopher server and Gopher client harder than it needs to be. This page offers a proposal:

=> RFC 1436

First, the server needs to know about a new item type which says: item is a file you can write to. Below is a Gopher menu for a wiki page. The selector for this menu is Alex/menu. The selector for writing a new page is Alex/write/text.

$ **echo Alex/menu | nc localhost 7070**
iThe text of this page:		localhost	7070
0Alex	Alex	localhost	7070
hAlex	Alex/html	localhost	7070
wReplace Alex	Alex/write/text	localhost	7070
1Page History	Alex/history	localhost	7070
i		localhost	7070
iThere are no links leaving this page.		localhost	7070
.

Let’s replace the text with something! We need to end our text with a period to let the server know that we don’t intend to send any more.

$ **nc localhost 7070 << 'EOF'
Alex/write/text
test
.
EOF**
iPage was saved.		localhost	7070
1Alex	Alex/menu	localhost	7070
.

Or post a text file. You need to add the selector at the top and the period at the end:

$ **( echo Alex/write/text; cat haiku.txt; echo . ) | nc localhost 7070**
iPage was saved.		localhost	7070
1Alex	Alex/menu	localhost	7070
.

Verify the page content:

$ **echo Alex | nc localhost 7070**
Quiet disk rattling
Keyboard clicking, then it stops.
Rain falls and I think.
.

The wiki also allows a different kind of upload: binary files. When uploading a file, the server has two problems:

1. When has the end of the file been uploaded? If the client hangs up after sending the file, it can’t read the response. If the client doesn’t hang up but does a shutdown on the socket, letting the server know it’s done writing, it would work – but you can’t do it using telnet or nc. That’s why we need to tell the server how long the file is going to be.

2. What’s the content-type of the file? Sure, we could depend on file name extensions. But that doesn’t always work. My wiki restricts upload to certain MIME types, for example. Other Gopher servers might not care and that’s why this part is simply a solution I’m going to use for my wiki. I think it might just as well not be part of the new item type. It’s an local experiment, so to say.

So anyway, how would it work? It did not make an appearance in the menu above, but the selector would be Alex/write/file. The content-length is appended to this selector, separated by a tab character. wc -c counts the number of bytes and when it does that for standard input, no filename is printed.

$ **echo -e "Alex/write/file\t"`wc -c < test.png` \
  | cat - test.png \
  | nc localhost 7070**
3Page was not saved: Files of type application/octet-stream are not allowed.	localhost	7070
.

When writing a file using this selector, a content-type of application/octet is assumed.

The wiki will extract the MIME type from the selector, if possible: Alex/image/png/write/file, for example.

$ **echo -e "Alex/image/png/write/file\t"`wc -c < test.png` \
  | cat - test.png \
  | nc localhost 7070**
iPage was saved.		localhost	7070
1Alex	Alex/menu	localhost	7070
.

Test it:

$ **echo Alex | nc localhost 7070 > test2.png**
$ **diff test.png test2.png**

Note that for my particular wiki, Alex/text/plain/write/file is not quite the same as Alex/write/text. The first will store a binary blob with MIME type text/plain which will be served like any other binary file: no parsing for links and all that. The second will edit a wiki page, which can be read using a Gopher client.

I understand that allowing file uploads can be tricky. It’s up to the Gopher server to handle this with care: store files in write-only upload directories, or provide all the peer review facilities like a wiki does, it’s up to you to write a good Gopher server. To paraphrase the description of item type 9: “Item is a binary file! Server will read until the TCP connection closes. Beware.”

Let’s get back to uploading text files and the problem of peer review facilities: Ideally, authors would provide some metadata as they upload their document. A summary of changes made, a name. My wiki will interpret a fenced off block at the very beginning of the document as metadata.

The following example sends the selector for writing a text file called Alex, a metadata block, and finally the new text.

$ **nc localhost 7070 <<'EOF'
Alex/write/text

username: Alex

summary: a different Haiku

Cold and dark outside
Remember all the sick beds
Good riddance old year
.
EOF**
iPage was saved.		localhost	7070
1Alex	Alex/menu	localhost	7070
.

Check it out:

$ **echo Alex | nc localhost 7070**
Cold and dark outside
Remember all the sick beds
Good riddance old year
.

Let’s check Recent Changes, too:

$ **echo do/rc | nc localhost 7070**
iRecent Changes		localhost	7070
1Show minor edits	do/rc/showedits	localhost	7070
i		localhost	7070
i2018-01-06		localhost	7070
i		localhost	7070
1Alex	Alex/menu	localhost	7070
i20:55 UTC by Alex from 127.0.0.1: a different Haiku		localhost	7070
i		localhost	7070
i2018-01-05		localhost	7070
i		localhost	7070
1test	test/menu	localhost	7070
i21:53 UTC by 127.0.0.1		localhost	7070
.

Yay! 👍

Sources:

• gopher-server.pl
• gopher-server.t (tests)
• gopher.el, the Gopher client for Emacs, supports writing back to the wiki on the write branch

=> gopher-server.pl | gopher-server.t | gopher.el

Try browsing the wiki:

=> this site, via Gopher

Note that you will not be able to edit pages because spam protection requires you to answer a question before you are allowed to save. The question is simply that you need to say hello. Luckily, we can use the metadata for text files to do this:

$ **nc alexschroeder.ch 70 << 'EOF'
SandBox/write/text

username: Alex

answer: hello

This is a test.
.
EOF**
iPage was saved.		alexschroeder.ch	70
1SandBox	SandBox/menu	alexschroeder.ch	70
.

Anyway, here’s a quick setup for local experiments. You need to get Oddmuse, of course. And then:

=> get Oddmuse

Run perl stuff/server.pl starts an Oddmuse Wiki on port 8080. You can edit pages on http://localhost:8080/. Data is stored in /tmp/oddmuse.

Run perl stuff/gopher-server.pl --port=localhost:7070 --wiki=./wiki.pl --wiki_dir=/tmp/oddmuse to start the Gopher server on port 7070.

Run lynx gopher://localhost:7070 to browse the same wiki.

​#Gopher ​#Wikis

Comments

(Please contact me if you want to remove your comment.)

⁂

“Phlogs” are Gopher Blogs. I think the SDF has the biggest collection of them (link goes via a proxy, the real deal is gopher://phlogosphere.org).

=> SDF | collection | gopher://phlogosphere.org

People don't like comments.

=> don't like comments

– Alex 2018-01-07 12:49 UTC

There are still problems with the code, though. I finally decided to document it as issue 15 on GitHub. For now, the interface will stay as it is. I just have issues getting this Mojo::IOLoop code to work. [This was later fixed by switching back to Net::Server.]

=> issue 15

– Alex Schroeder 2018-01-08 14:24 UTC

Stuff I need to check:

• [[image:foo]] links ✓
• pages with non-ASCII names ✓
• gopher links ✓

– Alex 2018-01-13 17:20 UTC