New Stuff in Tootik

=> My tootik instance

tootik is maturing quickly, and I use it for many things: for example, I follow bots that monitor news sites and publish summaries on Mastodon, allowing me to consume news (and replies) in plain text form through a Gemini client.

Thread View

When viewing a post, tootik shows mentioned users, links [...] and a compact lists of replies. Every reply is accompanied by a link to the same view but with the reply as the "post", so one can see replies to this reply. To read all posts in a thread, the user (yes, the human user) needs to follow this link recursively and this gets annoying very quickly, because it's easy to forget the context and one needs to find the link (which is not always in the same position on the screen) to follow it. Also, this doesn't scale: if a post has two replies, and both have more replies, it's hard to read one "branch" of replies, navigate back to the "junction" where this "branch" begins and repeat the recursion for other branches until you read everything, unless you have a "map" of the thread.

Now, tootik provides such a "map": posts with nested relies are accompanied by a link to a "thread view" shows all "branches" and sorts "sibling" replies chronologically. It's much easier to follow a conversation (or a heavy debate) now, especially if you use a client that marks already visited links, like Lagrange.

I think at least 3 users asked for this feature, and so far I haven't received negative feedback.

=> https://github.com/dimkr/tootik/commit/84a388d2bf5993492652b81dcc52441b4256cb84

Polls

tootik now allows users to vote on polls, view a poll results graph and publish Station-style polls.

Poll support is important from a project maturity viewpoint, because it creates many corner cases that must be handled (like the visibility of "vote posts").

=> https://github.com/dimkr/tootik/commit/0063bd52fb04cf129e708f8377ba343f8e06e78d | https://github.com/dimkr/tootik/commit/ce6fb8da66c4e7cca4ff41be2c23362293724ffe | https://github.com/dimkr/tootik/commit/1871ce83210b6cf04081e9d6905e0550760e2d50

Help Page

The addition of polls adds more "rules" one needs to know when publishing posts, making some kind of documentation that's integral to the tootik UI (as opposed to, the git repo) a necessity.

=> https://github.com/dimkr/tootik/commit/8bcdcc628044add5b5668c193d42186649ee4476

Shared Inbox

tootik now sets the endpoints.sharedInbox property on users (to the inbox of the special "nobody" user), so a tootik instance with two followers of the same user on another server receives only one request when this user publishes a new public post, or when a poll published by this user receives more votes. This reduces resource consumption a lot, and the number of requests tootik needs to handle has dropped significantly. (I still see some duplicate requests that get filtered; I guess some serveres still have old cache with unspecified endpoints.sharedInbox.)

In addition, when a user of a tootik instance has two followers on another server, both with the same shared inbox, tootik now sends only one request to this server when the the user publishes a new public post. This reduction in the number of outgoing requests speeds up delivery of posts to other servers.

Poll results are sent to servers with followers of the poll author, so a long list of active polls is not a big problem even if the poll author has many followers (assuming they have "shared inboxes" and actually share them).

=> https://github.com/dimkr/tootik/commit/b52c2b30025a7fa459369922754cf2ab0d2a3006 | https://github.com/dimkr/tootik/commit/cde8fff0a9d3eb50c4a7b4e7c9b3014f40d379d6 | https://github.com/dimkr/tootik/commit/18dc00c58096767f32df2e3ac64f06d0db772b28

Access Control

Previously, tootik allowed anyone who knows the "secret" hash of a post ID to see the post. Most post IDs contain something that's hard to predict, like a timestamp or a sequence number, but there are many corner cases where one can "discover" this "secret" (for example, through another post).

Now, tootik filters content by the "to" and "cc" fields, and all fixed corner cases are reproduced in newly added tests.

=> https://github.com/dimkr/tootik/commit/87d9ca2658f610c463d0599408ff68f42f107da8 | https://github.com/dimkr/tootik/commit/9265a8b803350175bd124b14cf8cb33f5c1f99f3 | https://github.com/dimkr/tootik/commit/2a3dc897a2bcdc702c1f464b7ee1b546379553dc | https://github.com/dimkr/tootik/commit/2fc6a7515784051f6896b091b178518d45675b85 | https://github.com/dimkr/tootik/commit/6003654115e572c872b81067e1f5e498c727eb3d

Performance Optimization

The slowest pages (/users and /users/inbox) are still slower than anything else, but they're much faster now. Previously, a big query used to match all posts within a time frame against the user's ID, followed users, followed groups and posts by the user (to show replies), and it's impossible to speed up such a query using an index. The query reads many columns from the posts table, forcing it to do a slow "scan" that checks for many conditions on many posts, and a big index would only slow down insertion of posts without speeding up the query. Now, each "why should I see this post" condition is a separate (and fast) query: search of posts by group can use one index, search of posts by author can use another, search for replies to posts by a specific author can use another, and all queries can run in parallel to maximize utilization of the disk's read bandwidth.

With all these new features, my instance still runs comfortably on a $5/mo VPS. Total RAM consumption (with the underlying OS and evreything) is around the 120 MB mark, and CPU consumption is usually <=5%.

=> https://github.com/dimkr/tootik/commit/c506d9457b1a4b6805e3bda4abe89ef362d14b2d | https://github.com/dimkr/tootik/commit/01921857d65719315eeb2fb13ab3489f7449f72f

Tests

tootik's test coverage has increased a lot: pretty much every change is now accompanied by tests and bugs are fixed once they have tests that reproduce them (then act as regression tests). It's much harder to write broken SQL now.

guppy:// Support

tootik now supports guppy://, in addition to Gemini, Gopher and Finger. This new protocol will be useful for low latency consumption of public content like news, and every user that consumes "feeds" (like the federated posts feed) over guppy:// instead of gemini:// reduces load on the server. I can see myself using this on the long way to work, if and when Lagrange for Android gains guppy:// support (I'm already using guppy:// on two devices!). TLS handshakes time out often due to dodgy mobile signal, while guppy:// allows pages to be received in a single burst of packets, without a preceding three-way handshake or key exchange.

=> https://github.com/dimkr/tootik/commit/bfb085ec82a655e04238da023c527d53ba0ff53d | The Guppy protocol homepage | Lagrange Guppy support pull request

Proxy Information
Original URL
gemini://gemini.dimakrasner.com/tootik-new.gmi
Status Code
Success (20)
Meta
text/gemini
Capsule Response Time
3922.587361 milliseconds
Gemini-to-HTML Time
2.291346 milliseconds

This content has been proxied by September (3851b).