2021-07-25 Gemini, CAPCOM, Antenna, and Tower

A few days ago I noticed that CAPCOM has become practically useless to me. Remember when @solderpunk reorganized CAPCOM a while ago?

=> @solderpunk

Each month, CAPCOM randomly selects 100 distinct URLs from its list of known feeds, and includes their content in its output. This makes it a nice way to discover new content in Geminispace.” – CAPCOM

=> CAPCOM

At the time I liked the idea. But of the 64 links visible right now, 20 are links to my blog. This doesn’t feel right. I sent Solderpunk an email and suggested he change the feed. I have multiple feeds and currently I think it shows all the regular changes to wiki pages, and to comment pages. This is not good. There’s a different feed that shows just new blog posts. I think that’s the one people should be subscribing to, if at all. Or just use my front page as a gemlog. 😁

=> My blog feed (Gemini links, Atom format) | Subscribing to a gemlog

Anyway, I’m not sure the wrong feed is the only reason I seem to be dominating CAPCOM right now. It’s true, I’m enjoying summer break these days and that means I’m not working – I have plenty of time to write blog posts. Like this one. It’s also true that some of the links would disappear if Solderpunk were to change my feed: comment pages would disappear, book club pages would disappear… I still fear that I might be taking up 10% of CAPCOM.

I have written two feed aggregators. Perhaps their implementations are simply better than what CAPCOM is doing at the moment.

Moku pona produces one line per feed it watches; when it works without bugs, that means it links to each feed exactly once, simply reordering them when updates come in. Active feeds are at the top. You don’t know whether you’ll like what’s new on the feed, but after a while you start remembering the names. I like it.

=> App::mokupona

Jupiter takes the four latest entries of every feed, sorts them all by date, and keeps the 100 most recent ones. Again, nobody can really overwhelm the feed. You can’t have more than 4/100 entries.

=> App::jupiter

@tinyrabbit said that they suspected many CAPCOM feeds to be abandoned. Thus, if CAPCOM picks a subset of the feeds it knows, many of them have no updates, so the ones that do (like mine) start dominating.

=> @tinyrabbit

We started talking about Antenna. Antenna is different in that it doesn’t watch pages for changes like my feed aggregators do. Feed authors have to ping Antenna if they have an update. Antenna updates the feeds it is pinged with, once every 10 minutes.

Let’s quickly compare numbers. Moku pona checks all the URLs for changes (gemlog front pages or actual feeds). I subscribe to 55 sites and check them twice a day, making 110 requests per day. But look at the updates I’m actually showing right now:

awk '{ print $3 }' updates.txt | uniq --count
      5 2021-07-25
      6 2021-07-24
      1 2021-07-23
      1 2021-07-21
      2 2021-07-20
      2 2021-07-19
      1 2021-07-18

That’s not a lot of changes. It seems that I should be able to make do with about 5 requests per day! So if all the services knew to a personal Antenna installation, they would make 5 pings, and my Antenna installation would make 5 requests, done for the day. 10 requests instead of 110.

Now it looks as if Antenna is built to be a central hub. But think about it. Doesn’t ActivityPub allow just this? Don’t email newsletters do just this? You “sign up” to get updates from a server, and when they have updates, the send you a ping, and you have some software that knows what to do with it. What Antenna does is get pings and it knows what to do with it: it requests the URL mentioned in the ping and adds it to the updates it knows about. What we’re missing is the sign up part.

Currently, the one Antenna installation asks authors to ping it, using tools of their own devising. It works surprisingly well and shows that Gemini makes it easy to write clients and servers and services. I love it!

But obviously, not every reader can ask all the authors to ping them. Or I guess they could, like all the newsletters you can subscribe to, like all the fediverse accounts you can follow using ActivityPub. What we’re missing is the right tools.

Let’s imagine this missing tool and call it Tower.

I have a blog, and it has a Tower installation. You want to get pinged when my blog updates, and you have an Antenna installation. You visit my blog, find the Tower link, and submit your Antenna URL. When my blog updates, it calls Tower, and Tower goes through all the Antenna URLs it has listed, and pings them. Then each Antenna installation can fetch the updated front page of my blog, or my feed, and add new entries to their list.

Now, we all have different Gemini hosting software, so perhaps it’s not as easy to implement. We could implement Tower as a service on the net! This Tower installation doesn’t just get called when my blog updates. Instead, it watches a bunch of URLs.

“Huh?” you ask. And you’d be right. Where’s the benefit? It’s true, if we don’t all implement a Tower convention, the use is limited.

Let’s use some imaginary numbers again. Let’s say we’re 100 people, each of us is following 50 gemlogs. Let’s say we all have moku pona installed and they all update twice a day, like mine does. Each of use makes 2×50=100 requests per day, us being 100 people results in 10,000 requests per day. This is when we all use self hosted feed readers.

Let’s assume somebody wrote the ominous Tower service and all 100 people signed up. They don’t actually add Tower to their blogs, they tell the one centralised tower service: this is the URL of my personal Antenna installation and these are the 50 URLs I want to follow (big privacy issue, for sure – let’s just continue with the thought experiment to get an idea of the number of requests this would result in). The Tower service checks all 100 URLs twice a day, making 2×100=200 requests per day.

It then pings all the signed up Antenna installations with the URLs they are following. We need to make some more assumptions. Let’s say that about 10% of all the blogs update on a given day: 10 of them. Since we’re all following 50 gemlogs, let’s assume that this numerical thought experiment is pretty egalitarian: there are no gemlog superstars, no long tail, instead we have a wonderful, uniform distribution. That means every person’s Antenna gets about 10×½=5 pings every day. All the Antenna installations then fetch each of these URLs, so 100×5=500 requests to update their data. Let’s hope the pings are spread out or else those poor servers are hit by all the Antenna installations at once!

Let’s add this all up: Tower makes 200 requests per day, sends out 500 requests per day, and the Antenna installations then make an additional 500 requests per day. That’s 200+500+500=1,200 requests per day instead of the 10,000 requests per day I mentioned above.

So clearly, something is good about this. Perhaps the numbers are worse because there are multiple Tower service installations. It’s still better than everybody running their own moku pona. It’s also a privacy issue, as I said. The Tower service operator would know which Antenna installation subscribed to what URLs. At least with private moku pona installations, it’s hard to know.

​#Gemini ​#Feeds