Ancestors

Toot

Written by 🇨🇦🇩🇪🇨🇳张殿李🇨🇳🇩🇪🇨🇦 on 2025-01-06 at 14:00

This is a weird question, probably, and it comes in two parts:

1. What would a Fediverse-style search system look like?

2. Is there such a thing yet?

Note: I'm not talking about metasearch engines. I'm talking about truly distributed search in the vein of how Mastodon is distributed microblogging or Pixelfed is distributed Instagram or the like.

=> More informations about this toot | More toots from zdl@mastodon.online

Descendants

Written by Andrew Helwer on 2025-01-06 at 14:30

@zdl like a search engine for the web as a whole, not just the fediverse?

It's an interesting idea. Crawling the web with bots for indexing is probably the hardest part, because the amount of data & traffic becomes quite large. I wonder whether operators would have to spend a lot of time overcoming anti-spam measures. Then you overlay some search algorithm over the data you've collected.

Unfortunately I don't see where the "federated" part of the search engine comes in. Perhaps people could split up & share the work of crawling & indexing, forming a sort of distributed internet archive. But search engines don't inherently involve any sort of sharing or communication so I don't know how it could be applicable beyond that.

=> More informations about this toot | More toots from ahelwer@discuss.systems

Written by 🇨🇦🇩🇪🇨🇳张殿李🇨🇳🇩🇪🇨🇦 on 2025-01-07 at 01:16

@ahelwer Yes. Basically a federated version of Google/DDG/Bing/Whatever.

The problem with having a single service that does Internet search is that you have a single point of failure and a single-vendor situation that provides incentives to enshittify the product. (Like Google, as bad as it has become, still finds more things than its next two competitors.)

A paid service would start off better, but remember that Ticketmaster used to be good once and now...

🧵 ▶️

=> More informations about this toot | More toots from zdl@mastodon.online

Written by 🇨🇦🇩🇪🇨🇳张殿李🇨🇳🇩🇪🇨🇦 on 2025-01-07 at 01:19

@ahelwer So any single-vendor solution, whether paid or ad-driven is going to enshittify. And probably really quickly given the speed at which enshittification has started to happen these days.

Metasearches are better but don't help the problem of a single site getting so much better that it can afford to start twisting the screws. Metasearches now use several search engines, but the one that still gets the most results is Google, and that gives Google motivation for shenanigans.

🧵 ▶️

=> More informations about this toot | More toots from zdl@mastodon.online

Written by 🇨🇦🇩🇪🇨🇳张殿李🇨🇳🇩🇪🇨🇦 on 2025-01-07 at 01:21

@ahelwer Human-curated sites are another solution, but have the problem of being either a) perpetually out of date or b) dominated by a clique of curators that set the tone and tenor of the site in their image: good results in places the clique has interest, bad results in places the clique doesn't care about, and on top of that inflicting their version of morality, say, on others.

🧵 ▶️

=> More informations about this toot | More toots from zdl@mastodon.online

Written by 🇨🇦🇩🇪🇨🇳张殿李🇨🇳🇩🇪🇨🇦 on 2025-01-07 at 01:25

@ahelwer (If you want an example of that for a human-curated site, look no farther than the utter Hellhole that is Wikipedia editing.)

So ... I started to think about what a distributed search that operated over some kind of federated system like Mastodon or Pixelfed or whatnot do. And rammed my head straight into a conceptual wall because I'm a dullard and a technophobe.

Hence my asking what a federated search system would even look like.

🧵 ⏹️

=> More informations about this toot | More toots from zdl@mastodon.online

Written by Jens Finkhäuser on 2025-01-15 at 18:59

@zdl @ahelwer Personally, I think something that captures personal recommendations as well as specialized search is for folk to publish indexes, and for people to subscribe to them. Your federation infrastructure would effectively distribute incremental index updates.

You still concentrate a lot into the hands of people running good indices, who might be prompted to try shenanigans. But since the output of their work is effectively open and distributed, it'd be relatively easy to fork it.

=> More informations about this toot | More toots from jens@social.finkhaeuser.de

Written by Jens Finkhäuser on 2025-01-15 at 19:00

@zdl @ahelwer The main downside is that you can't just download a whole search engine's worth of index data to your mobile phone, so there's some slice of indices you'll likely subscribe to in practice. This may be generated based on your search query!

=> More informations about this toot | More toots from jens@social.finkhaeuser.de