Migrating prosody internal storage to SQLite on OpenBSD

• Author: Solène
• Date: 18 August 2023
• Tags: prosody xmpp openbsd

=> Comment on Mastodon

Introduction

As some may know, I'm an XMPP user, an instant messaging protocol which used to be known as Jabber. My server is running Prosody XMPP server on OpenBSD. Recently, I got more users on my server, and I wanted to improve performance a bit by switching from the internal storage to SQLite.

Actually, prosody comes with a tool to switch from a storage to another, but I found the documentation lacking and on OpenBSD the migration tool isn't packaged (yet?).

The switch to SQLite drastically reduced prosody CPU usage on my small server, and went pain free.

=> Prosody documentation: Prosody migrator

Setup

For the migration to be done, you will need a few prerequisites:

• know your current storage, which is "internal" by default
• know the future storage you want to use
• know where prosody stores its files
• the migration tool

On OpenBSD, the migration tool can be retrieved by downloading the sources of prosody. If you have the ports tree available, just run make extract in net/prosody and cd into the newly extracted directory. The directory path can be retrieved using make show=WRKSRC.

The migration tool can be found in the subdirectory tools/migration of the sources, the program gmake is required to build the program (it's only replacing a few variables in it, so no worry about a complex setup).

In the migration directory, run gmake, you will obtain the migration tool prosody-migrator.install which is the program you will run for the migration to happen.

Prepare the configuration file

In the migration directory, you will find a file migrator.cfg.lua.install, this is a configuration file describing your current prosody deployment and what you want with the migration, it defaults to a conversion from "internal" to "sqlite" which is what most users will want in my opinion.

Make sure the variable data_path in the file refers to /var/prosody which is the default directory on OpenBSD, and check the hosts in the "input" part which describe the current storage. By default, the new storage will be in /var/prosody/prosody.sqlite.

Run the tool

Once you have the migrator and its configuration file, it's super easy to proceed:

• Stop the prosody server with rcctl stop prosody
• Modify /etc/prosody/prosody.cfg.lua to use the sql driver instead of internal

storage = "sql"
sql = {
    driver = "SQLite3";
    database = "prosody.sqlite";
}

• Backup /var/prosody in case something is going bad
• Run the migration with lua53 prosody-migrator.install --config ./migrator.cfg.lua.install
• Verify the file /var/prosody/prosody.sqlite exists and isn't empty
• Chown /var/prosody/prosody.sqlite to _prosody:_prosody
• Start the prosody server with rcctl start prosody, check everything is working fine

If you had an error at the migration step, check the logs carefully to check if you missed something, a bad path, maybe.

Conclusion

Prosody comes with a migration tool to switch from a storage backend to another, that's very handy when you didn't think about scaling the system correctly at first.

The migrator can also be used to migrate from the server ejabberd to prosody.

Thanks prx for your report about some missing steps!