Is there a https://github.com/oldmoe/litestack for #Django? The ORM already supports SQLite, what else can it be used for?
=> More informations about this toot | More toots from adamghill@indieweb.social
@adamghill I’m sure @anze3db could have this knocked up for you by Easter 🥳
=> More informations about this toot | More toots from carlton@fosstodon.org
@carlton @adamghill I think you can reproduce the whole of litestack in Django already:
Cache: DatabaseCache backend works with SQLite, right?
Queue/pub/sub: https://github.com/RealOrangeOne/django-tasks + DatabaseBackend
Full text search: ORM already supports it
But I guess we could wrap it all in a separate package for marketing purposes? 😂
=> More informations about this toot | More toots from anze3db@fosstodon.org
@anze3db @adamghill 🫠
Edit: a package which set it all up right for you, or served as an example for that, might be worth the juice.
=> More informations about this toot | More toots from carlton@fosstodon.org
@anze3db @carlton yeah that’s what I was trying to figure out. Is it… all there already?!
=> More informations about this toot | More toots from adamghill@indieweb.social
@anze3db @carlton @adamghill I'd say the only thing that is really missing here is support for SQLite's full-text search in the ORM. Litestack seems to use the FTS5 extension, but Django has no mention of it... yet? ;)
=> More informations about this toot | More toots from kytta@fosstodon.org
@anze3db @carlton no shame in “package for marketing purposes” imo 😅
=> More informations about this toot | More toots from adamghill@indieweb.social
@anze3db @carlton ok this is the part I don’t understand.
1 webserver -> 1 SQLite file (in the same Docker container?)
How do you scale to 2 webservers? I’ve read through litestream/marmot docs, but I’m missing something. They replicate to S3. Is a process running on all of the webservers keeping their SQLite db file up to date?
Sorry if this is a dumb question, I searched around a little bit but was looking for a step-by-step tutorial and failed. 🫠
=> More informations about this toot | More toots from adamghill@indieweb.social
@adamghill @anze3db @carlton You don't - it's one server only, with the database WAL (transaction log) being shipped to S3 for backup/restore.
You can run multiple Django processes to scale up on a single server, and of course get a server with more cores. The one limitation is that you can't have more than one process writing to an SQLite database at the same time... but you can have a lot of readers.
And as the database is local, the latency is super low.
=> More informations about this toot | More toots from alextomkins@fosstodon.org
@alextomkins @anze3db @carlton ok... so maybe I wasn't missing anything. 😂
https://github.com/maxpert/marmot looks like an interesting approach if I ever need more than one container I suppose.
=> More informations about this toot | More toots from adamghill@indieweb.social
@adamghill @alextomkins @carlton the premise of litestack is that you don't need to scale horizontally, because you can run a single server with 192cores which should be more than enough compute for most apps.
Your app will have low latency (no networking between db and code) and will generally be simpler, but you’ll never be able to guarantee higher nines of availability because you’ll eventually have to reboot that one server…
Distributed sqlite is a headache, better to use MongoDB instead 😂
=> More informations about this toot | More toots from anze3db@fosstodon.org
@anze3db @alextomkins @carlton “better to use MongoDB instead” no thanks once bitten twice shy
=> More informations about this toot | More toots from adamghill@indieweb.social
@adamghill @alextomkins @carlton I'm not a fan of Mongo either so this was exactly my point. If you want distributed writes you are going to have a much better experience with Mongo than SQLite.
=> More informations about this toot | More toots from anze3db@fosstodon.org
@anze3db @alextomkins @carlton I don’t really care about 9 9s or anything, I was mostly thinking about blue/green deployments.
I guess https://litestream.io/reference/restore/ could be called when the new container starts up?
=> More informations about this toot | More toots from adamghill@indieweb.social
@adamghill @anze3db @carlton Yup, that's pretty much how I've got some of our Django/SQLite sites setup in AWS:
litestream restore -if-replica-exists /database/django.db
sqlite3 /database/django.db "PRAGMA journal_mode = wal;"
Then run litestream, that then calls migrations and uWSGI/uvicorn after that.
=> More informations about this toot | More toots from alextomkins@fosstodon.org
@alextomkins @anze3db @carlton Gotcha! That makes sense, thanks. 🙏
=> More informations about this toot | More toots from adamghill@indieweb.social
@alextomkins @adamghill @carlton don't you loose all the writes that happen after the litestream restore command finishes and before all the traffic switches to the new container?
=> More informations about this toot | More toots from anze3db@fosstodon.org
@anze3db @adamghill @carlton It's always one container maximum, with a small bit of downtime during a deploy:
desired_count = 1
deployment_minimum_healthy_percent = 0
deployment_maximum_percent = 100
A few requests will fail during the deploy - but it doesn't take too long.
=> More informations about this toot | More toots from alextomkins@fosstodon.org
@alextomkins @adamghill @carlton ah, ok, with downtime during deployment this makes sense. 👍
=> More informations about this toot | More toots from anze3db@fosstodon.org
@adamghill @anze3db @alextomkins @carlton Yep, that’s how I do it in my setup: https://github.com/piepworks/blaze-starter/blob/main/start.sh
=> More informations about this toot | More toots from trey@indieweb.social This content has been proxied by September (ba2dc).Proxy Information
text/gemini