i think honestly what i want out of a storage solution is to like maybe upload stuff to some kind of object storage and then separately have a graph database or rdf quad store for the metadata. but i’m wondering if it can be made simpler without losing the ability to handle any general use case, or if this is as simple as it gets
(a filesystem is the wrong level of abstraction, i’m interested in logical handling of objects which may or may not have content that is either binary or text)
=> More informations about this toot | More toots from trwnh@mastodon.social
the area where my thinking seems to differ from most of what i've seen so far is that i don't make a real distinction between text and binary anymore. content is content. so this implies that the object storage is not just for media, but for everything, including text files. but the thing is that content should be strictly content. what most people get "wrong" with HTML for example is that they require an entire structural wrapper, which is itself wrapping not just a body but also a header
=> More informations about this toot | More toots from trwnh@mastodon.social
entirely natural, of course -- HTML is most often used to serialize documents that are browsed on the Web. but for the most part, that's all presentational stuff! if you removed it, you would lose aesthetics, but the core of the message is only some certain part, that can be extracted and handled on its own. the rest of it is just part of the view or (re)presentation.
i've talked before about how you can combine body content with header metadata and get a document that can itself be body,
=> More informations about this toot | More toots from trwnh@mastodon.social
and you can of course unwrap those layers of containers of head+body as well. when there's nothing left to unwrap or extract, that leaves you with some base atomic content, which may be just some plain text.
i argue that this is not meaningfully different than, say, a png file. it's just that text is often inlined into the presentation layer. we don't go about browsing the metadata and then linking to the content, we just stick the content directly into the same view as the metadata...
=> More informations about this toot | More toots from trwnh@mastodon.social
but just as we can inline text content, we can also separate it and link to it, we can stick it into object storage and address it as that literal text. and we can then arbitrarily package and wrap it into whatever containers or as many layers of containers as we care to do so. we could take some plaintext content and wrap it in some HTML template that itself gets wrapped in what eventually becomes an HTML document which itself gets wrapped in an HTTP message. this is more or less how we do rn.
=> More informations about this toot | More toots from trwnh@mastodon.social
the takeaway really, is that if we want the content to be replicable and manageable by some user(s), then it needs to be put into some data store, and it is exactly that data store that i am trying to model. the data model needs to support lossless serialization into whatever convenient format some person might ask for or need. i don't think "sql data dump" is an ideal export format, and i don't think "a really big json file" is ideal either. i want something that can be split up into atoms.
=> More informations about this toot | More toots from trwnh@mastodon.social
@trwnh its funny to me how we often reply to this by not the big json file, because it's impractical and a deeper structure would be even worse, but json-lines of individual activities
=> More informations about this toot | More toots from alice@gts.void.dog
@alice yeah this is likely because people just "know" json and how to work with it, but it's not at all efficient for large data stores lol
=> More informations about this toot | More toots from trwnh@mastodon.social
@trwnh @alice unironically more people need to learn how to work with goddamn binary formats and if not that MIME Multipart
=> More informations about this toot | More toots from erincandescent@erincandescent.net
@erincandescent @alice @trwnh
... if you're going to export huge amounts of data, really huge - have the decency to write a reader and encoder for the data you're trying to preserve. When someone will , in the distant future, try to decode your trove, they'll appreciate your foresight.
=> More informations about this toot | More toots from tuban_muzuru@ohai.social
@tuban_muzuru @erincandescent @alice in most cases "always bet on plain text" is good enough for that kind of thing imo. this is more about strategy and architecture of like... managing content. a sort of storage strategy, one that can handle abstract backends
it's probably going to look less like an sql database and a lot more like object storage in the end: the blob being the content (even if it's as simple as a literal string), and the metadata being whatever attribute-value pairs
=> More informations about this toot | More toots from trwnh@mastodon.social
@trwnh @erincandescent @alice
Yeah - I dislike writing a parser for anything but plain text.
=> More informations about this toot | More toots from tuban_muzuru@ohai.social
text/geminiThis content has been proxied by September (3851b).