Ancestors

Toot

Written by Kris on 2025-02-02 at 10:14

How aaS sucks, and how OpenAI broke theirs

Paraphrased from the Pandemic Coping Discord:

Kris> The latest OpenAI ChatGPT update is pretty broken. Everything using text generation and creative writing is defective now. Many other things that previously worked now don't.

Kris> Generally speaking, OpenAI has no strategy that doesn't change the existing model in preexisting deployments, preserving existing known-to-work workflows.

Kris> That is, I just canceled my OpenAI subscription.

H> O3 mini is optimized for "STEM" and "software development". It is Feb, somebody has a quarterly OKR "+XX% more software development customers by end of Q1", and that matches firing season (Feb to Apr in silicon valley).

I> (Preserving preexisting deployments) Never thought about that. Good point. Does any of these valleybros have that as an issue on their radar?

Kris> Ditched ollama. It appears that brew install lm-studio is superior. Also has an API server, and a CLI component. That and mistral-nemo kind of works, and so does Codestral.

Kris> mistral-nemo has up to 1M token context, if you have the memory. 16k uses 10 GB, 32k uses 22 GB, and then I am out. Maybe I do need this M4pro with 64 GB.

Kris> (never thought about that) Industry-wide problem, maybe with the exception of AWS. All the aaS should offer the latest version and the two previous versions to not break preexisting deployments. The oldest version, scheduled for deletion, should have a penalty price increase to incentivize migration to a more recent version. Almost no aaS offer has that.

=> More informations about this toot | More toots from isotopp@infosec.exchange

Descendants

Written by Michael on 2025-02-02 at 10:32

@isotopp Just the concept of a "fireing season" is totally crazy

=> More informations about this toot | More toots from mschfr@mastodon.social