New #scholarlypublishing book! PRC, Publish-Review-Curate, can replace the cloistered traditional journal management system (Cathedral) way. By breaking up the monolithic scholarly communication Cathedrals, not only do we reduce gatekeeping and speed dissemination, https://www.linkedin.com/posts/adamhyde_prc-vs-the-cathedral-activity-7284530548255965185-ldwk
we also can escape the dominance of corporate publishers by building on the infrastructure of non-profit preprint servers, including adding peer review.
I haven't actually read the book yet but knowing Adam Hyde and how he's worked with us at MetaROR (MetaResearch Open Review) and with Elife, I'm betting he discusses these things!
=> More informations about this toot | More toots from alexh@fediscience.org
@alexh i am a huge post-pub peer review advocate, read this yesterday, but was really not a fan of this. it's written by chatGPT and it shows - a bunch of nonsensical arguments peppered throughout, some factual inaccuracies that would have been resolved by simply looking at the thing he was writing about, and it feels like most of the hardest issues (profit motive, moderation, credit assignment, platform capture, etc.) were just not attempted. Not talking about moderation is a huge miss, like talking about going to the moon without talking about space being a vacuum. I couldn't tell if the FOSS analogy was strictly intended as an analogy (doing a cathedral/bazaar framing in 2025 is... rough) but there was a lot of it, and most of it makes it sort of clear that this person doesn't really engage much with the community, surface level to the point where the analogy is an active negative to credibility.
this is sort of a nitpick, but it also felt a little bit like when someone publishes a work about open access in a paywalled journal - publication formats, metadata, and, well, public review are all parts of post-pub peer review, and how are you gonna publish a novella about them as a busted ass PDF without means of doing review? the author even talks about hypothes.is and has written several blog posts about single source publishing but that's just... not there? He runs a company that makes publishing software that specializes in PRC models! Same thing re: talking a lot about preprint servers as a major player - i am positive that arXiv would not accept this, but a place like hcommons would, and merely trying to walk the walk here would have made the piece much richer with a discussion about some of the problems with preprint servers, their conflicting motivations, and the different ways they are moderated and governed (the several sections about the aesthetics of preprint servers were very odd and also factually wrong, but they were also the only parts where there was some criticism there).
I wanted to like it, definitely am on the same page re: post-pub peer review being a necessary move, but like i said, i was left with the impression that I always am with things that are written by an LLM - bland without much substance. The appendix on how this book was written handwaves at some "all ideas are collaborative" and "authorship is a myth" argument (again, making it pretty clear he has not engaged seriously with criticism of LLMs), claiming that it was a tool that helped him communicate the macro ideas as clearly as possible without worrying about the mechanics of writing. I want to tell him that the exact opposite is true - the writing was distractingly generic, introduced obvious errors and masked any original insight entirely.
=> More informations about this toot | More toots from jonny@neuromatch.social
@alexh i don't mean to dog on the guy, again same team and I don't even know him, but there is this big sense of columbus complex here and this is an example - I personally have been talking about preprints-only and how preprints should just be called prints in public since at least 2018 when i decided i didn't care about staying in academia and i am not even in the top 50% of people who think the most about this and was for sure at least a decade late on that. I'm pretty sure NISO is even considering this in the JAV standard it's that far along.
Releasing a book early and iterating is great! nobody expects it to be perfect! but if there is anything i have learned in this space it's to start by assuming someone is already doing something, do a ton of legwork to try and find them, give them credit for it, see if you can join up, and then assume you still only have like 5% visibility over the space. A little humility goes a long way - we don't need a movement name because it's been going for a long time already, and in any case why assume it should be you that names it?
sorry for addressing this to you, i did leave comments on the hypothes.is copy of the book but i am not gonna join linkedin to talk about this and i keep seeing it pop up.
=> More informations about this toot | More toots from jonny@neuromatch.social
@jonny @alexh brutal review Jonny. Proposal: books written with ChatGPT should have an appendix with just the prompts used in order. Shorter and fewer errors.
=> More informations about this toot | More toots from neuralreckoning@neuromatch.social
@neuralreckoning
@alexh
I do not want to just harp on that problem and did try to look past it, but that is the part I am genuinely interested in - the argument he was trying to make, but there were some parts that were very clearly GPT taking the wheel (the bioRxiv/hypothes.is part was particularly jarring because one can verify it's not the case by going to any biorXiv paper, but there is a ton of press coverage and blog posts that say it should be true that would have been in the training corpus). That made me doubt the entirety of the rest of the text because I couldn't tell what part was actually intentional, with some actual experience and insight that he as a human being has, and which part was just infill.
It turns into a fundamental issue when you structure a piece around a concept and then you're missing big pieces of it that get filled in halfheartedly by a language model. Like however you feel about cathedral/bazaar framing, it has tangible meaning. it also commits you to a number of structuring assumptions about what works - like even a journal mediated PRC process like eLife is not exactly a "bazaar-like" process. That space between "still journals mostly" and "self-coordinated public review" is the whole topic, but it only gets contended with piecemeal and without much treatment of the problem as a landscape of interrelated problems. As someone who has read a shitload of longform LLM prose masquerading as human text at this point on purpose, that kind of paragraph-level coherence but macro-level incoherence is a distinct hallmark.
=> More informations about this toot | More toots from jonny@neuromatch.social
@jonny Thank you Jonny, that's very thoughtful review! and so good to point to the bigger issues like you do here. Do you mind if I point Adam to this post? @neuralreckoning
=> More informations about this toot | More toots from alexh@fediscience.org
@alexh @neuralreckoning sure, no problem! like i said the only reason i wasn't responding directly to him is because i'm not gonna do something like post on linkedin.
=> More informations about this toot | More toots from jonny@neuromatch.social
@jonny @neuralreckoning thanks!.And holy smokez , I just discovered.you have a podcast! You should promote it more.
=> More informations about this toot | More toots from alexh@fediscience.org
@alexh @neuralreckoning me? or Dan's? https://braininspired.co/podcast/183/
=> More informations about this toot | More toots from jonny@neuromatch.social
@jonny @neuralreckoning i was talking about Preprints in Motion but I also didn't know about Dan's ! (Mind blown)
=> More informations about this toot | More toots from alexh@fediscience.org
@jonny @neuralreckoning oh I see maybe that is just an episode of another podcast interviewing dan?
=> More informations about this toot | More toots from alexh@fediscience.org
@alexh @jonny yeah it is. 😄
=> More informations about this toot | More toots from neuralreckoning@neuromatch.social
@alexh
@neuralreckoning
Must be someone else! I dont know what preprints in motion is :)
=> More informations about this toot | More toots from jonny@neuromatch.social
@jonny @alexh @neuralreckoning Not the same Jonny indeed :) https://jacoates.co.uk/
=> More informations about this toot | More toots from mstimberg@neuromatch.social
@jonny @neuralreckoning @alexh Having programmed collaboratively for years in a nearly as pure as it gets bazaar model (early Fiji software years), and being now an eLife editor: no, eLife's publication system very much follows the cathedral model.
=> More informations about this toot | More toots from albertcardona@mathstodon.xyz
@jonny @alexh You make it sound like the author didn't read his own book. And if that's true, it's the worst possible kind of sin in publishing.
=> More informations about this toot | More toots from albertcardona@mathstodon.xyz This content has been proxied by September (3851b).Proxy Information
text/gemini