Ancestors

Toot

Written by Sebastian Schöner on 2025-01-20 at 12:41

I chose dotnet/C# for a tool I wrote. It's running a bunch of processes and redirects their outputs. However, redirecting the output seems to block: even the async variants (BeginOutputReadLine, or manually reading async from the stream) just move the blocking calls onto a threadpool, meaning that the threadpool is clogging up. You can get fully async reading from pipes on Windows (https://blog.s-schoener.com/2024-06-16-stream-redirection-win32/). Do any of my C# friends know whether that is implemented somewhere in dotnet already?

=> View attached media

=> More informations about this toot | More toots from sschoener@mastodon.gamedev.place

Descendants

Written by Ritesh Oedayrajsingh Varma on 2025-01-20 at 14:51

@sschoener this is a trap in default StreamReader, FileStream etc. The ‘async’ part is that it’s not on the same thread, but that’s it. To make it actually async as you’d expect, you need to specify FileOptions.Asynchronous (https://learn.microsoft.com/en-us/dotnet/api/system.io.fileoptions?view=net-9.0) when creating the stream. That will turn the async operations into actual async overlapped IO.

=> More informations about this toot | More toots from rovarma@mastodon.gamedev.place

Written by Ritesh Oedayrajsingh Varma on 2025-01-20 at 14:53

@sschoener For StreamReader that means creating it with a FileStreamOptions (https://learn.microsoft.com/en-us/dotnet/api/system.io.filestreamoptions?view=net-9.0), though I’m not sure how to replace the default reader of a process object with a new, properly async, instance

=> More informations about this toot | More toots from rovarma@mastodon.gamedev.place

Written by Ritesh Oedayrajsingh Varma on 2025-01-20 at 14:55

@sschoener honestly, using any of the *Async functions on readers should throw an exception when called on what is effectively a sync reader (i.e. without the option specified), but I guess it’s too late to change that now :P

=> More informations about this toot | More toots from rovarma@mastodon.gamedev.place

Written by Daniel Collin on 2025-01-20 at 14:57

@rovarma @sschoener Maybe I'm oldschool, but I create my own threads and put the work there using a simple handle api to query if something is ready or not :)

=> More informations about this toot | More toots from daniel_collin@mastodon.gamedev.place

Written by Sebastian Schöner on 2025-01-20 at 15:28

@daniel_collin @rovarma I do that as well in my native code but C# already has an ecosystem where they create numerous threadpools. You'd think they'd have this sorted, but alas as so often I am punished for taking a dependency on anything... so yeah, probably time to rewrite this in my own stack.

=> More informations about this toot | More toots from sschoener@mastodon.gamedev.place

Written by Michael Vance on 2025-01-20 at 16:20

@sschoener @daniel_collin @rovarma there should be a term for the hope you have in using a library to make things easier or more canonical, and then the inevitable disappointment.

=> More informations about this toot | More toots from mtothevizzah@mastodon.gamedev.place

Written by Ritesh Oedayrajsingh Varma on 2025-01-20 at 16:52

@mtothevizzah @sschoener @daniel_collin NIH? :P

=> More informations about this toot | More toots from rovarma@mastodon.gamedev.place

Written by Sebastian Schöner on 2025-01-20 at 17:37

@rovarma @mtothevizzah @daniel_collin forward declared regret.

=> More informations about this toot | More toots from sschoener@mastodon.gamedev.place

Written by Sebastian Schöner on 2025-01-20 at 15:27

@rovarma thanks! OK, then I'll probably have to do it all manually.

=> More informations about this toot | More toots from sschoener@mastodon.gamedev.place