This page permanently redirects to gemini://gemini.complete.org/using-filespooler-without-queues-to-pass-more-metadata/.

Using Filespooler Without Queues to Pass More Metadata

One frustration people sometimes have with ssh or NNCP[1] is that they'd like to pass along a lot of metadata to the receiving end. Both ssh and nncp-exec allow you to pass along command-line parameters, but neither of them permit passing along more than that. What if you have a whole host of data to pass? Maybe a dozen things, some of them optional? It would be very nice if you could pass along the environment.

=> 1: /nncp/

Filespooler[2] lets you do just that. As mentioned in the Filespooler Reference[3], any environment variable beginning with FSPL_SET_ that's set when fspl prepare is called is written into the header and will also be set for the command run by fspl queue-process or fspl stdin-process.

=> 2: /filespooler/ | 3: /filespooler-reference/

The support added in Filespooler 1.0.8 for not using a sequence file on the sender makes this easier.

Consider this example:

FSPL_SET_FOO=123 fspl prepare -s - | \
  fspl stdin-process bash -- -c set | grep FSPL_SET_FOO
FSPL_SET_FOO=123

Of course, this can be combined with ssh:

FSPL_SET_FOO=123 fspl prepare -s - | \
  ssh otherhost fspl stdin-process bash -- -c 'set' | \
  grep FSPL_SET_FOO
FSPL_SET_FOO=123

Of course, you can both set the environment and use stdin. Let's say you wrote this script and saved it in ~/test.sh:

#!/usr/bin/env bash

set -euo pipefail

echo "Hex dump of $FSPL_SET_FILE:"
hd

Now you can run this service remotely:

echo hi | \
  FSPL_SET_FILE=hi.txt fspl prepare -i - -s - | \
  ssh otherhost fspl stdin-process ~/test.sh
Hex dump of hi.txt:
00000000  68 69 0a                                          |hi.|
00000003

This technique also works with nncp-exec; see also Using Filespooler over NNCP[4].

=> 4: /using-filespooler-over-nncp/

Links to this note

Using Filespooler over NNCP[5]

=> 5: /using-filespooler-over-nncp/

NNCP[6] is a powerful tool for building Asynchronous Communication[7] networks. It features end-to-end Encryption[8] as well as all sorts of other features; see my NNCP Concepts[9] page for some more ideas.

=> 6: /nncp/ | 7: /asynchronous-communication/ | 8: /encrypted/ | 9: /nncp-concepts/

Filespooler[10]

=> 10: /filespooler/

Filespooler lets you request the remote execution of programs, including stdin and environment. It can use tools such as S3, Dropbox, Syncthing[11], NNCP[12], ssh, UUCP[13], USB drives, CDs, etc. as transport; basically, a filesystem is the network for Filespooler.

Filespooler is particularly suited to distributed and Asynchronous Communication[14].

=> 11: /syncthing/ | 12: /nncp/ | 13: /uucp/ | 14: /asynchronous-communication/

Using Filespooler Without Queues to Pass More Metadata

Links to this note

More on www.complete.org