NNCP, or Node-to-Node Copy, is a suite of store-and-forward programs for Unix-based systems, created in 2017 by Russian developer Sergey Matveev. I first learned about NNCP from an excellent post by John Goerzen^, and I first gave it a try in early 2023. I now use it regularly to move files and execute tasks among my devices.
There are already a few guides for NNCP on Gemini, but NNCP can be a very confusing application to set up and configure correctly. In-depth documentation seems to be lacking, both on Gemini and on the Web: some helpful tips are buried in the nncp-devel mailing list and are not present in any guide I've seen. I want to address some of those shortcomings here.
This post will describe what NNCP is, how it works, and some examples on how to use it. Most of the examples are expanded versions of notes I wrote for myself. This is a very long document, but NNCP has a lot of depth and power, and I want to capture as much of that complexity as I can.
=> ^ Recovering Our Lost Free Will Online: Tools and Techniques That Are Available Now
NNCP is a set of CLI programs for copying files, requesting files and executing remote commands between computers. Many tools exist to do such things already, but most work synchronously: both computers must be online simultaneously and connected to each other to send data. NNCP, on the other hand, can work asynchronously.
According to the project's homepage^, NNCP is "intended to help build up small size (dozens of nodes) ad-hoc friend-to-friend (F2F) statically routed darknet delay-tolerant networks for fire-and-forget secure reliable files, file requests, Internet mail and commands transmission." What does this mean?
The project page further states: "All packets are integrity checked, end-to-end encrypted, explicitly authenticated by known participants public keys. Onion encryption is applied to relayed packets. Each node acts both as a client and server, can use push and poll behaviour model. Also there is multicasting areas support."
At its core, NNCP works similarly to FidoNet or SMTP. Data and requests are staged as "packets" in a spool directory. From the spool, nodes can either connect to each other and exchange packets directly (for synchronous transfers), the source node can transfer packets into a directory (for asynchronous transfers via a regular file system) or the source node can dump packets into "bundles" that can be piped into other applications (for asynchronous transfers via tarballs or other sequential transmission methods). Upon transferring, the destination node loads packets into its own spool; they can then be "tossed" to recover the data or request.
NNCP is a spiritual successor to UUCP (Unix-to-Unix Copy), another software suite for transferring files and remote command execution. UUCP was a common method for sending e-mail in the 1980s and '90s and for many years served as the backbone software for Usenet. UUCP was originally designed to send data over modems, only adding support of TCP/IP in later years. NNCP lacks the ability to communicate over modems, but does improve on UUCP in many other ways, including encryption, multicasting and chunking. Many tasks that can be done with UUCP can also be done with NNCP; for example, John Goerzen has a tutorial on how to run a Usenet leaf node using NNCP^^.
NNCP is fully decentralized, free and open source, and runs on any Unix-like system. It does not run on Windows systems; this is an intentional choice by the author. I use NNCP on computers running Debian, Ubuntu, Fedora Linux and Rocky Linux, as well as on Termux for Android.
Suppose node Alice trusts nodes Bob, Carol and Dan. One evening Alice decides to perform some tasks: she wants to send a file to Bob and to Dan, she wants to request a file from Carol, and she wants to execute a command for Bob.
A typical workflow for this situation would proceed as follows.
There are many possible workflows that can achieve the same results.
At time of writing, the most recent version of NNCP is 8.11.0, released on 2024-07-23. I currently run 8.10.0 on all of my devices, so this guide will focus on 8.10.0. NNCP is not included in most software repositories, so I installed it from source on my machines.
NNCP is written in Golang, so you need to install Go to compile it. 8.10.0 requires Go 1.20 or newer, while 8.11.0 requires Go 1.22 or newer. One of my computers is stuck on Debian 9, but I could still run a tarball of Go 1.20 downloaded from GitHub. Once Golang is installed, download the 8.10.0 tarball of NNCP from the downloads page^, verify its integrity, extract it, and cd to the source directory.
The source ./config file controls a few key components of NNCP, including the default locations for your personal configuration file and NNCP's spool directory. These can be overridden later, but if you know some components will be located in nonstandard places (i.e. when running in Termux), it may be handy to edit ./config now.
Some important default locations are:
Once you are happy with the ./config file, simply run
$ bin/build
to compile the code.
If the build process succeeds, the ./bin directory will contain the following 24 binaries:
hjson-cli nncp-ack nncp-bundle nncp-call nncp-caller nncp-cfgdir nncp-cfgenc nncp-cfgmin nncp-cfgnew nncp-check nncp-cronexpr nncp-daemon nncp-exec nncp-file nncp-freq nncp-hash nncp-log nncp-pkt nncp-reass nncp-rm nncp-stat nncp-toss nncp-trns nncp-xfer
Move the compiled binaries to your location of choice, and be sure to add the location to your $PATH if needed. Create the spool directory and log file in their default locations. You may need elevated permissions to perform these operations.
NNCP is now ready to configure.
NNCP supports two configuration modes: a single file in HJSON format^, or a directory structure. The default mode is a single HJSON file, and the file is created using a configuration generator tool.
$ nncp-cfgnew > $CONF_FILE
The default configuration file name is 'nncp.hjson', and its default location is '/usr/local/etc/nncp.hjson' (if you did not edit './config' before building). In this mode, all the data necessary for running NNCP is located in a single file, including all neighbor configuration and your private keys. This makes it easy to reinstall NNCP if needed and manage with (say) version-control tools. An example default configuration file, with all comments and blank lines removed, is below.
{
spool: /var/spool/local-nncp
log: /var/spool/local-nncp/log
mcd-listen: [".*"]
mcd-send: {.*: 10}
self: {
id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
exchprv: GCOYCD3AJ424FSUXKKQF2AX442NPTYFSCZ747I32SGJL6XF7P4RQ
signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
signprv: CUZS7I2AYXXLPEYFCUWQEBRZNH4ONUPQRVYOCVRWGDFQFNJ23Z3MENCZS2A4DL5JCK3FYR32FYND3B74AFKRD6FSOOKLKAEIUJLJ2XI
noiseprv: DMJNUAYLNGQ2QSIAYCKTG73HVV5CGM6TXP5Z2W7Q7Q2Y65P7QMNQ
noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
}
neigh: {
self: {
id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
exec: {
sendmail: ["/usr/sbin/sendmail"]
}
}
}
}
NNCP's directory-style configuration translates the structure of HJSON into a file hierarchy: objects become directories, keys become file names, and values become file contents. This makes configuration much easier to deploy, maintain and secure. To use the directory mode, you need to have an existing HJSON-formatted configuration file--use `nncp-cfgnew` to create one if needed. Then run the following command, replacing $CONF_FILE with the name of your file and $CONF_DIR with the name of a(n extant) directory to store the configuration.
$ nncp-cfgdir -cfg $CONF_FILE -dump $CONF_DIR
If the '-cfg' flag is omitted, `nncp-cfgdir` checks the default location for the configuration file and gives an error if the file is not found. It's possible to translate from directory mode back to single-file mode:
$ nncp-cfgdir -load $CONF_DIR > $CONF_FILE
At build time, NNCP can only be configured to use one configuration mode by default. If you want to use a configuration that is not the default mode or not in the default location, you can specify the configuration in each NNCP command by specifying the '-cfg' flag, or you can set the $NNCPCFG environment variable. When showing example commands, this guide will not include the '-cfg' flag. I personally like the flexibility that the directory mode provides. I have about a dozen NNCP nodes and share some common configuration information between them all. If I need to modify something on one node and propagate configuration changes to the other nodes, it's far easier to send out files containing a single change using rsync or Syncthing than it is to manually edit a dozen single-file configurations to reflect the change. It also allows me to fine-tune configuration permissions and harden my setup, as I can deploy secrets from my password vaults more easily. The only major downside to directory mode is that it cannot be encrypted at rest (more on this later). Now that you have a configuration file/directory, you can add a few initial settings. => https://hjson.github.io/ HJSON, a user interface for JSON (HTTPS) --- ### First Configuring The basic structure of an NNCP configuration is as follows.
(configuration file/directory)
├─ (general options)
├─ notify (mail notifications)
│ ├─ (packet options)
├─ self (private node data)
│ ├─ id
│ ├─ (public and private keys)
├─ neigh (public node data)
│ ├─ self (configuring yourself as a neighbor)
│ │ ├─ id
│ │ ├─ (public keys)
│ │ ├─ exec (command definitions)
│ │ ├─ addrs (where to reach the node)
│ │ ├─ incoming (where inbound files from this neighbor are saved)
│ │ ├─ freq (where the neighbor can access outbound files)
│ │ ├─ calls (scheduled connections to node)
│ │ ├─ (general options)
│ ├─ alice
│ │ ├─ (neighbor options)
│ ├─ bob
│ │ ├─ (neighbor options)
│ ├─ (neighbors)
├─ areas (multicasting data)
│ ├─ area1
│ │ ├─ id
│ │ ├─ (public and private keys)
│ │ ├─ subs (nodes to send area packets)
│ │ ├─ exec (command definitions)
│ │ ├─ (general options)
│ ├─ area2
│ │ ├─ (area options)
│ ├─ (areas)
For the moment, you only need to worry about the .neigh.self section of this configuration. Note that "self" appears twice in the configuration: once in a dedicated section holding private data, and again as a neighbor to configure local communication. When generating a new configuration, the "self" neighbor is automatically created. It includes an example `sendmail` command, but it does not specify an incoming or freq directory. This means that by default, your node cannot send or request files to itself--it can only run the example command. To enable sending files to yourself, add the following line inside the .neigh.self section, replacing $INCOMING_DIR with the location of your inbound-files directory. Paths must be absolute.
incoming: $INCOMING_DIR
If your configuration is using directory mode, create a file called 'incoming' in '$NNCPCFG/neigh/self/' (where $NNCPCFG is the location of your configuration directory). The only contents of 'incoming' are the location of the inbound-files directory. To enable requesting files from yourself, add the following lines inside the .neigh.self section, replacing $FREQ_DIR with the location of your outbound-files directory. Paths must be absolute.
freq: {
path: $FREQ_DIR
}
If your configuration is using directory mode, create a directory called 'freq/' in '$NNCPCFG/neigh/self/' (where $NNCPCFG is the location of your configuration directory). Create a file called 'path' in the 'freq/' directory whose only contents are the location of the outbound-files directory. I configure my NNCP nodes to send and request files to a landing point in the home directory of each device. For the "self" node, the locations I use for inbound and outbound files are, respectively:
/home/$USER/nncp/self/in/
/home/$USER/nncp/self/out/
In single-file mode, the .neigh section of your configuration should look similar to the following.
neigh: {
self: {
id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
incoming: /home/user/nncp/self/in
freq: {
path: /home/user/nncp/self/out
}
exec: {
sendmail: ["/usr/sbin/sendmail"]
}
}
}
The following directories should exist on your machine:
/home/user/nncp/self/in
/home/user/nncp/self/out
/var/spool/nncp
Let's try it out! --- ### First Packet You are now ready to create a test packet and transfer it to yourself. Your first packet will be a dummy file, and you will send it by transferring it into a directory and reading it back into the spool. Before you begin, check the current status of the spool. Run:
$ nncp-stat
self
* This command returns the list of neighbors (including self) and information about packets contained in the spool. Right now the spool is empty, so the command will only show 'self' as a neighbor. Create a dummy file; it can have any name and contain anything you like. The file does NOT need to be placed in the outbound-files directory specified in your configuration file; it can exist anywhere on the file system, and NNCP only needs permission to read it. For purposes of demonstration, I am using a file called "dummy.bin" that is 4 KiB in size. To create a packet with the dummy file, use the `nncp-file` command:
$ nncp-file ~/dummy.bin self:
2024-08-01T20:14:35Z Tx dummy.bin 4.5 KiB/4.5 KiB 100% (16 MiB/sec)
2024-08-01T20:14:35Z File ~/dummy.bin (4.3 KiB) is sent to self:dummy.bin
* The file is now wrapped inside a packet and sitting in the spool. * The 'self:' argument at the end of the command specifies that you are sending this packet to the 'self' node. When the packet is eventually tossed, the dummy file will arrive in the inbound-files directory specified for 'self' in the node configuration. If you want to send the file to a subdirectory in the inbound-files directory, append a relative path (including filename) after the colon:
$ nncp-file ~/dummy.bin self:sub/dir/dummy.bin
* The path will be created if necessary. Running `nncp-stat` again shows your packet in the spool.
$ nncp-stat
self
nice: B | Rx: 0 B, 0 pkts | Tx: 4.5 KiB, 1 pkts
* The stats you see for 'self' include niceness, the number and total size of packets in the receive (inbound) queue, and the number and total size of packets in the transfer (outbound) queue. Niceness will be covered later. Notice that the packet is slightly larger than the file being sent. In order to send files, both the origin and the destination must have enough free space to hold both the packet, which includes overhead data, and the tossed data as it's being processed. Relays only need enough space to hold the packet in transit. You need a transfer directory to place the transferred packet. For this guide, we'll make a temporary one. NNCP will give an error if a specified transfer directory does not exist.
$ mkdir /tmp/packets/
Transfer the packet into the directory using the `nncp-xfer` command:
$ nncp-xfer -tx -mkdir -node self /tmp/packets
2024-08-01T20:15:22Z Tx [redacted id] 4.5 KiB/4.5 KiB 100% (0 B/sec)
2024-08-01T20:15:22Z Packet transfer, sent to self: 56NJJCBXTQ7ICVW3YG636SEZA7PUJZYMX2YE7H53DBZDETHIT4A (4.5 KiB)
* When you created the packet, NNCP generated a random packet ID, which is the long alphanumeric string in the second line of output. You can use this ID to find the packet when running commands against the spool (to be discussed later). * The '-tx' flag tells `nncp-xfer` to only transfer outbound packets to the directory; omitting this flag lets NNCP also read the directory for inbound packets and load them into the spool. Specifying '-node self' limits the packets you transfer to those whose current destination (or next hop in a relay) is 'self'. * By default, this command removes packets from your spool. * `nncp-xfer` uses a tree of subdirectories inside the transfer directory to sort and store packet data, based on the IDs of the origin and destination. The '-mkdir' flag tells NNCP to make the necessary directories if they don't exist. If this flag is omitted and the directories don't already exist, `nncp-xfer` will SILENTLY fail! Confirm the packet is in transit by checking the spool again.
$ nncp-stat
self
To read the packet from the transfer directory back into the spool, use `nncp-xfer` again, but this time check for receiving packets.
$ nncp-xfer -rx -node self /tmp/packets/
2024-08-01T20:15:40Z Rx /tmp/packets/..E7H53DBZDETHIT4A 4.5 KiB/4.5 KiB 100% (0 B/sec)
2024-08-01T20:15:40Z Packet transfer, received from self: /tmp/packets/R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ/R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ/56NJJCBXTQ7ICVW3YG636SEZA7PUJZYMX2YE7H53DBZDETHIT4A (4.5 KiB)
Check with `nncp-stat`:
$ nncp-stat
self
nice: B | Rx: 4.5 KiB, 1 pkts | Tx: 0 B, 0 pkts
* The packet is now in the receive queue, having reached its destination node.
Your dummy file is now queued in the spool, but it hasn't reached its final location on the file system yet. You still need to process ("toss") the packet and recover the original file. The command to do so is `nncp-toss`:$ nncp-toss
2024-08-01T20:15:48Z Rx file 56NJJCBXTQ7ICVW3..E7H53DBZDETHIT4A 4.0 KiB/4.0 KiB 100% (0 B/sec)
2024-08-01T20:15:48Z Got file dummy.bin (4.0 KiB) from self
One final check of the spool shows that the packet is now gone:
$ nncp-stat
self
Congratulations, you've sent your first packet with NNCP! You can now delete the contents of '/tmp/packets/'. --- ### Adding a Neighbor If all you did with NNCP was send files to yourself, it would be a pretty useless tool. Let's look at how to add a neighbor to your node. Usually each machine you install NNCP on will have only one node configured for it. However, there's nothing preventing you from running multiple nodes on a single device--and that's what we'll do for testing purposes. Simply run `nncp-cfgnew` again to create a new configuration file with new node keys. From here on, I'll call this new configuration file the "secondary" node, and the original configuration file your "primary" node. Create a new spool directory (for example, /var/spool/2nncp for the secondary node and add its location to the secondary configuration. You will need to choose names for your nodes. It's best to use characters that don't require escaping, to make things like scripting easier. For this guide, we'll use the above names. Convert the secondary configuration to the same mode as your primary configuration. If you use single-file mode for your primary configuration, you can get neighbor information from the secondary in two ways. The most obvious method is to open the secondary configuration file in a text editor and copy the .neigh.self section from there. The second method uses the command `nncp-cfgmin` to output a minimal version of the secondary configuration to stdout. From there, the .neigh.self section can be copied to the primary configuration.
$ nncp-cfgmin -cfg $SECONDARY_NNCPCFG
Once you've copied neighbor information from the secondary to the primary, edit the primary's configuration file. Neighbor names must be unique, so change the neighbor name from "self" to "secondary". Configure incoming and freq directory values similar to how the primary is configured, and create their corresponding directories. Next, specify an address at which you can connect to the secondary node. This will just be your local host for now. Add the following lines to the .neigh.secondary section:
addrs: {
lan: 127.0.0.1:6627
}
* The port can be any value you wish. NNCP's TCP daemon listens on port 5400 by default, but I use 6627 because it translates to "NNCP" on standard E.161 keypad mappings. If you use directory mode, simply copy the neigh/self/ directory from the secondary to the primary. Be sure to rename the directory to neigh/secondary/ when you copy it, so you don't overwrite the self directory in your primary configuration! Create a directory named 'addrs' in the secondary directory, and create a file named 'lan' in neigh/self/addrs/. Its only contents should be "127.0.0.1:6627". Add the primary node as a neighbor to the secondary configuration using the same steps. If you are using the single-file configuration mode, the .neigh section of your primary configuration file should look similar to this.
neigh: {
self: {
id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
incoming: /home/user/nncp/self/in
exec: {
sendmail: ["/usr/sbin/sendmail"]
}
freq: {
path: /home/user/nncp/self/out
}
}
secondary: {
id: IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA
exchpub: TN5D74YCVY44CU3HGEEBP4YXSMO753DOWRHE24DXRDRABPIVXRTA
signpub: 4IX3LEIFI47SXCOIZZBCEW5MAH6KAIIQWBVDWCI3CBMJRDQA2LAA
noisepub: S72GC3TVPLAXSL2ODSERI2MG3LVY5EJHVSHW5NWAZ32HN6XNJQMA
incoming: /home/user/nncp/secondary/in
freq: {
path: /home/user/nncp/secondary/out
}
addrs: {
lan: 127.0.0.1:6627
}
}
}
The .neigh section of your secondary configuration file should look similar to this.
neigh: {
self: {
id: IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA
exchpub: TN5D74YCVY44CU3HGEEBP4YXSMO753DOWRHE24DXRDRABPIVXRTA
signpub: 4IX3LEIFI47SXCOIZZBCEW5MAH6KAIIQWBVDWCI3CBMJRDQA2LAA
noisepub: S72GC3TVPLAXSL2ODSERI2MG3LVY5EJHVSHW5NWAZ32HN6XNJQMA
incoming: /home/user/nncp2/self/in
exec: {
sendmail: ["/usr/sbin/sendmail"]
}
freq: {
path: /home/user/nncp2/self/out
}
}
primary: {
id: R7B23ZHLN4XPIBFDFRE7UH5S6A7SUE6CQ3TUUHOQXCDNB6ZQCPBQ
exchpub: RTX6MK2W6MKZEQZP4IUAVUX5CQXONT5EB6Y5I3KCW4B4ABYJ4ULQ
signpub: YI2FTFUBYGX2SEVWLRDXULQ2HWD7YAKVCH4LE44UWUAIRISWTVOQ
noisepub: XGCF7SR352IPKU7NJ3OGR5YOFBA2NS3Y4FOPLLL7UMFSDFXR3NLQ
incoming: /home/user/nncp2/primary/in
freq: {
path: /home/user/nncp2/primary/out
}
addrs: {
lan: 127.0.0.1:6627
}
}
}
The following new directories should exist on your machine:
/home/user/nncp/secondary/in
/home/user/nncp/secondary/out
/home/user/nncp2/primary/in
/home/user/nncp2/primary/out
/home/user/nncp2/self/in
/home/user/nncp2/self/out
/var/spool/2nncp
--- ### TCP Transmission and Bundles You can now try a second operation: requesting a file synchronously over TCP. First, create a file to request. As an example, I've created another 4 KiB file called "dummy2.bin". This file needs to sit somewhere inside the secondary node's outbound-files directory; for ease, put it at /home/user/nncp2/primary/out/dummy2.bin. Open the port you specified in the primary configuration's .neigh.secondary.addrs section in your firewall. Launch a second terminal and run the following command, replacing $SECONDARY_NNCPCFG with the location of your secondary configuration.
$ nncp-daemon -cfg $SECONDARY_NNCPCFG -bind [::]:6627
* This command starts NNCP's daemon to listen for TCP connections. * The daemon handles connections for the secondary node, so you need to specify the secondary configuration using the '-cfg' flag. If you have the $NNCPCFG environment variable set, it will overwrite this flag, so unset or change the environment variable first. * The '-bind' flag tells the daemon what to listen for--in this case, connections from any IPv4 or IPv6 address on port 6627. With the daemon running, your primary node can request a file from the secondary node. Return to the first terminal and run:
$ nncp-freq secondary:dummy2.bin
* A packet is added to the primary spool's transfer queue, as can be seen with `nncp-stat`. Call the secondary daemon with the following command:
$ nncp-call secondary
2024-08-02T03:57:48Z We have got for secondary: 1 packets, 490 B
2024-08-02T03:57:48Z Connection to secondary (127.0.0.1:6627)
2024-08-02T03:57:48Z Tx JCUI42ELYBO4EKJC..PMFBKYI5JFDA6RKA 490 B/490 B 100% (0 B/sec)
2024-08-02T03:57:48Z Packet JCUI42ELYBO4EKJCZ4SKICMYQK4W5MIPOI5LPMFBKYI5JFDA6RKA is sent
2024-08-02T03:57:59Z Finished call with secondary (0:0:11): 32 KiB received (32 KiB/sec), 33 KiB transferred (33 KiB/sec)
* `nncp-call` opens a TCP connection with the specified node. The node must be reachable via at least one address listed in the configuration file, and it must be listening for connections. Once a connection is established, NNCP uses a sync protocol based on Noise^ and using XDR serialization^^ to send data. The terminal running `nncp-daemon` outputs the following:
2024-08-02T03:57:48Z primary has got for us: 1 packets, 490 B
2024-08-02T03:57:48Z Connection with primary (127.0.0.1:43162)
2024-08-02T03:57:48Z Rx JCUI42ELYBO4EKJC..PMFBKYI5JFDA6RKA 490 B/490 B 100% (0 B/sec)
2024-08-02T03:57:48Z Got packet JCUI42ELYBO4EKJCZ4SKICMYQK4W5MIPOI5LPMFBKYI5JFDA6RKA 100% (490 B / 490 B): done
2024-08-02T03:57:59Z Finished call with primary (0:0:11): 33 KiB received (33 KiB/sec), 32 KiB transferred (32 KiB/sec)
After 10 seconds of inactivity, the connection closes automatically. The packet is now in the secondary spool's receive queue. You might wonder why you didn't use `nncp-daemon` to transfer your first test packet to yourself. The reason is that NNCP locks the spool when performing any operations on it in order to prevent conflicts. Running `nncp-daemon` also locks the spool, which prevents you from running `nncp-call` against the spool to connect to the daemon. Using an asynchronous transfer method like `nncp-xfer` allows you to lock the spool sequentially: first to take the packet out, and second to put the packet back in. The secondary node needs to process the packet and add the requested file to its spool's transfer queue. You use `nncp-toss` to do so:
$ nncp-toss -cfg $SECONDARY_NNCPCFG
2024-08-02T03:58:15Z Tx dummy2.bin 4.5 KiB/4.5 KiB 100% (14 MiB/sec)
2024-08-02T03:58:15Z File /home/user/nncp2/primary/out/dummy2.bin (4.3 KiB) is sent to primary:dummy2.bin
2024-08-02T03:58:15Z Got file request dummy2.bin to primary
* The secondary spool now has one packet in its transfer queue containing "dummy2.bin". At this point the primary node could call the secondary again to receive the file. Suppose, however, that the primary suddenly disconnects and needs to switch to an asynchronous transfer method. The secondary can dump the file packet into a bundle and pipe it into tools that can assist in the transfer. The tool to perform this action is our third packet transmission command, `nncp-bundle`.
Let's send the file you requested from the secondary back to the primary node using nncp-bundle, piping it into gzip to compress it first.
$ nncp-bundle -cfg $SECONDARY_NNCPCFG -tx -delete primary | gzip > /tmp/packets/response-packet.gz 2024-08-02T04:10:41Z Tx TTZG6HRTJOEWOWZ7..GSL24SA2SS6GNS7A 4.5 KiB/4.5 KiB 100% (0 B/sec) 2024-08-02T04:10:41Z Bundle transfer, sent to node primary TTZG6HRTJOEWOWZ7XYHCQXMBDSPMQEH4OGK7GSL24SA2SS6GNS7A (4.5 KiB)
nncp-xfer, nncp-bundle cannot read inbound packets and write outbound packets simultaneously. You must therefore specify a direction when using it.
nncp-bundle to remove them from your spool instead.
gzip, which compresses the incoming stream into "/tmp/response-packet.gz".
NNCP is able to check the integrity of the packets it writes and reads. One important property of nncp-bundle is that it does not do this check by default.
Ingest the packet on the primary node:
$ gunzip -c /tmp/packets/response-packet.gz | nncp-bundle -rx -check 2024-08-02T04:11:58Z check TTZG6HRTJOEWOWZ7..GSL24SA2SS6GNS7A 4.3 KiB/4.5 KiB 96% (0 B/sec) 2024-08-02T04:11:58Z Bundle transfer, received from IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA TTZG6HRTJOEWOWZ7XYHCQXMBDSPMQEH4OGK7GSL24SA2SS6GNS7A (4.5 KiB)
nncp-bundle not checking the integrity of the packet when writing it, the '-check' flag checks the packet before ingesting. The packet is valid, so it is added to the primary's spool.
Finally, toss the packet:
$ nncp-toss -seen 2024-08-02T04:12:06Z Rx file TTZG6HRTJOEWOWZ7..GSL24SA2SS6GNS7A 4.0 KiB/4.0 KiB 100% (0 B/sec) 2024-08-02T014:12:06Z Got file dummy2.bin (4.0 KiB) from secondary
NNCP has successfully transferred between two nodes.
=> ^ The Noise Protocol Framework (HTTPS)
=> ^^ XDR: External Data Representation Standard (HTTPS)
Niceness encodes the priority of a packet. Each packet can have a niceness level between 1 and 255 inclusive, 1 being the highest priority and 255 being the lowest. Niceness can be set as an integer directly, as one of four single-letter aliases (optionally plus or minus an integer), or as one of five multi-letter strings.
The four single-letter niceness aliases correspond to four of the five niceness strings:
The fifth string is:
Aliases plus or minus an integer take the following ranges:
After sending a packet with NNCP, you may want confirmation that the recipient received was able to toss it successfully. This can be especially important when sending packets asynchronously. Until you get such a confirmation, you will likely want to keep a copy of the packet in your spool, in case you need to send it again. NNCP provides such confirmation functionality through "ack" packets.
Let's start by sending a file called "dummy3.bin" from the primary node to the secondary using nncp-xfer. We'll use the '-noprogress' flag to suppress progress messages.
$ nncp-file ./dummy3.bin secondary: 2024-08-02T22:58:17Z File dummy3.bin (4.3 KiB) is sent to secondary:dummy3.bin $ nncp-xfer -noprogress -tx -keep -mkdir -node secondary /tmp/packets 2024-08-02T22:58:32Z Packet transfer, sent to secondary: 23LPTDWGCZSRPDSJV2ZYLJFNQOQHV7OFIMB7WO6AQ3TYDD3WHHEQ (4.5 KiB) $ nncp-xfer -cfg $SECONDARY_NNCPCFG -noprogress -rx -node primary /tmp/packets 2024-08-02T22:58:54Z Packet transfer, received from primary: /tmp/packets/TTZG6HRTJOEWOWZ7XYHCQXMBDSPMQEH4OGK7GSL24SA2SS6GNS7A/IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA/23LPTDWGCZSRPDSJV2ZYLJFNQOQHV7OFIMB7WO6AQ3TYDD3WHHEQ (4.5 KiB)
When the secondary tosses the packet, use '-gen-ack' to create an ack packet:
$ nncp-toss -cfg $SECONDARY_NNCPCFG -seen -noprogress -gen-ack 2024-08-02T22:59:05Z ACK to primary of 23LPTDWGCZSRPDSJV2ZYLJFNQOQHV7OFIMB7WO6AQ3TYDD3WHHEQ is sent 2024-08-02T22:59:05Z Got file dummy3.bin (4.5 KiB) from primary
@ nncp-stat -pkt
primary
tx 5VBM4YU32Z547EFWGA7H4S6N7FUHB6MGAAI7UYXSN2O5BVQXTPHA 480 B (nice: MAX)
nice: MAX | Rx: 0 B, 0 pkts | Tx: 480 B, 1 pkts
self
* Using '-pkt' allows you to see the IDs of individual packets in the spool. * ack packets are automatically assigned a niceness of MAX, meaning they have the lowest transfer and process priority. Send the ack packet back to the primary:
$ nncp-xfer -cfg $SECONDARY_NNCPCFG -noprogress -tx -mkdir -node primary /tmp/packets
2024-08-02T22:59:30Z Packet transfer, sent to primary: 5VBM4YU32Z547EFWGA7H4S6N7FUHB6MGAAI7UYXSN2O5BVQXTPHA (480 B)
$ nncp-xfer -noprogress -rx -node secondary /tmp/packets
2024-08-02T22:59:58Z Packet transfer, received from secondary: /tmp/packets/IXTVOWOLQ73QU5OJ24MKCNPLL7GRNIXR3UFIO7G2OO2LLJBP4MZA/TTZG6HRTJOEWOWZ7XYHCQXMBDSPMQEH4OGK7GSL24SA2SS6GNS7A/5VBM4YU32Z547EFWGA7H4S6N7FUHB6MGAAI7UYXSN2O5BVQXTPHA (480 B)
At this point the primary node has two packets in its spool:
$ nncp-stat
secondary
nice: B | Rx: 0 B, 0 pkts | Tx: 23 KiB, 1 pkts
nice: MAX | Rx: 480 B, 1 pkts | Tx: 0 B, 0 pkts
self
Toss the ack packet and check the spool:
$ nncp-toss
2024-08-02T23:00:03Z Got ACK packet from secondary of 23LPTDWGCZSRPDSJV2ZYLJFNQOQHV7OFIMB7WO6AQ3TYDD3WHHEQ
$ nncp-stat
secondary
self
NNCP detects that the ack packet is referring to the other packet sitting in the spool. Tossing the ack packet automatically removes the original packet, as it is no longer needed. Acks are not needed when using `nncp-call` if the packet is not a transit packet, since `nncp-call` performs integrity checks by default. There is an important quirk to note about acks, and it shows up in the example workflow at the beginning of this guide. There, Alice wants Dan to send an ack. Normally Alice would keep a copy of the packet in her spool, and when Dan's ack reaches her, the packet will automatically be removed from her spool. However, the first step in the route uses `nncp-call`. `nncp-call` does not have an option to keep packets, because if Alice calls Carol again, NNCP has no way to tell if Alice needs to resend the packet. If Alice sends Dan multiple packets, she would have to manually track all of the their IDs in order to know which one Dan's ack is for. The solution involves low-level work with routing, which we will cover in the next section. --- ### Routing I will include sample commands here based on the example workflow at the beginning of this guide. You don't need to run them. In the example workflow, Alice wants to send a file to Dan, but she can't communicate with Dan directly, either synchronously or asynchronously. She needs to send her packet to Carol, who will then relay the packet to Dan. This can be be done using the '-via' flag. When creating the file packet, Alice specifies that the packet needs to be relayed via Carol:
[alice]$ nncp-file -via carol $FILE dan:
* Dan remains the destination, as the packet is meant for him. The '-via' switch actually creates a transit packet out of the original packet, with a different ID than the original. The original packet is encrypted using Dan's keys, and the transit packet is encrypted using Carol's keys. If the packet was not originally created with the '-via' flag (and is therefore not a transit packet), Alice can make a transit packet manually by using `nncp-trns`. If the packet ID is $PACKET:
[alice]$ nncp-trns -via carol dan:$PACKET
* After creating the transit packet, the original packet remains in the spool. Alice calls Carol to send the packet to her:
[alice]$ nncp-call carol
When Carol tosses the packet, the transit packet is decrypted, leaving the original packet for Dan. Packets can be routed through multiple nodes. If Alice wants to route Dan's packet through Bob and then Carol, she can run:
[alice]$ nncp-file -via bob,carol $FILE dan:
If Alice knows she will always need to route through Carol to reach Dan, Alice can edit Dan's neighbor configuration to route through Carol by default. More information is in a later section. The '-via' flag overrides the value set in the configuration. '-via -' disables routing entirely. You can now revisit the problem presented in the previous section. To summarize, Alice wants to send a packet to Dan, and she wants to keep a copy of the packet in her spool for Dan to ack. However, the packet's first hop involves calling Carol, which always removes the packet from Alice's spool. The solution uses the `nncp-trns` command. Alice first creates a packet for Dan that does not route via Carol. She then uses `nncp-trns` to create the transit packet to route via Carol. This leaves two packets in her spool: one to send to Carol, and one for Dan to ack. Alice then sends the transit packet to Carol by calling, and when Dan's ack reaches her, the original packet is removed by the ack. --- ### Areas NNCP packets can have only one destination. So far we've seen NNCP assign nodes as destinations, so the packets we've made can only be received by one node each. Even a relayed packet technically sets its destination as the first hop in the relay, and only after traversing the relay will the packet's destination be set to the final node. This one-destination limitation means that if you wanted to send one file to 20 nodes, you would need to create 20 packets. Areas are how NNCP solves this problem. NNCP's configuration can add nodes to an "area" and send a packet to the area. When you process the area packet, NNCP automatically creates the necessary packets to send to nodes in the area. Suppose Alice wants to create an area called "friends" so she can send packets to Bob, Carol and Dan simultaneously. Alice first generates the area ID and keypair with:
alice$ nncp-cfgnew -area friends
* An HJSON document containing the area information is printed to stdout. This can be converted to directory mode using `nncp-cfgdir` the same way as full configurations. Alice edits her configuration, listing Bob, Carol and Dan as subscribers to the area. She then shares the area ID and keys with the others. Bob, Carol and Dan add the ID and keys, and they each specify a directory for incoming files from the area. When Alice sends a packet to 'friends', she prepends the destination with 'area:' to differentiate it from a node.
alice$ nncp-file $FILE area:friends:
If Alice runs `nncp-stat`, she will see a single initial packet in the 'self' queue. NNCP copies the outbound packets for the area nodes from this initial packet when Alice tosses it. This is to ensure integrity of all the area packets if Alice's node fails in the middle of packet creation. Alice now creates the outbound packets by tossing the initial packet.
alice$ nncp-toss
text/geminiThis content has been proxied by September (3851b).