GNUNet File Sharing Tutorial and an Alternative to IPFS

=> home

With Web3 on the headlines all the day. I want to shift my view onto more traditional technologies. Blockchain is a great. Really, especially for transactions. Once I had to wire some money to an overseas bank. It literally took a week from sending to confirmed receiving. The fees are high too. With XMR or Eitherium. I can send Crypto to the destination in like 15 minutes. With sane fees. But I think that's where blockchain's advitange ends that, I don't think there's applications that blockchain can do better than traditional client server architectures.

Back to Web3. A decentralized web is very attractive. We can solve the most annoying part of internet in one go - dead links. As long as enough machines are hosting the content. A decentralized web will always serve up the content even if the original hoster vanished into the ether. IPFS is probably the 1st thing people think of for this. However, there's a more old-fashioned system doing the same thing. With much less Cryptocurrency and tech fluff. GNUNet[1]

=> [1]: GNUNet home page

GNUNet is GNU's version of overlay network. Just like Tor and I2P. Yet, unlike Tor and I2P. GNUNet rethinks and reimplements the entire 7 OSI layers. Instead of TCP/IP they have CADET, instead of OSI transport layer they have gnunet-core. etc.. Despite that, GNUNet can run over TCP/IP. However, GNUNet has a very small userbase right now. It's a large project involving may aspects. From my recent experiments, The major GNUNet functionalities, GNUNet FS and GNS (GNU Name Service) works as advertised. This post will be focusing on the former.

* Ubuntu: `apt install gnunet`
* Arch (AUR): `yay -S gnunet-git`

After installing. Start GNUNet services using the `gnunet-arm` command. A `-s` flag tells it to launch subsystems. Running GNUNet is _like_ running an IPFS node that it joins file index and assists sharnig files to other nodes. Under the hood GNUNet does a lot more encryption and ananonymization compared to IPFS. Unrelated subsystem is also started.

❯ gnunet-arm -s

## Searching for files

You are ready to go if no error shows up. Let's search for LICENSE (project license) files that people may have published as tests with the `gnunet-search` command. The `-t` parameter denotes how long the file search goes on. By default it'll run forever, until you Ctrl-C out of it.

❯ gnunet-search LICENSE -t 4s

1:

gnunet-download -o "LICENSE" gnunet://fs/chk/759AD2T0092YBD5AFXJGEY08JC1TH4HKGGFR1MTH2Q7CMWPAPEATK0JYMXDAZQWNB7RFG33ZHWE6N3ZE6JR0E55MW8CHXZ7S2E338D8.0RP3SPX1SV66X6MV2WYJF70F8K20RK9PZ5PB78ZH9FR2WPWPQVD4B8B5FFBB2S8NDW7R5YQETNP03WNCR6W6BF002DGTBBCAXZBYHX0.1952

2:

gnunet-download -o "LICENSE" gnunet://fs/chk/EKDWDB2K3F5CB8M6JVBAKXTN1AS369KKXVEVR2650CKJQWQ3ECJRTQPNGC8EA1A2DS4XJCWE08RCS7QBTCGTW7FP3DAXWE1CW1WPNM8.ZSMGMEY8FP8MHS62DC1TH3TQ0Y55YP2SVR9PZVB6EMDG0HDASE2BHXRZXRT973X9ZCBBHW76VVJ41BMR968WC0F3XQJ1DA2GM1BBFBG.1063

You see several download commands on the console. Executing the command downloads the file from peers. If the download hangs. It's likely no peers currently have the file (GNUNet is quite small right now). In my case, the download is successful and it turns out to be the license file for OpenJS

❯ gnunet-download -o "LICENSE" gnunet://fs/chk/759AD2T0092YBD5AFXJGEY08JC1TH4HKGGFR1MTH2Q7CMWPAPEATK0JYMXDAZQWNB7RFG33ZHWE6N3ZE6JR0E55MW8CHXZ7S2E338D8.0RP3SPX1SV66X6MV2WYJF70F8K20RK9PZ5PB78ZH9FR2WPWPQVD4B8B5FFBB2S8NDW7R5YQETNP03WNCR6W6BF002DGTBBCAXZBYHX0.1952

100% [============================================================]

Downloading `LICENSE' done (1490 b/s).

❯ head LICENSE -n 3

Copyright OpenJS Foundation and other contributors https://openjsf.org/

Based on Underscore.js, copyright Jeremy Ashkenas,

You can also download my slides from COSCUP2022 about hacking Arch Linux onto ARM systems. I'm not sure if the download will be successful as I might be the only node with this file, thus prone to file missing.

❯ gnunet-search -t "4 s" coscup

1:

gnunet-download -o "coscup2022-slides.pdf" gnunet://fs/chk/DPT7S0RHZ4Q0SCTCJTNB43NFEAWQNC806ZE81BZ0QN43M63VDGJRT2ESV731KNCHNY7DH39BN2S74QMSG0H5CDCJW4980Y6E7GFV8J8.QMJ145WZ01FKG6NXZFRK5ATTQ3FBR3AN5JRWJ4BD91CW9JKW18KGYQK3R6XB1HD00JHKMT41H5NHM684H82GSW097A5JZCMY891KGS0.2051732

❯ gnunet-download -o "coscup2022-slides.pdf" gnunet://fs/chk/DPT7S0RHZ4Q0SCTCJTNB43NFEAWQNC806ZE81BZ0QN43M63VDGJRT2ESV731KNCHNY7DH39BN2S74QMSG0H5CDCJW4980Y6E7GFV8J8.QMJ145WZ01FKG6NXZFRK5ATTQ3FBR3AN5JRWJ4BD91CW9JKW18KGYQK3R6XB1HD00JHKMT41H5NHM684H82GSW097A5JZCMY891KGS0.2051732

100% [============================================================]

Downloading `coscup2022-slides.pdf' done (19 KiB/s).

## Publishing files

The `gnunet-publish` command allows users to upload files onto GNUNet. Like IPFS, shared files are chunked, encrypted, signed, and be stored on peers when applicable. GNUNet also comes with it's own DHT - R5N[2] (Which makes it funny now GNU has 2 projects with their own DHT. GNUNet's R5N and Jami's OpenDHT) to share data across nodes.

=> https://lsd.gnunet.org/lsd0004/ The R5N Distributed Hash Table

Let's say we want to upload the a Hello World text file onto GNUNet. We use the command `gnunet-publish hello.txt`. 

❯ echo "Hello World" > hello.txt

❯ gnunet-publish hello.txt

Publishing `/home/marty/hello.txt' done.

URI is `gnunet://fs/chk/W70H5ZWGHZNW7ECB2T9TDK9NCKNFHSF6S9H9T22DKW7BN694FJPDTWQ3D7ZRJG9SFGM0EG4ZYSNYCJZ913D1FBBVH94T58KC1T08DAG.MFFDY2YMMS07Y32T2AAQHHRCAJGA8PGQCAP0EGN4PGV4AKAFQJEAP8WQ58MWFRKARB2QEYTGJ41BVK5616KMCJMAJHTC7CYPRW0XK2R.12'.

Now searching for `hello.txt` will find this file

❯ gnunet-search -t 4s hello.txt

1:

gnunet-download -o "hello.txt" gnunet://fs/chk/W70H5ZWGHZNW7ECB2T9TDK9NCKNFHSF6S9H9T22DKW7BN694FJPDTWQ3D7ZRJG9SFGM0EG4ZYSNYCJZ913D1FBBVH94T58KC1T08DAG.MFFDY2YMMS07Y32T2AAQHHRCAJGA8PGQCAP0EGN4PGV4AKAFQJEAP8WQ58MWFRKARB2QEYTGJ41BVK5616KMCJMAJHTC7CYPRW0XK2R.12

But wait. We can append more flags to `gnunet-publish` to add more about our file. Like `gnunet-publish hello.txt -k hello -n`. The `-k` flag stands for keyword. People can search for "hello" for this file to show up. And `-n` makes GNUNet make a local copy of the file. So you can remove the original file once published. Let's try nother example. This time with the famous Lenna image.

❯ gnunet-publish lena_color.gif -n -k lenna -k "computer graphics"

Publishing `/home/marty/lena_color.gif' done.

URI is `gnunet://fs/chk/R8QTF12V2TYKVMESVPMP5D5GFX3HBSXRT6FNVZXDPCFN7AJG57BY45QCB318D3KVC49MTPMSDNJXE6S7Q8EQTPX3RM4AQPYH960G7E0.5XFQ8YVJ917W6GPDARJBZ2N7DN7E568YBV5JMBZ5Z3CRH5SHV51M4RG830W1CJS2GWHZT3PBNE7RTXPF5P4FW5RZT6BC77AFSGGB0VG.227335'.

❯ gnunet-search -t 4s "computer graphics"

1:

gnunet-download -o "lena_color.gif" gnunet://fs/chk/R8QTF12V2TYKVMESVPMP5D5GFX3HBSXRT6FNVZXDPCFN7AJG57BY45QCB318D3KVC49MTPMSDNJXE6S7Q8EQTPX3RM4AQPYH960G7E0.5XFQ8YVJ917W6GPDARJBZ2N7DN7E568YBV5JMBZ5Z3CRH5SHV51M4RG830W1CJS2GWHZT3PBNE7RTXPF5P4FW5RZT6BC77AFSGGB0VG.227335