Home

Awesome

BitWeb

This is a cntinuation of the P2PWeb project, using the bittorrent technology. In a few words, this is a proxy software that you can plug your browser with, and that will let you access torrents as if they were websites.

For example, when you point yout browser to http://dbbda49df0642ebc45f142059e60552a70cc3c5b.bitweb/index.html, the proxy will download the torrent that has the info_hash dbbda49df0642ebc45f142059e60552a70cc3c5b, will look at the file index.html and will serve it over http to your browser. This is the same technology as the magnet links for torrents.

The purpose of this is to have:

Features (not all there yet)

Future (to do):

Description and installation

The software is both a SOCKS4A proxy (that can reply to HTTP requests for .bitweb domains) and a bittorrent client.

In order to have your browser redirect to this proxy, create a proxy.js file containing:

function FindProxyForURL(url, host) {
    isp = "PROXY ip_address:port; DIRECT";
    tor = "SOCKS 127.0.0.1:9050";
    bit = "SOCKS 127.0.0.1:8878";
    if (shExpMatch(host, "*.onion")) return tor;
    if (shExpMatch(host, "*.bitweb")) return bit;
    return "DIRECT";
}

Put the url of this file (using the file: protocol) in the proxy settings of your browser for the autoconfiguration URL.

Usage

To start the proxy server on the default port (8878), run:

bitweb -D

To create a torrent suitable for websites, use the command line:

bitweb -Ct mywebsite.torrent -k privatekey.p8.der /path/to/my/website

You can add metadata to your pages:

bitweb -Ut mywebsite.torrent -k privatekey.p8.der -p index.html -H "content-type:text/html; charset=utf-8"

You can visualize your torrent:

bitweb -St mywebsite.torrent

You can seed your torrent:

info_hash=$(bitweb -St mywebsite.torrent | grep 'info hash' | cut -d' ' -f3)
torrent_name="$(bitweb -St mywebsite.torrent | grep 'info hash' | cut -d' ' -f3-)"

cp mywebsite.torrent ~/.cache/bitweb/$info_hash.torrent
mkdir ~/.cache/bitweb/$info_hash
cp -R /path/to/my/website "~/.cache/bitweb/$info_hash/$torrent_name"

(and restart your server, HUP signal doesn't work yet)

Future usage (using ed25519)

Generate a key pair:

$ bitweb -K
Public Key: c015b1438a5dd7b18d203758b576719ac768247e89f92444bd1e8b8e77565eca
File: c015b1438a5dd7b18d203758b576719ac768247e89f92444bd1e8b8e77565eca.key

Copy the generated .key file to ~/.local/share/bitweb

Further work, Bittorrent extensions

Padding files (implemented)

In order to be able to share pieces between different versions of a torrent, add padding files in the torrent to make sure that every legitimate file is aligned with a piece boundary. Use the libtorrent approach.

Torrent signature (implemented)

A info dict can contain a RSA public key and a signature that validates the owner of the corresponding private key authored the info dict. The signature is computed using the bencoding of the info dict (with the signature field removed).

For a request to a specific info_hash, the rule is to trust the key included in that info_hash when requesting future revisions.

TODO: Use BEP 35, signing instead? The problem is that the signature is outsie the info dict and may not be transmitted when initializing through magnet links. Or completely forget about this part as it will be taken care of by the torrent revisions

File metadata (implemented)

Add HTTP header information for each file in a torrent (the content-type for example).

Torrent versions

A torrent can reference previous versions. When referencing a previous version, a torrent must also reference recursively all the previous versions. See below for the implementation details.

TODO: maybe we might want to make this more general than bitweb. Avoid using the string "bitweb" when implementing that feature. maybe it could be proposed as a BEP.

Backlinks

A torrent can reference other torrents (or a specific file in other torrents). A link is composed of a source (torent info hash and filename), a relation type (uri that describes what relation the source has to the destination) and a destination (info hash of a torrent and a filename).

Forward links should be stored in the torrent itself and backlinks should be available in the DHT.

Encryption

torrents (info dict and content itself) should be able to be encrypted using a password (that can be embedded in HTML links). like freenet, this would allow anyone to share space and bandwidth for websites you can't know anything about. Notes would advertise that they are willing to share disk space or bandwidth and would download automatically such torrents.

Reference

Hacking

Small tasks:

Handle torrent versions using DHT store extension

Implementation note: session::dht_get_item() and session::dht_put_item()

In the info dict, add a "bitweb" entry containing a dict with:

New revisions are made by incrementing the seq number and adding the parent hash info to the parents list.

Revisions are advertised using the DHT store extension. A mutable item is stored with:

The cas field is not present, the id, sig and token fields are computed as defined in the DHT store extension.

To allow anyone to republish a version of the torrent, the torrent should contain outside of the info dict the signature required to republish the value. It cannot be included inside the info dict because the signature depends on the info hash value.

TODO: this signature should be obtained by nodes when they use magnet links to obtain the torrent. The client can check that the signature obtained is valid by checking it against the public key and the site id stored inside the info hash.

Other idea: instead of looking up websites by the info hash, look them up against sha1(info_hash + signature) and have this information stored as an immutable message through the DHT store extension. The problem with that is that it adds an indirection layer and will slow down downloading a torrent for the first time, and initial HTTP connection.

When obtained, and if a torrent file is written to disk, the signature should be included in a bitweb signature key at the same level of the info dict. Every bitweb client that share a website should regularly republish the latest signature.

URL conventions:

TODO: be notified in real time when a site is updated and do not require pooling frequently for updates.

Hardening code

Each socks client (source IP and port) in a separate process (in case the process crash). Avoid too much processes (one process per connection) and avoid crashing other users or the main daemon.

Have a separate process for the torrent node as well.

Simple related tasks includes:

Code issues

Crypto++

Can't load DER encoded key? Check it is in PKCS#8 format

openssl pkcs8 -in privkey.pem -out privkey.p8 -topk8 -nocrypt -outform der

See: http://crypto-users.996303.n3.nabble.com/Load-Private-RSA-key-td2851.html And: http://www.cryptopp.com/fom-serve/cache/62.html And: http://stackoverflow.com/questions/9869261/crypto-load-rsa-pkcs1-private-key-from-memory And: http://www.cryptopp.com/wiki/Keys_and_Formats

See: http://stackoverflow.com/questions/25441918/crypto-cant-der-encode-and-ber-decode-rsa-public-key-ber-decode-error