Home

Awesome

Encrypted File Storage System (EFSS)

EFSS stores data securely in a file structure without having to install or compile anything beyond PHP (i.e. not FUSE, not a PHP extension, and runs virtually everywhere).

EFSS is a file system with transparent encryption and compression implemented in userland code in the PHP scripting language. It comes with a robust suite of tools to manage EFSS data stores that include a command-line shell and a powerful library that can be used in various PHP programming projects. The specification for the file system is open to allow for portability to other programming languages. EFSS can be used to store hierarchical file-like data securely outside of a database in a low-write, moderate-read, PHP-based web environment.

Note that EFSS was originally intended to be used as a backup system. As a result, there are references to features that are primarily only seen in backup systems (e.g. incrementals) and those references and features should be largely ignored. For backup purposes, use Cloud Backup instead.

Donate Discord

Features

Command-line Tools

EFSS comes with a suite of command-line tools for performing common maintenance tasks with EFSS data stores. Each command-line tool utilizes the underlying 'support/efss.php' classes to manipulate and navigate EFSS data stores. Each tool has a number of options that can be seen by running the tool without any options. For example, running 'php create.php' will display the options for 'create.php'.

The following is a high-level breakdown of the command-line tools that are available:

Using the EFSS Class

Example usage:

<?php
	// Temporary root.
	$rootpath = str_replace("\\", "/", dirname(__FILE__));

	require_once "support/efss.php";

	// DO NOT use the keys and IVs here for your storage system.  Running 'create.php' will generate the necessary information.
	$key1 = pack("H*", "13c21eb0d95f5bcb49c1aef598efddc8eac7295092ffbd99c6c1f28d0b0dc9e3");
	$iv1 = pack("H*", "3d38ae9c7cf6bcb449cbf9c368103ec6");
	$key2 = pack("H*", "2e444c2eceafe40cbdf564ad48f6ca99dbb23d43f7b630d89646a48ef0d67a02");
	$iv2 = pack("H*", "95dfd6e8b753c594c52d95fef5cd4511");

	$efss = new EFSS();
	if (file_exists($rootpath . "/my_efss.dat"))  $result = $efss->Mount($key1, $iv1, $key2, $iv2, $rootpath . "/my_efss.dat", EFSS_MODE_EXCL);
	else  $result = $efss->Create($key1, $iv1, $key2, $iv2, $rootpath . "/my_efss.dat");
	if (!$result["success"])
	{
		echo "Unable to create/mount EFSS data store.  Error:  " . $result["error"] . "\n";

		exit();
	}

	$efss->mkdir("test/test2");
	$efss->file_put_contents("/test/test2/test.txt", "It works!");

	$result = $efss->file_get_contents("test/test2/test.txt");
	echo $result["data"];

	$efss->Unmount();
?>

More documentation can be found in the 'docs' directory of this repository.

Limitations

Okay, so EFSS is not perfect. There are a few limitations of this software that hopefully won't be show stopper issues for you.

Under PHP, there are theoretical, untested limits for EFSS data stores. Even though attempts have been made to avoid issues, the limit can be as low as 2GB under 32-bit PHP (64-bit PHP won't encounter a 2GB restriction). The normal limit of EFSS data stores is 8.7TB at the default block size of 4096 bytes (2^31 * 4KB). The absolute limit of EFSS data stores is 70.3TB at a block size of 32,768 bytes (2^31 * 32KB).

EFSS is designed for reading data, not writing data. Performance when writing new files (transparent encryption and compression, after all) and creating new directories is a bit on the sluggish end of things. Reading data, however, is a bit faster, as is to be expected.

If an EFSS data store becomes corrupted, especially the first three blocks, there is currently no easy way to fix the data store so that it will mount. I welcome pull requests if you can figure out how to solve the issues.

How EFSS Works

A minimal EFSS data store consists of four files: The main encrypted block file, a file containing last updated timestamps for each block in the main file, a hashes file containing a hash of each block (for rapid verification), and a 2KB file containing a unique serial. If you use 'create.php' to make an EFSS data store, it will also create a PHP file containing the key/IV information that the other command-line tools will use to be able to mount the file system (i.e. decrypt the block file).

EFSS tracks five different block types within the block file:

Linked lists start with a block number to the next block. The last node references block zero (an impossible reference) to indicate the termination point of the linked list.

A special type of file known as an "inline file" may be written as part of a directory block to avoid wasting space (i.e. file blocks are not used). The rule is that the file data must occupy less than 45% of the block size. If compression is available, the original, uncompressed data could potentially be slightly larger than 45% as long as the compressed data meets the 45% criteria. The shell can help identify inline files via the 'i' flag.

EFSS also does a lot of directory structure traversal operations to calculate absolute paths. A directory block cache exists to save blocks in RAM to avoid hitting the disk too frequently. The relevant directory block cache entry is wiped whenever an associated block is written to disk so that the block is loaded again on the next read operation. This approach does have a minor performance penalty but guarantees the stability of directory structures.

More Information

Documentation and examples can be found in the 'docs' directory of this repository.