A zero-details, privacy-focused in-app file system.

ZboxFS's goal is to help app store files securely, privately and reliably. It embeds in your app and provides an encrypted virtual file system.

Features

Everything is encrypted, including file content, directory and metadata
State-of-the-art cryptography: AES-256-GCM (hardware), XChaCha20-Poly1305, Argon2 and etc.
Support varieties of storages, including memory, OS file system, RDBMS and key-value object store
Files and directories are packed into same-sized blocks to eliminate metadata leakage
Content-based and file-based data deduplication
Data compression using LZ4 in fast mode

Data integrity is guaranteed by authenticated encryption primitives (AEAD crypto)
File contents versioning
Copy-on-write (COW) semantics
ACID transactional operations
Multiple programming languages binding
Built in Rust with safety and performance

Comparison

ZboxFS runs inside your application. It utilises different underlying storages and provides an encrypted virtual file system to application.

Many OS-level file systems support encryption, such as EncFS, APFS and ZFS. Some disk encryption tools also provide virtual file system, like TrueCrypt, LUKS and VeraCrypt.

The right diagram shows the difference between ZboxFS and them.

How it works

Advanced Encryption

Powered by libsodium, ZboxFS uses the most advanced cryptography in industry: AES-256-GCM (hardware), XChaCha20-Poly1305, Argon2 and etc.

256-bit master key is randomly generated. Then it is encrypted by the key derived from user-input password with adjustable costly Argon2 function and cryptographic random salt.

Sub-keys are derived indivisually from master key using BLAKE2 function. They are used to encrypt different parts, such as data blocks, index and WAL.

Encryption keys are kept in protected area and never leave application's memory. They are overwritten and destroyed immediately once not used.

No-Knowledge Storage

By abstracting IO access, ZboxFS supports a variety of underlying storages, including memory, OS file, RDBMS and key-value object store.

File data is packed and encrypted in same-sized blocks before writing to underlying storage. The storage has zero knowledge about file data.

A simple, generic and extensible storage interface makes it easy to write adaptor for your own storage.

Zbox Cloud Storage is a cloud file storage designed specifically to work with ZboxFS. Like the other storages, it is has no knowledge about your data. Backed by geo-distributed CDN, it runs on multiple platforms and utilises a local cache to boost IO performance.

In-App File System

ZboxFS has a simplified file system built in. It is specifically designed to store data securely and reliably. Like other file system, it has similar concepts like files, directories, metadatas and etc.

This file system co-operates with content store and transaction manager, performs IO tasks on top of volume and underlying storage.

Like Unix-style file system Inode, ZboxFS uses "Fnode" to describe file and directory object. It contains file metadata and links to different versions of contents in content store.

Transaction Management

Transaction management, which closely works with write-ahead logging (WAL), copy-on-write (COW) and append-only storage, is the core part of ZboxFS.

When mutating the file system, all affected objects will be copied then modified in a transaction (Tx). During the commiting phase, tx manager will persist a WAL record containing that tx to storage first. After that, tx will write all the newly changed objects and switch to them in COW wrapper.

Writing to storage is append-only, this enables transaction isolation and also makes aborting on failure easier.

Content Store

To localise content change, ZboxFS chunks file content using rolling hash algorithm, which produces variable-length chunks based on file content.

The chunk hash is computed when read in and used to dedup chunks in file. A merkle tree is also built on top of those chunk hashes. Content store saves the hash and uses it to dedup the whole file content.

A file can contain multiple versions of contents. Every change made to file content creates a new version of content. Content store keeps track of all contents and provides content IO functions.

API

ZboxFS provides a set of POSIX-like API to application, such as read(), write(), seek() and etc. The two core parts are Repo and File.

Repo represents a secure collection which consists of files, directories and all of their associated data. It provides API to interact with the file system inside.

File is a reference to an opened file in Repo. An instance of File can be read and/or written depending on what options it was opened with. File can also seek to its logical position internally.

The API currently has bindings for programming languages: Rust, JavaScript, NodeJs, C/C++, and Android Java.

How to use it

Requirements

Rust stable >= 1.32
libsodium >= 1.0.17

Installation

Add the following dependency to your Cargo.toml:

[dependencies]
zbox = "0.8.7"

If you don't want to install libsodium by yourself, simply specify libsodium-bundled feature in dependency, which will automatically download, verify and build libsodium.

[dependencies]
zbox = { version = "0.8.7", features = ["libsodium-bundled"]  }

Storage

ZboxFS supports a variety of underlying storages. Memory storage is enabled by default. All the other storages can be enabled individually by specifying its compiling feature.

Memory: storage-mem
OS File: storage-file
SQLite: storage-sqlite
Redis: storage-redis
Zbox Cloud Storage: storage-zbox-native

Example

extern crate zbox;

use std::io::{Read, Write, Seek, SeekFrom};
use zbox::{init_env, RepoOpener, OpenOptions};

fn main() {
    // initialise Zbox environment
    init_env();

    // create and open a repository
    let mut repo = RepoOpener::new()
        .create(true)
        .open("zbox://yb2CCTcmEuxenZVuZhVKMCJD@AWCpPaNkvG6vVW",
              "secret password")
        .unwrap();

    // create a file and write content to it
    let mut file = OpenOptions::new()
        .create(true)
        .open(&mut repo, "/hello_world.txt")
        .unwrap();
    file.write_once(b"Hello, World!").unwrap();

    // seek to the beginning of file
    file.seek(SeekFrom::Start(0)).unwrap();

    // read all content
    let mut content = String::new();
    file.read_to_string(&mut content).unwrap();
    assert_eq!(content, "Hello, World!");
}