How can we store data for 200 years?

Introduction

For the first time in human history, we possess the capability to pass down digital information for generations in a sustainable manner. While the preservation of knowledge and memories isn't a new concept, it's never been done quite like this before.

Traditional methods have been inherently limited and fall short of providing sustainable solutions for the long-term preservation of data. Personal hard drives and on-premise hardware are vulnerable to physical degradation and loss. Centralised cloud storage (CCS) solutions suffer from a lack of permanence and their cost is continuous and cumulative, increasing dramatically over time if you store large amounts of data.

In comes decentralized storage

A blockchain is designed to be a decentralized, digital ledger that records transactions in a transparent, secure and tamper-proof manner.

One such use-case of blockchain technology is decentralized storage. Decentralized storage providers are far more secure, cost-effective and censorship-resistant compared to CCS providers. Within decentralized storage providers, two models exist: contract-based and permanent storage.

Contract-based storage

Contract-based providers such as Filecoin, Sia and Storj provide a safer alternative to CCS. Through these providers, you’re able to retain ownership of your data. Even though they provide similar benefits to CCS providers, the obligation to continually renew contracts and ongoing monthly fees still apply.

Filecoin recently introduced the concept of perpetual storage with the launch of the Filecoin Virtual Machine (FVM). However, this model still relies on having sufficient funds to maintain the contract. Through this, you could essentially set an indefinite expiration date on your storage deal. However, as described in their documentation, this is contingent upon having enough funds to maintain the contract.

Essentially, this is still contract-based storage and as we do not know the total cost of ownership, the required funds to maintain your data for decades or longer cannot be known.

Permanent storage

Currently, Arweave is the only permanent decentralized storage provider. What it offers, at first glance, might seem too good to be true: users pay a one-time upfront fee in order to permanently store their data. Once uploaded, it will never be deleted and no hidden charges apply for data retrieval. Your data remains in its authentic state, tamper-proof and permanent for at least the next 200 years.

Once you understand how Arweave works, you’ll realise it's ability to store data permanently is thanks to it being a blockchain specifically designed to store data. The cornerstones of Arweave’s architecture, the 'blockweave' and its on-chain storage endowment, facilitate this permanent data storage.

Understanding Arweave’s Unique Architecture

Blockweave

Arweave’s unique data structure, the blockweave, holds all network-stored data. It employs a consensus mechanism, Succinct Random Proofs of Access (SPoRA), to verify whether miners are correctly storing the blockweave.

In the blockweave, every block connects to the previous, the subsequent, and a random data chunk, justifying its name. In order to add more data, miners must show they have access to a randomly chosen block, incentivizing them to store the entire blockweave in order to mine further blocks and earn rewards.

Moreover, your data will be replicated across the nodes housing a full replica of the blockweave. Currently, estimates show over 200+ complete replicas of the ~175 TiB blockweave, promising data access even if some nodes were to go offline. Your data's permanence is guaranteed.

Endowment

When you make your upfront payment, it is not fully distributed to the miners who store your data. A small portion, 16.67% to be exact, is paid initially while the rest goes into a storage endowment. The funds in this endowment will be paid out as needed to keep the block reward for storing data higher than the cost of storage.

On average, the cost of storage has declined by 30.5% per year over the past 50 years. In a highly conservative estimate, Arweave assumes a 0.5% annual decline when calculating data upload costs. Any time the cost of storage falls more than 0.5% a year, it extends out the lifetime that the protocol can pay to store at least 15 replicas (the minimum threshold) of your data.

Should the number of replicas approach the minimum threshold, the endowment would start paying out to incentivise miners to store more replicas and make the market more attractive for other miners to join the network. Since its inception in 2018, no tokens have been withdrawn from the endowment, demonstrating that the minimum threshold of replicas has never been approached and the overall efficacy of the protocol to date.

Co-founder of Arweave, Sam Williams, has run simulations, which indicate that the endowment would still retain tokens even after 10,000 years have elapsed since storing your data.

Scalability is ensured

In a typical contract-based system, every piece of data comes with unique specifications, such as the duration of storage, the number of copies and who is responsible for storing the data. Such a set up necessitates individual handling for each data commitment. As more data is added to the blockchain, this individual handling can lead to blockchain bloat and prevent the network from scaling.

Arweave does things differently. Instead of giving each piece of data its own rules, Arweave uses a single, overarching set of rules for all data on its network. This means it only has to handle one contract: the endowment. This approach streamlines the process and enables Arweave to scale regardless of the amount of data added to the network.

Future-proofing your data

According to Arweave’s yellow paper, the expectation is that when a more sophisticated permanent storage system arises, it will 'subsume' or incorporate the data preserved within Arweave. After the final block has been mined, the drive for data preservation will transition from financial incentives to social ones, a shift facilitated by the progressively decreasing relative cost of data storage.

The historic practice of 'nesting' or embedding retired archives within newer ones is predicted to be a continuing trend in Arweave's trajectory. For instance, Arweave currently hosts an archive of the 'Gopherspace', a precursor to the modern web, which itself contains archives of early Telnet and bulletin board-based discussion systems.

So, while the Arweave network as we know it may not persist indefinitely, the data safeguarded within it will.

Conclusion

The need for safe, long-term data storage is more pressing than ever. The conventional methods, whether it's personal hard drives, on-premise hardware or centralised cloud storage, come with their own set of problems – issues that get amplified when we consider the preservation of data for a span of 200 years. However, with the advent of blockchain technology, we are witnessing a transformative shift in how we approach data storage.

Arweave is an incredibly powerful primitive, unlocking a new technological paradigm: permanent storage. But it’s not necessarily a straightforward affair to access the network and make the most of this novel technology. This is where Akord comes in. We provide anyone, regardless of whether they’re a large organization or an individual without any technical expertise, with the tools to easily upload, manage, view and share their data within an highly intuitive app or suite of developer tools.

What’s more, Arweave is by design public, meaning all data is unencrypted and discoverable by anyone. But there are many instances where you may want to preserve data for generations in a private context. In this case, Akord offers quantum resistant end-to-end encrypted vaults (read about our approach to encryption on-chain here).

Whether you're an organization with large archives or someone looking for a way to save their family heritage, make your first upload to Arweave on us – get 100 MB free when you signup to Akord.

Keep learning

Still want to explore further? These articles may interest you:

• The Permaweb
• How to archive with Arweave?
• We, Arweave’s true inspiration?
• Arweave crosses 1 billion transactions