The web3 archive

Introduction

Let’s be upfront: archiving is not a hot topic. It’s not your go-to dinner table chat. And it’s not what most people think of when it comes to cutting edge tech and the blockchain space.

But in this article I’m going to show you how web3 technology, specifically Akord and Arweave, are completely reimagining what an archive can do and how it can be experienced. It’s actually pretty exciting, and I’ll prove it. 

The web3 archive has 11 fundamental properties which we’ll explore in this article: permanent, sustainable, decentralised, immutable, self-sovereign, secure, accessible, cost-efficient, scalable, collaborative and user-friendly.

Permanent and sustainable

Longevity is the key component in an archive. The more resilient and the longer an archive survives, the better.

When we stored information on clay tablets and papyrus as mediums for storing data, they were not particularly permanent for obvious reasons.

The printing press and computer technology represented massive leaps forward, increasing the degree of permanence to an archive by orders of magnitude.

Still, an archive without a shelf life and that didn't need constant maintenance has remained (until now) out of reach.

Current mainstream archiving solutions use either tape or centralised cloud-storage technology. Tapes have a 20–30 year lifespan, but the technology to manage them lasts ~10 years.

Centralised cloud storage depends on third party providers and managing monthly payments and complex contracts. Who knows if Google Cloud will exist in 30 years? Will you still want to pay their prices or agree to their terms next year, in 5 years, or 10?

Arweave delivers permanent, user-owned data through a decentralised network, like Bitcoin, and uses a blockchain consensus mechanism called the blockweave.

When you upload to Arweave, you pay once. And never again. That payment goes into an endowment that can pay for your data’s perpetual storage, because the cost of storage decreases in a predictable fashion over time.

The network incentivises miners to store all data uploaded to Arweave, and the endowment is structured to be funded for at least 200 years. Critically, for archives, this permanence means an end to data migrations, which are costly, time consuming and open the door to human error and data loss.

While the traditional data storage industry is scheming at the theoretical level about the anatomy of the 100 year archive, Arweave has 2x that ambition and realised the infrastructure with a sustainable model that will keep running without centralised control and maintenance.

These traditional models at long-term storage attempt to spread data across different geographical locations to increase resiliency. Arweave already has hundreds of nodes spread across the globe, making it an inherently more resilient network for long-term data storage.

In 2022, Arweave upgraded the protocol to version 2.6, declaring that work on the protocol is for the most part finished. Using Bitcoin as a parallel, a blockchain protocol does not unlock its full value until it is stable – a prerequisite for businesses and organisations to confidently build on the network.

But unlike Bitcoin, Arweave is an incredibly energy efficient network. In the words of founder Sam Williams, “it’s all about more data, less energy.” Arweave has a Proof of Access consensus mechanism, described as Proof of Work (Eg, Bitcoin) security, with Proof of Stake (Eg, Ethereum) efficiency.

According to Williams, The system uses ~800 SHA256 hashes per node, per second. For context, a macbook uses 8-10million hashes per second. Further, as the blockweave expands in size, the amount of electricity used in the mining process decreases.

The permanent storage that Arweave offers is zero-to-one, meaning this a completely new technological offering; a new paradigm that has been unlocked. It's why companies like Meta have decided to use Arweave to store certain types of data that have indefinite storage needs, because no other solution can offer them truly permanent storage.  

Decentralised and immutable

Arweave is a decentralised network, which means that no central authority controls or owns the data on the network. The contracts that govern the dynamics between the network and its participants are in open-source code, making their integrity independently verifiable.

Because the network is permanent and decentralised, it means the data is immutable and its integrity is guaranteed.

One of the core challenges facing modern archiving is to ensure that digital objects remain unaltered while stored in an archive. When a physical object is tampered with it will often present obvious clues, but this is not so with digital objects that are by their nature designed to be easily edited or overwritten.

Arweave and Akord solves this problem out-the-box. On the Arweave blockchain, every upload will forever be recorded, and stored, unaltered. And in every digital vault in Akord you can easily view a complete immutable timeline of events, where every action in the vault (a file uploaded, moved, renamed, a folder created, a message sent, and so on) can be located and is timestamped on-chain.

Anything in an Akord archive can be objectively proven and verified as being the original at any point in the future.

Self-sovereign

When data is permanent, decentralised and immutable, it follows that it will be self-sovereign, or ‘user-owned’.

If you’re archiving data with Microsoft Azure or Amazon S3 Glacier, you are effectively renting access to that data.

If you stop paying, you don’t get access and your data will be deleted. Considering that it can take up to two years to plan and execute a data migration, many archives are essentially at the mercy of their data landlords.

The web3 archive empowers businesses and organisations with ownership of the most valuable resource they possess – their data.

True ownership is a fundamental pre-requisite to permanent or perpetual storage – if you don’t own your data in a very tangible way, then you will forever be at the mercy of myriad external factors that will be out of your control.

Secure

The web3 archive can be public or private. If you choose to encrypt all your data, then it can be secure in the areas that traditional cloud storage services, like Amazon S3 Glacier, Microsoft Azure and Google Cloud, are famously prone to attack.

The encryption keys for these services are held on centralised databases vulnerable to being hacked or leaked through human error, which often results in terrabytes of data and hundreds of thousands, or even millions, of user records being exposed. 

With Akord, our encryption is quantum resistant and end to end. Most importantly, you own the keys to that encryption. We can’t see your data, ever. There are no backdoors. This level of privacy hands you the control and ownership over your digital assets, eliminating the need to trust third parties to properly protect your data.

Decentralised storage is also resistant to ransomware attacks by the very nature of its architecture. As data is spread across hundreds or even thousands of nodes across the globe, hackers would have to attack all nodes simultaneously in order to steal and hold data to ransom.

In The State of Ransomware 2021 report by Sophos, 37% of respondents’ organizations were hit by ransomware that year, and of those over half said the cybercriminals were successful. The average bill for rectifying a ransomware attack, considering downtime, people time, device cost, network cost, lost opportunity, ransom paid etc was US$1.85 million.

Cloud services try to protect against ransomware through measures that further lock down the data and tighten access control, with the consequence that data becomes much less readily accessible.

Accessible

Archiving is a serious, meticulously undertaken process with a long history. But the idea around what is involved in this process is changing with the possibilities of digital technology and the interactivity that web2 has ushered in.

Archiving used to be characterised with a file-it-and-forget-it mentality. The focus and majority of work was centred on the initial archiving away of data. Physical archives have been passive and static, difficult to access due to a fixed geographic location.

But digital archives don’t have these physical restrictions; they have collapsed space, requiring only an internet connection, in theory. Digital technology and user-friendly apps have opened up the idea that archives don’t necessarily need to be passive; they can be active, living and breathing tools, where their intrinsic value can be more readily accessed and used by many people, not just a few gatekeepers.

But web2 has failed to deliver on this potential. The big cloud services offer special archiving tiers for their storage, but this data can take up to 48 hours to retrieve and cost you money to do so.

These services are so complex to manage, that centralised cloud archiving has become the domain of IT departments or specialist contractors, locking out or creating huge barriers of entry for a company, organisation or public at large.

Cost-efficient and scalable

Archives, ideally, last for a long time. A very long time. There are many areas of archiving where storing data “forever” is mission critical: social and cultural, media and entertainment, science and research, to name the obvious.

With this long-term view in mind, it’s worth thinking about your archival solution’s cost over decades, not years.

“Total cost of ownership”, or TCO, is a phrase used to define the lifetime cost of buying an asset. In the case of data storage, to define the TCO we need to know how long we want to store the data for. With cloud storage solutions, it is impossible to define the total cost of ownership if you pay monthly for storage, and have an indefinite retention period for that data. Also, as discussed previously, even if you did have a fixed period of time in mind, you would never truly own that data.

With Akord and Arweave, TCO is effectively known on day 1. That’s because you pay upfront for storage and never again. And you control the data, not a centralised third party.

This means you pay what may feel like a premium to start your archive, but after a decade you've already started paying more with monthly payments on a traditional cloud service. That doesn't even take into consideration all the costs associated with managing that service and inevitable data migrations as services change terms and prices. After 20 years the difference is stark. 

With a service like Microsoft Azure or Google Cloud, the cumulative cost of every GB of data stored never stops going up over time. And the more data you add to the archive, the more you compound the effect, spiralling the cost of your archive over the years.

This is why Akord is a truly scalable archiving solution, especially for organisations that are continually archiving more and more data. Many archives are funded through unpredictable sources – grants, public funding, private donations from patrons. You can't scale an archive when you're beholden to gatekeepers whose continued support cannot be counted on.

Without knowing the TCO for the data in an archive it’s difficult to scale an archive without potentially risking its very existence by arriving at some point in the future where the business or organisation can no longer afford its maintenance.

What's more, the Arweave protocol itself is designed to be scalable. As the Arweave team themselves explain:

Pay once, store forever. This is the only way an archive can confidently scale.

Collaborative and user-friendly

Akord’s vaults are beautifully designed, interactive and collaborative.

You can invite other team members and set access-level rights, send them messages and receive notifications informing you of activity in the vault.

Managing the data in a vault can be handled through our Command Line Interface and developer tools, or through our web app. Akord, and specifically the web app, is designed to remove all the normal web3 barriers: you don’t need tokens or third party crypto wallets – create your Akord wallet on signup in a few clicks and pay for storage with your credit card. We work with organisations or businesses who are working with large amounts of data to buy storage on the network at a competitive price. 

The Akord app works just as well on your mobile as it does on your desktop. We’ve focused on making the overall user experience feel intuitive and straightforward, taking the best of web2 app experiences and combining that with web3 technology under the hood.

With the web3 archive anyone in your team, regardless of their technical skills, can easily access and contribute to the archive.

Akord’s versatile media viewer is particularly powerful for public archives, where sharing and presenting digital objects in an accessible way to the public is a priority.

Our media viewer supports many file and image formats as well as video and audio. You can switch between focusing on single image at a time or a gallery mode, zoom in on images and PDFs, flick through the pages of a PDF document, change between light and dark mode, and print directly from the media viewer.

Supporting archives that matter most

Archives no longer need to be hidden away. They can be accessed instantly by anyone in your organisation, or with anyone across the globe if that’s what you want.

We believe that a great archive needs a great experience. If your data is valuable enough to be stored forever, then you should do it the justice of being easily accessed and presented in the best possible light.

The tools we have are just the beginning. We’re constantly working on improving the Akord experience, and we have exciting new features in our roadmap for 2023 to further elevate your web3 archives.

If you’re archiving data that is valuable for the social good, preserving what matters most to humanity, then you may be eligible for specialist support and even storage grants for you project. Please reach out and let me know more about your work.