Open Index Protocol

From OIP Wiki
Revision as of 20:34, 13 August 2018 by Bitspill (talk | contribs) (Reverted edits by Bitspill (talk) to last revision by Devon)
Jump to: navigation, search
Open Index Protocol
An 'open jungle' blockchain specification for a worldwide database
Initially Developed By Blockchain Technology Group LLC
Live Demo

The Open Index Protocol is a specification for decentralized publishing, distribution and payments of any digital media. The specification includes:

  • How a proof-of-work blockchain[1] and interoperable transport protocols create a shared data layer for digital content
  • How metadata, terms of use, file addressing, and payment information are stored in the blockchain
  • How to search and browse the shared data layer
  • How the blockchain security incentive is connected to the value of the content published to the shared data layer
  • How file persistence & legal compliance are ensured
oip simplified stack
comparison of web vs oip stack

Open Index Protocol uses the Flo blockchain for the meta-data index and interoperable transport protocols for file storage/distribution and payments. Current implementation supports IPFS for file storage/distribution and Bitcoin, Litecoin and Florincoin tokens for payments; support for additional file and value transport protocols is on the roadmap.


In February 2014 Devon & Amy James proposed decentralized applications ArchiveChain[2] and MovieCoin[3] on the Ethereum community forum. Early development proved a shared data layer for any digital content is more efficient than separate appcoins for each, so these initial ideas were combined to become Open Index Protocol. Significant contributions to the specification have been made by Ryan Jordan, Ryan Taylor, Skylar Young, Jeremiah Buddenhagen and lead Florincoin developer Joseph Fiscella.

The initial proof of concept archived social media data into a blockchain and visualized the data as an interactive word-cloud. It was demonstrated in October 2014 at Inside Bitcoins, Las Vegas[4]. Next, streaming audio and video were added using the BitTorrent network. On Feb 19, 2015 the first piece of media was published and retrieved using a 100% decentralized system. Bitcoin has been used for paid content since the initial public release in April 2015 [5]. In May 2015, the primary file storage/distribution network was changed to IPFS and support for BitTorrent was deprecated. At each stage, a user-facing browser or visualizer was built alongside specification development [6]. Initially the browser was known as ‘The Decentralized Library of Alexandria’ and the specification as the ‘Alexandria protocol.’ In a demonstration at the Decentralized Web Summit hosted by the Internet Archive in June 2016, Sir Tim Berners-Lee said the names were confusing and suggested they be changed [7]. The browser name was changed to ‘Alexandria’ and the specification name changed to ‘Open Index Protocol.’


Central points of failure plague current digital distribution architecture (Figure 1). The system is vulnerable to attack demonstrated by problems like hacked personal information[8], spying[9][10], deplatforming, demonetization and censorship[11], as well as direct technical attacks on network infrastructure itself[12]. Current architecture also suffers from wasteful redundancy[13] which increases operational costs[14]. Network speed is fragile because speed and popularity are negatively correlated (Figure 2).
A sketch of network shapes from On Distributed Communications.
Figure 1
A sketch of peer download speeds comparing HTTP to p2p networks
Figure 2


Zero central points of failure. Every component of Open Index Protocol is distributed. The specification is a permissionless system[15] that creates an ‘open jungle’ where anyone can publish, distribute and sell. The Open Index Protocol changes the fundamental economics of content distribution by transforming individually-negotiated, contract-based services into digital commodities that can be exchanged with fungible tokens. The system expands the sharing economy[16] to new services including front end platforms, social media influencers, proof-of-work miners and file storage/distribution providers.

Why Now?

Twenty years ago, print media changed forever when the ‘open jungle’ of the internet democratized information exchange[17]. Walled garden services[18] like America Online, CompuServe & Prodigy did not initially use HTTP, but eventually adopted it because the open jungle of the World Wide Web was growing in popularity[19]. Today, walled garden content distribution services like YouTube[14][20][21], Spotify[22][23][24] & Netflix[25][26][27] are ripe for similar disruption by the open jungle of Open Index Protocol.

The current digital content distribution industry is in crisis. Artists blame confusing contract terms. Audiences are so frustrated by the difficulty of accessing content they resort to piracy[28][29]. The industry as a whole is struggling.

The root of these problems is twofold: 1) current decentralized hub and spoke distribution architecture cannot support market demand as efficiently as a distributed network, and 2) current solutions were constructed when the technology required for a decentralized and open system did not yet exist.

Since 1999, p2p file sharing[30][31], blockchain[32] and other decentralized technologies have been invented to solve these problems. Open Index Protocol defines a specification for how to use these technologies to create a permissionless system for decentralized publishing, distribution, access and payments of any digital media.


  • Open - Permissionless system for publishing, distribution & payments of any digital media. The shared data layer functions as a public open index; blockchain protects metadata, peer-to-peer file storage/distribution ensures access, and digital currency streamlines payments. Transport layers for file storage/distribution and payments are fully interoperable.
  • Efficient - Distributed networking reduces system overhead, cryptographic tokens decrease administrative management of content & payments, p2p file storage/distribution increases playback performance and user payment process is simplified.
  • Independent - Unique Terms of Use controlled by individual user, increased revenue channels for publishers (including direct payments in social media).
  • Anticensorship - Open Index Protocol is fully transparent. Full global state replication of index data, transparency of all publish attempts including fails, and human readable index data ensure content data is transparent and auditable. Prevents deplatforming, demonetization and censorship.
  • Secure - Index is protected by proven proof-of-work blockchain and contributing to security is incentivized. The commercial value of content is connected to and drives the security incentive. System enables collective defense against attack, ensures liquidity of tokens for publishers, and automates reliable profit margins for miners and traders.
  • Antifragile - System has positive sensitivity to increases in volatility. Free market economics strengthened by interdependent incentive structure; system fees drive network security. Chaotic market inputs cause convex response, improving network incentive for all participants (publishers, platforms, index mining and storage/distribution of files). Value capture is split between subjective/objective services which benefits overall network growth. Interoperable transport layers can adapt to changes in market preferences.

Design Philosophy

The system design objective is to serve the digital media distribution market with an open specification using decentralized technology. To meet the needs of this large and diverse industry, the specification is:

  • An "open jungle"[33]
  • As interoperable as possible with as few rules as possible
  • Capable of serving all distribution and monetization models: free, ad based, pay per view, subscription, metered, non-financial exchange (seeding), rental, lifetime access, stream, download.

Example uses: backend shared data layer for blog entries, Wikipedia articles, Netflix shows, YouTube videos, Spotify songs, SoundCloud remixes.


  • A sufficiently diverse marketplace will include rational actors acting in their own best interest who understand it will benefit them to use standards & obey the law
  • A proof-of-work blockchain-based shared data layer is the most efficient technology available to protect information freedom and resist entropy of information access
  • Incentive structures influence outcomes. A trust-minimized, permissionless system with a sustainable market-based incentive structure fosters cooperation, competition, and iteration; continuously improving product-market-fit
  • At present, some human governance is necessary. Open Index Protocol Working Group is a potential central point of failure, vulnerable to social attack. This risk is minimized with a transparent governance structure, weighing feedback from all system participants: publishers, platforms, influencers and miners


To ensure system integrity, the specification was built with two inviolable principles: permissionless system design and limited standardization

Permissionless system design

All networks used by Open Index Protocol are permissionless, anyone can join the networks without requiring authorization from another party. Examples of other permissionless networks include BitTorrent, Bitcoin, IPFS and Ethereum. Permissioned layers can be built on top of a permissionless system for use-cases where identity and permissioning are appropriate, however the opposite is not possible.

Open Index Protocol is a permissionless system because freedom of information requires a free and open shared data layer for digital media to ensure media persistence, censorship resistance, security, and sustainability.

Media persistence and censorship resistance are the foundational values behind the permissionless system design. Index and file persistence are secured via: decentralized networks, incentive alignment with positive sensitivity to market chaos, and transparent governance. Censorship resistance is protected via: full global state replication of index, human readable data, auditable records (all publish attempts including fails), and transparent immutable ledger.

Security vs convenience friction is reduced through permissionless system design. API endpoints are universal; they can be looked up through a localhost or through any protocol compliant host. When security is prioritized over convenience, the end user or service provider runs a local protocol daemon and full node. When convenience is prioritized over security, the end user or service provider accesses the system through hosted full nodes. Both local and hosted addresses have the same url format, making host addresses interchangeable.

Sustainability of the system is balanced and maintained with interdependent incentive alignment; the commercial value of the media is connected to and drives the system security incentives.

Limited Standardization

Open Index Protocol specification limits standardization such that the incentive structure benefits assimilation, and does not threaten negative restriction. A salutary approach is used; the specification defines standard use of objective data (index, file addressing, payments), it does not standardize subjective, service-based uses of the data (proprietary discovery algorithms, user experience, user interface). The co-opetitive model benefits from cooperation on objective data, competition of subjective services, and collaboration for iteration; system incentive design improves potential for product-market-fit.

Standardization is powerful; Open Index Protocol benefits from all three fundamental characteristics of standardization:

An example of the power of standards is the rate of google searches compared to emails sent. Google is the most popular individual website in the world, but it’s index and search algorithm are proprietary, whereas email is a standard; for each google search[35] performed, about 40 emails[36] are sent.

Economic Principals

Thin and Fat Protocols

Websites compared to Blockchains: How value is distributed across systems and corresponding value capture opportunity, as coined by Joel Monegro.

A sketch of thin vs fat protcols.
Joel Monegro's Fat Protocol's.
  • Thin Protocol: HTTP/World Wide Web is a thin protocol because all of the value (usually in the form of data - ie Google, Facebook) is captured at the application layer and none is captured at the protocol layer.
  • Fat Protocol: Bitcoin is a fat protocol because the majority of value capture happens in the protocol layer (by miners), and the application layer can only capture value by offering additional services beyond the protocol function.

The Web's thin protocol has caused the walled-garden content distribution marketplace, and the recent introduction of fat protocols threatens to perpetuate the problem. As explained by Vitalik Buterin, fat protocol systems are

Fat protocols offer no direct incentive for the developer community to create application layer services, thus network effect is limited by lack of competitive services to draw users at the application layer.

The pioneers of the internet understood that open source standardization is key to information freedom. Standardized specifications open new markets for competitive services.

Salutary Protocol

  • Salutary protocol: Open Index Protocol is a salutary protocol because value capture is divided between the protocol and application layers.
A sketch of a salutary protcol.
OIP's Salutary Protocol.

Objective services like index security and file storage/distribution capture value at the protocol layer; subjective services like user interface and content discovery capture value at the application layer. Value at the protocol layer comes from blockchain tokens awarded for fungible kinds of work; value at the application layer comes from salutary requirement that a percent of value transmitted is assigned to the application layer for subjective kinds of work.

Both thin and fat protocols govern a single class of asset exchange (ie; hypertext transport or value transport). Salutary protocols employ these single purpose protocols and standardize their use for dual or multi purpose asset exchange (ie; file transmitted in exchange for value transmitted). All three protocol types have the components associated with positive network effect: limited standardization and interoperability. Salutary protocols have the unique characteristics of metered system fees, user-based governance and an eco system in which a unified open set of data (back end) is financially incentivized to be used/distributed by more than one competing interface (front end).

Salutary protocols are more resilient.

  1. Free market-determined components:
    1. Market determined selection of single purpose protocols used within salutary system. Increases stability and flexibility within the multi-equilibrium token market & eliminates de facto fee associated with “implicit cost” of holding tokens in fat protocol systems. Example: Current version of OIP uses Bitcoin & IPFS because they are the most popular, but it can support payment & storage/distribution in any single asset class protocol, and can adapt to changes in market preferences.
  2. System fees:
    1. Open market with closed loop system fees incentivizes a sustainable ecosystem. OIP stabilizes the relationship between 1) the value of media in the system with 2) the security of the system by using metered fees. Salutary protocols have an interdependent fee structure with market pressures from all stakeholders; the effect is similar to the “sink” (fee) as defined by Vitalik Buterin because salutary protocol system fees are “transparent and explicit.” Salutary protocol system fees are also stable and easy to calculate “instead of the highly variable and difficult to calculate ‘defacto fee’" associated with the ‘implicit cost’ of holding tokens. In this way, salutary protocol system fees function like a sink, because they are “a more transparent and explicit way to figure out what the value of the protocol tokens should be.” Example: Open Index Protocol calculates & enforces the minimum cost associated with publishing the data to the blockchain and associated storage/distribution cost and then connects the markets of: front end/artist, front end/audience, miner/artist, miner, front end, artist/influencer, artist/audience.

Comparison of digital distribution services

Service Offering Cost to Users Revenue Creator Share of Revenue
Open Index Protocol Self Publishing of Downloadable and Streaming Music, TV, Film and Videos A La Carte, Subscription and Ad-Based Varies based on use Completely up to creator As much as 100% to creator
iTunes Purchasable Music A La Carte Only $0.99-$2.19 per track Varies based on use 70% to creator
Spotify Streaming Music Subscription & Ad-Based $9.99 per month ~$0.0072 per play 70% to creator
Pandora Algorithm-Based Streaming Music Subscription & Ad-Based $4.99 per month ~$0.0014 per play 70% to creator
SoundCloud Self Publishing of Streaming Music Subscription & Ad-Based $7-$15 per month Unknown Unknown
Netflix Streaming TV/Film Subscription Only $7.99 per month Unknown Unknown
YouTube Self Publishing of Streaming Video Subscription & Ad-Based $9.99 per month ~$0.0005 per play 55% to creator
Crackle Streaming TV/Film Ad-Based Only $0 Unknown Unknown

See also

  • Alice and Bob: An explanation of the various kinds of users and service providers in the Open Index Protocol eco-system.
  • Specification: The rules, variables, formulae and thresholds that make up Open Index Protocol.
  • Message Protocol: The JSON schema of various kinds of OIP messages and definitions of all terms used within them.
  • Security Considerations: the questions, solutions & security considerations that have been analyzed.
  • Back to the Main Page

Other Open Index Protocol Applications in Development

  • streaming music player
  • Caltech/Jensen Lab's Tomography Public Database
  • Robo3D's "3DRM" marketplace
  • video archiver
  • eVue Digital Labs


  5. Alexandria v0.4.0 Video Demo
  6. Alexandria v0.5.1 Video Demo
  7. Presentation at the Decentralized Web Summit
  8. Washington Post: Data of 143 million Americans exposed in hack of credit reporting agency Equifax
  9. 9 Examples of How Smartphones Are Eavesdropping on You
  10. Wired: A Hacker Can Turn an Amazon Echo into a Wiretap
  11. BBC: Turkey social media ban raises censorship fears
  13. Juan Benet IPFS Presentation
  14. 14.0 14.1 YouTube: 1 Billion Viewers, No Profit; The Wall St Journal
  15. Blockchain – What is Permissioned vs Permissionless?
  16. Sharing Economy;
  34. What does “permissionless” mean?