Something went wrong. Try again.
This repository has no description
Something went wrong. Try again.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221<!DOCTYPE html><html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>Content-Addressable aRchives (CAR)</title> </head> <body> <div id="abstract"> <p> The CAR format offers a serialized representation of set of content-addressed resources in one single concatenated stream, alongside a header that describes that content. </p> </div> <section> <h2>Introduction</h2> <p> The CAR format (Content Addressable aRchives) is used to store series of content-addressable objects as a sequence of bytes. It packages that stream of objects with a header. </p> <p> Much of the content of this specification was initially developed as part of the <a href="https://github.com/ipld/ipld">IPLD</a> project. This specification was developed based on demand from the community to have just the one simplified document. Note that a CARv2 specification was developed at some point to add support for an index trailer, but it met with limited adoption and so was not considered when bringing CAR into DASL. </p> </section> <section> <h2>Parsing CAR</h2> <p> The CAR format is made of a Header followed by a Body. The Header is a length-prefixed chunk of DRISL ([[drisl]]) and the Body is a sequence of zero or more length-prefixed blocks that contain a tuple of a DASL CID ([[cid]]) which is always 36 bytes long and the data addressed by that CID. The length prefix in a CAR is encoded as an unsigned variable-length integer ([[varint]], a variant of <a href="https://en.wikipedia.org/wiki/LEB128">LEB128</a>). This integer specifies the number of remaining bytes, excluding the bytes used to encode the integer. </p> <pre><code>|--- Header --| |------------------- Body -------------------|[ int | DRISL ] [ int | CID | data ] [ int | CID | data ] … </code></pre> <p> The steps to <dfn>parse a CAR</dfn> are: </p> <ol> <li> Accept a byte stream <var>bytes</var> that is consumed with every step that reads from it. </li> <li> Run the steps to <a>parse a CAR header</a> with <var>bytes</var> to obtain <var>metadata</var>. </li> <li> Set up array <var>blocks</var> and run these substeps: <ol> <li>If <var>bytes</var> is empty, terminate these substeps.</li> <li> Run the steps to <a>parse a CAR block</a> with <var>bytes</var> to obtain <var>cid</var>, <var>data size</var>, and <var>data</var>. </li> <li> Push an entry onto <var>blocks</var> containing <var>cid</var>, <var>data size</var>, and <var>data</var>. </li> <li>Return to the beginning of these substeps.</li> </ol> </li> <li> Return <var>metadata</var> and <var>blocks</var>. </li> </ol> <p> Note that the CAR header contains a near-arbitrary DRISL object that is to be treated as metadata ([[drisl]]). For historical reasons, there are two constraints on the header: </p> <ul> <li> The object must contain a <code>version</code> map entry, the value of which is always integer-type <code>1</code>. Version numbers in data formats are an anti-pattern, and as a result this number is guaranteed never to change. </li> <li> The object must contain a <code>roots</code> entry, which must be of type array. It may be empty, but if it isn't then it must be an array of CIDs encoded using tag 42 ([[cid]]). A CAR can be used to contain one or more DAGs of [[drisl]] content and the purpose of the <code>roots</code> is to list one or more roots for those DAGs. The array may be empty if you do not care about encoding DAGs. </li> </ul> <p> Some historical implementations will only return <var>version</var> and <var>roots</var>, but it is recommended that they make the entire <var>metadata</var> object available. A best practice for authors is to use the <var>metadata</var> to capture MASL content, which is able to provide metadata and a pathing mapping for the entire content of the CAR stream if needed ([[masl]]). </p> <p> The steps to <dfn>parse a CAR header</dfn> are: </p> <ol> <li>Accept a byte stream <var>bytes</var>.</li> <li>Read an unsigned varint <var>length</var> from <var>bytes</var> ([[varint]]).</li> <li>If <var>length</var> is 0, throw an error.</li> <li> Read <var>length</var> bytes from <var>bytes</var> and decode them as DRISL ([[drisl]]) into <var>metadata</var>. If <var>metadata</var> is not a map, throw an error. </li> <li> If <var>metadata</var> does not have a <code>version</code> key entry with integer value <code>1</code>, throw an error. Otherwise, store <code>version</code> in <var>version</var>. </li> <li> If <var>metadata</var> does not have a <code>roots</code> key entry that is an array, or if that array contains anything other than DASL CIDs, throw an error. Otherwise, store <code>roots</code> in <var>roots</var>. </li> <li> Return <var>metadata</var>. (For implementations that only report <code>version</code> and <code>roots</code>, return those.) </li> </ol> <p> After its header, CAR contains a series of blocks each of which is length-prefixed. </p> <p> The steps to <dfn>parse a CAR block</dfn> are: </p> <ol> <li>Accept a byte stream <var>bytes</var>.</li> <li>Read an unsigned varint <var>length</var> from <var>bytes</var> ([[varint]]).</li> <li>If <var>length</var> is less than 36, throw an error.</li> <li> Read a CID ([[cid]]) from <var>bytes</var> and store it in <var>cid</var>. Note: the length of the CID is always 36 bytes. </li> <li>Set <var>data size</var> to <var>length</var> minus 36 (the CID length).</li> <li> Read <var>data size</var> bytes from <var>bytes</var> and store the result in <var>data</var>. </li> <li> Return <var>cid</var>, <var>data size</var>, and <var>data</var>. </li> </ol> </section> <section> <h2>Additional Considerations</h2> <section> <h3>Conformance</h3> <p> A CAR stream must only feature DASL CIDs. </p> <p> A CAR stream must have CIDs that match the data body that follows them. A CAR implementation should verify that CIDs match block body data, though it may delegate verification to other components. (Keep in mind that not verifying at all negates the value of content addressing.) </p> <p> A CAR stream's stated roots must match CIDs contained in the Body. However, implementations frequently operate in a streaming fashion such that they have no way of knowing whether a CAR stream conforms to this requirement before having processed the entire stream. Checking correctness with respect to this requirement may therefore be more readily performed via a warning (at end of processing) or a dedicated validator. </p> </section> <section> <h3>Determinism</h3> <p> Deterministic CAR creation is not covered by this specification. However, deterministic generation of a CAR from a given graph is possible and is relied upon by certain uses of the format, most notably, <a href="https://filecoin-project.github.io/specs">Filecoin</a>. dCAR may be the topic of a future specification. </p> <p> Care regarding the ordering of the <code>roots</code> array in the Header and avoidance of duplicate blocks may also be required for strict determinism. </p> </section> <section> <h3>Security & Verifiability</h3> <p> The <code>roots</code> specified by the Header of a CAR is expected to appear somewhere in its Body section, however there is no requirement that the roots define entire DAGs, nor that all blocks in a CAR must be part of DAGs described by the root CIDs in the Header. Therefore, the <code>roots</code> must not be used alone to determine or differentiate the contents of a CAR. </p> <p> The CAR format contains no internal means, beyond the blocks and their CIDs, to verify or differentiate contents. Where such a requirement exists, this must be performed externally, such as creating a digest of the entire CAR (and refer to it using a CID). </p> </section> </section> <section> <h2>Appendix: Media Type</h2> <p> The media type for CAR is <a href="https://www.iana.org/assignments/media-types/application/vnd.ipld.car"><code>application/vnd.ipld.car</code></a>. </p> <p> The conventional file extension for CAR is <code>.car</code>. </p> </section> </body></html>