opensheaf-py, a literate program

Draft. OpenSheaf 0.1 is a draft specification: until version 1.0, a revision may change the format in incompatible ways, and these tools follow the latest draft.

This is the source of opensheaf-py (the osheaf command), the reference tools of the OpenSheaf specification, written as a literate program. It is meant to be read: the program is explained in the order in which it is best understood, with the reasons for what it does and the sections of the specification it implements. The Python files of the package are tangled out of these documents (see tangling) and kept in the repository, so that the package installs and runs like any other; the documents are also woven into this website (see weaving). One edits the documents, never the files.

An OpenSheaf archive is a folder of plain files: an OCFL storage root in which every document is an object, every version of the document is kept with its sources and its PDF, fixed by SHA-512 digests, and described by DataCite metadata; collections list documents, of the same archive or of others, pinned by digests; a discovery document and a website make it all findable. The chapters follow that order: first the archive and its documents, then collections and copies across archives, the website, the services the tools talk to (arXiv and Zenodo), the command line, and the tests.

Reading the code

Code is set in grey boxes. Above each box stands the name of its chunk, with ≡ for the first fragment of a chunk and +≡ for the ones that continue it; a name in angle brackets inside the code, like ⟨the syntax as patterns⟩, is a reference to another chunk, and a link to it. Below each box: where the chunk is used, and where it continues. The index of chunks lists them all.

The archive

1. The archive: configuration and storage: the settings, the OCFL storage root, inventories, object versions; the structure of the core module.
2. Document versions, history and dates: document versions as opposed to object versions, the history and the current version, notices carried forward, dates.
3. TeXmacs: typesetting and reading markup.
4. Depositing a document.
5. Description-only versions and completions: notices, withdrawals, relations, licences; sources found later.
6. The main collection.
7. The discovery document and the index.
8. Verifying an archive.
9. Creating an archive.

Across archives

1. Collections across archives.
2. Mirrors.
3. Accepted copies.

The website

1. The website.
2. The templates of the website.

Services

1. Importing from arXiv: fetching.
2. Importing from arXiv: the metadata.
3. Importing from arXiv: the object versions.
4. Zenodo: DOIs for document versions.

The command line and the tests

1. The command line, with options read from YAML files.
2. The tests.

The literate tools

1. Tangling: from the documents to the program.
2. Weaving: from the documents to the website.

What writing it down found

Explaining a program to a reader is also reading it carefully. Writing these chapters, with the code left exactly as it was, brought up a lag of the tools behind the specification as amended on 13 November 2026 (collections of two kinds, selections, mirrorOf, the main collection pinning heads), a mirror that collection update did not refresh, and a resolution that did not try the mirrors of an archive's discovery document. These have since been dealt with, in the chapters where they occur; nothing found remains open.