14 May 2021

Reading Folded Documents

Using an X-ray microtomography scanner and computational algorithms, an international team of researchers read unopened, securely and intricately folded 17th-century letters without unsealing or damaging them in any way.

Computer-generated unfolding sequence of 300-year-old sealed letter (from Unlocking History Research Group, www.eurekalert.org/pub_releases/2021-03/qmuo-sos030121.php).

Welcome back. Before I warm up, I must credit researchers affiliated with MIT, Adobe Research, King’s College London, Queen Mary University of London and The Netherlands’ Utrecht, Leiden and Radboud universities for the fascinating study I’m about to review.

Their research was a twofer for me; maybe for you, too. Though generally familiar with tomography from both geologic seismic surveys and X-ray CT or CAT scans (computer tomography aka computerized axial tomography), I was new to X-ray microtomography. Likewise, I’d come across articles on reconstructing documents but had never heard of letterlocked messages, much less applying X-ray microtomography to reveal their contents. Suppose I jump right in.

X-Ray Microtomography (XMT)
Tomography is a method of producing three-dimensional images of the internal structure of solid objects (e.g., the human body or earth) by recording how that internal structure affects energy waves passing through the object. With CAT scans, the energy waves are imposed X-rays; with geology, the seismic waves may be generated by earthquakes or explosions. 

Schematic of X-ray microtomography (from www.sciencedirect.com/science/article/abs/pii/S1044580310002706).

X-ray microtomography, in use since the early 1980s, is tomography that produces high-resolution images with micrometer-size pixels. Among a wide range of applications (e.g., archaeology, fossils, electronics) is biomedical, including dentistry, where XMT can produce high-quality images from the outer to innermost structure of teeth and surrounding structures. The XMT scanner used to read letterlocked messages for this study was developed for dental research.

The Letterlocking Dictionary and Wikipedia help me define letterlocking as folding and securing a written message, whether on paper or another material, to serve as its own envelope or sending device.

Dating to the 13th century in Western history, letterlocking fastens a letter with slits, tabs or holes, various folding techniques and the possible addition of sealing wax or string. It’s all to prevent the “letterpacket” from being read without breaking the securing features or leaving other signs of tampering. Although letterlocking might involve intricate folding with artistic elements that suggest a sender of elite status, the procedure was used by all social classes before the proliferation of mass-produced envelopes.

Letterlocking categories based on manipulations and assigned security score; filled category numbers indicate historic originals exist; unfilled numbers indicate hypothetical categories; letterlocking formats show up to 12 edges with indicative examples (www.nature.com/articles/s41467-021-21326-w).

The study examined four letters from the Brienne Collection, a postmaster’s trunk of undelivered letters sent from all over Europe to The Hague, the Netherlands, between 1689 and 1706. The trunk contains 3,148 items, including 1,706 opened or partially opened letterpackets and 577 unopened letterpackets, which may themselves contain additional items. One of the four tested letterpackets was opened.

One of four tested letterpackets (DB-1627) from Brienne Collection: (a) outside front and back, (b) transparent view through volumetric XMT data, (c) crease pattern visible if fully opened, (d) proposed step-by-step letterlocking process to make packet from a flat piece of paper based on virtual unfolding results, (e) letterlocking category and format (www.nature.com/articles/s41467-021-21326-w).

Virtually Unfolding Letterlocked Messages
The computational algorithms used to unfold the four XMT-scanned letterpackets built upon earlier work that extracted text from historical documents--primarily scrolls, books and once or twice folded artifacts, particularly highly damaged documents that could not be physically opened and read.

Given that the letterpackets were largely undamaged homogenous paper material, the focus was to automate and extend computational flattening frameworks to address more intricately folded documents than had previously been considered. The general steps were as follows:

- XMT scanning produces a volumetric dataset representing material density in 3D space.
- Segmentation identifies and separates layers of writing substrate.
- Flattening computes a distortion-minimizing, 2D embedding of the segmentation result, corresponding to the document’s unfolded state.
- Hybrid mesh propagation repairs discontinuities on the folded and flattened embeddings and automatically merges flattened connected components into a common reference frame.
- Texturing produces a 2D image of the virtually unfolded letter by mapping voxels (essentially 3D pixels) from the volumetric scan to their corresponding flattened destination.

Wrap Up
Virtual unfolding produced nearly complete reconstructions of all four letterpackets. Each had a distinct folding sequence, including inner folds angled diagonally to their outer silhouette. For two with a separate paper lock, unfolding reconstructed the primary writing substrate and the paper lock as two pieces.

The results demonstrated the applicability of the approach for letterlocked messages and likely for many other types of historical texts (letters, scrolls and books).

So, was it a twofer for you? At least a one? Thanks for stopping by.

Study of virtual unfolding of sealed documents in Nature Communications journal: www.nature.com/articles/s41467-021-21326-w
Letterpackets from study: brienne.org/unfolding
Article on study on EurekAlert! website: www.eurekalert.org/pub_releases/2021-03/qmuo-sos030121.php
X-ray microtomography background: en.wikipedia.org/wiki/X-ray_microtomography

If you're interested in letterlocking, here are two rather involved (very cool) techniques:

Letterlocking with a spiral lock (photo from 5:29 minute video www.youtube.com/watch?v=1ZRA8KZrNTQ).
Letterlocking with dagger-trap pleated letter (photo from 12:11 minute video www.youtube.com/watch?v=16GAIaYN_Gk).

No comments:

Post a Comment