Designing a network view for cases too big to draw
The Workbench's first network view tried to draw a whole case at once, and on a sample case it could not be read. Research into why found that none of the tools investigators already use does that: they start from a search or a single party and expand step by step, or gather many entities into one. The replacement, the Wire, is one map of a case at three altitudes.
The failure was on the 1MDB demo case, a sample investigation in the Workbench, which then had 81 parties (the people and companies in a case) and 108 links. The layered “who controls whom” layout fitted everything on one screen and could not be read. Real cases reach 500 to 5,000 parties. On 11 October I treated it as a design problem that needed research before any fix.
The tools investigators use never show everything
OpenAleph, the open-source platform the Workbench reads its files with, builds its diagrams by hand and expands them party by party. The ICIJ's Offshore Leaks database, run on Linkurious for the Pandora Papers, goes from a search to one entity to its neighbours, one step at a time. Maltego, a link-analysis tool, gathers 25 or more entities of one type into a single “collection node”.
Neo4j Bloom, a graph explorer, starts empty, with a search. Kumu, a mapping tool, grows the view one degree at a time.
The research papers agreed. Path-finding, following a route from one party to another, gets significantly harder above about 50 nodes in a dense graph or 100 in a sparse one. Node-link drawings (dots joined by lines) are best for following a path, which is the journalist's main question; matrices, a grid with a row and a column per party, are better for dense groups. Edge bundling, which draws many lines as one bundle, hides which line is which, so it cannot be used where each link's status has to show.
Start from the most-flagged party, grouped by default
I decided the view should start from the most-flagged party, falling back to the most-linked. It groups by default: accounts roll up into the party that holds them, and a shell chain, a line of companies owning companies, collapses into one node. The mouse wheel zooms.
For the whole-case overview above about 300 parties, the view uses Sigma.js, a library for drawing large graphs, and keeps React Flow, a library for node diagrams, for the focus view. Still open, with suggested values: a cap of 50 visible parties and 10 per expansion.
The Wire shows one map at three altitudes
The same day produced a prototype called The Wire (the name was a proposal). Whole case: parties sit in soft patches found by community detection, which finds groups of parties more linked to each other than to the rest, and each patch is named “Around” its most-linked member.
Around a party: the view opens on the most-flagged one, Tanore Finance with 9 flags, with owners and directors to the west, what it owns to the east, money in to the north and money out to the south. A second ring of dots is chosen by degree of interest, flags times three plus links. One link: two parties at the ends of a straight wire, each record a bead on a date axis.
Money runs as current: lights travel along payment lines the way the money went, thickness is the amount on a log scale, and there are no arrowheads. Accounts show as beads on their holder's card. That exposed a round trip, Malaysian Official 1 to Tanore and back, labelled “ROUND TRIP · TEXTBOOK” where the round-trip rule fired.
Built into the app but not yet designed
On 12 October the Wire was built from the prototype into See, the part of the app for exploring a case. The changes include amounts under a million written in full (9,800 rather than “10 k”) and a time strip that spans the case's own dates. The build had not yet been tested on 1MDB, whose records could not be downloaded in the cloud environment I was building in.
The same day, the project's build list recorded that the two Wire screenshots I had sent with the mockups were work in progress, needed much more design work than the other screens, and were not targets. Both were to be redesigned into the Wire as a new mockup first.
Still open
The notes of that day do not agree on whether the prototype itself was approved: the backlog calls it approved, while the project's start-here note says the first of those two screens still waits on its mockup. Still unchecked in the build list: every link labelled with its sentence and a certainty symbol, and line styles for shown, not yet shown and disputed.
Sources and notes (10)
- OpenAleph: network diagrams — how OpenAleph builds diagrams by hand and expands party by party
- ICIJ: Offshore Leaks database how-to — search, then one entity, then neighbours one step at a time
- Linkurious: Pandora Papers investigation — how the ICIJ's Pandora Papers graph was explored
- Maltego: Collections — collection nodes gathering 25 or more entities
- Neo4j Bloom: scene interactions — starts empty with a search; expands on demand
- Kumu: Focus — grows the view one degree at a time
- van Ham and Perer, Degree-of-interest graphs (2009) — degree of interest to cap the view and pick what to open next
- Yoghourdjian et al. (2020), graph size and path-finding — path-finding gets harder above about 50 dense or 100 sparse nodes
- Network view research — the notes this was written from
- The Wire prototype notes — the notes this was written from
Journalism WorkbenchAll 12 parts →
