Eunoia: Euler and Venn Diagrams in Five Languages

Ten years ago I wrote eulerr as part of my Bachelor’s thesis (Larsson 2018) in statistics—an R package for area-proportional Euler diagrams. At the time it was the first package to feature ellipses beyond three sets, EulerAPE (Micallef and Rodgers 2014) being the first package to introduce ellipses. eulerr has been surprisingly popular, having been used in (at least) 600 academic papers. It was written in a mixture of R and C++, with the latter doing the heavy work of computing the layouts, whereas the R code handled the interface, plotting, and organization of the optimization pipeline (which includes both an initial layout and a final fit).

This architecture proved good enough to beat most of the competitors in terms of both performance and accuracy¹, but a few things have kept gnawing at me over the years. Specifically:

¹ See Larsson and Gustafsson (2018) for comparisons and benchmarks.

The mix of C++ and R code meant that it was hard to port the code to other languages, particularly because eulerr relied on R’s built-in optimization routines (nlm() in particular), which meant that it was hard to make a faithful port.
The web app that built on top of eulerr was based on Shiny. And although Shiny is a convenient and powerful framework, it meant that I had to maintain a dedicated server to run the app, which had to be manually accessed and updated in order to keep it aligned with the latest version of eulerr.

I always knew that the way out of this would be to rewrite the entire fitting engine in a single language, so that I could maintain thin wrappers in the high-level languages (R, Python, Julia, etc.) around a single core. And that that core should be in a language that compiles to WebAssembly, so that I could build a static web app around it without the need of a server. Up til recently I always imagined that this rewrite would be a rewrite in C++. But in the recent months I have been learning Rust, and have grown to love both the language and, perhaps in particular, its tooling.

The core engine now lives in a standalone Rust library called Eunoia and eulerr is just one of several thin bindings on top of it. The same core also drives packages for Python, Julia, and JavaScript ², plus a web app you can use without installing anything. This is the project that pulled Basin, my optimization library, into existence, and it is finally ready for a proper introduction.

² Developed in the main repository for Eunoia.

What It Does

You hand Eunoia a set of quantities—the sizes of some sets and the sizes of their intersections—and it lays out circles, ellipses, squares, or rectangles whose overlapping areas match those quantities as closely as possible. When the numbers admit an exact diagram, you get one. When they do not, which is the common case for three or more sets, you get the best approximation the optimizer can find, together with residuals and goodness-of-fit statistics so you can tell whether the picture is trustworthy.

Venn diagrams are also included. A Venn diagram is just an Euler diagram in which every intersection has to be drawn whether or not it is empty, and Eunoia knows the canonical arrangements—circles for up to three sets, ellipses for four and five.

One Core, Many Languages

The whole point of the rewrite is that there is now a single implementation to maintain, and everything else is a binding. Table 1 shows the family as it stands today.

Language	Package	Install
Rust	`eunoia`	`cargo add eunoia`
R	`eulerr`	`install.packages("eulerr")`
Python	`eunoia`	`pip install eunoia`
Julia	`Eunoia.jl`	from GitHub (see below)
JavaScript	`@jolars/eunoia`	`npm install @jolars/eunoia`
Web app	eunoia.bz	build diagrams in the browser

Table 1: The family of Eunoia packages, all powered by the same Rust core.

Because every package calls the same Rust code, they fit identical diagrams from the same specifications. The only differences are in the plotting. To show this by example, here is the same three-set example—a tiny movie library tagged by genre—fit twice, once in R and once in Python. First, the R version:

library(eulerr)

genres <- euler(
  c(
    "Adventure" = 20,
    "Comedy" = 14,
    "Drama" = 18,
    "Adventure&Comedy" = 6,
    "Adventure&Drama" = 5,
    "Comedy&Drama" = 4,
    "Adventure&Comedy&Drama" = 2
  ),
  shape = "ellipse"
)

plot(genres)

An area-proportional Euler diagram of three overlapping ellipses labelled Adventure, Comedy, and Drama, fitted by the R package eulerr. — Figure 1: eulerr (R)

Next, the Python version:

import eunoia as eu
import matplotlib.pyplot as plt

genres = eu.euler(
    {
        "Adventure": 20,
        "Comedy": 14,
        "Drama": 18,
        "Adventure&Comedy": 6,
        "Adventure&Drama": 5,
        "Comedy&Drama": 4,
        "Adventure&Comedy&Drama": 2,
    },
    shape="ellipse",
)

ax = genres.plot()
ax.figure.subplots_adjust(left=0, right=1, top=1, bottom=0)

plt.show()

An area-proportional Euler diagram of the same three sets, fitted by the Python package eunoia, visually identical to the R version. — Figure 2: eunoia (Python)

The two pictures are essentially the same picture because the fit is the same fit. The plotting is where the bindings diverge—eulerr draws through R’s grid graphics, the Python package through Matplotlib—but the layout is the same. And not only that, but the actual geometry of the fitted shapes, including how their labels are laid out, comes from the Rust library. The downstream packages (R, Python, Julia, JavaScript) just call the core to get the layout and drive a fixed-point plotting routine to settle the shapes and labels into their final positions.

That core takes the same specification everywhere. The keys name sets and their intersections with an ampersand, the values are the sizes of the disjoint regions, and a shape argument picks the family. Here is that single spec written in each language.

use eunoia::geometry::shapes::Ellipse;
use eunoia::{DiagramSpecBuilder, Fitter, InputType};

let spec = DiagramSpecBuilder::new()
    .set("Adventure", 20.0)
    .set("Comedy", 14.0)
    .set("Drama", 18.0)
    .intersection(&["Adventure", "Comedy"], 6.0)
    .intersection(&["Adventure", "Drama"], 5.0)
    .intersection(&["Comedy", "Drama"], 4.0)
    .intersection(&["Adventure", "Comedy", "Drama"], 2.0)
    .input_type(InputType::Exclusive)
    .build()
    .unwrap();

let layout = Fitter::<Ellipse>::new(&spec).seed(1).fit().unwrap();

library(eulerr)

euler(
  c(
    "Adventure" = 20,
    "Comedy" = 14,
    "Drama" = 18,
    "Adventure&Comedy" = 6,
    "Adventure&Drama" = 5,
    "Comedy&Drama" = 4,
    "Adventure&Comedy&Drama" = 2
  ),
  shape = "ellipse"
)

import eunoia as eu

eu.euler(
    {
        "Adventure": 20,
        "Comedy": 14,
        "Drama": 18,
        "Adventure&Comedy": 6,
        "Adventure&Drama": 5,
        "Comedy&Drama": 4,
        "Adventure&Comedy&Drama": 2,
    },
    shape="ellipse",
)

using Eunoia

euler(
    Dict(
        "Adventure" => 20, "Comedy" => 14, "Drama" => 18,
        "Adventure&Comedy" => 6, "Adventure&Drama" => 5, "Comedy&Drama" => 4,
        "Adventure&Comedy&Drama" => 2,
    );
    shape = "ellipse",
)

import { euler } from "@jolars/eunoia";

euler({
  sets: {
    Adventure: 20, Comedy: 14, Drama: 18,
    "Adventure&Comedy": 6, "Adventure&Drama": 5, "Comedy&Drama": 4,
    "Adventure&Comedy&Drama": 2,
  },
  shape: "ellipse",
  inputType: "exclusive",
});

As you can see, the specifications are nearly identical across languages. The only differences are in the syntax of the language itself, and in the fact that the Rust version is more verbose because it is a lower-level language than the others.

Next, I will show some of the improvements that the rewrite in Rust brings to the table.

What’s New

The rewrite in Rust comes with a number of improvements over the original C++ implementation. The major ones are:

A new algorithm for fitting diagrams that is both more robust and accurate.
An overall faster and more efficient implementation, thanks to exact analytical gradients and sparse intersection-handling.
Additional shapes: squares and rectangles (including rotated ones).
Collision-aware labeling that avoids overlaps with boundaries and other labels.

In the following section, I will discuss each of these items.

The New Algorithm

The fundamental algorithm in Eunoia is unchanged from that in eulerr:

We start with an initial layout that uses multi-dimensional scaling (MDS) to lay out the shapes to minimize the differences between the distances between shapes in the current fit compared to the ones that would be required from an exact fit. This idea comes from Wilkinson (2012), and his seminal venneuler package for R. It was later improved by Frederickson (2015), who introduced the idea of using a relaxation of the loss function for disjoint and nested sets.
This initial layout is fed to a second optimization step that computes a loss over the differences between the areas of intersections of the diagram with those from the set specification. The optimizer then iteratively adjusts the shapes to minimize this loss.

Because the intial layout sometimes gets stuck in local minima, the original eulerr implementation would run this initial optimization step for a number of random starting layouts, and then pick the best one to feed to the final optimization step. Specifically for the case of ellipses and three sets, the algorithm would also (optionally) resort to a fallback optimization step using GenSA (Xiang et al. 2013) if the final loss was above a certain threshold. The diagram in Figure 3 shows this entire (old) pipeline from eulerr.

flowchart TD
  A1["n random starting layouts<br/>(runif, run sequentially)"]
  A2["Initial optimization<br/>best of 10 nlm() restarts on optim_init"]
  A3["Final optimization<br/>nlm() on the chosen loss"]
  A4{"Ellipses and three sets,<br/>and diagError > threshold?"}
  A5["Fallback: GenSA()<br/>simulated annealing"]
  A6["Solution"]
  A1 --> A2 --> A3 --> A4
  A4 -->|Yes| A5 --> A6
  A4 -->|No| A6

Figure 3: The original fitting pipeline in eulerr.

This worked well enough to beat the competition (Larsson and Gustafsson 2018), but when I rewrote the code for Eunoia, I wanted to introduce a few improvements.

Instead of only running the initial optimization step for several random starting layouts and then picking the best one, we now run the entire optimization pipeline for these initial layouts. And because this is embarrassingly parallel, we also run these attempts in parallel. The reasoning is that there may actually be multiple initial layouts that have more or less the same initial loss but that lead to very different final losses.

In addition, this also opens up for forwarding curated starting layouts to the optimizer, and currently we always feed a Venn diagram layout as one of the layouts carried forward from the initial optimization step. For set combinations where all intersections are present, this is usually the best starting layout and will quickly lead to an exact fit if the numbers admit one.
The default optimizer is now Levendberg–Marquardt (LM; changed from R’s nlm()). It took some effort to finally find an algorithm that worked better than nlm(), but the LM algorithm is designed precisely to handle the default loss function—a sum of squared differences—and it shows, being both faster and more accurate than nlm()’s more general trust region method.
The fallback optimization step is now enabled by default—for all shapes and any size of diagram. It uses a memetic variant of the CMA-ES algorithm (Hansen and Ostermeier 1996), using LM as the meme (local search) to polish the best solution found by CMA-ES.
For non-squared, but continuous, loss functions, Eunoia uses L-BFGS-B as the default optimizer, which (like LM) is a quasi-Newton method, but doesn’t rely on the loss being squared residuals. For non-continuous loss functions, Eunoia uses the Nelder–Mead simplex method (in fact an improved version of it). eulerr used nlm() for these losses too, computing gradients numerically, but for losses like the maximum absolute error, the gradient is essentially flat over large regions of the parameter space, which makes nlm() struggle. There is also the options to use MADS (Mesh Adaptive Direct Search) for non-continuous losses, which is typically more robust than Nelder–Mead, but slower. The default is still Nelder–Mead for that reason, but this might possibly change in the future.

You can see the new pipeline in Figure 4.

flowchart TD
  B1["n restarts (default 10)<br/>run in parallel"]
  B2["Initial layout: MDS via<br/>Levenberg-Marquardt<br/>(slot 0: Venn warm-start)"]
  B3["Final optimization<br/>L-BFGS / LM"]
  B4{"loss > threshold?"}
  B5["Fallback: CMA-ES global<br/>escape, then LM polish"]
  B6["keep lowest-loss attempt"]
  B1 --> B2 --> B3 --> B4
  B4 -->|Yes| B5 --> B6
  B4 -->|No| B6

Figure 4: The new fitting pipeline in Eunoia.

The new algorithm is both more robust and more accurate than the old one.

Performance and Efficiency

Another upgrade that comes with Eunoia is that the optimizer now uses exact analytical gradients for the smooth loss functions, which provides a significant speedup over the numerical gradients used in eulerr. In addition, and unlike eulerr, Eunoia also keeps track of only the intersection areas that are actually present in the diagram. This didn’t use to be the case with eulerr, which generated a 2^n-1 sized list of all combinations for n sets, and meant that it choked on large set combinations. Eunoia (and the new eulerr) only keeps track of the intersections that are actually present, which makes it possible to fit diagrams with many sets, as long as the number of non-empty intersections is not too large.

Additional Shapes

eulerr only supported two shapes: circles and ellipses. Euonia adds squares and recangles (optionally rotated) to the mix. Here you can see the shapes in action, and also the recent plot composition features of eulerr that—in patchwork style—allow you to combine multiple diagrams into a single figure.

library(eulerr)

combo <- c(
  "Adventure" = 20,
  "Comedy" = 14,
  "Drama" = 18,
  "Adventure&Comedy" = 6,
  "Adventure&Drama" = 5,
  "Comedy&Drama" = 4,
  "Adventure&Comedy&Drama" = 2
)

fit_ellipse <- euler(combo, shape = "circle")
fit_ellipse <- euler(combo, shape = "ellipse")

Euler Diagrams in Your Browser

The JavaScript package ships the whole fitting engine compiled to WebAssembly. This makes it possibly to run Eunoia directly in your browser, without the use of a server to host a Shiny instance (for instance). The diagram below is fit live by the npm package, using Observable, which is natively supported in Quarto (which this site is built with). Drag the sliders to change the region sizes or pick a different shape family, and the layout re-fits as you go.

Observable cells to load the Eunoia WebAssembly module and set up the inputs.

eunoia = {
  // `/web` is the wasm fitting engine; `/svg` is the wasm-free serializer that
  // turns a fitted layout into an SVG string. Both ship in the same package.
  const [web, svg] = await Promise.all([
    import("https://cdn.jsdelivr.net/npm/@jolars/[email protected]/web.js"),
    import("https://cdn.jsdelivr.net/npm/@jolars/[email protected]/svg.js")
  ]);
  await web.init(); // instantiate the embedded WebAssembly module, once
  return { euler: web.euler, toSvg: svg.toSvg };
}

viewof shape = Inputs.select(["ellipse", "circle", "square", "rectangle"], {
  value: "ellipse",
  label: "Shape"
})

viewof adventure = Inputs.range([0, 30], { step: 1, value: 20, label: "Adventure" })
viewof comedy = Inputs.range([0, 30], { step: 1, value: 14, label: "Comedy" })
viewof drama = Inputs.range([0, 30], { step: 1, value: 18, label: "Drama" })
viewof ac = Inputs.range([0, 15], { step: 1, value: 6, label: "Adventure & Comedy" })
viewof ad = Inputs.range([0, 15], { step: 1, value: 5, label: "Adventure & Drama" })
viewof cd = Inputs.range([0, 15], { step: 1, value: 4, label: "Comedy & Drama" })
viewof acd = Inputs.range([0, 10], { step: 1, value: 2, label: "All three" })

layout = eunoia.euler({
  sets: {
    Adventure: adventure,
    Comedy: comedy,
    Drama: drama,
    "Adventure&Comedy": ac,
    "Adventure&Drama": ad,
    "Comedy&Drama": cd,
    "Adventure&Comedy&Drama": acd
  },
  shape: shape,
  output: "regions",
  inputType: "exclusive",
  seed: 1
})

Code

diagram = {
  const div = document.createElement("div");
  div.style.maxWidth = "480px";
  div.innerHTML = eunoia.toSvg(layout, {
    fontFamily: "system-ui, sans-serif",
    setOrder: ["Adventure", "Comedy", "Drama"]
  });
  const node = div.querySelector("svg");
  node.setAttribute("width", "100%");
  node.removeAttribute("height");
  node.style.height = "auto";
  return div;
}

Figure 5: An Euler diagram fit in your browser by the Eunoia WebAssembly build.

Code

md`**Loss:** ${layout.metrics.loss.toFixed(5)} &nbsp;·&nbsp;
   **diagError:** ${layout.metrics.diagError.toFixed(5)} — both closer to zero is better.`

If you push the sliders to a configuration that no arrangement of shapes can satisfy, you will see the loss incrase: that number is the optimizer telling you how far short the best diagram falls. It is the same diagnostic the other packages return, and a good habit to check before trusting any Euler diagram.

A Note on Each Binding

Rust

The core, published as eunoia. It exposes the fitter, the shape families, Venn arrangements, and a plotting module that turns a fitted layout into region polygons and label anchors ready to draw as SVG. The full API is on docs.rs.

R

eulerr is the original package, now at version 8.1.0 and the package this whole effort grew out of. The interface that long-time users know is unchanged; what changed is that the C++ backend is gone, replaced by the Rust core through a small C ABI. If you install from source you will need a Rust toolchain, but CRAN’s binary builds do not.

One thing that has changed is that Eunoia includes functionality that we previously relied on external packages for. Because the optimization routines now all use the same Rust core, we have been able to drop the dependency on GenSA (Xiang et al. 2013). We also now handle poly-clipping through the iOverlay library, which means we no longer need the polyclip package, nor do we need my own polylabelr package since we now host our own algorithm for finding the visual center of a polygon. This actually means that eulerr has no dependencies at all except for base packages and, if compiling from source, the Rust toolchain.

Python

The Python package for eunoia is eunoia on PyPI, a pip install eunoia away. It mirrors the eulerr API as closely as Python idiom allows, plots through Matplotlib, and ships prebuilt wheels so there is no compiler in your way.

Julia

The Julia package is Eunoia.jl. It is the newest member of the family. Install it from the general registry with

using Pkg
Pkg.add("Eunoia")

Plotting uses a Makie extension that loads as soon as you import a backend, so the core package stays free of plotting dependencies until you actually want a figure.

Writing the Julia bindings required a bit of extra work because there is no equivalent to PyO3/maturin (Python) or rextendr/extendr (R), which meant we had to create our own C ABI wrapped around the Rust core. But that also means that there is now a full-featured C ABI for Eunoia, so if you are interested in writing your own bindings for, say, C#, Go, or any other language that can call a C library, you can do so quite easily.

JavaScript

Is @jolars/eunoia. The default entry is built for bundlers, but there is also a self-contained @jolars/eunoia/web entry—a single ES module with the WebAssembly inlined—that loads from a CDN with no build step, which is exactly what powers the demo above. A separate @jolars/eunoia/svg entry serializes a layout to SVG in pure JavaScript. And if you would rather not write any code at all, the same engine backs the web app at eunoia.bz/app.

Try It

Eunoia and its bindings are open source and dual-licensed under MIT and Apache-2.0. The Rust core, the JavaScript package, and the Julia binding live in the eunoia repository; eulerr and the Python package have their own. Narrative documentation for the whole family is at eunoia.bz/docs.

Whichever language you reach for, I would love to hear how it goes. File an issue or start a discussion on whichever repository fits, and contributions of any kind are very welcome.

References

Frederickson, Ben. 2015. “A Better Algorithm for Area Proportional Venn and Euler Diagrams.” Ben Frederickson, June 29. https://www.benfrederickson.com/better-venn-diagrams/.

Hansen, N., and A. Ostermeier. 1996. “Adapting Arbitrary Normal Mutation Distributions in Evolution Strategies: The Covariance Matrix Adaptation.” Proceedings of IEEE International Conference on Evolutionary Computation, May, 312–17. https://doi.org/10.1109/ICEC.1996.542381.

Larsson, Johan. 2018. “Eulerr: Area-Proportional Euler Diagrams with Ellipses.” Bachelor thesis, Lund University. https://lup.lub.lu.se/student-papers/record/8934042.

Larsson, Johan, and Peter Gustafsson. 2018. “A Case Study in Fitting Area-Proportional Euler Diagrams with Ellipses Using Eulerr.” Proceedings of International Workshop on Set Visualization and Reasoning (Edinburgh, United Kingdom) 2116 (June): 84–91. https://ceur-ws.org/Vol-2116/paper7.pdf.

Micallef, Luana, and Peter Rodgers. 2014. “eulerAPE: Drawing Area-Proportional 3-Venn Diagrams Using Ellipses.” PLOS One 9 (7): e101717. https://doi.org/10.1371/journal.pone.0101717.

Wilkinson, L. 2012. “Exact and Approximate Area-Proportional Circular Venn and Euler Diagrams.” IEEE Transactions on Visualization and Computer Graphics 18 (2): 321–31. https://doi.org/10.1109/TVCG.2011.56.

Xiang, Yang, Sylvain Gubian, Brian Suomela, and Julia Hoeng. 2013. “Generalized Simulated Annealing for Global Optimization: The GenSA Package.” The R Journal 5 (1): 13–28. https://doi.org/10.32614/RJ-2013-002.