Declarative HTML5 Canvas
Bringing an SVG-like API to Canvas rendering through the magic of front-end JavaScript frameworks.
Recently, I got the itch to resurrect a project I had put on the backburner about a year ago: a cross-platform font editor called Glyphy . After running into some difficulties transforming to and from SVG coordinate space to render an interactive Bézier curve editor, I decided to give the Canvas API a try.
The resulting math was much easier to reason about, but the Canvas element uses an imperative API, and I missed the nice, declarative view descriptions you get with SVG markup. This is an overview of how I squared that circle to get the best of both worlds: using UI components to abstract Canvas draw calls.
Background
When I last left off with Glyphy, I had gotten as far as completing most of the tedious setup work — parsing OpenType’s binary format to XML via FontTools , deserializing the XML to an in-memory data structure with a custom setup (inspired by Rust’s Serde framework ), parsing the CharString code for each glyph to a bytecode stack, and bootstrapping the front-end using my design system library, Electric — but I hadn’t actually managed to render any glyphs yet. Taking a look at the long list of PostScript operators I would need to implement left me feeling pretty intimidated, and ultimately I got distracted by the next shiny object that entered my field of vision.
But recently I was inspired to pick it back up again. Doing some work in Unreal Engine to implement vector graphics rendering had given me the confidence to take another look at my PostScript interpreter, and selecting the typefaces to use for this blog reminded me what a huge nerd I am for typography — and how disappointing it is that there aren’t many viable options for hobby-level type design software on Windows.
As it turns out, the litany of PostScript operators essentially boils down to the same handful of commands that should be familiar to anyone who’s worked with 2D graphics libraries before:
MoveTo(x, y)
to set the initial coordinates for a new contour.LineTo(x, y)
to draw a straight line.BezierCurveTo(cpx1, cpy1, cpx2, cpy2, x, y)
to draw a Bézier curve — with thecp__
parameters representing the “handles” you would manipulate in an application like Adobe Illustrator to adjust the velocity of the curve at a given point.
The rest of the couple dozen or so path construction operators defined by the spec are just different expressions of those same commands, designed to slim down the resulting file size by avoiding repetition or making some operands implicit. After expanding them
to the familiar set of MoveTo
/ LineTo
/ BezierCurveTo
commands, we can trivially apply them to either an HTMLCanvasElement
’s 2D rendering context, or an SVGPathElement
by stringifying them to its d
attribute.
The “problem” with SVG
But there is one minor hiccup with either option, and an additional one when it comes to SVG in particular.
- OpenType glyphs are defined in the classical Cartesian coordinate space, with the
(0,0)
origin point at the bottom left (technically, at the font’s baseline in the Y axis), with Y values ascending as you move up, whereas the DOM places the origin at the top-left, with Y values ascending as you move down. - For SVG, you also need to wrangle the
viewBox
attribute , which defines the bounding box of the SVG’s viewport with respect to the coordinates specified on the SVG child elements. If you force the parent SVG element to render at a wider or narrower aspect ratio than that specified by theviewBox
, the browser seems to implicitly center the specified viewport within the actual viewport, adding an additional layer of complexity.
I am, at the core of my essence, either an artist with an underdeveloped right-brain or an engineer with an underdeveloped left-brain, depending on how you want to look at it. In a fight, my analytical side would probably win out, but only barely. This unique position enables me to fill an occupational niche that I really love, at the intersection of design and engineering. But when it comes to solving exceptionally challenging problems in either domain…
What I’m trying to say is that I kind of suck at math. I’ve managed to brute-force my way to a pretty decent working intuition for linear algebra (mostly by banging my head against 3D game development problems for the better part of the past five years), so I have no problem building a matrix to transform between glyph-space coordinates and browser-space coordinates and position everything exactly where I want it to appear on the screen. But trying to account for the SVG viewBox
situation on top of that was enough to collapse my mental model of the problem like a house of cards.
I managed to hack together this solution, which is workable for rendering a single glyph on its own:
But when it comes to building an interactive Bézier curve editor, with a rich, dynamic UI and informative data readouts like the exact glyph-space coordinates
of the mouse cursor at any given time, I need to have a firm grasp on the math at play to implement features in a robust way and at a reasonable pace. I wrote the code above less than a month ago, and I would have a seriously hard time explaining, for example, why that y
expression produces the desired result. It may as well be an arcane incantation that pleases the eldritch spirits in my GPU.
With the Canvas API, we don’t have to worry about that extra viewport layer. We have the canvas element, which uses the familiar DOM coordinate system (with one confounding variable, the devicePixelRatio
, which is trivial to deal with), and then we have our glyph-space coordinates, which we can project into the canvas using a relatively straightforward matrix transformation.
A declarative Canvas API
The one bummer about migrating from SVG to Canvas is the API. Canvas is imperative, in contrast to the declarative nature of XML markup. To “draw” a circle with SVG, you would write something like this:
To draw the same circle on a canvas, you would write something like this:
You could certainly come up with a more declarative JavaScript API for interacting with the canvas via functions, but I really like the XML family of markup languages for describing UI. The labelled closing tags make it possible to define deeply nested hierarchies of UI elements without losing track of your place in the tree, and attributes are named rather than positional like JS function arguments, so it’s always clear what the value assignments actually represent.
The goal we’re aiming for is something that ends up being very close to SVG for base geometric primitives. In Angular (my daily driver of choice), it looks like this:
The equivalent in React would look something like this:
Implementation
The key to making this work is hierarchical dependency injection. This is a seldom-explored feature in user-land Angular, though the Angular team uses it extensively in the Common and Forms modules and throughout Angular Material. If you’re coming from React, Vue, or SolidJS, you may know this pattern as the “Context API.”
Whatever the host framework, the idea is the same: An instance of some interface is provided at a particular node of the view tree, and then any children of that node who depend on that interface can get a reference to the provided instance by requesting it from the runtime.
We’ll use this pattern to make the canvas host aware of its children who want to render to it. When canvas children are added or removed from the view tree, or when existing canvas children undergo some change that requires them to re-render, they’ll notify the host, which will clear the canvas and invoke a callback for each child, giving it an opportunity to issue draw calls to the canvas.
That’s a pretty abstract explanation and probably more than a little unclear, so we’ll walk through a concrete implementation in Angular (very similar to the one Glyphy is using ), and then we’ll do the same for React. Finally, we’ll look at a clever technique to implement hierarchical dependency injection with plain-old web components, so you can get in on the fun even if you’re not operating in the context of a framework runtime.
Angular
In Angular, we’ll take advantage of the fact that dependency injection is actually a two-way street. That is, not only can a child inject dependencies provided by its parents, but a parent can “query” for dependencies in its children. This is a little odd, but it does make sense for our use case. Since the canvas host is the one driving the render loop, it needs to invoke methods on its child elements rather than the other way around.
Canvas Component
We’ll start by defining the interface through which the host will interact with its children, and an “injection token” which gives the host a generic identifier to query for in lieu of a concrete type.
We’ll start the canvas host by setting up the typical boilerplate for an HTML canvas.
If you’re not overly familiar with Angular: @ViewChild
issues a query for some dependency within our component’s template. In this case, we use a string that matches the #canvas
template variable to get a reference to the canvas element. (This is conceptually similar to React’s ref
attributte.)
static: true
indicates that the element isn’t conditional or dynamic in any way — it should always be present when this component is instantiated — which makes it available in the OnInit
hook, before our view has fully initialized.
<ng-content>
is Angular’s implementation of the <slot>
concept — basically analogous to React’s children
prop.
Next, we’ll add our DI query, and a #render
method to render each of our child RenderElement
s.
@ContentChildren
is similar to @ViewChild
, with two major differences:
- It operates on the nodes passed into our
<ng-content>
slot, rather than the nodes that we directly declared in our template - It maintains a dynamic list of results, rather than a single one
Here we’re querying for the RENDER_ELEMENT
token we declared earlier, which means that Angular will populate the QueryList
with providers of that token.
We want to call our #render
method under a few circumstances:
- When the
QueryList
changes (i.e., becauseRenderElement
s were added, removed, or changed their location in the view at runtime) - When one or more of the
RenderElement
s emits achanges
event, indicating that it needs to be re-rendered - When our canvas element’s size changes, we’ll want to resize its pixel buffer accordingly and redraw everything
In Angular, the primary mechanism for this type of reactive programming today is the third-party RxJS library . To be honest, RxJS is enormously complex, and its role in Angular looks to be soon largely superseded by Angular’s forthcoming Signals implementation , so I’m going to gloss over a lot of the details here.
All you really need to know are that both Angular’s QueryList
and our RenderElement
interface have a changes
property holding an RxJS observable, which is essentially an event emitter. We can subscribe
to those observables to be notified when they emit. We’ll use those subscriptions to call our #render
method, but we’ll throttle the event streams by window.requestAnimationFrame
first so that we only render at most once per frame.
@ContentChildren
queries are only available for us to read beginning with the AfterContentInit
lifecycle hook, so we’ll implement that and use it to set up our subscriptions. (Annoyingly, we’ll also need to monkey-patch Angular’s QueryList
type definition to address a longstanding issue that will trigger an “implicit any” error from TypeScript if we try to use it as-is.)
RxJS is really great at three things:
- Effortlessly managing complex combinations of event streams from multiple sources like we’re doing here
- Confusing the hell out of anyone without an encyclopedic knowledge of the operators it provides
- Making your application leak like a sieve
We’ll address the third point there by making sure none of our subscriptions can outlive our component. To do that, we would have historically used another observable as a notifier for RxJS’s takeUntil
operator, which we would emit once in an OnDestroy
hook. But Angular recently added its own utility operator, takeUntilDestroyed
, that does basically the same thing.
Now, the only thing remaining to finish our CanvasComponent
is to handle element resizing.
To test that everything is working as expected so far, we can render a blank magenta background in our #render
method, and add the component to our app.
If there is now a full-page wall of magenta searing your retinas, we’re good to move on to our first child element!
Circle Component
Let’s keep it simple to start, and implement the circle from our API mockup. Technically, this will be a Directive
instead of a Component
, because it won’t have any meaningful template or CSS styling. I will, however, give it an element selector — this is atypical for directives, which normally use attribute selectors. But we really don’t care about the host element at all — it’s just a vehicle for the API we’re trying to achieve.
We could hypothetically make these elements useful by using the ARIA API to give screen readers an idea of what our canvas represents — and that is something I actually plan to investigate for Glyphy — but it’s beyond the scope of this toy example.
A few things worth noting here:
-
The
providers
array in theDirective
decorator is how this class hooks into the DI framework so that it can be discovered by the Canvas’s@ContentChildren
query.The array element can take multiple forms , but this is typically the one you would use in cases like this, where you want to provide some abstract token that represents an interface implementation rather than a concrete
@Injectable
type. -
We declared
changes
as anObservable<void>
inRenderElement
, but we’re defining it as an AngularEventEmitter<void>
here. What gives?Angular’s
EventEmitter
class is sneakily an RxJS observable under the hood. Declaring it as anEventEmitter
with the@Output
decorator means that we could bind a listener to that event in our app template if we really wanted to. We don’t have a use for that, but it’s a free bonus that could theoretically be useful for debugging, so why not? -
If you’re wondering why we’ve been multiplying everything by
devicePixelRatio
, that’s because the DOM uses a device-independent “pixel” unit virtually everywhere, which doesn’t necessarily correspond to the physical pixels on the user’s display. But the canvas’swidth
andheight
properties configure the size of an actual pixel buffer that will be blitted to the canvas surface as a raster bitmap.If we didn’t scale our canvas coordinates to account for the difference between the DOM’s
px
unit and the display’s physical device pixels, we would end up with blurry rendering on displays with high DPIs, or even for users who use the browser-zoom feature to scale up the document.
With that done, we can add a circle to the canvas in our app template and take a look at the results:
That puts a static black circle on the screen at a fixed position relative to the top-left corner of the canvas. Which is… neat, I guess? Let’s make it a little more interesting — and prove that our update logic is working the way we expect — by making it interactive.
Adding some interactivity
First, let’s move the circle inputs to our controller, and add a rudimentary hit test to change the cursor style when we’re “hovering” over it.
You should now see the cursor turn into a little grabby hand when you mouse over the circle in the canvas. Now let’s try and move the circle around.
Now you should be able to grab onto the circle and drag it around the viewport! For one last flourish, let’s make the scroll wheel resize the circle.
React
For the React implementation, I’m going to assume you’re already pretty familiar with the framework (because, for better or worse, most front-end developers are).
The trickiest part here will be translating our setup to React’s Context API. As mentioned at the start of the Angular implementation, the parent component that hosts the canvas element needs to be the driver of the render loop — it should invoke some callback provided by its children whenever a re-render is called for. But we can’t query our children for a particular interface implementation like we did in Angular.
Canvas Context
I’m admittedly much less familiar with React than I am with Angular, so I would welcome any suggestions for how to do this more idiomatically, but the solution I arrived at was to provide a class instance as a context object to all children of the canvas element. When children who are interested in rendering to the canvas are added to the view tree, they’ll register their onDraw
callback with the context object. We’ll then invoke each of those callbacks whenever React renders our Canvas component. Here’s what that looks like:
There are a few interesting optimizations we need to do here that we didn’t really have to think about in the Angular implementation:
-
Virtually everything in React is transient by default, but it’s not trivially cheap to initialize a canvas element’s rendering context or to set its
width
andheight
properties, so we want to avoid needing to do that every time our component renders. To that end, we instantiate ourCanvasContextImpl
class withuseMemo
(dependent only on thecanvasRef
) to keep it reference-stable between React renders. -
If we can avoid it, we also don’t really want to thrash the garbage collector by reallocating the array of child callbacks every frame, but each of our children will invoke the context object’s
onDraw
method anew every time our canvas re-renders. To mitigate the performance impact, we’ve added a pseudo-private_reset
method for our host component to invoke, which will “reset” the array without actually resizing or disposing it. We keep a separate “pointer” into the array just past its last populated index, and as long as the array still has some unused capacity remaining, we insert new callbacks at that index instead ofpush
ing them.To be honest, this is the type of micro-optimization that’s likely out of place in a JavaScript app — the browser’s JS runtime may already be doing that sort of optimization for us, and we may even be preventing it from doing more effective optimizations by making the array heterogeneous to accomodate the
null
entries. Profiling would be a good idea here, but since this is a toy example for a blog post and not actual production code, I haven’t gone to the trouble. If you decide to look into it, do let me know what you find!
We’ll round out the Canvas implementation by handling resizes, much like we did in Angular:
Porting from Angular
The remainder of the React implementation is a pretty straightforward port of the Angular code, so I won’t spill too much ink over it. Maybe the one notable quirk is that our child Circle needs to return an empty React.Fragment
to avoid TypeScript complaining that it doesn’t qualify as a component (though React itself didn’t seem to mind the omission).
A pesky issue with React.StrictMode
One mildly infuriating problem, which I don’t have a good answer to at the moment, is that React’s StrictMode
forces every component to render twice in dev builds as a debugging aide. Angular does something similar by running an extra change detection cycle in development, but the fact that we’re relying on React’s component renders to collect draw calls from our children means that we’ll end up with two onDraw
callbacks for every single RenderElement
added to our canvas. Memoization and useCallback
are no help either — onDraw
gets invoked with two equivalent but referentially-unique callbacks no matter what we try to do about it in user-land.
This sounds like just a performance issue at first, but canvas draw calls are alpha-blended, so it actually leads to a visibly different result in development vs production, which is really not ideal — you don’t want to merge and deploy a PR only to find that it looks significantly different than you expected out in the real world.
I ultimately resorted to removing the StrictMode
wrapper in my test project, but if you were going to try something like this in real-world production code, you would definitely want to come up with a more robust solution. Adding an if
branch to skip every other callback when not in production is hacky and inelegant, but would likely do the trick.
Bonus: Web Components
One of the biggest hassles of working on the web today is how fragmented the entire ecosystem is. Multiple bundlers and build systems, multiple TypeScript compilers, multiple mechanisms for module import/export/resolution, and of course a zillion or so mutually incompatible front-end frameworks. I’ve walked through solutions for the two most popular frameworks as of writing, but what if you’re working in Vue, or SolidJS, or even just vanilla JS/HTML/CSS?
The web components spec is widely implemented across modern browsers, which gives us a way to build UI features that are framework-agnostic. The only hiccup is that the crux of our solution is designed around hierarchical dependency injection. Dependency injection as part of the web components spec doesn’t even really make sense to talk about: the whole point of dependency injection is to enable dependency inversion — depending on abstract interfaces rather than concrete types — which is a concept that’s inseparable from static type systems.
Fortunately, the web components API gives us all of the tools we need to bootstrap a really straightforward DI implementation! I wish I could take credit for this idea: I yoinked it from Justin Fagnani , a Google engineer working on Lit . I recommend checking out the talk that inspired this solution on YouTube.
Dependency injection without a framework
The seed of the idea is that custom elements (web components) can dispatch custom events. CustomEvent
is a generic derivative of Event
, with a detail
property that we can populate with arbitrary data. An implementation of the classic bottom-up DI model (which Justin covers in his talk) goes something like this:
- A custom element with an abstract dependency emits a custom event, populated with some identifier representing the interface it depends on.
- A custom element which provides a dependency is listening for that event type. If it provides a match for the requested identifier, it mutates the custom event data to populate it with the provided instance, and then stops the event from propagating any further up the tree.
- Since DOM events bubble synchronously, the dependent can retrieve the provided instance from the event immediately after dispatching it.
From there, it’s a short walk to get to the top-down “query” model we used in our Angular implementation:
- In addition to listening for events from below, the dependency provider emits its own event, populated with both the interface ID and the concrete instance.
- To query for dependencies provided by descendants, the ancestor element listens for the provider event and retrieves any instances they’re interested in.
I’ll use Lit for the WC implementation of our drag-the-circle demo, but you could follow the steps above with (or without) any web components framework, as long as you have some mechanism for managing (and preferably batching) reactive updates. You’ll notice some striking similarities between this and the Angular implementation, which is no surprise since Angular, Lit, and the web components spec itself all have Googley genes in their DNA.
Porting from Angular
First, we’ll knock out the stuff that should be pretty familiar at this point:
Implementing dependency injection
Now we can add in the DI magic. First some helper types:
Then we’ll create our custom event types. We’re only actually using the top-down querying method in our toy example, so InjectionRequest
will go unused, but I figured it wouldn’t hurt to round out the example.
We also need a way to inform the Canvas host when a RenderElement
’s properties change at runtime, so we’ll add another custom event type for that.
We’ll create a UniqueToken
as a runtime representation of our RenderElement
interface, just like we did with Angular’s InjectionToken
.
Then we’ll dispatch the appropriate events in our Circle element’s lifecycle callbacks.
And finally, we’ll handle those events in our Canvas host.
You may notice that we’re not doing any throttling or debouncing here, unlike the Angular implementation. In Lit, updates are automatically batched via microtask scheduling so that re-renders occur at most once per frame. Attaching the @state
decorator to our _elements
array and updating it immutably has the same effect as manually calling requestUpdate
: an update is queued, and any subsequent update requests that occur in the meantime are coalesced into the same update cycle.
We’ve already tied our #canvasRender
calls to LitElement
’s render
method, so we’ll redraw all canvas elements exactly once for every Lit update cycle. To be honest, the logic here is so elegant and easy to follow that this might be my favorite implementation of the bunch, despite the extra boilerplate.
Abstracting our DI solution with decorators
If we wanted to abstract all the event handling and dispatching to get a general-purpose DI solution for Lit, we could whip up a few decorators to do the job. The decorator implementations are honestly pretty ugly, but the whole purpose of metaprogramming constructs like decorators is to front-load the cognitive overhead of performing these kinds of repetitive tasks. As a reward for the effort, we end up with a nice, declarative way to mix in functionality without needing to remember each step of the ritual every time. Here’s what our elements look like using decorators to replace the manual event dispatching/handling:
I won’t copy the decorator implementations here, because it’s a lot of code (and well out of scope for this post), but you can explore them in all their inscrutable glory in the Stackblitz demo here .
Evaluating the result
As mentioned at the top of this entry, my motivation for designing a declarative Canvas API was to use it for my font editing application, Glyphy. I’ve now used the framework described here to build out most of the core functionality of Glyphy’s main glyph editor UI, so we’ll close out this entry by taking a closer look at a real-world, nontrivial use case.
Glyphy’s main editor UI has a lot of moving parts — literally and figuratively. Here’s what it looks like in action currently:
The vast majority of the UI there — everything below and to the right of the tabs along the top and left edges — is rendered on an HTML canvas using the abstractions described here.
A (mostly) unified API for UI elements
It really helps with productivity when I don’t need to context-switch between my go-to Angular component patterns and the imperative patterns that tend to arise out of more traditional canvas abstractions. If it has a visual representation, it’s an Angular component, and you can mostly expect it to behave like one.
That said, the “mostly” caveat deserves some explanation:
- Unlike SVG (or HTML), Glyphy’s canvas components don’t interact with CSS at all. This isn’t a huge problem — my design system declares all of its tokens in a POJO configuration file and reflects them to CSS variables when the application is bootstrapped, so theme colors can be queried just as easily (and efficiently) from TypeScript as they can from SCSS. Still, I mostly like to keep my templates and my stylesheets separate, so this makes for a slightly awkward discrepancy.
- A bigger problem is event handling. Much like our toy drag-the-circle demo, Glyphy is doing things like hit-testing for interactive controls manually in the component controllers. This isn’t a huge departure from my typical workflow — I often use RxJS instead of template bindings to handle events, because my library components usually need to support non-trivial interaction patterns for accessibility. But those components don’t burden their consumers with that complexity like the canvas components do currently. A nice usability upgrade might be to abstract some of the hit-testing/clicking/dragging logic into custom events that emit from the individual canvas components.
The reason I haven’t yet taken a serious look at remedying these limitations is that I want to avoid falling into the trap of trying to reinvent the DOM from scratch. The uncanny valley effect of “it’s exactly like the DOM API you already know, except for all the subtle ways it violates your expectations” is (frankly) my biggest pet peeve about working with React, and that’s not an experience I’m eager to replicate.
Simplified math without a viewBox
Without having to worry about the SVG’s confusing viewBox
attribute, we can transform glyph coordinates into DOM coordinates and back using a set of relatively straightforward matrix operations.
The GlyphEditorComponent
defines a few matrix observables:
glyphToCanvas$
takes the raw glyph coordinates and transforms them into the DOM/client’s coordinate space (which is just canvas-space divided bydevicePixelRatio
)canvasToGlyph$
takes DOM/client coordinates and transforms them back into the glyph’s coordinate spacepanAndZoom$
captures the translation and scaling produced by the user panning and zooming in the glyph editor
panAndZoom$
is actually an RxJS BehaviorSubject
, which we manually update by listening for user input:
glyphToCanvas$
starts with a nice default framing for the glyph, and then incorporates the pan and zoom:
And canvasToGlyph$
, unsurprisingly, just takes the inverse of glyphToCanvas$
:
The Matrix
class is a straightforward 3x3 matrix implementation representing affine 2D transformations, which you can peruse at your leisure here
.
The canvas-rendering components all take a transform
matrix input, which can be passed to the CanvasRenderingContext2D
after converting to the expected format:
And the GlyphEditorComponent
provides its glyphToCanvas$
observable as that
transform:
The inverse matrix, canvasToGlyph
, is mainly used for hit-testing. To avoid transforming every glyph point into client-space every frame to test each point’s distance to a PointerEvent
’s clientX
and clientY
, we transform the pointer coords into glyph space, sort the glyph points by square-distance to those glyph-space pointer coords, and then transform only the nearest point into client-space to check if it’s within a target radius measured in DOM px
units.
This all sounds like a lot — and truthfully, it is. But the majority of this math needed to be done in the SVG implementation as well, so just imagine trying to reliably and consistently account for the SVG viewBox
on top of it all, and you can begin to understand why I was desperate for a simpler reference frame.