Want to perform or record remotely with other musicians, in real time? Bassist John J. Williamson offers a guide to the world of online real-time music (ORM) software. This is an edited version of an article to appear on John’s website.

Introduction

Since the COVID-19 lockdown, there has been a flush of interest in applications allowing musicians to rehearse, record or livestream together, online, in real time. Engineer Doug Hunt’s livestreamed recording of the band Mama’s Gun shows what is becoming possible through ORM software.

Inspired by Dan Tepfer’s livestreamed duets, I recently produced a DIY tutorial on ORM with pianist Sam Leak and have since been researching this field further, and talking to some of the key players.

In this guide, I won’t advocate a particular app, or give a tutorial. Most software I’ll mention has documentation and YouTube tutorials, while my tutorial details common technical points. My aim is simply to help musicians become ‘informed consumers’ of this software – most of which is free.

What is online real-time music?

Next time you’re on a Zoom call, count a bar, ‘1, 2, 3, 4’ in time, and have the other person say the ‘1’ of the next bar. It won’t even be close. Conferencing apps have too much latency (or delay) for musicians. Why?

The internet is not a free-flowing superhighway. The data packets our computers send and receive battle for priority – your Zoom call against someone else’s Netflix stream. Maybe your data packet gets lost and has to be sent again, due to WiFi interference from your microwave. Apps present a smooth experience by quietly compensating for packet loss, at the cost of latency.

ORM software is different. It just blindly fires data packets, not checking whether, or in what order, they arrive. In the right conditions, this method gives low enough latency for real-time music.

For years, real-time music has been something of a niche interest. There was ‘no business case, hardly any demand, and internet speed was not good enough,’ Christian Hoene of Soundjack told me. But now, a new generation of professionals is interested, and broadband speeds make ORM viable. Doug Hunt sums it up: ‘We feel like the infrastructure has matured to the point where we can think of this as a real thing.’

The basics

ORM software is more the same than different, and you’ll meet two types of problem: no sound, or glitchy sound. No sound means you set something up wrong. Glitchy sound is the internet’s fault, though it can be mitigated. A few key points:

• Use a wired (ethernet) connection from computer to your internet router. WiFi won’t work.
• Google ‘speed test’ and look for 10Mbps ‘upload bandwidth’ and 20Mbps ‘download’, ‘ping’ in single figures, and low ‘jitter’. Not all that bandwidth gets used, but lower values can indicate congestion. If you’re not getting what you pay for, phone your internet provider!
• Be within a few hundred miles of the person(s) you’re trying to play with, for close-to-real-time latency.
• Use a USB audio interface with low latency to connect your instrument to your computer.
• Read the documentation of the ORM app you use, and look for settings like ‘buffer size’ and ‘jitter buffer’, which mitigate packet loss (reducing glitches) at the cost of increased latency. This crucial balance depends on your needs, and your internet.

ORM software is not completely ‘plug and play’, and is subject to effects outside its control, such as internet speed. Some developers wisely address this by making their apps as automatic as possible and having them notify the user of external issues.

Most ORM software doesn’t do video. The easiest thing is to use Zoom or similar for video, and perhaps voice communications too.

Beyond duets: Client-server or peer-to-peer

Consider a ‘peer-to-peer’ trio, where everyone is connected to everyone else – three points of a triangle. Your internet needs twice as much bandwidth as for a duet, because each person is sending and receiving two streams. It works, but gets unwieldy.

In a ‘client-server’ model, you each connect to a shared server. This simplifies adding more musicians. But public servers can be busy, or too far away for acceptable latency.

Most ORM software can also run in ‘server mode’, where one of your party becomes a private server for everyone else. They will need to set up ‘port-forwarding’ on their router for whatever port(s) the particular ORM software uses, or contact their internet provider for help.

In a duet, there’s no point using an external server unless you have to (for example because you can’t set up port-forwarding).

Getting technical

Pick an app, read the instructions, watch tutorials, and get going! The rest of this article is for those who have a basic understanding of home recording and wish to make serious use of ORM for rehearsing, livestreaming, or recording.

Latency

With ORM, you’re often near the edge of what’s possible. On a bad internet day, you might experience glitching, and need to change settings to give higher latency with fewer glitches.

Some latency is natural: two people nine metres apart experience around 25 milliseconds, easily achievable using ORM. Some ORM software tells you your latency in-app, so keep an eye on it. Two people far apart in a room can rely on fixed latency, but a poor internet connection might give variable latency that makes it harder to ‘lock in’ rhythmically.

The style of music matters. Jazz bassist Ferg Ireland experienced just-noticeable latency in his first forays into ORM: ‘Playing ballads was fine, but above a certain tempo it didn’t really work’.

In my experience, with perfect conditions, ORM can be fast enough for anything you want to play. In imperfect conditions, you can compromise to suit the music.

Livestreaming

This handy video outlines one approach to livestreaming a duet.

One person routes their instrument, and the live feed from their colleague, into separate tracks of a digital audio workstation (DAW) such as Logic. ‘Looping-back’ the colleague’s output to an input is accomplished with software like Soundflower, or through the mixer of some audio interfaces, or by connecting an output of your interface to an input (careful with monitoring to avoid feedback).

You mix the tracks in your DAW and send the output in turn to a streaming program, like OBS, with a separate video stream from Zoom or similar. Older computers might struggle. You could send the audio into another computer dedicated to handling streaming.

Occasional glitches are fine in rehearsal, but you’d hope to minimise them for your livestream audience. Some ORM apps use the ‘Opus’ compression codec, which attempts glitch-concealment, or a plug-in for your DAW might detect and mitigate glitches before outputting to the streaming software.

Recording

At present, even a paying audience would accept the occasional glitch to see musicians playing in real time. But what if you want to record glitch-free?

Non-real-time remote recording is nothing new, but sucks the life out of certain music. Jazz relies on real time communication: bass and drums pushing and pulling together, etc.

This video wonderfully demonstrates ‘both-ends’ real time recording. Musician A records the signal from Musician B, and their own instrument, locally (hence glitch-free). Musician B does the same, the other way around. The two locally-recorded tracks are combined afterward, and no audio that was transmitted over the internet remains in the final mix.

What’s out there?

All ORM software is based on similar principles, so there’s little room for any app to distinguish itself. Most, though not all, is available on both Windows and Mac. This list is not exhaustive.

• JamKazam suffered initially from being too easy. Users would get started without understanding the limitations of ORM, try and fail to play with someone 2,000 miles away using WiFi, and quickly give up. But it can be set up to work well, London musician Adam Spiers says, ‘and they’ve come up with a package solution for livestreaming, video and recording.’

• Soundjack arose from a research project that included a demonstration over 5G with Jamie Cullum. It has an optional hardware solution, the ‘Soundjack Box’, which simplifies things, takes some heavy lifting away from your computer, and notifies you about sources of latency. In the near future, they plan to use a GPS clock to resolve small timing offsets between sound cards, removing another possible source of glitches.
• Jamulus is open-source software, and the renewed interest in ORM has led to feverish development.
• Jacktrip is basic but high-quality, and is run from the command line (‘Terminal’ on a Mac). Nothing’s automatic, but it can be quite powerful. Jacktrip sends truly uncompressed audio. This video exhaustively details a set-up.
• The TPF-Tools are a modular set of software that helps automate beyond-duet setups. They handle video separately and have a guided set-up process for installing and running the programs needed for a streaming or recording workflow.
• Source-Connect and Audiomovers have long competed with the ISDN industry-standard for remote studio connections. They now offer free versions for limited time or limited participants.
• Doug Hunt (the engineer behind the Mamas Gun live record) is developing his own software which he intends to be ‘like Zoom for ORM’. Meanwhile, clients book him to oversee their real-time livestreams. Sign up here for updates on his software.

Looking to the future

ORM has a spectrum of uses: informal jamming, virtual rehearsals, and serious recording and livestreaming.

These potential uses will remain beyond COVID-19, and an app that attracts users now will have a head start. ORM might become a tool that musicians have to summon the willpower to learn, like home recording or taxes.

Some musicians will have already bounced off. I’m ambivalent myself; the closer you get to that ‘in-the-room’ feel, the further away it still seems. I’ll certainly jump at the next chance to play with someone in person.

But things are ever-changing, and internet speeds are ever-improving. At some point, most jobbing musicians will encounter a situation where online real-time music would be helpful. What better time to look into it than now?

Thanks to Sam Leak, Adam Spiers, Joe Pearson, Ferg Ireland, Conall Mulvenna, Christian Hoene, Doug Hunt, Roman Haefeli and Johannes Schütt.

John J. Williamson is a professional bassist and currently works part-time in scientific research at the Francis Crick Institute.