Are there viable ways to play together in real time, on-line from different locations

[slewis]

(I always try to do a search for existing threads on topics but haven't found anything on this, although it may be because I'm not searching for the correct terms.) But I'm working on a project with another musician hundreds of miles away, where it would be extremely helpful if we could play together on-line in real time (Zoom, Facetime, Skype, whatever). I've heard that it's been done successfully (I think) with no latency in the connection, but I've also heard it's pretty hard or impossible to do. Any advice, tips, suggestions would be very much appreciated. TIA.....

[Chipotle]

Originally Posted by slewis

(I always try to do a search for existing threads on topics but haven't found anything on this, although it may be because I'm not searching for the correct terms.) But I'm working on a project with another musician hundreds of miles away, where it would be extremely helpful if we could play together on-line in real time (Zoom, Facetime, Skype, whatever). I've heard that it's been done successfully (I think) with no latency in the connection, but I've also heard it's pretty hard or impossible to do. Any advice, tips, suggestions would be very much appreciated. TIA.....

There are/were a few companies working on solutions for this. I believe most have custom hardware with minimal latency, connecting directly to the company's servers and sent to their hardware on the other end(s), and proprietary data packet configuration. Supposedly it can work, IF you have a good internet connection and the physical distance isn't too far.

In the end, you are still subject to the latency of the internet in between, the physical speed the signals can travel, the time it takes to collect packets and process them... that's why it's a tough problem to solve.

[Rudy4]

Originally Posted by slewis

(I always try to do a search for existing threads on topics but haven't found anything on this, although it may be because I'm not searching for the correct terms.) But I'm working on a project with another musician hundreds of miles away, where it would be extremely helpful if we could play together on-line in real time (Zoom, Facetime, Skype, whatever). I've heard that it's been done successfully (I think) with no latency in the connection, but I've also heard it's pretty hard or impossible to do. Any advice, tips, suggestions would be very much appreciated. TIA.....

If there was one good thing that came out of the pandemic it was a wish to come up with a way to interactively play with others over the internet. The general consensus was close, but no horseshoes. The best possible result comes with really fast connections and participants that are in close proximity. That means within a few blocks of each other.

Cockos, the folks who birthed the Reaper DAW, spent time on a solution a few years back and the closest they could come up with was Ninjam, which is a whole other way of collaborating which works by setting delays to be greater, matching measures instead of actual real time streaming. Further reading on the Cockos website, but that's not what you're asking for.

[Doug Young]

I tried several of these during the pandemic, and most were iffy at best. The one that does seem to work is JackTrip. Originally it was software you downloaded and then did massive tweaking of your computer network settings (as in, they had a week-long Stanford class on how to do it!). But someone got smart and put it all in a tiny box, pre-configured. Plug it into your wired internet connection, use headphones, and connect audio to it, and you're set. It basically eliminates nearly all the local, controllable latency on each end. You're left with the latency of the internet, which no one can do anything about - physics, the speed of light, etc. But, if you have decent latency between you and your collaborators, it can work.

Teja Gerken and I have used it quite a bit for practicing, and I know a band that uses it with multiple people on a regular basis. Teja and I have about 20ms latency between us (we live about an hour apart). That's like being in the same room, sitting 20 feet from each other. The sound quality is usually pretty good, and the interface and features have been steadily improving. I've seen people with recordings they've claimed to have made with it - what I've experienced would not be good enough that I'd want to do that, but it's plenty good enough for practicing, jamming etc. One thing I have noticed is that Teja and I tend to start a tune, and it seems to get slower and slower as we go on. My theory is that even 20ms, while not noticeable, is enough that I'm subconsciously holding back to sync with him, and then he's holding back to sync with me, and endless cycle, we end up playing at 3/4 speed by the end of a tune. But again, fine for practicing.

Anyway, there are some newer ones I see advertized that I have not tried, but my guess is that nothing is likely to beat the JackTrip solution, especially if it runs on your own computer, where you have lots of latency issues to start with. The Jacktrip box costs $250, if I recall, I got mine on Amazon.

[slewis]

Originally Posted by Chipotle

There are/were a few companies working on solutions for this. I believe most have custom hardware with minimal latency, connecting directly to the company's servers and sent to their hardware on the other end(s), and proprietary data packet configuration. Supposedly it can work, IF you have a good internet connection and the physical distance isn't too far.

In the end, you are still subject to the latency of the internet in between, the physical speed the signals can travel, the time it takes to collect packets and process them... that's why it's a tough problem to solve.

Thank you!

[slewis]

Originally Posted by Rudy4

If there was one good thing that came out of the pandemic it was a wish to come up with a way to interactively play with others over the internet. The general consensus was close, but no horseshoes. The best possible result comes with really fast connections and participants that are in close proximity. That means within a few blocks of each other.

Cockos, the folks who birthed the Reaper DAW, spent time on a solution a few years back and the closest they could come up with was Ninjam, which is a whole other way of collaborating which works by setting delays to be greater, matching measures instead of actual real time streaming. Further reading on the Cockos website, but that's not what you're asking for.

Thank you!

[Rudy4]

Doug, Jacktrip looks interesting, and there's a totally free version to try out the technology. The website makes a point of relating that there's no need for additional hardware purchase, so I'm interested in finding out about the box purchased from Amazon.

Jacktrip HERE

[ljguitar]

Originally Posted by slewis

…I've also heard it's pretty hard or impossible to do. Any advice, tips, suggestions would be very much appreciated.

Hey slewis
Be sure to check out Doug Young's post. He's using it NOW and explains its ins/outs in his post to you.

[David Eastwood]

An old friend and I started down this route during the pandemic also. He's in New York, and I'm in Minnesota, so we're about 1000 miles apart, roughly speaking.

Zoom was a dismal failure. It's optimized for one participant producing sound at a time, so really just doesn't work for ensemble playing.

He found Jamkazam, which we tried - and have ended up using ever since. In the early days, it, too, was not very good, but has improved significantly in the 2+ years we've been using it. It does have its bad days when internet latency is worse than usual, but more often than not it's just fine for the two of us. On those bad days we just chat and play to each other, rather than with.

We occasionally add a third (based in Atlanta), and that can get more challenging, but we've learned that the secret is for us not to be all banging away at once - which, in turn, has made the music we produce much more enjoyable.

It's not perfect, by any means - and if any of us were at Doug and Teja's level of ability, I'm sure it wouldn't be satisfactory at all. If that were the case, then Jacktrip with the additional hardware piece would be seriously under consideration.

Jacktrip and Jamkazam appear to have similar subscription offerings and costs, outside of the hardware piece, and both have free options.

One thing that is worth mentioning - don't even think about trying this over wifi. You will need a hardwired connection from whatever computer you're using to the internet.

[Songbook19]

I teach on-line and after dealing with the frustration of trying to do anything via Zoom (and to a lesser extend Google Meet) and the endless tweaking of settings, someone on the Zoom Community FB Group mentioned Rock Out Loud Live (ROLL) which I now use whenever possible. It's still not perfect but I find even their basic setup better than either of the aforementioned.

And they have something called an E.A.R mode, which if you're only playing/conducting a session with two people, AND you've got a relatively good internet connection, AND you're both using headphones (to eliminate a feedback loop), AND you're both using Google Chrome on a PC or Mac . . . then the experience comes as close to zero latency as possible.

It's $10 a month, pay as you go, and there's a 10-day trial period so you could sign up, kick the tires, and move on if it's not what you were hoping for. It's a little light on documentation but there's a "Book a Demo" button where you can schedule a 30-minute one-on-one session with a live person to ask questions.

https://rockoutloud.live/login

https://www.facebook.com/groups/rockoutloudlive

[strangebloom]

Originally Posted by Doug Young

I tried several of these during the pandemic, and most were iffy at best. The one that does seem to work is JackTrip. Originally it was software you downloaded and then did massive tweaking of your computer network settings (as in, they had a week-long Stanford class on how to do it!). But someone got smart and put it all in a tiny box, pre-configured. Plug it into your wired internet connection, use headphones, and connect audio to it, and you're set. It basically eliminates nearly all the local, controllable latency on each end. You're left with the latency of the internet, which no one can do anything about - physics, the speed of light, etc. But, if you have decent latency between you and your collaborators, it can work.

Teja Gerken and I have used it quite a bit for practicing, and I know a band that uses it with multiple people on a regular basis. Teja and I have about 20ms latency between us (we live about an hour apart). That's like being in the same room, sitting 20 feet from each other. The sound quality is usually pretty good, and the interface and features have been steadily improving. I've seen people with recordings they've claimed to have made with it - what I've experienced would not be good enough that I'd want to do that, but it's plenty good enough for practicing, jamming etc. One thing I have noticed is that Teja and I tend to start a tune, and it seems to get slower and slower as we go on. My theory is that even 20ms, while not noticeable, is enough that I'm subconsciously holding back to sync with him, and then he's holding back to sync with me, and endless cycle, we end up playing at 3/4 speed by the end of a tune. But again, fine for practicing.

Anyway, there are some newer ones I see advertized that I have not tried, but my guess is that nothing is likely to beat the JackTrip solution, especially if it runs on your own computer, where you have lots of latency issues to start with. The Jacktrip box costs $250, if I recall, I got mine on Amazon.

That sounds promising. Thanks for the info. Yes, if its in a box without a computer getting in the way, that's probably the best you can hope for to practice remotely.

[Rockysdad]

If you BOTH have Apple devices, Mac or such wouldn't FaceTime work?

[Doug Young]

Originally Posted by Rudy4

Doug, Jacktrip looks interesting, and there's a totally free version to try out the technology. The website makes a point of relating that there's no need for additional hardware purchase, so I'm interested in finding out about the box purchased from Amazon.

Yes, you can use Jacktrip with free software - that's how it started, a research project at Stanford, if I remember right. The issue is that most computers have quite a bit of latency in the signal path - your audio interface, drivers, internet interface and so on. It can easily be 10ms, so 20ms in total for 2 people, before you even get into the internet's latency. Jacktrip had a Stanford class on how to configure your computer for minimum latency (not sure if that's still available), and I started down that path during the pandemic. But it got pretty geeky fast, even for someone who's a computer programmer, and I quickly decided I'd rather spend the time playing guitar, as well as realizing I was unlikely to do more than trim off a few ms of latency from my Mac anyway.

What the box does (it's sold by the Jacktrip org) is bundle a stripped down computer running Linux (or some variant), a 2-channel audio interface, and ethernet connection. There's no controls, no interface, just plug it in. Tiny little thing, about the size of a guitar effects pedal. That is all pre-configured for the lowest latency they can achieve on that part of the path. The box also has a USB port, which connects to the computer, and you control your interactions, starting a session, connecting with others, etc, via a web interface. You plug in audio and headphones, and you're up and running.

Overall, it does work, subject to the whims of the internet. And yes, you need a direct ethernet connection, the box doesn't even support wifi, which is inherently problematic as far as latency goes, anyway.

[Doug Young]

Originally Posted by Rockysdad

If you BOTH have Apple devices, Mac or such wouldn't FaceTime work?

No. There are multiple issues. One is latency - you play, your partner hears you some time later and plays along, then you hear them back even later.

The other issue is duplexing. Most of these chat programs only send sound one direction at a time, partly to avoid feedback. It can be switching very fast, so when you're talking you don't notice. But play music, and you quickly discover that when person 1 is playing, person 2's sound is cut off. In practice, it sounds like everything's breaking up.

And yet another issue is sound quality. These things are optimized for speech, and music tends to simple sound like crap. Zoom addressed this by adding a "high fidelity music mode", which you have to turn on. Works best with headphones, as it disables some of the echo and feedback cancellation features. But what it mostly helps with is lessons and live musical webcasts, improving the sound quality quite noiticeably. It doesn't help with the other issues.

Now, I have seen some people sort of make this work anyway. Muriel Anderson started hosting sort of interviews with various musicians during the pandemic over zoom, and she almost always does some sort of duet with her host. But what I think is going on is that she generally has her guest play something, and she plays along with them, improvising over their song. Since the broadcast is going out from her side, the guest's sound is delayed reaching her, but she plays along at the time at which the sound arrives to her, and she broadcasts that. The guest just has to sort of ignore what they hear from her, if they even hear it due to the duplexing, etc.

There are other tricks like this you can play to *simulate* playing together. One friend I know was doing sessions by using 2 different programs, one direction on facetime, the other on zoom, for example, with each person muting the mic on the opposite program. This got around the duplexing issue. But not the sound quality or latency. I experimented with doing something like this during the pandemic, with the idea of webcasting a concert with my duo partner, Teja Gerken. If he could connect with my via Skype or Facetime, then I could capture that video and audio stream, send it into a Zoom session via OBS, add my part, and stream the result, which would be in sync. The problem was that this would only work for tunes where Teja could play his part totally standalone, with me playing along and him basically not even hearing what I was playing. Since most of our stuff involves a lot of back and forth, that was going to be pretty limiting, so I never got past just experimenting. And again, none of this is satisfying from the perspective of "playing together", just a sort of trick to create a distributed performance.

[Doug Young]

Originally Posted by David Eastwood

A
He found Jamkazam, which we tried - and have ended up using ever since.

JamKazaam's an interesting story. They were a startup long before the pandemic, which died apparently for a combination of lack of interest and so-so results. They actually sold a hardware device originally, I think I still have one in the garage, but it was just a budget audio interface. When the pandemic hit, it was resurrected by one of its former employees in his free time, and of course suddenly got intense interest. There were a bunch of people trying to help out. I know a guy who got involved to help them write a better manual, and who tried to help them get investment money to restart the company.

My efforts to use it were mixed. It was very cool that it worked at all, but the sound and latency as well as a very crude and flakey interface made it less than usable. one thing that was interesting was all the stats it showed, so you could actually see "180ms latency on this connection" and so on, which demonstrated the challenge very clearly. I noticed a lot of improvements over time, but never had a really successful session that was more than "maybe this will work someday". I haven't looked at it now in years, glad to hear its improved.