Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Grain/Popping/Crackling audio when playing several Players #953

Open
kaua-melo opened this issue Sep 23, 2021 · 11 comments
Open

Grain/Popping/Crackling audio when playing several Players #953

kaua-melo opened this issue Sep 23, 2021 · 11 comments

Comments

@kaua-melo
Copy link

Hi, first of all thank you very much for the effort to making the library! It was an amazing finding!

I'm getting some popping/crackling sounds when playing several Players fast, one after the other.
Here's a live example. Press on the top left button first and then hover over the squares. If you hover over the squares slowly, you'll see that the samples play fine, without poppings (it might take few seconds to load all samples). But if you move your mouse fast, we get some annoying poppings.

I initialize tone as:
Tone.start();

and start the Players as:
players["s" parseInt((13 * cValue) / 100)].start()

Source code can be found under "View Page Source" of the page.

I tried adding fadeIn/fadeOut and also tried to use the restart() method in case the Player was already playing, but the problem persists.

I found a similar scenario example where the same problem exists: https://musiclab.chromeexperiments.com/Song-Maker/ (choose 'Woodwind' instrument). And a similar scenario where the problem doesn't exist: https://musiclab.chromeexperiments.com/Shared-Piano/#lmC19e8Jq. But I can't look into their code source.
I found this issue but I couldn't reproduce it, so I didn't know if that was the same. That's why I decided not to reopen it.

Is there any way I can play several samples (let's say from 10 to 15), one after the other, very fast, without getting this popping sounds?

I'm not experience with sound, so I'm sorry in advance if I'm not using the lib as expected.


Tested on:
macOs 10.14.6
Chrome 94.0.4606.54
Firefox 92.0

@dirkk0
Copy link

dirkk0 commented Sep 27, 2021

My guess is that you are overloading the system. You could try two things:
a) put a setTimout with a value of 1 or 0 around the player start, maybe this helps.
b) check if the player is already playing and restart it only when it's not.

@kaua-melo
Copy link
Author

Hey dirkk0,

Thank you very much for taking the time to reply. I appreciate it!

a) I just put the .start() inside setTimout as follows but the result was the same. I changed the delay from 0 to 1000 but couldn't see any difference in the popping sounds.

      setTimeout(function(){ 
          players["s"   parseInt((13 * cValue) / 100)].start();
      }, 10);

b) I also tried .restart() the player when they are already playing but unfortunately the result is the same.

      setTimeout(function(){ 

        // If the player is already playing, let's restart it.
        if(players["s"   parseInt((13 * cValue) / 100)].state === "started"){
          players["s"   parseInt((13 * cValue) / 100)].restart();
        }
        else{
          players["s"   parseInt((13 * cValue) / 100)].start();
        }

      }, 3);

You suggested to restart it only when it's NOT playing. It seems we can't apply .restart() to a Player when its state is "stopped". Maybe you meant the opposite?

I checked the link you sent but they suggest stopping and starting the audio through Tone.Transport.pause();. Unfortunately I can't do that since I might have many other Players playing at some point. I would like to restart only one, if it's actually already playing.

What I think it's strange is that it seems to be possible to do that without Popping sounds, as we can hear on this example. We can play the samples there in the same speed as in my example but we do not hear any popping...

@dirkk0
Copy link

dirkk0 commented Sep 28, 2021

Still I think that the stuttering comes from too many concurrent players playing, since the crackling doesn't happen when you move 'slow enough'.
I would recommend that you create an array as a simple player pool with N elements (let's say N=25) and once the 25th player is playing you stop player zero and start it with the new value (that's want I meant with re-start), thus guaranteeing that only N players are playing at the same time.

@kaua-melo
Copy link
Author

I see what you are saying now...

1- Here's a codesandbox demo with the pool suggestion.
I limited the number of simultaneous Players to 2 and unfortunately we can still hear the poppings/stuttering.
[I tried to use 'onstop' callback event but didn't manage to get it working. It seems other people are having issues with that as well: #519. That's why I remove the Player from the playingNow array through setTimeout 2 seconds]

2- That makes me think that the problem isn't really that we have several Players playing at the same time, but actually the fact that we start the Players too fast, one after the other, and for some reason tone.js has some issue with that.

I created this sandbox demo in order to make sure that we have a minimum delay between the Players and it indeed reduced the poppings a lot. The problem is that it needs a huge delay (around 80 microseconds) to remove the poppings. The interaction feels quite bad with that delay, so that can't be a good solution for the problem.

Here's what the code does:

The function play(p) start() a Player in case it's stopped, and restart() in case it has already been started.

      function play(p) {
        // If the player is already playing, let's restart it.
        if (players[p].state === "started") {
          lastTimePlayed = new Date().getTime();
          players[p].restart();
        }
        // Otherwise let's just start it.
        else {
          lastTimePlayed = new Date().getTime();
          players[p].start();
        }
      }

The function delayPlay(p) wait a bit before we play the Player[p], making sure we have a minimum delay (minDelay) between the Players:

      function delayPlay(p) {
        setTimeout(function () {
          // How much time has passed between the last time a Player was started to now?
          var d = new Date().getTime() - lastTimePlayed;

          // If we are still trying to play too fast, let's wait a bit more.
          if (d < minDelay) {
            delayPlay(p);
          } else {
            console.log(
              " # delayedPlay. Delay: "  
                (new Date().getTime() - lastTimePlayed)
            );
            play(p);
          }
        }, 1);
      }

Finally, when the user tries to play a Player, we check whether there's a minimum delay with the previous Player played.
if the delay is too small, we call delayPlay(p), otherwise we just play it. [We are console logging the delays between the players if you wanna check].

      // If the delay between two Players is too small, let's wait a bit before playing it.
      if (delay < minDelay) {
        delayPlay(playerKey);
      }
      // Otherwise let's just play it.
      else {
        console.log(
          " - Played 1. Delay: "   (new Date().getTime() - lastTimePlayed)
        );
        play(playerKey);
      }

It seems to me that that's not the right way to go since it needs such a huge delay that it makes the interaction really bad.
There must be a way of playing many samples very fast without those poppings if this example is using Tone.js: https://musiclab.chromeexperiments.com/Shared-Piano/#lmC19e8Jq, but I really couldn't find the way to achieve that.

Any ideas on how we can do that with Tone.js?

@dirkk0
Copy link

dirkk0 commented Sep 29, 2021

I am not sure what was the issue, but I think it was the stopping and immediately restarting.
However, I think I could solve it by throwing more players at it (maybe even go to a hundred), and make sure that every player has the chance to play to the end.
Additionally, I lowered the individual volumes a little bit, so that no overdrive happens:
https://codesandbox.io/s/player-pool-forked-go5zl?file=/index.html

I can't hear any crackling anymore.

@kaua-melo
Copy link
Author

Hey again dirkk0!

Thank you so much for taking the time to look into the code. It really does reduce the poppings!
It seems that the problem is a sum of:

  1. Restarting a Player which is currently playing
  2. The loud volume of the players.

Although this solution of creating several Players with the same .wav works in some cases, it might be strange to have to create like 30 Players with the same audio just because restart(), or doing stop() and start(), creates these popping sounds. Imagine if we have 30 samples, that would result in 900 Players.

I'm not sure whether this is actually a common issue when dealing with sound (I have no experience with sound), but I wonder whether it might be possible to simply restart(); Players without having those popping sounds.

I'll follow with your solution for now. Thank you very much, I really appreciate it!

@dirkk0
Copy link

dirkk0 commented Oct 1, 2021

You are very welcome!
Yes, this might seem strange, but you really have an edge case there with 100*20=2000 tiles that can be played on mouse move almost immediately.
You can propably do some further optimisation by loading the samples into buffers and feeding these to the players.

@tambien
Copy link
Contributor

tambien commented Oct 6, 2021

Thank you for your help @dirkk0 🙏🏻, i really appreciate it.

We can close this issue if it seems like it's been addressed?

@dirkk0
Copy link

dirkk0 commented Oct 6, 2021

You are welcome, too! :) Yes, I think @kaua-melo would agree that we can close this.

@kaua-melo
Copy link
Author

Hey! Sorry for the delay to respond!

My specific case of the issue was solved, but I'm not sure how I would address the problem if I had many more samples.

In sum, the issue is: Restarting a Player which is currently playing makes a pop sound.
And we didn't understand why nor how to fix it.

What @dirkk0 suggested (and worked really well! Thank you super much!) was to create several Players with the same sample. If that's a common solution when dealing with sound, then it's 100% solved. But if that sounds like a workaround, it could possibly be interesting to investigate if there's any way of stopping and restarting a player without having those poppings. I think I'm not in a position to judge if that's a real issue or if that's a common limitation when dealing with sound. So feel free to close or keep the issue open.

Here's a demo of the problem: https://codesandbox.io/s/relaxed-noyce-xoto8?file=/index.html

Thank you very much for the attention. I really appreciate it!

@dirkk0
Copy link

dirkk0 commented Dec 1, 2021

possibly related to #912 with possible fix #968

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants