audio cuts out randomly while streaming from spotify

[tobiasmcnulty]

I'm running spopd on volumio 1.5 on a Raspberry PI, and for the most part it plays fine. However, several times per song (say 2-5 on average), the sound will go silent for a variable period of time (from less than a second to 2-3 seconds). The intervals between the moments of silence is also inconsistent (anywhere from 20-30 seconds to a couple minutes). I've watched the system logs and dmesg for any events that might be happening around the same time, and top for any other processes that might be hogging the CPU, but I don't see anything. The moments of silence also don't seem to correspond to the log entries in spopd.log for downloading song parts. For example, audio will stop and start right in the middle of the time period between two "download complete" messages, i.e., sound doesn't start up again immediately upon logging of a "download complete":

2015-02-07 23:19:13 [SPTF] 23:19:13.594 I [file_streamer_simple.cpp:769] Request for file 4a445c666dc00f50c8705ff1e3cc411c4c369ce5 complete (code: 0)
2015-02-07 23:19:26 [SPTF] 23:19:26.109 I [file_streamer_simple.cpp:769] Request for file 4a445c666dc00f50c8705ff1e3cc411c4c369ce5 complete (code: 0)
2015-02-07 23:19:36 [SPTF] 23:19:36.765 I [file_streamer_simple.cpp:769] Request for file 4a445c666dc00f50c8705ff1e3cc411c4c369ce5 complete (code: 0)
2015-02-07 23:19:47 [SPTF] 23:19:47.847 I [file_streamer_simple.cpp:769] Request for file 4a445c666dc00f50c8705ff1e3cc411c4c369ce5 complete (code: 0)
2015-02-07 23:19:59 [SPTF] 23:19:59.930 I [file_streamer_simple.cpp:769] Request for file 4a445c666dc00f50c8705ff1e3cc411c4c369ce5 complete (code: 0)

To make things worse, the behavior is the same whether or not high_bitrate is set to true. The Raspberry PI is not overloaded; spopd is only taking about 7-14% of the CPU, and no other process is taking near that amount. The load average on the machine is around 0.2 - 0.3.

Here's a sample log of the "blips" and how they correspond to the spopd.log entries: https://docs.google.com/spreadsheets/d/1PL8W1K1uCoqHYxMiykPq9z4FS8vfoIIvjX6z6mTTOJs/edit#gid=0

[Du7chManiac]

Same here, audio cuts out randomly. Using volumio 1.5 on a raspberry pi 2 using hifiberry dac+.
Its not the hardware because when i play something using mpd from anything local then it doesn't happen.

[Du7chManiac]

Since there is no response on this I made some free time and looked into the issue myself. I have solved it for my setup...I will share my solution.

I installed monitoring tools on my pi to see the disk io (to microsd) and I saw it spiking alot when the audio cuts out.

During regular mpd usage this does not happen (also no audio cut outs). So to verify my idea I mounted a cifs share on the pi. The share is served by my windows hp microserver. Then I configured the cache path of the spopd daemon to use the share as cache.
And voila, my problem is fixed. So I guess getting a faster microsd then class 10 (or a more expensive one) will also work but Ill stick with the cifs share.

I hope this helps you out to. If you need any help on how to do this on volumio just ask me here and Ill help you out. I work as a it administrator over linux servers so for me this is very basic stuff.

[tobiasmcnulty]

Neat. My music is mounted over NFS anyways so I'll try saving the cache
there and report back.
On Feb 20, 2015 6:53 PM, "Du7chManiac" [email protected] wrote:

Since there is no response on this I made some free time and looked into
the issue myself. I have solved it for my setup...I will share my solution.

I installed monitoring tools on my pi to see the disk io (to microsd) and
I saw it spiking alot when the audio cuts out.

During regular mpd usage this does not happen (also no audio cut outs). So
to verify my idea I mounted a cifs share on the pi. The share is served by
my windows hp microserver. Then I configured the cache path of the spopd
daemon to use the share as cache.
And voila, my problem is fixed. So I guess getting a faster microsd then
class 10 (or a more expensive one) will also work but Ill stick with the
cifs share.

I hope this helps you out to. If you need any help on how to do this on
volumio just ask me here and Ill help you out. I work as a it administrator
over linux servers so for me this is very basic stuff.

—
Reply to this email directly or view it on GitHub
#40 (comment).

[Du7chManiac]

The builtin function from volumio mounts the shares as a ro (read only), you can change this in the advanced settings of your library in the volumio gui. Just change ro to rw.

[tobiasmcnulty]

Where are you changing cache settings? My changes to cache_path and settings_path in /etc/spopd.conf didn't seem to get picked up by spopd (even after killing and restarting it), so I symlinked /.cache and /root/.cache to my NFS partition and verified files are getting created there. I'm still getting intermittent blips... though maybe fewer, it's hard to say...so I'm worried spop may have started and opened some files before the NFS partition was mounted.

For others stuck in a similar situation, note that when you start spop over SSH it'll put its cache in /root/.cache. When Volumio starts it on boot, it puts it in /.cache, presumably because HOME is not set.

[Du7chManiac]

Run a ps -ef | grep spopd
You will see a line spopd -c xxx where xxx is the config file spopd uses (I
can't give exact path I'm not at home).

Edit that config file and then kill spopd and launch it again using that -c
option to that config.

Symlinking will still use your sd which will not fix it.

I did all this under root account.
On 21 Feb 2015 17:52, "Tobias McNulty" [email protected] wrote:

Where are you changing cache settings? My changes to cache_path and
settings_path in /etc/spopd.conf didn't seem to get picked up by spopd
(even after killing and restarting it), so I symlinked /.cache and
/root/.cache to my NFS partition and verified files are getting created
there. I'm still getting intermittent blips... though maybe fewer, it's
hard to say...so I'm worried spop may have started and opened some files
before the NFS partition was mounted.

For others stuck in a similar situation, note that when you start spop
over SSH it'll put its cache in /root/.cache. When Volumio starts it on
boot, it puts it in /.cache, presumably because HOME is not set.

—
Reply to this email directly or view it on GitHub
#40 (comment).

[tobiasmcnulty]

spopd -c /etc/spopd.conf (and that's file I'm editing). Still not picking up the config changes..

[Du7chManiac]

You restart it using that file as well?
If you start it regular it will use another config file so either you edit
that to or you use the -c option

Else check the spopd log for an error in your config file. I also only used
the cache option not the settings one
On 21 Feb 2015 17:58, "Tobias McNulty" [email protected] wrote:

spopd -c /etc/spopd.conf (and that's file I'm editing). Still not picking
up the config changes..

—
Reply to this email directly or view it on GitHub
#40 (comment).

[Schnouki]

Hi all,

First, well done. I wasn't able to reproduce this issue on my laptop (no RPi yet) so I could only suspect that it was a libspotify issue, which doesn't help much. Thanks a lot for your research @Du7chManiac, it explains a lot: slow IO cause libspotify to delay calls to the audio delivery callback, which causes audio buffer underruns.
Moving the cache to a faster IO device makes sense, and with the cache for network filesystems it makes sense to use CIFS or NFS as well.
Another option may be to use a tmpfs, but this may prove difficult with the low amount of RAM on a RPi.
Yet another option would be to disable the cache by setting the cache path to an empty string (cache_path =).
And yet another option would be to try to increase the number of audio buffers (BUFNB in https://github.com/Schnouki/spop/blob/master/plugins/ao.c#L38), but it's more complicated (requires a recompilation) and I have no idea if it can be effective against such an IO issue.

@tobiasmcnulty, if spopd doesn't pick your config changes, you can try quitting it from the JSON interface: just telnet SPOP_IP 6602 and type quit, then restart it as usual (I'm not sure how it's started on Volumio).

Meanwhile I'll try to reproduce this issue and add a workaround in spop. Thanks a lot for your messages @Du7chManiac and @tobiasmcnulty!

[Du7chManiac]

@Schnouki No problem for helping you out, basically I'm just helping someone who helped me out by delivering this daemon :)

In volumio spop is configured without a cache, the cache_path is not in the config. Using it that way is what is causing me the stuttering. It somehow does heavy IO without the cache config. But I'm not involved in developing Volumio so I'm just stating what I see from looking at the config files.

Volumio also makes a ram filesystem but it is very small, I think only 256 MB.

I think though that another solution would be to use the spop interface and set the playlists to offline mode. Maybe that will soften the IO load as well.

I'm sticking with my current setup, I haven't suffered a single stuttering since I configured the cache.

PS: On volumio you control spop the same way, just telnet to localhost on 6602

[keab]

Update: Skip this and read my next post below

I had the same problems, using Volumio 1.5. I solved it by limiting the cache size to 20MB and moving it to RAM. It's a no cost solution and has worked fine for a few days now.
As written above, setting the cache path in spopd.conf has no effect, so I used a symlink instead.Below is my recipe, which works fine as long as no other program is also using the /.cache folder (in which case you can create a symlink from /.cache/spop instead, but that will clutter /var/run. There are probably better ideas)

#First limit cache size to 20MB
$sudo nano /etc/spopd.conf
#Add/edit this setting at the end:
cache_size = 20

#Turn off the spotify client to avoid problems during changes
$telnet localhost 6602
#Type:
quit

#check that /.cache is not used by any other program. The only thing listed should be "spop"
$ls /.cache

#Remove the old cache
sudo rm -r /.cache

#Create symlink to ramdisk instead
sudo ln /var/run /.cache

#Restart Volumio

[reaganry]

keab solution working for me so far(along with preventing nginx from logging)- thanks!

[tobiasmcnulty]

This works for me too! Note on my system (Volumio 1.55), the Ramdisk is mounted at /run/shm. /var/run is a symlink to /run, also a tmpfs parition, but using /run/shm provides one less level of indirection. You can inspect your system to see where the Ramdisk is mounted with the mount command.

Also, I think the last command should read sudo ln -s /var/run /.cache (note addition of -s).

Thanks for the fix!

[tobiasmcnulty]

One more thing: I think the cache_size setting should be put under the [spop] header, not the [sox] header, correct? [sox] is lower down in my file.

[pod2g]

I confirm that this fix works perfectly. Thanks keab.

@tobiasmcnulty: correct

[Zundrium]

Fix working here as well, thanks @keab.

[Mov-zz-ingDo-zz-ts]

Made an account to say thanks to everyone who found a solution.

@keab, your solution also works fine here (rpi 1B with hifiberry)

[keab]

Great that so many were helped. However I keep learning, so here's an update which simplifies a lot: The symlink is not needed! Just do the following:
$sudo nano /etc/spopd.conf
Add this line under [spop]:
cache_path = /run/shm
Save the file and restart.
I had tried this before but it didn't work then because I didn't put the line under [spop]. Thanks @tobiasmcnulty for making me think in this direction.
(Note that some files will still be put under /.cache, but that looks like some user data of much smaller size.)

Regarding the cache_size, I have made some experiments and spop seems to ignore the setting, even if I put it under [spop]. Leaving it out completely should limit the cache to 10% of available disk space (See https://github.com/Schnouki/spop/blob/master/spopd.conf.sample) but spop did not stop at 10%.
I also tried setting cache_path= "" which should disable caching. It did not anymore put audio files under /.cache but I could see that the size of the rootfs increased during playback so that setting didn't seem to work either.

So if anyone found a way to limit the cache that actually works, please post it! I'm not sure what will happen if/when the ramdisk becomes full but hopefully it will just delete the oldest files and continue.