Jump to content

m3u Playlists regularly being rewritten


ToG

Recommended Posts

My relative path m3u Playlists are regularly being changed to absolute path names, &/or having a bunch of track listings at the end duplicated.

At this stage its cost me hours of manually rewriting the erroneous playlists.

Link to comment
Share on other sites

  • Replies 29
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

The only thing that should cause PA to write M3U playlists back to storage is making any changes to it - such as adding or removing any songs, or changing the sequence order, for example. Any such alteration will cause the whole file to be re-written, there is no facility to append to an existing file if (for example) you happen to just add one new song at the end.

PA has only one format for saving playlists - which uses the filename and its full Android path, plus any ratings information. It doesn't have a relative path feature. If you need to keep the original file, with no changes and relative paths, please retain a master copy of the file elsewhere.

PA doesn't really treat M3U files as either relative or absolute by the way, so you can store your playlist files anywhere you like as long as PA can scan that location. The scanner pattern matching checks for just the filename and the name of the first layer of containing folder (if there is one).

Andre

Link to comment
Share on other sites

@andrewilley  I am aware of this function, but the m3u's get changed when I just play them. If any changes were made it's definitely been by accident. It may well be that I accidentally cause a change to the file that is rewriting the playlists to absolute paths, but I am being increasingly careful to avoid this.

Also some playlists just add the last few tracks to the end again, which is definitely not something I'm doing.

I've attatched a file showing the duplicates being added at the end of the list.

I'm prepared to accept that I'm causing these complications.

Another thing is that I've been using relative paths because I've got a cloud sync to musicbee on my home media pc. So I really need to keep my current structure.

Ideally, for my needs I'd like to have a read only function added to Poweramp for the m3u files, as I only intend to alter playlists created by Poweramp, & then I rewrite them to relative paths manually. Once I've done this I would much prefer for Poweramp to refuse any changes made to the playlists.

I can always copy the tracks to a new PA playlist, which I then export & do my manual alterations & save the resulting file

2021 rhyme.m3u

Link to comment
Share on other sites

@ToG The physical M3U files should not be modified by playback, only by making changes. Could you maybe establish what action you might be doing that is causing the file to be updated?

You could try setting your M3U files to read-only. Android doesn't support sensible things like chmod without rooting (thanks Google) but you could set the permission by putting the SD Card onto a PC and changing its status to Read-Only there. PA will still allow changes to be made internally to the playlist (all such actions are handled internally in the database) but writing those changes back to the file will fail so a Full Rescan will revert to the original file contents.

I didn't know the specific operation of just adding a song to the end of a list could append the new content, I thought it wrote the whole file again from scratch. [I just tested it, and for me adding just one song to the end of an existing M3U playlist, the whole file got rewritten using Android paths]

I've seen the occasional issue of some songs at the end of an M3U playlist getting duplicated too. It seems to occur very occasionally - in fact, coincidentally, I caught it yesterday after making some external changes to an M3U due to some character set issues. I don't know what causes it, but normally I just either do a Full Rescan, or edit the file externally to Poweramp so it gets read again as a 'new' file. But yes, that does seem to be a bug.

Internally by the way, Poweramp does not use paths from playlist files at all (either relative or absolute) to avoid issues when people were copying absolute path files from one device to another. When PA scans in an M3U file it tries to match the filename and also the first containing folder with the songs that have already been scanned into its database. Technically you could create files that break this process, but the naming would have to be pretty convoluted - e.g. if I had two versions of movie soundtracks in separate subfolders, then Star Wars/Extended Version/End Credits.mp3  and  Back to the Future/Extended Version/End Credits.mp3 would be ambiguous.

Andre

Link to comment
Share on other sites

@andrewilley so, I've done some testing, & Poweramp will sometimes rewrite playlists to absolute paths if I Add a playlist to the queue.

It also will ad duplicates of songs from the end of a playlist if it does a playlist rescan. Maybe this is due to the music being on a 512gb Microsd... I have a roughly 450gb MP3 library. I'm considering that the library is more cumbersome than Poweramp can handle. Sometimes it will suddenly not see any music in certain folders if I add more music to an adjacent folder.

I'm absolutely certain that these issues are not caused by me altering the playlists myself.

I hope this is useful info. I would dearly love to solve these annoyances.

Link to comment
Share on other sites

Adding playlist contents to the Queue (either as individual songs or the whole playlist) shouldn't change the contents of the original playlist M3U file. However...

I've just tested several scenarios to make sure and it seems that merely opening the PA playlist editor screen - by long-pressing on any song in a playlist - will then auto-save the M3U file again when you exit from the editor screen. This is generally sensible behaviour to ensure that any potential changes aren't lost of course, but it does happen even if you haven't actually made any changes or you just used the editor screen to add a few songs to the Queue (which I suspect is what is triggering it for you). Not an issue normally, as for most people's general use it doesn't matter at all, but it will change the formatting of the lines in the M3U file, which in your case is a concern.

Could you perhaps do a few tests and see if this is what is occurring in your case? Just select a few songs in a playlist and then Back out of the editor screen without doing anything with those songs, and see if that causes your M3U file rewrites.

Although 450Gb on a phone is probably one of the largest music collections I've come across, large libraries shouldn't be an issue - other than Full Rescans might take a while.

I have very occasionally seen the duplication of a few songs at the end of a Playlist myself (although it's rare) but copying the original file back again and Full rescanning always seems to fix it.

Andre

Link to comment
Share on other sites

1 hour ago, andrewilley said:

Although 450Gb on a phone is probably one of the largest music collections I've come across, large libraries shouldn't be an issue - other than Full Rescans might take a while.

I haven’t noticed this playlist behaviour before but will check today. I wouldn’t be too concerned if it made changes based on pointing to relative or absolute paths only. Adding songs arbitrarily seems unusual though. 

I have 512GB Sandisk cards with over 500GB/24k music files on each. Poweramp handles this fine on both my LG G8thinQ phone and iBasso DX160 digital audio player. initial Full Rescans take about 10 minutes with these, but a regular scan for changes after only takes a few seconds.

I’m keeping an eye on pricing for 1TB cards, they are getting closer to affordable now.

Link to comment
Share on other sites

2 hours ago, MotleyG said:

Adding songs arbitrarily seems unusual though. 

I've seen it happen very, very occasionally. What happens is a bunch of songs at the end of a playlist get duplicated a second time, added to the end of the list. No rhyme or reason as to when or why, or even how many. Anecdotally, I've only seen it happen while I've been doing repeated testing, editing M3Us on my PC and copying them back to the phone, then usually doing a Full Rescan. But it's so rare that it would be hard to tie down a specific cause.

Andre

Link to comment
Share on other sites

@andrewilley  yes, the rewrite is definitely caused by selecting tracks within the playlist. I recreated the issue several times doing your test. 

It's unfortunate as I regularly use this feature & As you said it wouldn't be a problem except I rely on having relative paths for use across various devices. Sadly making the card read only isn't a viable option for me. I have a cloud sync app set up to sync my playlists and library between said devices... I wonder if adding a toggle feature to stop Poweramp making changes to m3u's would be possible, though I'm worried that it would be such a low priority that the effort wouldn't seem worth it to the dev crew.

 

The duplicates at the end of m3u's is also seeming to be happening as you said. Multiple changes & rescanning being the cause. the library size does appear to be irrelevant. I think my "read only toggle" suggestion might solve this as well.

At least I know why these issues occur.

Thank you for the help.

Edited by ToG
Clarification
Link to comment
Share on other sites

@ToG There are a couple of things at play here:

1) There is definitely a very rare bug where M3U playlists can get incorrectly extended by repeating the last few songs. I've seen it happen too, but so infrequently that it could be quite hard to track down.

2) The Playlist Editor feature in PA always re-saves playlist files, even when no changes have been made - e.g. if you only used the song selection feature to look at some song information, update cover art, or add something into another list or Queue. This one seems counter-intuitive to me, and I think physical saving of files should only ever occur after an actual change has been made. That ought to be pretty simple to implement, and I agree it would be a good idea because randomly saving M3U files into Poweramp's format can be quite destructive to the original format and it wipes any #EXTINF lines and other extended embedded data.

Andre

Link to comment
Share on other sites

  • 3 weeks later...

My ideal would be to add the option to disable Poweramp ability to change m3u files. When creating or altering a playlist I create a new Poweramp playlist & then export it to m3u, so I never want Poweramp to alter my m3us anyway.

This would solve both of the problems I have mentioned

Link to comment
Share on other sites

@ToG The problem with not updating the source M3U file when a change is made (a real change, not just viewing the Info/Tags, etc) is that the database and the file would get out of sync. Then if the file was ever re-read (during a full scan for example, or if your sync software updates the file from an external source) all of your internal changes would get lost. If you don't want real modifications auto-written back to the source files, use internal playlists rather than external ones as they are not auto-saved.

However once @maxmp fixes the 'rewrite playlist file after only viewing' issue, this should no longer be a problem for you.

Andre

Link to comment
Share on other sites

@andrewilley  I understand this, but it would render my entire system of cloud synced playlists redundant, which is not a viable option for me. I never plan on altering the m3u files through Poweramp. I copy them to internal playlists for editing purposes then I use an external playlist manager to export these playlists as relative path m3u files.

As you say fixing the "rewrite after only viewing issue" will solve this issue for me.

Aside from the intermittent duplication of songs in playlists bug, which is also a big hindrance to my system.

Unfortunately it's all seeming like I'm just going to be stuck undoing poweramps inbuilt playlist management systems for various reasons...

Just noticed that a select few songs are consistently ignored when added to playlists, which is doing my head in. I've verified the file names are accurate & they aren't read only. Poweramp will play the songs manually, but acts like they're nonexistant when reading from the playlists. Also verified that these songs are read perfectly well in other music apps, so again I'm very confused by this.

Poweramp is my favourite Android music option in many regards, but man is it causing me a lot of excess library curation, which just isn't an existing issue in any other app I've tried.

Tom

Link to comment
Share on other sites

8 minutes ago, ToG said:

Poweramp will play the songs manually, but acts like they're nonexistant when reading from the playlists.

Are these missing songs actually present in Poweramp's Library - i.e. has the scanner picked them up and read their metadata so you can play them normally from PA's Folders view? Or do you mean you can play them via PA when launched from an external app such as a file explorer? Playlists can only show and play songs that already exist in the PA music database.

Assuming the songs are present in the Library, could you check the path/filename details in PA's 'Info/Tags' display and compare that against the equivalent text line in the playlist? Are there any unusual (non-ASCII) characters in either the name or path? 

Andre

Link to comment
Share on other sites

  • 1 month later...

@andrewilley  hi, sorry for the delayed response. I've been quite sick for a while.

I had already checked the file names against the m3u file path. Everything is as it should be. The file path contains these characters () but there are multiple songs within this same file path that are all recognised by pa. The song is recognised by pa's folder view.

If I recreate the playlist with all the same songs pa will recognise the relevant song file at first, but eventually it just stops showing it in the m3u. I have the exact same library file structure & m3u's mirrored over the cloud to my pc, which is able to recognise the song file with no problems.

I honestly cannot think of a single reason why this is happening.

Link to comment
Share on other sites

  • 4 weeks later...

I was just looking through my playlists for something, and I noticed the original two bugs referred to at the start of this thread are still present in build 911.

Most of my file-based playlists have now been partially corrupted over time. I had copied them all over from my clean computer master copies back in June, while we were doing the various tests for this thread, and they all worked fine then.

But now, pretty much all of them seem to have had the 'random-appending' issue that was discussed earlier - some several times over by the look of it. For example, this is the end of a 'soft rock / ballads' playlist. All of the earlier content in the file is fine - but here at the end, PA keeps re-appending the last few lines back onto the end of the file again. The last genuine item is the \\diskstation entry for "Where Peaceful Waters Flow", the rest has been appended by PA.

\\diskstation\music\Andre MP3s\(Singles)\My Immortal - Evanesence (Band Version).mp3
\\diskstation\music\Andre MP3s\(Singles)\Seasons in the Sun - Terry Jacks.mp3
\\diskstation\music\Andre MP3s\(Singles)\Nights In White Satin - The Moody Blues.mp3
\\diskstation\music\TV, Films & Shows\(Film Music)\Take My Breath Away - Berlin.mp3
\\diskstation\music\Andre MP3s\(Singles)\Waiting for a Girl Like You - Foreigner.mp3
\\diskstation\music\Andre MP3s\(Singles)\Whiter Shade of Pale - Procol Harum.mp3
\\diskstation\music\Andre MP3s\Barclay James Harvest\(1981) Turn Of The Tide\05 - Echoes And Shadows - Barclay James Harvest.mp3
\\diskstation\music\Andre MP3s\Chris de Burgh\(1989) Spark to a Flame [Collection]\15 Where Peaceful Water Flow - Chris de Burgh.mp3
Folders/(40) Films/(Film Music)/Take My Breath Away - Berlin.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/(Singles)/Waiting for a Girl Like You - Foreigner.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/(Singles)/Whiter Shade of Pale - Procol Harum.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/Barclay James Harvest/(1981) Turn Of The Tide/05 - Echoes And Shadows - Barclay James Harvest.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/Chris de Burgh/(1989) Spark to a Flame [Collection]/15 Where Peaceful Water Flow - Chris de Burgh.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(40) Films/(Film Music)/Take My Breath Away - Berlin.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/(Singles)/Waiting for a Girl Like You - Foreigner.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/(Singles)/Whiter Shade of Pale - Procol Harum.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/Barclay James Harvest/(1981) Turn Of The Tide/05 - Echoes And Shadows - Barclay James Harvest.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/Chris de Burgh/(1989) Spark to a Flame [Collection]/15 Where Peaceful Water Flow - Chris de Burgh.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(40) Films/(Film Music)/Take My Breath Away - Berlin.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/(Singles)/Waiting for a Girl Like You - Foreigner.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/(Singles)/Whiter Shade of Pale - Procol Harum.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/Barclay James Harvest/(1981) Turn Of The Tide/05 - Echoes And Shadows - Barclay James Harvest.mp3
#EXT-X-RATING:0
D0D8-84ED/MusicFolders/(60) Music/Chris de Burgh/(1989) Spark to a Flame [Collection]/15 Where Peaceful Water Flow - Chris de Burgh.mp3

 

Also, playlist files are still being completely overwritten by PA following the simple act of long-pressing on any contained song title - for example to view Info/Tags. IMHO, file rewrites should only occur when a content change has been made to that playlist.

Andre

Link to comment
Share on other sites

  • 2 months later...

Thanks Max, I hope I managed to capture a log while it happened - previously I only ever noticed it some time later once the file-corruption had already occurred.

If you could also solve the issue of entire Playlists always being re-saved to storage whenever a contained song is long-pressed too, I'm sure that would make @ToG very happy too. Ideally saving back to storage (which can modify the original M3U file structure unnecessarily) should only occur when some sort of change was made to the list contents or ordering, not just because the user wanted to view some Info/Tags etc. Thanks.

Andre 

Link to comment
Share on other sites

@andrewilley based on your report, playlist handling will be improved in the next beta. The issue is the indirect (SAF, Storage Access Framework) access to files, where app can't safely do "atomic" update to a file (write to temp file and rename it then to the final file name). On Androids 5-7 SAF is just broken in this regard, so no changes there, though SAF is used only for sd card/usb storage playlists there (internal storage is accessed directly on those).

For the later Androids SAF is hopefully able to do atomic update more or less, so this was added to the next build 915.

Please note that if you actively upload files to the device or edit them right there, on the device, file update is never atomic and Poweramp rescan can catch and parse semi-uploaded/semi-updated file. Though, later rescans will re-parse the playlist in its final state.

Link to comment
Share on other sites

Good old Android, how can it mess up such a fundamental file operation? The method for safely updating a file's contents hasn't really changed since the days of DOS:

  1. Create a temporary work file ; write all content to it ; finalise/close temporary file
  2. Delete original file
  3. Rename temporary file back to same name as the original file

How can it get such a simple operation wrong I wonder?
 

Yes, if a file is still being written or edited when PA wants to read it, unless the file is exclusively locked for access I can see how PA would load a partial version.

Andre

 

Link to comment
Share on other sites

Thanks for updating this in build 915 Max, I can confirm that playlists no longer seem to get rewritten automatically by long-pressing on a song title - unless you actually make a real change. I'll keep an eye out for the duplicated entries at the end of playlists bug, but it was always pretty random so could take a while to confirm.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...