MusicLibraryRequest to get all tracks from a playlist (iOS 16 beta)

Hi there,

tl;dr: What's the best way to get all tracks (with catalog IDs) from a playlist that has more than 100 tracks, using MusicLibraryRequest.

I'm doing something dumb, not understanding something, and possibly both.

I've got an existing, kinda okay function that uses the MusicDataRequest and the Apple Music API to fetch all tracks from a playlist, with pagination like this:

func getTracksFromAppleMusicLibraryPlaylist(playlist: AppleMusicPlaylist) async throws -> [MusicKit.Track]? {

    var tracksToReturn: [MusicKit.Track] = []
    var libraryTracks: [AppleMusicLibraryTrack] = []

    @Sendable
    func fetchTracks(playlist: AppleMusicPlaylist, offset: Int) async -> AppleMusicPlaylistFetchResponse? {

        do {
            let playlistId = playlist.id
            var playlistRequestURLComponents = URLComponents()
            playlistRequestURLComponents.scheme = "https"
            playlistRequestURLComponents.host = "api.music.apple.com"
            playlistRequestURLComponents.path = "/v1/me/library/playlists/\(playlistId)/tracks"

            playlistRequestURLComponents.queryItems = [
                URLQueryItem(name: "include", value: "catalog"),
                URLQueryItem(name: "limit", value: "100"),
                URLQueryItem(name: "offset", value: String(offset)),
            ]

            if let playlistRequestURL = playlistRequestURLComponents.url {
                let playlistRequest = MusicDataRequest(urlRequest: URLRequest(url: playlistRequestURL))
                let playlistResponse = try await playlistRequest.response()
                let decoder = JSONDecoder()

                // print("Get Tracks Dump")
                // print(String(data: playlistResponse.data, encoding: .utf8)!)
           
                let response = try decoder.decode(AppleMusicPlaylistFetchResponse.self, from: playlistResponse.data)
                return response
            } else {
                print("Bad URL!")
            }
        } catch {
            print(error)
        }
        return nil
    }

    Logger.log(.info, "Fetching inital tracks from \(playlist.attributes.name)")
    if let response = await fetchTracks(playlist: playlist, offset: 0) {
        if let items = response.data {
            libraryTracks = items
        }
        if let totalItemCount = response.meta?.total {
            Logger.log(.info, "There are \(totalItemCount) track(s) in \(playlist.attributes.name)")
            if totalItemCount > 100 {
                let remainingItems = (totalItemCount - 100)
                let calls = remainingItems <= 100 ? 1 : (totalItemCount - 100) / 100
                Logger.log(.info, "Total items: \(totalItemCount)")
                Logger.log(.info, "Remaining items: \(remainingItems)")
                Logger.log(.info, "Calls: \(calls)")
                await withTaskGroup(of: [AppleMusicLibraryTrack]?.self) { group in
                    for offset in stride(from: 100, to: calls * 100, by: 100) {
                        Logger.log(.info, "Fetching additional tracks from \(playlist.attributes.name) with offset of \(offset)")
                        group.addTask {
                            if let response = await fetchTracks(playlist: playlist, offset: offset) {
                                if let items = response.data {
                                    return items
                                }
                            }
                            return nil
                        }
                    }
                    for await (fetchedTracks) in group {
                        if let tracks = fetchedTracks {
                            libraryTracks.append(contentsOf: tracks)
                        }
                    }
                }
            }
        }
    }
    // props to @JoeKun for this bit of magic
    Logger.log(.info, "Matching library playlist tracks with catalog tracks...")
    for (i, track) in libraryTracks.enumerated() {
        if let correspondingCatalogTrack = track.relationships?.catalog?.first {
            tracksToReturn.append(correspondingCatalogTrack)
            print("\(i) => \(track.id) corresponds to catalog track with ID: \(correspondingCatalogTrack.id).")
        } else {
            Logger.log(.warning, "\(i) => \(track.id) doesn't have any corresponding catalog track.")
        }
    }
    if tracksToReturn.count == 0 {
        return nil
    }
    return tracksToReturn
}

While not the most elegant, it gets the job done, and it's kinda quick due to the use of withTaskGroup .esp with playlists containing more than 100 songs/tracks.

Regardless, I'm kinda stuck, trying to do something similar with the new MusicLibraryReqeust in iOS 16.

The only way I can think of to get tracks from a playlist, using MusicLibraryRequest, having read the new docs, is like this:

@available(iOS 16.0, *)
func getAllTracksFromHugePlaylist(id: MusicItemID) async throws -> [MusicKit.Track]? {
    do {
        var request = MusicLibraryRequest<MusicKit.Playlist>()
        request.filter(matching: \.id, equalTo: id)
        let response = try await request.response()
        if response.items.count > 0 {
            if let tracks = try await response.items[0].with(.tracks, preferredSource: .catalog).tracks {
                Logger.log(.info, "Playlist track count: \(tracks.count)")
                return tracks.compactMap{$0}
            }
        }
    } catch {
        Logger.log(.error, "Could not: \(error)")
    }
    return nil
}

The problem with this is that .with seems to be capped at 100 songs/tracks, and I can't see any way to change that.

Knowing that, I can't seem to tell MusicLibraryRequest that I want the tracks of the playlist with the initial request, where before I could use request.properties = .tracks, which I could then paginate if available.

Any help setting me on the right course would be greatly appreciated.

Accepted Reply

I've been trying to get better at structured concurrency, so I took a stab at a function that collects all of the tracks from a library playlist. I think the key here is using the hasNextBatch property on MusicItemCollection. I'm far from on expert in this world, so I'm sure this code can be improved. In the meantime, here's what I came up with. It correctly returns the number of songs on playlists with more than 100 songs.

   func getTracksFromPlaylist(name: String) async throws {
        var request = MusicLibraryRequest<Playlist>.init()

        request.filter(text: name)
        let result = try! await request.response()

        if let first = result.items.first {
            let withTracks = try await first.with(.tracks)

            guard let startingTracks = withTracks.tracks else {
                return
            }          

            try await getAllTracksFromPlaylist(startingTracks: startingTracks, group: nil)

        }

    }

     func getAllTracksFromPlaylist(startingTracks: MusicItemCollection<Track>, group: ThrowingTaskGroup<[Track],Error>?) async throws  {

     //If the group didn't come in as nil, that means startingTracks has a next batch that needs to be fetched and processed
        if var group = group {
                if let currentSetToProcess = try await startingTracks.nextBatch() { 
                    //Add the tracks from this batch to the group
                     group.addTask {
                        return Array(currentSetToProcess)
                    }
					//If this set has a next batch, run the function again. Once currentSetToProcess.hasNextBatch returns false, the group will complete since it is not awaiting anything else.
                    if currentSetToProcess.hasNextBatch {
                       try await getAllTracksFromPlaylist(startingTracks: currentSetToProcess, group: group)
                    }

                }

           //If the group came in as nil, create the group and work through the first batch of tracks.
        } else {
            try await withThrowingTaskGroup(of: [Track].self, body: { group in
				
             //The array to store the final set of tracks
                var allTracks: [Track] = []
             
				//Add the starting tracks to the group
                group.addTask {
                    return Array(startingTracks)
                }

				//If there are more than 100 tracks on the playlist, this is true and means we need to loop through and get the other tracks.
                if startingTracks.hasNextBatch {
                    try await getAllTracksFromPlaylist(startingTracks: startingTracks, group: group)
                }

                                
				//As arrays of tracks are added to the group, append them to our allTracks array.
                for try await tracks in group {
                    allTracks.append(contentsOf: tracks)
                }
          
				//Finally, when the group has returned all of the tracks, print the final amount.
                print(allTracks.count)      

            })
        }

    }
    

Replies

I've been trying to get better at structured concurrency, so I took a stab at a function that collects all of the tracks from a library playlist. I think the key here is using the hasNextBatch property on MusicItemCollection. I'm far from on expert in this world, so I'm sure this code can be improved. In the meantime, here's what I came up with. It correctly returns the number of songs on playlists with more than 100 songs.

   func getTracksFromPlaylist(name: String) async throws {
        var request = MusicLibraryRequest<Playlist>.init()

        request.filter(text: name)
        let result = try! await request.response()

        if let first = result.items.first {
            let withTracks = try await first.with(.tracks)

            guard let startingTracks = withTracks.tracks else {
                return
            }          

            try await getAllTracksFromPlaylist(startingTracks: startingTracks, group: nil)

        }

    }

     func getAllTracksFromPlaylist(startingTracks: MusicItemCollection<Track>, group: ThrowingTaskGroup<[Track],Error>?) async throws  {

     //If the group didn't come in as nil, that means startingTracks has a next batch that needs to be fetched and processed
        if var group = group {
                if let currentSetToProcess = try await startingTracks.nextBatch() { 
                    //Add the tracks from this batch to the group
                     group.addTask {
                        return Array(currentSetToProcess)
                    }
					//If this set has a next batch, run the function again. Once currentSetToProcess.hasNextBatch returns false, the group will complete since it is not awaiting anything else.
                    if currentSetToProcess.hasNextBatch {
                       try await getAllTracksFromPlaylist(startingTracks: currentSetToProcess, group: group)
                    }

                }

           //If the group came in as nil, create the group and work through the first batch of tracks.
        } else {
            try await withThrowingTaskGroup(of: [Track].self, body: { group in
				
             //The array to store the final set of tracks
                var allTracks: [Track] = []
             
				//Add the starting tracks to the group
                group.addTask {
                    return Array(startingTracks)
                }

				//If there are more than 100 tracks on the playlist, this is true and means we need to loop through and get the other tracks.
                if startingTracks.hasNextBatch {
                    try await getAllTracksFromPlaylist(startingTracks: startingTracks, group: group)
                }

                                
				//As arrays of tracks are added to the group, append them to our allTracks array.
                for try await tracks in group {
                    allTracks.append(contentsOf: tracks)
                }
          
				//Finally, when the group has returned all of the tracks, print the final amount.
                print(allTracks.count)      

            })
        }

    }
    

Hi @talkingsmall,

Thanks much for your thorough and thoughtful reply.

I was unaware (or blanking on) of the .hasNextBatch on MusicItemCollection for some reason. I appreciate you bringing that to my attention, as it helped me come up with a solution (extra logging added for clarity):

@available(iOS 16.0, *)
func getAllTracksFromPlaylistId(id: MusicItemID) async throws -> [Track]? {

    func getAllTracks(tracks: MusicItemCollection<Track>) async throws -> [Track]? {
        var hasNextBatch = true
        var tracksToReturn = tracks.compactMap{$0}
        var currentTrackCollection = tracks

        Logger.log(.info, "Initial track count: \(tracks.count)")

        do {
            while hasNextBatch == true {
                if let nextBatchCollection = try await currentTrackCollection.nextBatch(limit: 300) { // 300 is max here
                    tracksToReturn = tracksToReturn + nextBatchCollection.compactMap{$0}
                    Logger.log(.info, "Fetched \(nextBatchCollection.count) track(s) from next batch")
                    Logger.log(.info, "Current track subtotal: \(tracksToReturn.count)")
                    if nextBatchCollection.hasNextBatch {
                        Logger.log(.info, "nextBatchCollection has nextBatch.  Continuing while loop...")
                        currentTrackCollection = nextBatchCollection
                    } else {
                        Logger.log(.info, "nextBatchCollection has no nextBatch.  Breaking from while loop")
                        hasNextBatch = false
                    }
                } else {
                    Logger.log(.info, "No results from nextBatch()!  Breaking from while loop")
                    hasNextBatch = false
                }
            }

            if tracksToReturn.count > 0 {
                Logger.log(.info, "Returning \(tracksToReturn.count) track(s)")
                return tracksToReturn
            } else {
                Logger.log(.info, "tracksToReturn is empty!")
                return nil
            }
        } catch {
            Logger.log(.error, "Could not get next batches!")
        }
        return nil
    }

    do {
        var request = MusicLibraryRequest<MusicKit.Playlist>()
        request.filter(matching: \.id, equalTo: id)

        let response = try await request.response()

        if let playlist = response.items.first {
            if let tracks = try await playlist.with(.tracks, preferredSource: .catalog).tracks {
                if let allTracks = try await getAllTracks(tracks: tracks) {
                    Logger.log(.success, "\(playlist.name) has \(allTracks.count) tracks")
                    return allTracks
                } else {
                    Logger.log(.fire, "Could not fetch any tracks for \(playlist.name)")
                }
            } else {
                Logger.log(.fire, "With tracks on \(playlist.name) returns nil for tracks")
            }
        } else {
            Logger.log(.warning, "Could not find playlist with id: \(id)!")
        }
    } catch {
        Logger.log(.error, "Could not: \(error)")
    }
    return nil
}

It might be noticed that I've not attempted any concurrency, instead using a while loop to fetch all tracks, serially, in batches.

The reason for this is because, I don't think it speeds things up at all, because I don't think we can't use .nextBatch() in parallel async calls.

In the Apple Music API MusicDataRequest example in my initial post, I can get the total number of tracks in a playlist by looking at the meta.total value and then leverage async calls, using the right offsets to get "next batch" tracks in parallel calls. This significantly improves the speed on playlists with a large number of tracks.

I'm not sure that a MusicItemCollection can provide us with something equivalent to meta.total, and it doesn't appear that .nextBatch() takes an offset argument. So I'm not sure how to speed things up better than what I've come up with here. Of course, it's highly probable that I am mistaken about that.

That said, I can now get all tracks using MusicLibraryRequest to fetch a playlist and using .with on the resultant MusicItemCollection<Playlist>, followed by using .hasNextBatch and .nextBatch(), which is what I was looking for.

Thanks again, @talkingsmall!

Hello @Kimfucious and @talkingsmall,

Thank you for your feedback about using MusicItemCollection's hasNextBatch and nextBatch(…).

I'd like to refer you to an older thread about getting tracks from a catalog playlist.

I would also like to quote an important part of my response:

Nevertheless, while this is technically possible, as I just showed you, let me reiterate that we would generally hope that you never need to eagerly fetch all the possible next batches of a MusicItemCollection in this way.

What we would hope you do in your app is precisely what you said, to fetch the next batch as the users scroll further down. I'll leave it up to you to implement this behavior in your UI code using MusicItemCollection's nextBatch().

I hope this helps.

Best regards,

Thanks for the feedback, @JoeKun! I always appreciate your help 😊

My usage for this is not for displaying in the UI, but for various other things.

For example, if I want to use the following to update the tracks in a playlist, where items is all existing tracks in the playlist plus some new tracks or minus a specific subset.

let updatedPlaylist = try await MusicLibrary.shared.edit(playlist, items: items)

At present, I don't see a way to use the .edit method otherwise. I think I need all those items ahead of time.

Adding tracks could be done without the use of .edit, using a loop and multiple .adds; it's the removal of tracks that seems tricky to me.

Of course, I could be missing something. If that's the case, I'd love to hear a better way to achieve this.

Hi Kimfucious,

You can easily load all of the tracks found in the library of a playlist. You can just use the .with(_,preferredSource:) method.

let detailedPlaylist = try await playlist.with(.tracks, preferredSource: .library)

which would load all of the tracks found in the user’s library.

-David

HI @david-apple,

Thanks for the pro advice!

In my initial post on this thread, I mentioned getting all tracks from a playlist with over 100 tracks. Sorry, I know that post was a bit wordy, so it's easy to overlook that point.

My findings are that .with, in this scenario, has a cap of 100 tracks, which is why I started looking at ways to get more than 100.

This is an edge case, but I do need to accommodate for this scenario.

I found that .nextBatch() allows for a limit of 300, which is nice and results in less calls.

If I'm mistaken, and that is entirely possible, kindly advise.

PS: Kindly note that using .preferredSource: .library or omitting that parameter results in the .edit method failing, as I've mentioned here.

I'm loving the new iOS 16 stuff for MusicKit and hope my inputs help.

Got it, thanks @Kimfucious.

@david-apple's advice is definitely correct, although I understand you cannot use it right now because of the issue described in that other thread.

When that issue is resolved, you should definitely use with(_:preferredSource:) with .library.

Please stay tuned.

And thanks for the kind words. Your feedback certainly helps. Thanks for catching these issues and reporting them promptly while we still have a bit of time to iron them out in the beta period for iOS 16.

Cheers,

@david-apple @JoeKun As a follow-up to this conversation, I was curious when about fetching nextBatch as a user scrolls down a list. Let's say, for example, the user tapped on the top songs link on Coldplay's artist page in the Apple Music app, that would load a MusicItemCollection<Song> with many next batches. Now let's say they tapped on a song to play in that collection. It is my understanding that the you would need to pass the MusicItemCollection<Song> to the ApplicationMusicPlayer along with the song to play. However, if the next batches have not been fetched, then not all songs will be passed to the ApplicationMusicPlayer. My first instinct would be to append next batches to the tail of the ApplicationMusicPlayer queue as more batches are fetched.

However, the question that comes up for me in this instance is how would I be able to tell the difference between a user tapping the same song in an album versus a collection of top songs? This is fairly important because I shouldn't be adding top songs to the tail of the ApplicationMusicPlayer queue if they are playing Coldplay's Yellow from the album Parachutes versus Yellow in the top songs category (when I would want to append songs to the tail as more next batches are fetched). If there isn't an easy way to tell when to append top songs in a next batch to the ApplicationMusicPlayer queue (beyond comparing all songs in the queue against all songs in the top songs collection to verify they are the same), then I would assume if user taps to play a song in a collection, it would be wise to go ahead and fetch the entire collection if the user plays a song from that collection and append it to the queue at that point.

Let me know if I'm missing something obvious here or if my question is unclear! Thank you for any help/pointers.