I see the same behavior and created a thread about it a couple of months ago. I gave up on using Google Assistant to play music. I cue up what I want to listen to before I start driving. The idea of voice commands to play music is really cool but it doesn't work for me. The way it will play a song randomly within the requested album is strange.
I'm using a Samsung Galaxy S7 running whatever the latest version of android is for that particular phone.