Yes, what you want is 2 clicks away (or taps).
When you select a song I. Your library, you are given the option to see it in Apple Music. Select Go to Album. It will take you to the album, but only list the songs you have added.
From there, there’s a link at the bottom of the song list Show Complete Album. Select that and you’ll be taken to the full album. Hit the giant play button at the top and it’ll start from track 1 and right through.
You can also search for the artist/album manually and simply play the entire album in the same way.