I saw this before I posted. Was very excited to start with, but it seems that Nextcloud driver actually downloads all files and stores them on the phone. This limits the usefulness because the phone's storage is simply not big enough.
This could be a problem in how Nextcloud is handling this, but still, it makes it impractical for now. I have submitted a question to the Nextcloud issue tracker asking about this behaviour. https://github.com/nextcloud/android/issues/5814