I’ve created a ssh user on a test host which uses the ssh public key to identify specific users and forward the connections direct into the container. Its very simple and I’ve created a GitHub repo which demonstrates the concept.
What this is nice about this concept is you don’t need to worry about status IP address or routing to the container. In fact your can still SSH into the container if it has no network. 
Note that this script works for ssh and sftp I’ve not yet got scp working yet… Once I do I’ll update the repo.