Ping is failing in containers with 'ping: socket: Operation not permitted'

Ah yeah, that could be it. I suspect the setuid workaround would likely work and is how ping was shipped in distros for a very long time (and so not particularly risky).