X11 freezes for a few seconds in LXD container

Is that from the snap package?

I can only suggest to have a look at /var/snap/lxd/common/lxd/logs/mycontainer/ which has logs pertaining to your container. There are separate log files for the LXD sockets, therefore check the proxy.X0.log for any hints.

You can have top or htop running in a terminal window, so that when you get this freezing, you can get a hint whether it is due to a forkproxy process (relates to the LXD proxy devices). You should see momentarily a process with very high load.
If you do not see a process with very high load, then the freezing would likely be some I/O issue; some resource becomes unavailable, it blocks the whole system until it becomes available again.