That’s usually an indication that the network namespace of your container never expired, which normally indicates an issue with the kernel.
When the last process using a network namespace goes away, the namespace is destroyed, which causes all virtual interfaces to be destroyed and physical interfaces to be moved back to the host network namespace.
veth is a virtual device, the network namespace expiring will cause the container side device to be deleted, which then causes the host side device to also be deleted.
What you’re reporting here is consistent with the container’s network namespace still being active, keeping that veth device around.