Trying to do nixos-rebuild switch inside a fresh nixos container fails with Failed to start Firewall

Y.M.O · February 24, 2024, 12:10am

Hi folks.

trying to do nixos-rebuild switch inside a fresh nixos container fails with `Failed to start Firewall.`

opened 09:06PM - 23 Feb 24 UTC

closed 09:58PM - 23 Feb 24 UTC

# Required information * Distribution: NixOS * Distribution version: 23.11 host 23.11 guest * The output of "incus info": attached because it's so long # Issue description once i was able to enter the shell of a container i tried to rebuild the fresh nixos system by doing: `nixos-rebuild switch` doing `apt update && apt dist-upgrade` worked flawlessly in a non nixos (ubuntu 23) guest container. I only tried `ubuntu 23` guest container # Steps to reproduce 1. `incus launch images:nixos/23.11 nix00 -c security.nesting=true` 2. once inside run `nixos-rebuild switch` 3. `incus launch images:ubuntu/23.10 ub00 -c security.nesting=true` 4. `apt update && apt dist-upgrade && apt-autoremove` 5. above works inside the ubuntu conainer # Information to attach ``` [root@nixos:~]# nixos-rebuild switch building Nix... building the system configuration... activating the configuration... setting up /etc... setting up tmpfiles Cannot set file attributes for '/var/empty', value=0x00000010, mask=0x00000010, ignoring: Operation not permitted warning: the following units failed: firewall.service × firewall.service - Firewall Loaded: loaded (/etc/systemd/system/firewall.service; enabled; preset: enabled) Drop-In: /nix/store/8q3psx2yj9y8092zq8pjlbhkgh7r31gv-system-units/service.d └─zzz-lxc-service.conf Active: failed (Result: exit-code) since Fri 2024-02-23 21:03:31 UTC; 519ms ago Process: 15945 ExecStart=firewall-start (code=exited, status=4) Main PID: 15945 (code=exited, status=4) CPU: 50ms Feb 23 21:03:30 nixos systemd[1]: Starting Firewall... Feb 23 21:03:31 nixos firewall-start[16073]: Warning: Extension icmp revision 0 not supported, missing kernel module? Feb 23 21:03:31 nixos firewall-start[16073]: iptables v1.8.10 (nf_tables): RULE_APPEND failed (No such file or directory): rule in chain nixos-fw Feb 23 21:03:31 nixos systemd[1]: firewall.service: Main process exited, code=exited, status=4/NOPERMISSION Feb 23 21:03:31 nixos systemd[1]: firewall.service: Failed with result 'exit-code'. Feb 23 21:03:31 nixos systemd[1]: Failed to start Firewall. warning: error(s) occurred while switching to the new configuration ``` I see that it is complaining about firewall so i am linking it to #525 which is what i faced up until now. I had to disable all the iptables firewalls in the host. Again not sure if related

I filed the above issue in the incus github issues. I was advised to report them here to get a wider audience. Basically something is not running correctly when i try to mix incus and nixos on.

github.com/lxc/incus

No Internet on Initial Setup; Instructions Unclear

opened 06:35AM - 23 Feb 24 UTC

closed 10:01PM - 23 Feb 24 UTC

ddanon

Incomplete

# Required information * Distribution: NixOS * Distribution version: Unstable * The output of "incus info" (attached because it's so long) # Issue description Following [the documentation](https://linuxcontainers.org/incus/docs/main/tutorial/first_steps/) , I set up my first Incus VM with Nixos Stable (images:nixos/23.11 [name] --vm). I see that three IPV6 addresses are assigned to the single NIC that is in the VM by default. Unfortunately, I am unable to reach the internet (which prevents me from changing configurations in NixOS). I'm including some of the relevant logs on this initial post, but not going too far overboard into detail because I assume it's a simple fix. Maybe all that is needed is a little clarification in the docs? # Steps to reproduce 0. Start state = NixOS with no incus config 1. add `incus.enable = true;` to nixos config 2. add `"incus-admin"` group to a user (if necessary) 3. run `sudo incus admin init` 4. run `incus launch images:nixos/23.11 [name] --vm` 5. find that secureboot needs to be disabled; run `incus config set [name] security.secureboot=false` 6. run `incus start [name]` 7. run `incus exec [name] -- bash 8. in the vm now, run `ping 1.1.1.1`, see error `Host Unreachable` # Information to attach - [ ] Any relevant kernel output (`dmesg`) - [x] Container log (`incus info NAME --show-log`) ```shell $ incus info nixos-stable-vm --show-log Name: nixos-stable-vm Status: RUNNING Type: virtual-machine Architecture: x86_64 PID: 960399 Created: 2024/02/22 22:50 MST Last Used: 2024/02/22 22:58 MST Resources: Processes: 17 CPU usage: CPU usage (in seconds): 2 Memory usage: Memory (current): 233.39MiB Network usage: enp5s0: Type: broadcast State: UP Host interface: tap575b0379 MAC address: 00:16:3e:24:81:55 MTU: 1500 Bytes received: 11.08kB Bytes sent: 98.05kB Packets received: 81 Packets sent: 686 IP addresses: inet: 169.254.175.78/16 (link) inet6: fd42:ea3f:6012:3cd7:70d:ee3a:f046:92af/64 (global) inet6: fd42:ea3f:6012:3cd7:216:3eff:fe24:8155/64 (global) inet6: fe80::216:3eff:fe24:8155/64 (link) lo: Type: loopback State: UP MTU: 65536 Bytes received: 13.74kB Bytes sent: 13.74kB Packets received: 159 Packets sent: 159 IP addresses: inet: 127.0.0.1/8 (local) inet6: ::1/128 (local) Log: ``` - [x] Container configuration (`incus config show NAME --expanded`) ```shell $ incus config show nixos-stable-vm --expanded architecture: x86_64 config: image.architecture: amd64 image.description: Nixos 23.11 amd64 (20240223_01:03) image.os: Nixos image.release: "23.11" image.requirements.secureboot: "false" image.serial: "20240223_01:03" image.type: disk-kvm.img image.variant: default security.secureboot: "false" volatile.base_image: 2ed7aae7c079f5c96daa671347caf5ce8a42edf9ee6d9cb3d70b34627986c6ed volatile.cloud-init.instance-id: b3c99049-84b3-4bef-9da7-0f9f4e8d110b volatile.eth0.host_name: tap575b0379 volatile.eth0.hwaddr: 00:16:3e:24:81:55 volatile.last_state.power: RUNNING volatile.uuid: 67f8d7bf-725a-4fc5-a47b-261d9ae5a779 volatile.uuid.generation: 67f8d7bf-725a-4fc5-a47b-261d9ae5a779 volatile.vsock_id: "632919396" devices: eth0: nictype: bridged parent: incusbr0 type: nic root: path: / pool: default type: disk ephemeral: false profiles: - default stateful: false description: "" ``` - [ ] Main daemon log (at /var/log/incus/incusd.log) - [ ] Output of the client with --debug - [ ] Output of the daemon with --debug (alternatively output of `incus monitor --pretty` while reproducing the issue)

The above is another issue also being faced by another user. Its also the original issue which i was facing until i removed all the firewalling that is based on iptables altogether. I still dont know what firewall rules are required by incus on nixos.

I would appreciate any suggestions and help you can provide. If you have it already working please share your relevant nix configuration for hosts/guests.

@adamcstephens I am pinging you here since you offered Thank you so much for accepting that and providing your help. Its really much appreciated !

Best Regards