# Pastebin TDSZOR2D [ 172.246803] ironic-python-agent[534]: 2025-02-24 21:21:35.263 534 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): podman pull --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest execute /opt/ironic-python-agent/lib/python3.11/site-packages/oslo_concurrency/processutils.py:349 [ 179.538925] podman[627]: 2025-02-24 21:21:36.488863966 +0000 UTC m=+0.941195445 image pull docker://172.24.4.1:5000/cleaning-image:latest [ 179.575871] systemd[1]: var-lib-containers-storage-overlay.mount: Deactivated successfully. [ 179.598178] ironic-python-agent[534]: 2025-02-24 21:21:42.618 534 DEBUG oslo_concurrency.processutils [-] CMD "podman pull --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest" returned: 0 in 7.355s execute /opt/ironic-python-agent/lib/python3.11/site-packages/oslo_concurrency/processutils.py:372 [ 179.607030] ironic-python-agent[534]: 2025-02-24 21:21:42.629 534 DEBUG ironic_python_agent.utils [-] Command stdout is: "d6cda735b4ad39bdec6f0fef913da14bfe15afa0dec4441ef9aa4367e9c1abdd [ 179.608732] ironic-python-agent[534]: " _log /opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/utils.py:106 [ 179.609674] ironic-python-agent[534]: 2025-02-24 21:21:42.630 534 DEBUG ironic_python_agent.utils [-] Command stderr is: "Trying to pull 172.24.4.1:5000/cleaning-image:latest... [ 179.610541] ironic-python-agent[534]: Getting image source signatures [ 179.611628] ironic-python-agent[534]: Copying blob sha256:b0f6f1c319a1570f67352d490370f0aeb5c0e67a087baf2d5f301ad51ec18858 [ 179.612809] ironic-python-agent[534]: Copying config sha256:d6cda735b4ad39bdec6f0fef913da14bfe15afa0dec4441ef9aa4367e9c1abdd [ 179.621324] ironic-python-agent[534]: Writing manifest to image destination [ 179.622935] ironic-python-agent[534]: Storing signatures [ 179.623572] ironic-python-agent[534]: " _log /opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/utils.py:113 [ 179.629540] ironic-python-agent[534]: 2025-02-24 21:21:42.632 534 DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest execute /opt/ironic-python-agent/lib/python3.11/site-packages/oslo_concurrency/processutils.py:349 [ 180.401301] podman[642]: 2025-02-24 21:21:43.32542027 +0000 UTC m=+0.478486669 image pull docker://172.24.4.1:5000/cleaning-image:latest [ 180.521219] podman[642]: [ 180.608676] podman[642]: 2025-02-24 21:21:43.634798368 +0000 UTC m=+0.787864554 container create d16eae1fd511bbf6a2cb3835ff8e63e6e2b022bfdd238e686cb98157fb6837f1 (image=172.24.4.1:5000/cleaning-image:latest, name=cranky_driscoll, io.buildah.version=1.33.7) [ 180.831195] systemd[1]: var-lib-containers-storage-overlay-volatile\x2dcheck2396159-merged.mount: Deactivated successfully. [ 181.637168] systemd[1]: Created slice machine.slice - Virtual Machine and Container Slice. [ 181.682226] systemd[1]: Started libpod-d16eae1fd511bbf6a2cb3835ff8e63e6e2b022bfdd238e686cb98157fb6837f1.scope - libcrun container. [ 181.703077] systemd[1]: tmp-crun.DDKQiR.mount: Deactivated successfully. [ 181.952573] systemd[1]: libpod-d16eae1fd511bbf6a2cb3835ff8e63e6e2b022bfdd238e686cb98157fb6837f1.scope: Deactivated successfully. [ 182.401230] systemd[1]: tmp-crun.SUgUfU.mount: Deactivated successfully. [ 182.713153] systemd[1]: var-lib-containers-storage-overlay\x2dcontainers-d16eae1fd511bbf6a2cb3835ff8e63e6e2b022bfdd238e686cb98157fb6837f1-userdata-shm.mount: Deactivated successfully. [ 182.771494] systemd[1]: var-lib-containers-storage-overlay-d7f5f75c1e86c83392a04d9e8304a25c5008f0a192e2acb726fea02c0b548779-merged.mount: Deactivated successfully. [ 183.137907] podman[642]: 2025-02-24 21:21:46.160901115 +0000 UTC m=+3.313967419 container remove d16eae1fd511bbf6a2cb3835ff8e63e6e2b022bfdd238e686cb98157fb6837f1 (image=172.24.4.1:5000/cleaning-image:latest, name=cranky_driscoll, io.buildah.version=1.33.7) [ 183.179655] ironic-python-agent[534]: 2025-02-24 21:21:46.200 534 DEBUG oslo_concurrency.processutils [-] CMD "podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest" returned: 126 in 3.568s execute /opt/ironic-python-agent/lib/python3.11/site-packages/oslo_concurrency/processutils.py:372 [ 183.189399] ironic-python-agent[534]: 2025-02-24 21:21:46.214 534 DEBUG oslo_concurrency.processutils [-] 'podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest' failed. Not Retrying. execute /opt/ironic-python-agent/lib/python3.11/site-packages/oslo_concurrency/processutils.py:423 [ 183.200321] ironic-python-agent[534]: 2025-02-24 21:21:46.225 534 DEBUG ironic_python_agent.utils [-] Command stdout is: "" _log /opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/utils.py:106 [ 183.212922] ironic-python-agent[534]: 2025-02-24 21:21:46.238 534 DEBUG ironic_python_agent.utils [-] Command stderr is: "Error: OCI runtime error: crun: pivot_root: Invalid argument [ 183.214220] ironic-python-agent[534]: " _log /opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/utils.py:113 [ 183.311474] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root [-] Error during cleanup: Unexpected error while running command. [ 183.313351] ironic-python-agent[534]: Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.314662] ironic-python-agent[534]: Exit code: 126 [ 183.315274] ironic-python-agent[534]: Stdout: '' [ 183.315854] ironic-python-agent[534]: Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. [ 183.321024] ironic-python-agent[534]: Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.321877] ironic-python-agent[534]: Exit code: 126 [ 183.322498] ironic-python-agent[534]: Stdout: '' [ 183.323068] ironic-python-agent[534]: Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n' [ 183.330331] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root Traceback (most recent call last): [ 183.331124] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root File "/opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/hardware_managers/container.py", line 126, in manage_container_cleanup -- [ 183.355362] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root result = processutils.execute(*cmd, **kwargs) [ 183.359450] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [ 183.365381] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root File "/opt/ironic-python-agent/lib/python3.11/site-packages/oslo_concurrency/processutils.py", line 388, in execute [ 183.367280] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root raise ProcessExecutionError(exit_code=_returncode, [ 183.368040] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. [ 183.369110] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.369948] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root Exit code: 126 [ 183.371090] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root Stdout: '' [ 183.372177] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n' [ 183.377386] ironic-python-agent[534]: 2025-02-24 21:21:46.250 534 ERROR root [ 183.418646] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root [-] Unexpected error dispatching manage_container_cleanup to manager : Unexpected error while running command. [ 183.420399] ironic-python-agent[534]: Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.422147] ironic-python-agent[534]: Exit code: 126 [ 183.422761] ironic-python-agent[534]: Stdout: '' [ 183.423824] ironic-python-agent[534]: Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. [ 183.429919] ironic-python-agent[534]: Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.430795] ironic-python-agent[534]: Exit code: 126 [ 183.431387] ironic-python-agent[534]: Stdout: '' [ 183.431952] ironic-python-agent[534]: Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n' [ 183.441549] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root Traceback (most recent call last): [ 183.444032] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root File "/opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/hardware.py", line 3662, in dispatch_to_managers -- [ 183.474744] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root result = processutils.execute(*cmd, **kwargs) [ 183.476006] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [ 183.482406] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root File "/opt/ironic-python-agent/lib/python3.11/site-packages/oslo_concurrency/processutils.py", line 388, in execute [ 183.483362] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root raise ProcessExecutionError(exit_code=_returncode, [ 183.488472] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. [ 183.489685] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.490558] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root Exit code: 126 [ 183.497133] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root Stdout: '' [ 183.498018] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n' [ 183.498903] ironic-python-agent[534]: 2025-02-24 21:21:46.347 534 ERROR root [ 183.504938] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root [-] Unexpected exception performing clean step manage_container_cleanup. ProcessExecutionError: Unexpected error while running command. [ 183.506183] ironic-python-agent[534]: Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.507980] ironic-python-agent[534]: Exit code: 126 [ 183.513642] ironic-python-agent[534]: Stdout: '' [ 183.514892] ironic-python-agent[534]: Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n': oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. [ 183.515828] ironic-python-agent[534]: Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.521232] ironic-python-agent[534]: Exit code: 126 [ 183.521962] ironic-python-agent[534]: Stdout: '' [ 183.523218] ironic-python-agent[534]: Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n' [ 183.528997] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root Traceback (most recent call last): [ 183.530253] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root File "/opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/extensions/clean.py", line 76, in execute_clean_step -- [ 183.573592] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root result = processutils.execute(*cmd, **kwargs) [ 183.574392] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [ 183.575600] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root File "/opt/ironic-python-agent/lib/python3.11/site-packages/oslo_concurrency/processutils.py", line 388, in execute [ 183.582233] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root raise ProcessExecutionError(exit_code=_returncode, [ 183.583062] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. [ 183.583885] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.589019] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root Exit code: 126 [ 183.589774] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root Stdout: '' [ 183.590506] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n' [ 183.591284] ironic-python-agent[534]: 2025-02-24 21:21:46.453 534 ERROR root [ 183.610050] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root [-] Command failed: execute_clean_step, error: Clean step failed: Unexpected exception performing clean step manage_container_cleanup. ProcessExecutionError: Unexpected error while running command. [ 183.611467] ironic-python-agent[534]: Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.612515] ironic-python-agent[534]: Exit code: 126 [ 183.614002] ironic-python-agent[534]: Stdout: '' [ 183.614631] ironic-python-agent[534]: Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n': ironic_python_agent.errors.CleaningError: Clean step failed: Unexpected exception performing clean step manage_container_cleanup. ProcessExecutionError: Unexpected error while running command. [ 183.623961] ironic-python-agent[534]: Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.626161] ironic-python-agent[534]: Exit code: 126 [ 183.626844] ironic-python-agent[534]: Stdout: '' [ 183.627411] ironic-python-agent[534]: Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n' [ 183.631890] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root Traceback (most recent call last): [ 183.636958] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root File "/opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/extensions/clean.py", line 76, in execute_clean_step -- [ 183.681639] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root result = processutils.execute(*cmd, **kwargs) [ 183.683976] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [ 183.691568] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root File "/opt/ironic-python-agent/lib/python3.11/site-packages/oslo_concurrency/processutils.py", line 388, in execute [ 183.695313] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root raise ProcessExecutionError(exit_code=_returncode, [ 183.698968] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. [ 183.699972] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.703790] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root Exit code: 126 [ 183.706605] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root Stdout: '' [ 183.707794] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n' [ 183.714370] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root [ 183.715140] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root During handling of the above exception, another exception occurred: -- [ 183.728867] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root result = self.execute_method(**self.command_params) [ 183.730146] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [ 183.731357] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root File "/opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/extensions/clean.py", line 87, in execute_clean_step [ 183.737217] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root raise errors.CleaningError(msg) [ 183.738227] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root ironic_python_agent.errors.CleaningError: Clean step failed: Unexpected exception performing clean step manage_container_cleanup. ProcessExecutionError: Unexpected error while running command. [ 183.739346] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root Command: podman run --rm --network=host --tls-verify=false docker://172.24.4.1:5000/cleaning-image:latest [ 183.744821] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root Exit code: 126 [ 183.745586] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root Stdout: '' [ 183.746723] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root Stderr: 'Error: OCI runtime error: crun: pivot_root: Invalid argument\n' [ 183.747930] ironic-python-agent[534]: 2025-02-24 21:21:46.529 534 ERROR root [ 187.203468] ironic-python-agent[534]: 2025-02-24 21:21:50.224 534 DEBUG ironic_python_agent.ironic_api_client [-] Heartbeat: announcing callback URL https://10.0.0.28:9999, API version is 1.68 heartbeat /opt/ironic-python-agent/lib/python3.11/site-packages/ironic_python_agent/ironic_api_client.py:186