run in bash, not zsh! zsh will not recognize the comment correctly
kubectl get pods -A -o json |
jq -r '.items[].spec.containers[].image' |
sort -nr |
uniq |
grep '.*/.*/.*' | # images may omit a registry, in which case we'd incorrectly use the image name as the registry
cut -d '/' -f 1 |
uniq
kubectl get pod -A |
grep ContainerStatusUnknown |
awk '{print "-n "$1" "$2}' |
xargs -L1 kubectl delete pod
kubectl get pods -A -o json |
jq -r '.items[] | select(.status.reason!=null) | select(.status.reason | contains("Evicted")) | "-n \(.metadata.namespace) \(.metadata.name)"' |
xargs -L1 kubectl delete pod
docker login ghcr.iokubectl create secret generic ghcr --dry-run=client \
--from-file=.dockerconfigjson=$HOME/.config/docker/config.json \
--type=kubernetes.io/dockerconfigjson -o yaml > ghcr-pull-secret.local.yml
metadata:
annotations:
sealedsecrets.bitnami.com/cluster-wide: "true"
cat ghcr-pull-secret.local.yml | kubeseal -w ghcr-pull-secret.yml -o yaml & cleanup resulting filecontainerd-shim uses a ton of inotify user instances. This can be checked using this command (make sure to run as root!):
for foo in /proc/*/fd/*; do readlink -f $foo; done | grep inotify | cut -d/ -f3 | xargs -I '{}' -- ps --no-headers -o comm {} | sort | uniq -c | sort -nr
Source: https://github.com/k3s-io/k3s/issues/10325#issue-2340098457
To fix this issue, bump the inotify limits:
/etc/sysctl.conf:fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=524288
sysctl -psysctl fs.inotify.max_user_instancesSource:
https://github.com/k3s-io/k3s/issues/10325#issuecomment-2155008661
https://www.suse.com/support/kb/doc/?id=000020048