Hallo liebe Leute, da der Thread ja noch nicht so alt ist und ich nicht extra einen neue aufmachen wollte, nutze ich diesen mal um vielleicht eine Lösung für mein Problem zu bekommen. Ich habe mir nach dieser (
https://mariushosting.com/monitor-your-synology-with-grafana-and-prometheus-dashboard/ ) Anleitung auch mal Grafana/Prometheus installiert. Hat auch alles soweit sehr gut geklappt und im Dashboard sind auch alle Systemwerte zu sehen. Doch leider werden mir meinen ganzen Docker-Container nicht angezeigt.
Im Portainer steht hinter dem Prometheus-cAdvisor Container exited - code 255 ( Neustart ohne Erfolg )
Die logs vom Container sehen so aus:
I1123 13:38:44.559577 1 cadvisor.go:123] enabled metrics: app,cpu,cpuLoad,disk,diskIO,memory,network,oom_event,percpu,perf_event
I1123 13:38:44.559649 1 storagedriver.go:55] Caching stats in memory for 2m0s
I1123 13:38:44.559785 1 manager.go:169] cAdvisor running in container: "/docker/e63dd2c9b7c0303401a9919ffddd777a49a1eb3ecffdba89bf510aca57f27eaa"
W1123 13:38:44.613725 1 fs.go:216] stat failed on /dev/vg1/volume_1 with error: no such file or directory
I1123 13:38:44.613964 1 fs.go:133] Filesystem UUIDs: map[]
3:38:44.613978 1 fs.go:134] Filesystem partitions: map[/dev:{mountpoint:/dev major:0 minor:514 fsType:tmpfs blockSize:0} /dev/loop0:{mountpoint:/rootfs/tmp/SynologyAuthService major:7 minor:0 fsType:ext4 blockSize:0} /dev/md0:{mountpoint:/rootfs major:9 minor:0 fsType:ext4 blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:517 fsType:tmpfs blockSize:0} /dev/vg1/volume_1:{mountpoint:/ major:0 minor:35 fsType:btrfs blockSize:0} /proc/acpi:{mountpoint:/proc/acpi major:0 minor:518 fsType:tmpfs blockSize:0} /proc/kcore:{mountpoint:/proc/kcore major:0 minor:514 fsType:tmpfs blockSize:0} /proc/keys:{mountpoint:/proc/keys major:0 minor:514 fsType:tmpfs blockSize:0} /proc/sched_debug:{mountpoint:/proc/sched_debug major:0 minor:514 fsType:tmpfs blockSize:0} /proc/scsi:{mountpoint:/proc/scsi major:0 minor:519 fsType:tmpfs blockSize:0} /proc/timer_list:{mountpoint:/proc/timer_list major:0 minor:514 fsType:tmpfs blockSize:0} /rootfs/dev/shm:{mountpoint:/rootfs/dev/shm major:0 minor:17 fsType:tmpfs blockSize:0} /rootfs/run:{mountpoint:/rootfs/run major:0 minor:18 fsType:tmpfs blockSize:0} /rootfs/run/user/196791:{mountpoint:/rootfs/run/user/196791 major:0 minor:34 fsType:tmpfs blockSize:0} /rootfs/sys/fs/cgroup:{mountpoint:/rootfs/sys/fs/cgroup major:0 minor:19 fsType:tmpfs blockSize:0} /rootfs/tmp:{mountpoint:/rootfs/tmp major:0 minor:30 fsType:tmpfs blockSize:0} /rootfs/volume1/
@Docker/btrfs/subvolumes/7fb3d6fada3eb2d371075a557edb38c194adc4449332ec1042067e5560e0c128/dev:{mountpoint:/rootfs/volume1/
@Docker/btrfs/subvolumes/7fb3d6fada3eb2d371075a557edb38c194adc4449332ec1042067e5560e0c128/dev major:0 minor:514 fsType:tmpfs blockSize:0} /rootfs/volume1/
@Docker/btrfs/subvolumes/7fb3d6fada3eb2d371075a557edb38c194adc4449332ec1042067e5560e0c128/dev/shm:{mountpoint:/rootfs/volume1/
@Docker/btrfs/subvolumes/7fb3d6fada3eb2d371075a557edb38c194adc4449332ec1042067e5560e0c128/dev/shm major:0 minor:517 fsType:tmpfs blockSize:0} /rootfs/volume1/
@Docker/btrfs/subvolumes/7fb3d6fada3eb2d371075a557edb38c194adc4449332ec1042067e5560e0c128/sys/fs/cgroup:{mountpoint:/rootfs/volume1/
@Docker/btrfs/subvolumes/7fb3d6fada3eb2d371075a557edb38c194adc4449332ec1042067e5560e0c128/sys/fs/cgroup major:0 minor:19 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:18 fsType:tmpfs blockSize:0} /run/docker.sock:{mountpoint:/run/docker.sock major:0 minor:18 fsType:tmpfs blockSize:0} /run/user/196791:{mountpoint:/run/user/196791 major:0 minor:34 fsType:tmpfs blockSize:0} /sys/firmware:{mountpoint:/sys/firmware major:0 minor:520 fsType:tmpfs blockSize:0} /sys/fs/cgroup:{mountpoint:/sys/fs/cgroup major:0 minor:19 fsType:tmpfs blockSize:0}]
I1123 13:38:44.617323 1 sysinfo.go:303] Found cpu without cache information, cpuPath: [/sys/devices/system/cpu/cpu0 /sys/devices/system/cpu/cpu1]
I1123 13:38:44.617722 1 manager.go:217] Machine: {Timestamp:2024-11-23 13:38:44.617474961 +0000 UTC m=+0.093555860 CPUVendorID:GenuineIntel NumCores:2 NumPhysicalCores:2 NumSockets:1 CpuFrequency:2001000 MemoryCapacity:18702553088 SwapCapacity:13369266176 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[] MachineID:3bc10ccffef54e58b900d5806c277a44 SystemUUID:12345678-1234-5678-90AB-CDDEEFAABBCC BootID:e6e8606c-fdb3-4401-9ede-03f6d017ce08 Filesystems:[{Device:/proc/scsi DeviceMajor:0 DeviceMinor:519 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/rootfs/run DeviceMajor:0 DeviceMinor:18 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/rootfs/volume1/
@Docker/btrfs/subvolumes/7fb3d6fada3eb2d371075a557edb38c194adc4449332ec1042067e5560e0c128/dev DeviceMajor:0 DeviceMinor:514 Capacity:67108864 Type:vfs Inodes:2283026 HasInodes:true} {Device:/run/user/196791 DeviceMajor:0 DeviceMinor:34 Capacity:1870258176 Type:vfs Inodes:2283026 HasInodes:true} {Device:/proc/acpi DeviceMajor:0 DeviceMinor:518 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/proc/timer_list DeviceMajor:0 DeviceMinor:514 Capacity:67108864 Type:vfs Inodes:2283026 HasInodes:true} {Device:/run/docker.sock DeviceMajor:0 DeviceMinor:18 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/proc/kcore DeviceMajor:0 DeviceMinor:514 Capacity:67108864 Type:vfs Inodes:2283026 HasInodes:true} {Device:/dev/vg1/volume_1 DeviceMajor:0 DeviceMinor:35 Capacity:19189226684416 Type:vfs Inodes:0 HasInodes:true} {Device:/dev DeviceMajor:0 DeviceMinor:514 Capacity:67108864 Type:vfs Inodes:2283026 HasInodes:true} {Device:/sys/fs/cgroup DeviceMajor:0 DeviceMinor:19 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/rootfs/run/user/196791 DeviceMajor:0 DeviceMinor:34 Capacity:1870258176 Type:vfs Inodes:2283026 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:18 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/rootfs/tmp DeviceMajor:0 DeviceMinor:30 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/dev/loop0 DeviceMajor:7 DeviceMinor:0 Capacity:28296192 Type:vfs Inodes:8192 HasInodes:true} {Device:/rootfs/volume1/
@Docker/btrfs/subvolumes/7fb3d6fada3eb2d371075a557edb38c194adc4449332ec1042067e5560e0c128/sys/fs/cgroup DeviceMajor:0 DeviceMinor:19 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/proc/keys DeviceMajor:0 DeviceMinor:514 Capacity:67108864 Type:vfs Inodes:2283026 HasInodes:true} {Device:/proc/sched_debug DeviceMajor:0 DeviceMinor:514 Capacity:67108864 Type:vfs Inodes:2283026 HasInodes:true} {Device:/sys/firmware DeviceMajor:0 DeviceMinor:520 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:517 Capacity:67108864 Type:vfs Inodes:2283026 HasInodes:true} {Device:/dev/md0 DeviceMajor:9 DeviceMinor:0 Capacity:8387944448 Type:vfs Inodes:524288 HasInodes:true} {Device:/rootfs/sys/fs/cgroup DeviceMajor:0 DeviceMinor:19 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/rootfs/dev/shm DeviceMajor:0 DeviceMinor:17 Capacity:9351274496 Type:vfs Inodes:2283026 HasInodes:true} {Device:/rootfs/volume1/
@Docker/btrfs/subvolumes/7fb3d6fada3eb2d371075a557edb38c194adc4449332ec1042067e5560e0c128/dev/shm DeviceMajor:0 DeviceMinor:517 Capacity:67108864 Type:vfs Inodes:2283026 HasInodes:true}] DiskMap:map[135:240:{Name:synoboot Major:135 Minor:240 Size:125829120 Scheduler:cfq} 248:0:{Name:dm-0 Major:248 Minor:0 Size:12582912 Scheduler:none} 248:1:{Name:dm-1 Major:248 Minor:1 Size:19988777795584 Scheduler:none} 249:0:{Name:zram0 Major:249 Minor:0 Size:5610930176 Scheduler:none} 249:1:{Name:zram1 Major:249 Minor:1 Size:5610930176 Scheduler:none} 8:0:{Name:sata1 Major:8 Minor:0 Size:20000588955648 Scheduler:cfq} 8:16:{Name:sata2 Major:8 Minor:16 Size:20000588955648 Scheduler:cfq} 9:0:{Name:md0 Major:9 Minor:0 Size:8589869056 Scheduler:none} 9:1:{Name:md1 Major:9 Minor:1 Size:2147418112 Scheduler:none} 9:2:{Name:md2 Major:9 Minor:2 Size:3989802582016 Scheduler:none} 9:3:{Name:md3 Major:9 Minor:3 Size:15999788908544 Scheduler:none}] NetworkDevices:[{Name:eth0 MacAddress:90:09:d0:25:be:5b Speed:1000 Mtu:1500} {Name:eth1 MacAddress:90:09:d0:25:be:5c Speed:65535 Mtu:1500} {Name:sit0 MacAddress:00:00:00:00 Speed:0 Mtu:1480}] Topology:[{Id:0 Memory:0 HugePages:[] Cores:[{Id:0 Threads:[0] Caches:[] UncoreCaches:[] SocketID:0} {Id:1 Threads:[1] Caches:[] UncoreCaches:[] SocketID:0}] Caches:[] Distances:[]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
I1123 13:38:44.618070 1 manager.go:233] Version: {KernelVersion:4.4.302+ ContainerOsVersion:Alpine Linux v3.18 DockerVersion: DockerAPIVersion: CadvisorVersion:v0.49.1 CadvisorRevision:6f3f25ba}
I1123 13:38:44.619197 1 factory.go:219] Registration of the podman container factory failed: failed to validate Podman info: response not present: Get "
http://d/v1.0.0/info": dial unix /var/run/podman/podman.sock: connect: no such file or directory
I1123 13:38:44.619248 1 factory.go:219] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
I1123 13:38:44.619269 1 factory.go:55] Registering systemd factory
I1123 13:38:44.619278 1 factory.go:221] Registration of the systemd container factory successfully
I1123 13:38:44.619354 1 factory.go:219] Registration of the crio container factory failed: Get "http://%2Fvar%2Frun%2Fcrio%2Fcrio.sock/info": dial unix /var/run/crio/crio.sock: connect: no such file or directory
I1123 13:38:44.620233 1 factory.go:219] Registration of the mesos container factory failed: unable to create mesos agent client: failed to get version
I1123 13:38:44.639791 1 factory.go:352] Registering Docker factory
I1123 13:38:44.639816 1 factory.go:221] Registration of the docker container factory successfully
I1123 13:38:44.639909 1 factory.go:103] Registering Raw factory
I1123 13:38:44.639988 1 manager.go:1196] Started watching for new ooms in manager
W1123 13:38:44.640005 1 manager.go:306] Could not configure a source for OOM detection, disabling OOM events: open /dev/kmsg: no such file or directory
W1123 13:38:44.640301 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/\""
I1123 13:38:44.640521 1 manager.go:319] Starting recovery of all containers
W1123 13:38:44.672700 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/51fb9e7161a0b53876927fb9982be4133bbfbfda8269c340a37dd4da8b7f1148\""
W1123 13:38:44.674728 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/29ce1b727ec46ccece7f06cc631bf93bd966ab23d6a982f116cf3bf2ba339393\""
W1123 13:38:44.676835 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/43d03c25a381134bd987b7e554aa5ddaa7a3d22fc1b028d21728b34e656590e7\""
W1123 13:38:44.678286 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/3c271bb07f4b37252c7d169e058092ce30c495b9fcda7293b6fc40ccb4bf9544\""
W1123 13:38:44.680148 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/b9669999deb92485b150c1b3357851c31c86003105924f6974847f1f33e9e495\""
W1123 13:38:44.682715 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/024613a42d18761ce30505fec484d4f23ac3dd6825289bb763329e605ff8e495\""
W1123 13:38:44.684575 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/9de95aff211462161d59fd180b4145324c8b106c8749d789e53a5369513f897a\""
W1123 13:38:44.686345 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/4ee87a34ca9565f078befe57e8c9aa165a196b1721327a6103255a53cea7d215\""
W1123 13:38:44.688244 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/e3f4c7cae5dd8472ffb769ef44a9ba90df323be02f8a9973e38fd559afb5c0dc\""
W1123 13:38:44.934931 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/e63dd2c9b7c0303401a9919ffddd777a49a1eb3ecffdba89bf510aca57f27eaa\""
W1123 13:38:44.946048 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/795e1b5ec1e6005dc693cb56f239dcd1684b428d103f335bf7a2f8b79eb5d487\""
W1123 13:38:44.957696 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/355bb4fcedb9a01b9d626888003be7c313d6a3666a713937f1691f9f1ff5ee46\""
W1123 13:38:44.970357 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/5d1be5248350b79b2678120c68ea48a3f8778862485b3565d367480aa563c75d\""
W1123 13:38:44.988828 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/ef8014deba59a37320e8a712a4817bb9e53608fc0ba633908f2f2f77939572a4\""
W1123 13:38:44.995599 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/68d0d38058c5ac8a5b9be827d5b13ba9f0473fa263f7912df2911a310a462eb2\""
W1123 13:38:45.004010 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/11ebc4b4653fc35e4cbcc32bbd84a9fd24394f7a1965ff91b4b2da8e5505efe6\""
W1123 13:38:45.006335 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/0798dd08597f835cc1f27f9877b9722836f849130f14b25cddd906fb785a6844\""
W1123 13:38:45.008400 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/9d301ca8af0b4ebfd1fec6b15c4890d9f22dceb4492d81112609d638a9116971\""
W1123 13:38:45.009957 1 manager.go:946] Error getting perf_event cgroup path: "could not find path for resource \"perf_event\" for container \"/docker/99b20e6c4ecc18e02a0b5498fc29012418f290d86e2f95bc4108a749889526ae\""
I1123 13:38:45.010146 1 manager.go:324] Recovery completed
E1123 13:38:45.014520 1 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/cpu/system.slice/syno-notify-bundle-drive-update.service": inotify_add_watch /sys/fs/cgroup/cpu/system.slice/syno-notify-bundle-drive-update.service: no space left on device
E1123 13:38:45.014593 1 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/cpu/system.slice": inotify_add_watch /sys/fs/cgroup/cpu/system.slice/syno-notify-bundle-drive-update.service: no space left on device
F1123 13:38:45.014616 1 cadvisor.go:170] Failed to start manager: inotify_add_watch /sys/fs/cgroup/cpu/system.slice/syno-notify-bundle-drive-update.service: no space left on device
Hat da evt. jemand eine Idee wie das Problem zu beheben ist?