Every now and then, a host in your Zabbix system will turn unavailable. It results in notifications like this:
Trigger: Zabbix agent on myhost is unreachable for 5 minutes
Trigger status: PROBLEM
And logs like this:
1461:20140808:074300.517 Zabbix agent item "vfs.fs.size[/home/deploy/sites/mc/shared/voip,free]" on host "myhost" failed: first network error, wait for 15 seconds
1466:20140808:074345.134 temporarily disabling Zabbix agent checks on host "myhost": host unavailable
What could be a cause? No, it’s not because a host could not be reached. That would be too easy.
Once upon a time, you need to send a value to Zabbix from a script. Zabbix_sender comes to the rescue:
zabbix_sender -z "server IP address" -p 10051 -s "host in zabbix" -k "item key" -o "item value"
But, what happens if for some reason Zabbix won’t accept the value (say, wrong item type?). It’ll just fail (non-zero), without any error message. Well, let’s try to debug (notice the added “-vv”):
zabbix_sender -vv -z "server IP address" -p 10051 -s "host in zabbix" -k "item key" -o "item value"
Info from server: "Processed 1 Failed 1 Total 1 Seconds spent 0.003079"
sent: 1; skipped: 0; total: 1
Oh wow, that’s helpful. I already know it’s failed. I want to know why.