alert when pod is restarted: Prometheus on windows containers
You can use kube-state-metrics like you said. From the Kubernetes control plane point of view, a pod/container restart is no different whether you are using Linux or Windows containers.
Keep in mind that the control plane is only supported on Linux so in case you only have Windows nodes on your cluster you can run the kube-state-metrics pod/container in your master(s), otherwise, you will need a Linux node. Alternatively, you can build the kube-state-metrics Windows Go binary and run it on a Windows pod/container, but that could be more troublesome.
You can use something like this with Alertmanager as an alert.