Kubernetes AWS Cloudwatch adapter not fetching custom metric value for EKS HPA autoscaling

Worked with OP on this out-of-band and still had the tab open for this question later in the day, so posting the outcome here for posterity for anyone that stumbles upon it.

The root cause of the issue was a timezone conflict. The metrics monitor was based on "current" metrics, but the the following line from the metric generator script was producing time stamps without a timezone specified and also was in a local timezone.

            'Timestamp': datetime.now(),

Since there was "no data" for the current timezone (only data X hours in the past due to a -X UTC offset), the system did not initiate scaling because there was a value of "0"/nil/null effectively. Instead, a UTC time string can be specified to ensure the generated metrics are timely:

            'Timestamp': datetime.utcnow(),

A secondary consideration was that the Kubernetes Nodes need access to poll the metrics from CloudWatch. This is done by attaching this policy to the nodes's IAM role:

{    "Version": "2012-10-17",    "Statement": [        {            "Effect": "Allow",            "Action": [                "cloudwatch:GetMetricData"            ],            "Resource": "*"        }    ]}