After several successes, Elasticsearch queries suddenly fail

every scroll request should use the most recent scroll id i.e scroll_id returned in previous scroll response.

Looking from the code excerpt looks like you are using the the scroll id from the first response probably changing that to use the most recent scroll_id should help

i.e. in the while block you would need $scrollID = $decoded->{ "_scroll_id" };

while ( $lengthOfArray > 0 ) {        $ua = LWP::UserAgent->new;        $serverEndpoint = "http://localhost:9200/_search/scroll?scroll=1m&scroll_id=" . $scrollID;        $req = HTTP::Request->new( POST => $serverEndpoint );        $req->header( 'content-type' => 'application/json' );        $req->content( $post_data );        #!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!                         $resp = $ua->request( $req );        #!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!        $myResults = $resp->content();        # Changes string (as returned by http request) to JSON format that is compatible with Perl        $decoded = JSON::XS::decode_json( $myResults );        $scrollID =  $decoded->{ "_scroll_id" };        @tempResponse = @{ $decoded->{ "hits" }{ "hits" } };        #print "@tempResponse\n";        @data = ( @data, @tempResponse );        $lengthOfArray = scalar @tempResponse;    }

probably this should help.