Keras - .flow_from_directory(directory)
According the Keras documentation.
flow_from_directory(directory), Description:Takes the path to a directory, and generates batches of augmented/normalized data. Yields batches indefinitely, in an infinite loop.
With shuffle = False, it takes the same batch indefinitely. leading to these accuracy values. I changed shuffle = True and it works fine now.