How to resolve "iterator should return strings, not bytes" How to resolve "iterator should return strings, not bytes" python-3.x python-3.x

How to resolve "iterator should return strings, not bytes"


request.FILES gives you binary files, but the csv module wants to have text-mode files instead.

You need to wrap the file in a io.TextIOWrapper() instance, and you need to figure out the encoding:

from io import TextIOWrapperf = TextIOWrapper(request.FILES['filename'].file, encoding=request.encoding)

It'd probably be better if you took the charset parameter from the Content-Type header if provided; that is what the client tells you the character set is.

You cannot work around needing to know the correct encoding for the file data; you can force interpretation as ASCII, for example, by providing a errors keyword as well (setting it to 'replace' or 'ignore'), but that does lead to data loss:

f = TextIOWrapper(request.FILES['filename'].file, encoding='ascii', errors='replace')

Using TextIOWrapper will only work when using Django 1.11 and later (as this changeset added the required support). In earlier versions, you can monkey-patch the support in after the fact:

from django.core.files.utils import FileProxyMixinif not hasattr(FileProxyMixin, 'readable'):    # Pre-Django 1.11, add io.IOBase support, see    # https://github.com/django/django/commit/4f474607de9b470f977a734bdd47590ab202e778            def readable(self):        if self.closed:            return False        if hasattr(self.file, 'readable'):            return self.file.readable()        return True    def writable(self):        if self.closed:            return False        if hasattr(self.file, 'writable'):            return self.file.writable()        return 'w' in getattr(self.file, 'mode', '')    def seekable(self):        if self.closed:            return False        if hasattr(self.file, 'seekable'):            return self.file.seekable()        return True    FileProxyMixin.closed = property(        lambda self: not self.file or self.file.closed)    FileProxyMixin.readable = readable    FileProxyMixin.writable = writable    FileProxyMixin.seekable = seekable


In python 3, I used:

import csvfrom io import StringIOcsvf = StringIO(xls_file.read().decode())reader = csv.reader(csvf, delimiter=',')

xls_file being the file got from the POST form. I hope it helps.


Fuse your two methods, this never fails in Python 3.5.2 and Django 1.9

delimitador = list_delimitadores[int(request.POST['delimitador'])][1]try:    text = TextIOWrapper(request.FILES['csv_x'].file, encoding='utf-8 ', errors='replace')    reader = csv.reader(text, delimiter=delimitador)except:    text = StringIO(request.FILES['csv_x'].file.read().decode())    reader = csv.reader(text, delimiter=delimitador)