Get data from pandas into a SQL server with PYODBC

For the 'write to sql server' part, you can use the convenient to_sql method of pandas (so no need to iterate over the rows and do the insert manually). See the docs on interacting with SQL databases with pandas: http://pandas.pydata.org/pandas-docs/stable/io.html#io-sql

You will need at least pandas 0.14 to have this working, and you also need sqlalchemy installed. An example, assuming df is the DataFrame you got from read_table:

import sqlalchemyimport pyodbcengine = sqlalchemy.create_engine("mssql+pyodbc://<username>:<password>@<dsnname>")# write the DataFrame to a table in the sql databasedf.to_sql("table_name", engine)

See also the documentation page of to_sql.
More info on how to create the connection engine with sqlalchemy for sql server with pyobdc, you can find here:http://docs.sqlalchemy.org/en/rel_1_1/dialects/mssql.html#dialect-mssql-pyodbc-connect

But if your goal is to just get the csv data into the SQL database, you could also consider doing this directly from SQL. See eg Import CSV file into SQL Server

Python3 version using a LocalDB SQL instance:

from sqlalchemy import create_engineimport urllibimport pyodbcimport pandas as pddf = pd.read_csv("./data.csv")quoted = urllib.parse.quote_plus("DRIVER={SQL Server Native Client 11.0};SERVER=(localDb)\ProjectsV14;DATABASE=database")engine = create_engine('mssql+pyodbc:///?odbc_connect={}'.format(quoted))df.to_sql('TargetTable', schema='dbo', con = engine)result = engine.execute('SELECT COUNT(*) FROM [dbo].[TargetTable]')result.fetchall()

Yes, the bcp utility seems to be the best solution for most cases.

If you want to stay within Python, the following code should work.

from sqlalchemy import create_engineimport urllibimport pyodbcquoted = urllib.parse.quote_plus("DRIVER={SQL Server};SERVER=YOUR\ServerName;DATABASE=YOur_Database")engine = create_engine('mssql+pyodbc:///?odbc_connect={}'.format(quoted))df.to_sql('Table_Name', schema='dbo', con = engine, chunksize=200, method='multi', index=False, if_exists='replace')

Don't avoid method='multi', because it significantly reduces the task execution time.

Sometimes you may encounter the following error.

ProgrammingError: ('42000', '[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]The incoming request has too many parameters. The server supports a maximum of 2100 parameters. Reduce the number of parameters and resend the request. (8003) (SQLExecDirectW)')

In such a case, determine the number of columns in your dataframe: df.shape[1]. Divide the maximum supported number of parameters by this value and use the result's floor as a chunk size.