Pandas to_sql fails on duplicate primary key
There is unfortunately no option to specify "INSERT IGNORE". This is how I got around that limitation to insert rows into that database that were not duplicates (dataframe name is df)
for i in range(len(df)): try: df.iloc[i:i+1].to_sql(name="Table_Name",if_exists='append',con = Engine) except IntegrityError: pass #or any other action
please note that the "if_exists='append'" related to the existing of the table and what to do in case the table not exists.The if_exists don't related to the content of the table.see the doc here: https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.to_sql.html
if_exists : {‘fail’, ‘replace’, ‘append’}, default ‘fail’ fail: If table exists, do nothing. replace: If table exists, drop it, recreate it, and insert data. append: If table exists, insert data. Create if does not exist.
Pandas has no option for it currently, but here is the Github issue. If you need this feature too, just upvote for it.