ETL - Automate Data Transfer from One Database to another ETL - Automate Data Transfer from One Database to another json json

ETL - Automate Data Transfer from One Database to another


Here's how we do that; we have customers that pay for our services in different banks. Each of those banks delivers TXT files that contains payments from the previous day; those files are located in a directory (folder).

There's the operating system scheduled job (on MS Windows, that's Task Scheduler) which runs every few minutes and check whether there's a new file in that directory.

If so, it runs a (MS DOS) batch script which calls Oracle SQL*Loader utility. It then loads data into an Oracle table and copies the TXT file into the archive directory.

Not exactly a rocket science, but - works for years now.

I'm sure that someone else will have different examples & suggestions.