Why can't you use cat to read a file line by line where each line has delimiters

The problem is not in cat, nor in the for loop per se; it is in the use of back quotes. When you write either:

for i in `cat file`

or (better):

for i in $(cat file)

or (in bash):

for i in $(<file)

the shell executes the command and captures the output as a string, separating the words at the characters in $IFS. If you want lines input to $i, you either have to fiddle with IFS or use the while loop. The while loop is better if there's any danger that the files processed will be large; it doesn't have to read the whole file into memory all at once, unlike the versions using $(...).

IFS=''for i in $(<file)do echo "$i"done

The quotes around the "$i" are generally a good idea. In this context, with the modified $IFS, it actually isn't critical, but good habits are good habits even so. It matters in the following script:

old="$IFS"IFS=''for i in $(<file)do   (   IFS="$old"   echo "$i"   )done

when the data file contains multiple spaces between words:

$ cat fileabc                  123,         commathe   quick   brown   foxjumped   over   the   lazy   dogcomma,   comma$ 

Output:

$ sh bq.shabc                  123,         commathe   quick   brown   foxjumped   over   the   lazy   dogcomma,   comma$

Without the double quotes:

$ cat bq.shold="$IFS"IFS=''for i in $(<file)do   (   IFS="$old"   echo $i   )done$ sh bq.shabc 123, commathe quick brown foxjumped over the lazy dogcomma, comma$

You can use IFS variable to specific you want a newline as the field separator:

IFS=$'\n'for i in `cat file`do   echo $idone

the for loop coupled with a change of the internal field separator(IFS) will read file as intended

for an input

abc 123, commathe quick brown foxjumped over the lazy dogcomma, comma

For loop coupled with an IFS change

old_IFS=$IFSIFS=$'\n'for i in `cat file`do        echo $idoneIFS=$old_IFS

results in

abc 123, commathe quick brown foxjumped over the lazy dogcomma, comma