I’m running an awk command in a shell script, and it’s breaking the datafile. All of the data doesn’t load, and there’s a newline character at the end of each line.
awk -v name="$filename" -v batch="$BATCHNAME" -F'|' 'BEGIN{OFS="|"}{$2=batch;print > (name".locked.concur")}' $filename.locked.concur
If I run this as a command, it works perfectly.
I’ve just found that i can use fflush() and it seems to have solved the issue.
So now i have:
awk -v name="$filename" -v batch="$BATCHNAME" -F'|' 'BEGIN{OFS="|"}{$2=batch;print > (name".locked.concur") fflush()}' $filename.locked.concur
Am I setting myself up for disaster?
Also, the file is renamed to $filename.locked.concur0 — I’m guessing this zero is the exit code that’s returned from fflush? How do I get rid of this 0?
full code:
for filename in `cat ${INFILEDIR}/file_list_concur` do rm -f $OHAP_DAT/bad/$filename.bad ##Create Batchname based on filename BATCH_GROUP=`ls $filename.locked.concur |cut -d '_' -f 1` BATCH_DATE=`ls $filename.locked.concur |cut -d '_' -f 2` BATCHNAME=${BATCH_GROUP}_Concur_${BATCH_DATE} echo 'Batch Date = '$BATCH_DATE echo 'Batch Group = '$BATCH_GROUP echo 'Batch Name = '$BATCHNAME echo 'File Name = ' $filename gawk -v name="$filename" -v batch="$BATCHNAME" -F'|' 'BEGIN{OFS="|"}{$2=batch;print > (name".locked.concur");}' $filename.locked.concur ##End Batchname sqlldr $ORACLE_PW control=$XXOH_TOP/bin/XXOH_AP_CONCUR_IMPORT_CTL.ctl data=$OHAP_DAT/in/$filename.locked.concur log=$OHAP_DAT/log/$filename.log bad=$OHAP_DAT/bad/$filename.bad bindsize=512000 << end_of_sqlldr end_of_sqlldr if [ -f $OHAP_DAT/bad/$filename.bad ] then echo !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! echo !! 'date' echo !! SQL*Loader bad file was found!! echo !! Script $filename Failed - BAD FILE! echo !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Advertisement
Answer
the problem is indeed that you are reading from and writing to the same file. this should work better:
gawk ... print > (name".locked.concur.tmp")}' $filename.locked.concur mv -f $filename.locked.concur.tmp $filename.locked.concur
better still:
gawk ... print}' $filename.locked.concur > $filename.locked.concur.tmp mv -f $filename.locked.concur.tmp $filename.locked.concur