I am trying to point iconv to a directory and all files will be converted UTF-8 regardless of the current encoding
I am using this script but you have to specify what encoding you are going FROM. How can I make it autdetect the current encoding?
dir_iconv.sh
#!/bin/bash ICONVBIN='/usr/bin/iconv' # path to iconv binary if [ $# -lt 3 ] then echo "$0 dir from_charset to_charset" exit fi for f in $1/* do if test -f $f then echo -e "nConverting $f" /bin/mv $f $f.old $ICONVBIN -f $2 -t $3 $f.old > $f else echo -e "nSkipping $f - not a regular file"; fi done
terminal line
sudo convert/dir_iconv.sh convert/books CURRENT_ENCODING utf8
Advertisement
Answer
Maybe you are looking for enca
:
Enca is an Extremely Naive Charset Analyser. It detects character set and encoding of text files and can also convert them to other encodings using either a built-in converter or external libraries and tools like libiconv, librecode, or cstocs.
Currently it supports Belarusian, Bulgarian, Croatian, Czech, Estonian, Hungarian, Latvian, Lithuanian, Polish, Russian, Slovak, Slovene, Ukrainian, Chinese, and some multibyte encodings independently on language.
Note that in general, autodetection of current encoding is a difficult process (the same byte sequence can be correct text in multiple encodings). enca
uses heuristics based on the language you tell it to detect (to limit the number of encodings). You can use enconv
to convert text files to a single encoding.