This command works fine in Local linux
gzip -d omega_data_path_2016-08-10.csv.gz
I would like to decompress a file with extension .csv.gz to HDFS location.
I tried the below command and i get this error
[cloudera@client08 localinputfiles]$ gzip -d omega_data_path_2016-08-10.csv.gz | hadoop dfs -put /user/cloudera/inputfiles/ gzip: omega_data_path_2016-08-10.csv already exists; do you wish to overwrite (y or n)? DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it. put: `/user/cloudera/inputfiles/': No such file or directory
Could someone help me to fix this?
Advertisement
Answer
To make gzip
write the output on standard output, use -c
flag.
So the command would be,
gzip -dc omega_data_path_2016-08-10.csv.gz | hdfs dfs -put - /user/cloudera/omega_data_path_2016-08-10.csv