This command works fine in Local linux I would like to decompress a file with extension .csv.gz to HDFS location. I tried the below command and i get this error Could someone help me to fix this? Answer To make gzip write the output on standard output, use -c flag. So the command would be,

Error while decompress a file from Local Linux to HDFS

This command works fine in Local linux

gzip -d omega_data_path_2016-08-10.csv.gz

JavaScript
​x
 
gzip -d omega_data_path_2016-08-10.csv.gz​

I would like to decompress a file with extension .csv.gz to HDFS location.

I tried the below command and i get this error

 [cloudera@client08 localinputfiles]$ gzip -d omega_data_path_2016-08-10.csv.gz | hadoop dfs -put /user/cloudera/inputfiles/
gzip: omega_data_path_2016-08-10.csv already exists; do you wish to overwrite (y or n)? DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

put: `/user/cloudera/inputfiles/': No such file or directory

JavaScript
 
 [cloudera@client08 localinputfiles]$ gzip -d omega_data_path_2016-08-10.csv.gz | hadoop dfs -put /user/cloudera/inputfiles/gzip: omega_data_path_2016-08-10.csv already exists; do you wish to overwrite (y or n)? DEPRECATED: Use of this script to execute hdfs command is deprecated.Instead use the hdfs command for it.​put: `/user/cloudera/inputfiles/': No such file or directory​

Could someone help me to fix this?

Answer

To make gzip write the output on standard output, use -c flag.

So the command would be,

gzip -dc omega_data_path_2016-08-10.csv.gz | hdfs dfs -put - /user/cloudera/omega_data_path_2016-08-10.csv

JavaScript
 
gzip -dc omega_data_path_2016-08-10.csv.gz | hdfs dfs -put - /user/cloudera/omega_data_path_2016-08-10.csv​

Advertisement

Answer