hadoop fs -getmerge

The hadoop fs -getmerge command is used to merge the files present in a directory in Hadoop Distributed File System (HDFS) and store the merged output in a local file system.

Syntax:

hadoop fs -getmerge <src> <localdst> [addnl]

• <src>: Specifies the source directory in HDFS that contains the files to be merged.
• <localdst>: Specifies the local file system path where the merged output will be stored.
• [addnl]: Optional parameter that specifies if a newline character should be added between each file while merging.

Example:

Suppose we have three files present in a directory named input in HDFS:

input/file1.txt
input/file2.txt
input/file3.txt

To merge these files and store the output in a local file named output.txt, we can use the following command:

hadoop fs -getmerge input output.txt

This will merge the files file1.txt, file2.txt, and file3.txt present in the input directory and store the merged output in a local file named output.txt