textfile类型小文件合并:
./2022-08-18/part-00199-e61cc02d-7c0b-4297-828d-10cb14e12c96.c000
合并text小文件:原文件不删除,产生新的合并文件。时间戳:yyyymmddhhMMss
hadoop jar filecrush-2.2.2-SNAPSHOT-jar-with-dependencies.jar com.m6d.filecrush.crush.Crush
-input-format text
-output-format text
–compress=none
./2022-08-18/ /tmp/999testall/ 20241220000001
合并结果:
hdfs dfs -ls /tmp/999testall/
/tmp/999testall/crushed_file-20241220000001-0-0