Improve Hive Memory Usage using Hadoop Archive
Hadoop hdfs is designed in such a way that, number of hdfs files directly affects the memory consumption in the namenode as it must keep track of all files in the hdfs environment. It does not affect if cluster is small, memory usage may cause problem on cluster when file count crosses 50 to 100 million files. Hadoop ecosystem performs best with fewer number of files. Now, let us check Improve Hive Memory Usage using Hadoop Archive. Related reading: Hadoop HDFS Architecture Improve Hive Memory Usage using Hadoop Archive You can…