When running a Hive query, get the following error in the jobhistory:

MAP capability required is more than the supported max container capability in the cluster. 
Killing the Job. mapResourceRequest: <memory:1638, vCores:1> maxContainerCapability:<memory:1200, vCores:2>


This is caused by the following settings in YARN:

mapreduce.map.memory.mb => 1638
yarn.scheduler.maximum-allocation-mb => 1200
yarn.nodemanager.resource.memory-mb => 1300

The solution is to setup the settings mentioned above in the following way:

mapreduce.map.memory.mb < yarn.nodemanager.resource.memory-mb < yarn.scheduler.maximum-allocation-mb

Then the problem should be resolved.

Leave a Reply

Your email address will not be published. Required fields are marked *