`
XMaster
  • 浏览: 36832 次
  • 性别: Icon_minigender_1
  • 来自: 北京
社区版块
存档分类
最新评论

Hive memory error:beyond the cluster container capability

    博客分类:
  • hive
 
阅读更多
问题:
java.lang.Exception: java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex's TaskResource is beyond the cluster container capability,Vertex=vertex_1517380657411_0232_1_00 [Map 1], Requested TaskResource=<memory:3750, vCores:1>, Cluster MaxContainerCapability=<memory:2250, vCores:8>


look at the values for following two variables mapreduce.map.memory.mb and mapreduce.reduce.memory.mb

and you could make the vaules as same as the container size.

there is nice HCC article explaining how this allocations work here

alternately if you don't have any more resources you may use the mr-engine instead of Tez ( though it is slow will complete the tasks with lesser concurrent memory utilization) // to change the engine run "hive.execution.engine=mr" for this query.
分享到:
评论

相关推荐

Global site tag (gtag.js) - Google Analytics