How to configue session timeout in Hive

This article explains how to configure the following settings in Hive: hive.server2.session.check.interval hive.server2.idle.operation.timeout hive.server2.idle.session.timeout 1). hive.server2.idle.session.timeout Session will be closed when not accessed for this duration of time, in milliseconds; disable by setting to zero or a negative value. For example, the value of “86400000” indicate that the session will …

How to create a Hive multi-character delimitered table

We have the following data: ROW1~|`900~|`3103~|`2015~|`AA~|`1002003829 ROW2~|`900~|`3103~|`2015~|`BB~|`1002005103 ROW3~|`900~|`3103~|`2015~|`CC~|`1002003829 ROW4~|`900~|`3103~|`2015~|`DD~|`1002003829 ROW5~|`900~|`3103~|`2015~|`FF~|`1002003829 How to load the data into a Hive table with delimiter “~|`”? Well, it is pretty straightforward, just use the “MultiDelimitSerDe” which is available since CDH5.1.4, example as folllows: CREATE TABLE test_multi (a string, b string, c string, d string, …

How to drop Hive’s default partition (__HIVE_DEFAULT_PARTITION__) with “int” partition column

We know that Hive will create a partition with value “__HIVE_DEFAULT_PARTITION__” when running in dynamic partition mode and the value for the partition key is “null” value. However, depending on on the partition column type, you might not be able to drop those partitions due to restrictions in the Hive …

Application log shows container “is running beyond physical memory limits” error

This post explain what you need to do when you see container “is running beyond physical memory limits” error in the hive application log. Symptoms: Hive query failed with the following error in the application log: 2015-07-06 09:53:30,004 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1435155934213_16721_m_002655_0: Container [pid=46028,containerID=container_e54_1435155934213_16721_01_003666] is …