Impala Reported Corrupt Parquet File After Failed With OutOfMemory Error

Recently I was dealing with an issue that impala reported Corrupt Parquet File after it failed with OutOfMemory error, however, if it does not fail, no corruption will be reported. See below error message reportd in Impala Daemon logs: Memory limit exceeded HdfsParquetScanner::ReadDataPage() failed to allocate 65535 bytes for decompressed …

How to Use Beeline to connect to Impala

You can certainly connect to Impala using Hive Driver from beeline, like below command: beeline -u 'jdbc:hive2://<impala-daemon-host>:21050/default;auth=noSasl' However, the result output format does not work properly: > show tables; customers dim_prod mansi sample_07 sample_08 small web_logs +——-+–+ | name | +——-+–+ +——-+–+ Notice the output is not inside the columns? …

Impala query failed with error “IllegalStateException”

This article examples ONE of the possible causes for the issue that Impala query failed with IllegalStateException error. Recently I was dealing with an Impala issue that when runnnig a simple SELECT query against a table failed with IllegalStateException error: SELECT * FROM <table_name>; Query: SELECT * FROM <table_name> ERROR: IllegalStateException: …

How to confirm Dynamic Partition Pruning works in Impala

This article explains how to confirm Impala’s new Dynamic Partition Pruning feature is effective in CDH5.7.x. Dynamic Partition Pruning is a new feature introduced from CDH5.7.x / Impala 2.5, where information about the partition is collected during run time and impala prunes unnecessary partitions in the ways that were impractical …