Recently when I was testing Sqoop1 command in my CDH cluster, I kept getting “java.io.IOException: No columns to generate for ClassWriter” error.

The full command was like below:

sqoop import --connect jdbc:mysql://<mysql-host>/test 
    --table test 
    --username <username> 
    --password <password> 
    --target-dir sqoop_test 
    -m 1

And full stacktrace:

16/08/20 03:03:13 ERROR manager.SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@7cd1be26 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@7cd1be26 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931)
	at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899)
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619)
	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569)
	at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1524)
	at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003)
	at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602)
	at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445)
	at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:305)
	at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
	at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246)
	at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327)
	at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1846)
	at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1646)
	at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
16/08/20 03:03:13 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter
	at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1652)
	at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:488)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

Although I do not have the RCA yet for this issue, I do have a workaround, which is adding “––driver com.mysql.jdbc.Driver” to the Sqoop parameters. So the full command becomes:

sqoop import --connect jdbc:mysql://<mysql-host>/test 
    --table test 
    --username <username> 
    --password <password> 
    --target-dir sqoop_test 
    -m 1 
    --driver com.mysql.jdbc.Driver

Hopefully this can help with anyone who might have the same issue.

4 Comments

  1. arun singh

    Hi I am new in bigdata and i am using –>sqoop import –connect jdbc:mysql://xxx.xxx.xx.xx:3306/FBI_DB –table FBI_CRIME –hbase-table H_FBI_CRIME –column-family cd –hbase-row-key ID –m 1 –username root -P. But I am not able to import data. Once I had used (––driver com.mysql.jdbc.Driver) but still didn’t get success . Always showing error —->>>ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter.

    Please help, What is wrong.

    1. Eric Lin

      Hi Arun,

      Thanks for visiting my blog and posting question. In order to find out why it is still not working for you, can you please share the full –verbose output of your Sqoop command output and email to my address eric at lin dot fit. I will need to check the full error details to be able to determine what else to look for.

      Cheers

Leave a Reply to srinivas Cancel reply

Your email address will not be published. Required fields are marked *