sqoop导入数据到hdfs报错

sqoop导入数据到hdfs报错

sqoop import --connect jdbc:mysql://192.168.56.1:3306/blade

尝试过的解决方式:

?serverTimezone=UTC --username root --password 123456 --table blade_code --target-dir /out1 --delete-target-dir --num-mappers 1 --fields-terminated-by '\t'

Warning: /data/soft/sqoop-1.4.7.bin__hadoop-2.6.0/../hbase does not exist! HBase imports will fail.

Please set $HBASE_HOME to the root of your HBase installation.

Warning: /data/soft/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

Warning: /data/soft/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.

Please set $ACCUMULO_HOME to the root of your Accumulo installation.

Warning: /data/soft/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.

Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.

2021-08-01 17:32:42,692 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7

2021-08-01 17:32:42,778 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

2021-08-01 17:32:42,897 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.

2021-08-01 17:32:42,897 INFO tool.CodeGenTool: Beginning code generation

Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.

2021-08-01 17:34:50,476 ERROR manager.SqlManager: Error executing statement: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure


The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure


The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

        at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)

        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)

        at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:835)

        at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:455)

        at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:240)

        at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:199)

        at java.sql.DriverManager.getConnection(DriverManager.java:664)

        at java.sql.DriverManager.getConnection(DriverManager.java:247)

        at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:904)

        at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:59)

        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)

        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)

        at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)

        at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)

        at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246)

        at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327)

        at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1872)

        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1671)

        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)

        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)

        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)

        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)

        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)

        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)

        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure


The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

        at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)

        at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)

        at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)

        at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167)

        at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:91)

        at com.mysql.cj.NativeSession.connect(NativeSession.java:152)

        at com.mysql.cj.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:955)

        at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:825)

        ... 24 more

Caused by: java.net.ConnectException: 连接超时 (Connection timed out)

        at java.net.PlainSocketImpl.socketConnect(Native Method)

        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)

        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)

        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)

        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)

        at java.net.Socket.connect(Socket.java:589)

        at com.mysql.cj.protocol.StandardSocketFactory.connect(StandardSocketFactory.java:155)

        at com.mysql.cj.protocol.a.NativeSocketConnection.connect(NativeSocketConnection.java:65)

        ... 27 more

2021-08-01 17:34:50,481 ERROR tool.ImportTool: Import failed: java.io.IOException: No columns to generate for ClassWriter

        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1677)

        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)

        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)

        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)

        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)

        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)

        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)

        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)

        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

尝试过的解决方式:

重启本地mysql服务 但是还是访问不上数据库  按照老师给的代码增加了远程访问的权限 但是还是不起作用 是因为我没有在这台节点上安装启动hive的原因?


正在回答

登陆购买课程后可参与讨论,去登陆

1回答

错误提示的是连不上mysql

sqoop命令会转化成mapreduce任务,最终会提交到hadoop集群中运行,所以需要确保能够在hadoop集群的所有节点上都能连上mysql才行

可能的问题是下面两个
1.没有开启mysql的远程访问权限,在hive课程中的安装部署那一块讲了如何开放mysql的远程访问权限,具体操作在电子书中
2.有可能是windows的防火墙没有关闭,这个也是必须要关闭的

  • 慕哥9561112 提问者 #1

    多谢老师 我忘了windows自身的防火墙了  我在另外一个虚拟机搭建了一个mysql服务 通过虚拟机之间的网络联通是可以进行sqoop导入数据的  多谢老师 很久没有在windows本上通过虚拟机操作服务器了

    2021-08-01 21:03:47
问题已解决,确定采纳
还有疑问,暂不采纳

恭喜解决一个难题,获得1积分~

来为老师/同学的回答评分吧

0 星
请稍等 ...
意见反馈 帮助中心 APP下载
官方微信

在线咨询

领取优惠

免费试听

领取大纲

扫描二维码,添加
你的专属老师