flume不能上传文件到hdfs中,主要是telnet连接不上,请求老师帮助
相关截图:
相关代码:
2022-04-28 09:54:02,806 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:95)] Component type: SINK, name: k1 started 2022-04-28 09:54:02,806 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.source.SpoolDirectorySource.start(SpoolDirectorySource.java:85)] SpoolDirectorySource source starting with directory: /data/log/studentDir 2022-04-28 09:54:02,820 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.register(MonitoredCounterGroup.java:119)] Monitored counter group for type: SOURCE, name: r1: Successfully registered new MBean. 2022-04-28 09:54:02,821 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:95)] Component type: SOURCE, name: r1 started 2022-04-28 09:54:02,827 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.hdfs.HDFSDataStream.configure(HDFSDataStream.java:57)] Serializer = TEXT, UseRawLocalFileSystem = false 2022-04-28 09:54:02,943 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:246)] Creating hdfs://192.168.159.129:9000/flume/studentDir/stu-.1651110842828.avro.tmp 2022-04-28 09:54:04,069 (Thread-8) [INFO - org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.checkTrustAndSend(SaslDataTransferClient.java:239)] SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false 2022-04-28 09:54:32,364 (Log-BackgroundWorker-c1) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFile.beginCheckpoint(EventQueueBackingStoreFile.java:230)] Start checkpoint for /opt/apps/apache-flume-1.9.0-bin/data/studentDir/checkpointDir/checkpoint, elements to sync = 6 2022-04-28 09:54:32,368 (Log-BackgroundWorker-c1) [INFO - org.apache.flume.channel.file.EventQueueBackingStoreFile.checkpoint(EventQueueBackingStoreFile.java:255)] Updating checkpoint metadata: logWriteOrderID: 1651110842399, queueSize: 0, queueHead: 4 2022-04-28 09:54:32,370 (Log-BackgroundWorker-c1) [INFO - org.apache.flume.channel.file.Log.writeCheckpoint(Log.java:1065)] Updated checkpoint for file: /opt/apps/apache-flume-1.9.0-bin/data/studentDir/data/log-8 position: 282 logWriteOrderID: 1651110842399 2022-04-28 09:54:32,370 (Log-BackgroundWorker-c1) [INFO - org.apache.flume.channel.file.LogFile$RandomReader.close(LogFile.java:520)] Closing RandomReader /opt/apps/apache-flume-1.9.0-bin/data/studentDir/data/log-1 2022-04-28 09:54:32,378 (Log-BackgroundWorker-c1) [INFO - org.apache.flume.channel.file.LogFile$RandomReader.close(LogFile.java:520)] Closing RandomReader /opt/apps/apache-flume-1.9.0-bin/data/studentDir/data/log-2 2022-04-28 09:54:32,384 (Log-BackgroundWorker-c1) [INFO - org.apache.flume.channel.file.LogFile$RandomReader.close(LogFile.java:520)] Closing RandomReader /opt/apps/apache-flume-1.9.0-bin/data/studentDir/data/log-3 2022-04-28 09:54:32,390 (Log-BackgroundWorker-c1) [INFO - org.apache.flume.channel.file.LogFile$RandomReader.close(LogFile.java:520)] Closing RandomReader /opt/apps/apache-flume-1.9.0-bin/data/studentDir/data/log-4 2022-04-28 09:54:32,397 (Log-BackgroundWorker-c1) [INFO - org.apache.flume.channel.file.LogFile$RandomReader.close(LogFile.java:520)] Closing RandomReader /opt/apps/apache-flume-1.9.0-bin/data/studentDir/data/log-5 2022-04-28 09:54:32,404 (Log-BackgroundWorker-c1) [INFO - org.apache.flume.channel.file.LogFile$RandomReader.close(LogFile.java:520)] Closing RandomReader /opt/apps/apache-flume-1.9.0-bin/data/studentDir/data/log-6
11
收起
正在回答
1回答
先解决第1个问题:
这里提示无法识别node3这个主机名,需要检查下node1这个节点的/etc/hosts中node3是否配置,以及对应的ip是否正确。
最后还要检查下其他所有节点中的/etc/hosts中的信息,需要保持一致。
大数据工程师 2024 版
- 参与学习 1151 人
- 提交作业 5960 份
- 解答问题 1144 个
不用Java初级内容充数!不用与大数据岗位无关内容占课时!我们做的就是“精华版”大数据课程
了解课程
恭喜解决一个难题,获得1积分~
来为老师/同学的回答评分吧
0 星