本文根据flume官网提供的Flume 1.8.0 User Guide整理得出的,主要内容是总结了flume1.8.0支持的各种sink,见下表。
sink名称 | type | 数据目的地 | 工作方式 | 备注 |
HDFS Sink | hdfs | HDFS | ||
Hive Sink | hive | hive | ||
Logger Sink | logger | 自定义 | 日志级别为INFO,主要用于调试 | |
Avro Sink | avro | 外部avro客户端 | 将接收到的events转换成avro events 并发送给指定地址的avro客户端 | |
Thrift Sink | thrift | 外部thrift客户端 | 将接收到的events转换成thrift events 并发送给指定地址的avro客户端 | |
IRC Sink | irc | IRC目的地 | ||
File Roll Sink | file_roll | 本地文件系统 | ||
Null Sink | null | 无 | 丢弃所有数据 | |
Hbase Sink | hbase | Hbase | 这两个sink同属一个大类:HbaseSinks | |
AsyncHbaseSink | asynchbase | Hbase(异步模式) | ||
MorphlineSolrSink | org.apache.flume. sink.solr.morphline. MorphlineSolrSink | Apache Solr servers | This sink extracts data from Flume events, transforms it, and loads it in near-real-time into Apache Solr servers, which in turn serve queries to end users or search applications. | |
ElasticSearchSink | org.apache.flume. sink.elasticsearch. ElasticSearchSink | elasticsearch cluster | ||
Kite Dataset Sink | org.apache.flume. sink.kite.DatasetSink | Kite Dataset | ||
Kafka Sink | org.apache.flume. sink.kafka.KafkaSink | Kafka topic | ||
HTTP Sink | http | 一个URI地址(POST) | ||
Custom Sink | 自定义FQCN | 自定义 |
flume学习(三)---sink
原文地址:https://www.cnblogs.com/walkingzq/p/8423217.html