实践一：regex filter interceptor
1、目标场景
regex filter interceptor的作用：
1）将event body的内容和配置中指定的正则表达式进行匹配
2）如果内容匹配，则将该event丢弃
3）如果内容不匹配，则将该event放行
2、Flume Agent配置文件
# 01 define agent name, source/sink/channel a1.sources = r1a1.sinks = k1a1.channels = c1# 02 source,http,jsonhandlera1.sources.r1.type = httpa1.sources.r1.bind = mastera1.sources.r1.port = 6666a1.sources.r1.handler = org.apache.flume.source.http.JSONHandler# 03 regex filter interceptor, match event body for filtera1.sources.r1.interceptors = i1 ?a1.sources.r1.interceptors.i1.type = regex_filter ?a1.sources.r1.interceptors.i1.regex = ^[0-9]*$ # filter matched event a1.sources.r1.interceptors.i1.excludeEvents = true ?# 04 logger sinka1.sinks.k1.type = logger# 05 channel,memorya1.channels.c1.type = memorya1.channels.c1.capacity = 1000a1.channels.c1.transactionCapacity = 100# 06 bind source,sink to channela1.sources.r1.channels = c1a1.sinks.k1.channel = c1
3、验证regex filter interceptor
1) 通过curl -X POST -d ‘json数据‘ 发送带有不同body的HTTP请求，其中有1个满足regex
2）观察终端打印出的event，body为1234的event被过滤， 并没有出现
4、regex filter interceptor的官方文档
实践二：regex extractor interceptor
1、目标场景
regex extractor interceptor的作用：
1）将event body的内容和配置中指定的正则表达式进行匹配
2）如果内容匹配，将配合配置文件中给定的key, 组成key:value添加到event的header中
3）event body中的内容不会变化
2、Flume Agent的配置文件
# 01 define agent name, source/sink/channel a1.sources = r1a1.sinks = k1a1.channels = c1# 02 source,http,jsonhandlera1.sources.r1.type = httpa1.sources.r1.bind = mastera1.sources.r1.port = 6666a1.sources.r1.handler = org.apache.flume.source.http.JSONHandler# 03 regex extractor interceptor,match event body to extract character and digitala1.sources.r1.interceptors = i1 ?a1.sources.r1.interceptors.i1.type = regex_extractora1.sources.r1.interceptors.i1.regex = (^[a-zA-Z]*)\\s([0-9]*$) ?# regex匹配并进行分组，匹配结果将有两个部分, 注意\s空白字符要进行转义# specify key for 2 matched parta1.sources.r1.interceptors.i1.serializers = s1 s2# key namea1.sources.r1.interceptors.i1.serializers.s1.name = worda1.sources.r1.interceptors.i1.serializers.s2.name = digital # 04 logger sinka1.sinks.k1.type = logger# 05 channel,memorya1.channels.c1.type = memorya1.channels.c1.capacity = 1000a1.channels.c1.transactionCapacity = 100# 06 bind source,sink to channela1.sources.r1.channels = c1a1.sinks.k1.channel = c1
3、验证regex extractor interceptor
1) 通过curl -X POST -d ‘json数据‘的方式发送HTTP请求，body中的内容为"shayzhang 1234", 其中shayzhang，1234将被正则表达式匹配
2) 观察logger打印到终端的event，header中将增加两部分 word:shayzhang, digital:1234
07_Flume_regex interceptor实践
原文地址：http://www.cnblogs.com/shay-zhangjin/p/7966452.html