本文章将会继承上一篇文章，主要讲通过工具来进行日志的收集与发送，《ELK系列~NLog.Targets.Fluentd到达如何通过tcp发到fluentd》
Nxlog是一个日志收集工具，它将系统日志，或者指定的日志文件，统配符文件找到，然后加工，最后发送到目标位置。而目标位置有很多种，如文件系统，fluentd系统等，下面我们介绍一个使用场景，也是经常涉及到的场景。
- log4产生日期，日期文件名，统一后缀，按日志级别命名
- nxlog工具，配置，启动，发送数据
- fluentd配置，接受数据，打印数据
一 log4产生日期，日期文件名，统一后缀，按日志级别命名
<?xml version="1.0"?><configuration> ?<configSections> ??? <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler,log4net" /> ?</configSections> ?<system.web> ???<compilation debug="true" targetFramework="4.0" /> ?</system.web> ?<log4net> ???<logger name="ErrorLog"> ?????<level value="ERROR"></level> ?????<appender-ref ref="ErrorLog"></appender-ref> ???</logger> ???<appender name="ErrorLog" type="log4net.Appender.RollingFileAppender"> ?????<param name="File" value="logs\\" /> ?????<param name="AppendToFile" value="true" /> ?????<param name="StaticLogFileName" value="false" /> ?????<rollingStyle value="Date" /> ?????<datePattern value="yyyyMMdd"Error.log"" /> ?????<layout type="log4net.Layout.PatternLayout"> ?????</layout> ???</appender> ???<logger name="InfoLog"> ?????<level value="INFO"></level> ?????<appender-ref ref="InfoLog"></appender-ref> ???</logger> ???<appender name="InfoLog" type="log4net.Appender.RollingFileAppender"> ?????<param name="File" value="logs\\" /> ?????<param name="AppendToFile" value="true" /> ?????<param name="StaticLogFileName" value="false" /> ?????<rollingStyle value="Date" /> ?????<datePattern value="yyyyMMdd"Info.log"" /> ?????<layout type="log4net.Layout.PatternLayout"> ?????</layout> ???</appender> ???<logger name="WarnLog"> ?????<level value="Warn"></level> ?????<appender-ref ref="WarnLog"></appender-ref> ???</logger> ???<appender name="WarnLog" type="log4net.Appender.RollingFileAppender"> ?????<param name="File" value="logs\\" /> ?????<param name="AppendToFile" value="true" /> ?????<param name="StaticLogFileName" value="false" /> ?????<rollingStyle value="Date" /> ?????<datePattern value="yyyyMMdd"Warn.log"" /> ?????<layout type="log4net.Layout.PatternLayout"> ?????</layout> ???</appender> ???<logger name="FatalLog"> ?????<level value="Fatal"></level> ?????<appender-ref ref="FatalLog"></appender-ref> ???</logger> ???<appender name="FatalLog" type="log4net.Appender.RollingFileAppender"> ?????<param name="File" value="logs\\" /> ?????<param name="AppendToFile" value="true" /> ?????<param name="StaticLogFileName" value="false" /> ?????<rollingStyle value="Date" /> ?????<datePattern value="yyyyMMdd"Fatal.log"" /> ?????<layout type="log4net.Layout.PatternLayout"> ?????</layout> ???</appender> ?</log4net></configuration>
产生日志的方法如下
???????????log4net.Config.XmlConfigurator.Configure(new FileInfo("log4.config")); ???????????for (int i = 0; i < 10; i++) ???????????{ ???????????????LogHelper.Info("test"); ???????????} ???????????Console.WriteLine("end");上面代码将会产生日志文件，命名比较固定，方法日志的收集工作。
二 nxlog工具，配置，启动，发送数据
安装：https://docs.fluentd.org/v0.12/articles/windows
配置：去掉了log4产生日志每行后面的\r标记，如果不进行处理，在json转换时会有问题
## This is a sample configuration file. See the nxlog reference manual about the## configuration options. It should be installed locally and is also available## online at http://nxlog.org/docs/## Please set the ROOT to the folder your nxlog was installed into,## otherwise it will not start.#define ROOT C:\Program Files\nxlogdefine ROOT C:\Program Files (x86)\nxlogModuledir %ROOT%\modulesCacheDir %ROOT%\dataPidfile %ROOT%\data\nxlog.pidSpoolDir %ROOT%\dataLogFile %ROOT%\data\nxlog.log <Extension json> ???Module ?????xm_json</Extension><Extension _syslog> ???Module ?????xm_syslog</Extension><Input in> ???Module ???im_file ???File ???"c:\dotnet\20*.log" ???Exec $raw_event = replace($raw_event, "\r\n", " "); ???Exec $raw_event = replace($raw_event, "\r", " "); ???Exec $raw_event = replace($raw_event, "\n", " "); ???Exec $raw_event = replace($raw_event, "0x0A", " "); ???Exec $raw_event = replace($raw_event, "0x0DA", " "); ???Exec $raw_event = replace($raw_event, "0x0D", " "); </Input><Output out> ???Module ?????om_tcp ???Host ???????192.168.200.214 ???Port ???????24224 ???Exec $raw_event =$raw_event + "\n";</Output><Route 1> ????Path ???????in => out</Route>
启动：nxlog -f -c conf/nxlog.conf
查看：打开fluentd端，查看它的日志，发现我们的日志已经过来了
这个东西，我在google上找了好久，最后总算是功夫不负有心人，让我找对了！当然也证明了最初的猜想是正确的，即在output中对字符进行过滤！
三 fluentd配置，接受数据，打印数据
1 运行脚本，升级需要的文件夹和文件
mkdir /scripts/fluentd -pcd /scripts/fluentdcat ?> Dockerfile << ‘EOF‘FROM fluent/fluentd:v0.12-onbuildENV TZ=Asia/ShanghaiRUN echo "http://mirrors.aliyun.com/alpine/v3.5/main" >/etc/apk/repositories ????????&& echo "http://mirrors.aliyun.com/alpine/v3.5/community" >>/etc/apk/repositories ????????&& apk add --update tzdata ????????&& apk add curl ????????&& ln -snf /usr/share/zoneinfo/$TZ /etc/localtime ????????&& echo $TZ > /etc/timezoneRUN apk add ?--virtual .build-deps ????????sudo build-base ruby-dev && sudo gem sources --clear-all && apk del .build-deps && rm -rf /var/cache/apk/* \
/home/fluent/.gem/ruby/2.3.0/cache/*.gemEOFmkdir plugins -pmkdir -p /srv/volume/fluentd/cat > /scripts/fluentd/fluent.conf << ‘EOF‘<source> ?@type forward ?port ?24224 ?bind 0.0.0.0</source><match **> @type stdout
</match>EOF
2 生成镜像
docker build --no-cache --pull -t pilipa/tools/fluentd ./
2 docker上直接运行
docker run --privileged=true -v /scripts/fluentd/fluent.conf:/fluentd/etc/fluent.conf pilipa/tools/fluentd
启动后出现了配置和日志相关的输出信息
日志收集总算可以告一段落了！
感谢各位的阅读！1024节日快乐！
ELK系列~Nxlog日志收集加转发(解决log4日志换行导致json转换失败问题)
原文地址：http://www.cnblogs.com/lori/p/7723243.html