streamsets 包含了丰富的组件，origin processer destination
测试例子为集成了http client 以及json 处理
启动服务
使用docker
- 创建pipeline
- 添加http client 组件
- 配置http client
http 服务地址： https://raw.githubusercontent.com/rongfengliang/json-learning/master/user3.json ?
- 添加json parse
- 配置json parse
注意路径是/ 开头的，应该使用的是类似jsonpath 以及json patch 的方式，具体要看代码
添加local fs 组件
输出到临时文件系统
?连接组件
运行
转换后的数据
{"name":"json-patch","version":"1.0.0","main":"index.js","license":"MIT","dependencies":{"generate-schema":"^2.6.0","json-merge-patch":"^0.2.3","json-templater":"^1.2.0","json8-patch":"^0.4.1","jsonpatch":"^3.0.1","jsonschema":"^1.2.4","uglify-js":"^3.4.5","uglifyify":"^5.0.1","uglifyjs":"^2.4.11"},"apps":"{\"users\":{\"name\":\"dalong\",\"age\":33}}","type":"dGhpcyBpcyBhIGV4YW1wbGU=","scripts":{"t":"node t","p":"node p","s":"node s","gs":"node gs","build":"browserify t.js | uglifyjs -c > bundle.js","live":"live-server","m":"node m"},"devDependencies":{"browserify":"^16.2.2","live-server":"^1.2.0"},"result":{"users":{"name":"dalong","age":33}}}参考资料
https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Origins/Origins_overview.html#concept_hpr_twm_jq
streamsets http client && json parse && local fs 使用
原文地址：https://www.cnblogs.com/rongfengliang/p/9505133.html