StreamSets 部署 Pipelines 到 SDC Edge

Easter79
• 阅读 600

可以使用如下方法:

  • 下载edge 运行包并包含pipeline定义文件.
  • 直接发布到edge 设备.
    在data colelctor 机器配置并配置了edge server 地址(主要需要网络可访问)
  • 导出edge pipeline 配置,并移动到edge 设备

直接发布pipeline

可以通过ui 部署

发布单个pipeline

  • Open the edge pipeline in the pipeline canvas.
  • On the General tab, set the Data Collector Edge URL property to the URL of the SDC Edge instance that you want to deploy the pipeline to.
  • The default URL is http://localhost:18633.
  • In the top toolbar, click the Publish Pipeline to Data Collector Edge icon:
    StreamSets 部署 Pipelines 到 SDC Edge
    Data Collector publishes the pipeline to the <sdcedge_home>/data/pipelines directory on the edge device where SDC Edge is installed.

发布多个pipeline

  • On the General tab of each pipeline that you want to publish, set the Data Collector Edge URL property to the URL of the SDC Edgeinstance that you want to deploy the pipeline to.
  • The default URL is http://localhost:18633.
  • From the Data Collector Home page, select multiple pipelines from the list, click the More icon, and then click Publish Pipelines to Data Collector Edge.
    Data Collector publishes the pipelines to the <sdcedge_home>/data/pipelines directory on the edge device where SDC Edge is installed.

导出pipeline

  • Open the edge pipeline in the pipeline canvas.

  • From the Properties panel, click the More icon (StreamSets 部署 Pipelines 到 SDC Edge ) to view the pipeline configuration REST response data:
    StreamSets 部署 Pipelines 到 SDC Edge

  • The browser displays the pipeline configuration in JSON format.

  • In the browser, copy the complete contents of the pipeline configuration and save it to a text file named pipeline.json.

  • In the address bar of the browser, type ?get=info after the pipeline ID.
    For example:

       https://localhost:18630/rest/v1/pipeline/EdgeSending76d9d551-217d-4b44-8b03-8fbe444962c5?get=info
    
  • The browser displays the pipeline information in JSON format.

  • In the browser, copy the complete contents of the pipeline information and save it to a text file named info.json.

  • On the edge device where SDC Edge is installed, create a folder named with the pipeline ID in the following directory:

    <SDCEdge_home>/data/pipelines
    

    For example:

    <SDCEdge_home>/data/pipelines/EdgeSending76d9d551-217d-4b44-8b03-8fbe444962c5 
    
  • Copy the pipeline.json and info.json files that you created for the pipeline into this folder.

  • Now you can start and manage the pipeline on SDC Edge as described in Sending Commands to SDC Edge.
    If you modify the same pipeline in Data Collector, view the pipeline configuration REST response data and then copy the updated data into the existing pipeline.json in the SDC Edge installation. You do not need to update the info.json file after you originally create it in the pipeline folder.

参考资料

https://streamsets.com/documentation/datacollector/latest/help/datacollector/UserGuide/Edge_Mode/EdgePipelines_Deploy.html#concept_ldf_ycf_3db

点赞
收藏
评论区
推荐文章
blmius blmius
3年前
MySQL:[Err] 1292 - Incorrect datetime value: ‘0000-00-00 00:00:00‘ for column ‘CREATE_TIME‘ at row 1
文章目录问题用navicat导入数据时,报错:原因这是因为当前的MySQL不支持datetime为0的情况。解决修改sql\mode:sql\mode:SQLMode定义了MySQL应支持的SQL语法、数据校验等,这样可以更容易地在不同的环境中使用MySQL。全局s
皕杰报表之UUID
​在我们用皕杰报表工具设计填报报表时,如何在新增行里自动增加id呢?能新增整数排序id吗?目前可以在新增行里自动增加id,但只能用uuid函数增加UUID编码,不能新增整数排序id。uuid函数说明:获取一个UUID,可以在填报表中用来创建数据ID语法:uuid()或uuid(sep)参数说明:sep布尔值,生成的uuid中是否包含分隔符'',缺省为
待兔 待兔
5个月前
手写Java HashMap源码
HashMap的使用教程HashMap的使用教程HashMap的使用教程HashMap的使用教程HashMap的使用教程22
Jacquelyn38 Jacquelyn38
3年前
2020年前端实用代码段,为你的工作保驾护航
有空的时候,自己总结了几个代码段,在开发中也经常使用,谢谢。1、使用解构获取json数据let jsonData  id: 1,status: "OK",data: 'a', 'b';let  id, status, data: number   jsonData;console.log(id, status, number )
Stella981 Stella981
3年前
KVM调整cpu和内存
一.修改kvm虚拟机的配置1、virsheditcentos7找到“memory”和“vcpu”标签,将<namecentos7</name<uuid2220a6d1a36a4fbb8523e078b3dfe795</uuid
Wesley13 Wesley13
3年前
mysql设置时区
mysql设置时区mysql\_query("SETtime\_zone'8:00'")ordie('时区设置失败,请联系管理员!');中国在东8区所以加8方法二:selectcount(user\_id)asdevice,CONVERT\_TZ(FROM\_UNIXTIME(reg\_time),'08:00','0
Wesley13 Wesley13
3年前
00:Java简单了解
浅谈Java之概述Java是SUN(StanfordUniversityNetwork),斯坦福大学网络公司)1995年推出的一门高级编程语言。Java是一种面向Internet的编程语言。随着Java技术在web方面的不断成熟,已经成为Web应用程序的首选开发语言。Java是简单易学,完全面向对象,安全可靠,与平台无关的编程语言。
Stella981 Stella981
3年前
Django中Admin中的一些参数配置
设置在列表中显示的字段,id为django模型默认的主键list_display('id','name','sex','profession','email','qq','phone','status','create_time')设置在列表可编辑字段list_editable
Wesley13 Wesley13
3年前
MySQL部分从库上面因为大量的临时表tmp_table造成慢查询
背景描述Time:20190124T00:08:14.70572408:00User@Host:@Id:Schema:sentrymetaLast_errno:0Killed:0Query_time:0.315758Lock_
Python进阶者 Python进阶者
11个月前
Excel中这日期老是出来00:00:00,怎么用Pandas把这个去除
大家好,我是皮皮。一、前言前几天在Python白银交流群【上海新年人】问了一个Pandas数据筛选的问题。问题如下:这日期老是出来00:00:00,怎么把这个去除。二、实现过程后来【论草莓如何成为冻干莓】给了一个思路和代码如下:pd.toexcel之前把这
Easter79
Easter79
Lv1
今生可爱与温柔,每一样都不能少。
文章
2.8k
粉丝
5
获赞
1.2k