Design a Data Flow

How To Design a Data Flow in Edge Signal

Three main types of Nodes are used to design data flows:

1. Input nodes allow you to input data into a flow; used to connect data from other services or to manually input data into a flow using the inject node

2. Output nodes allow you to send data outside of a flow; used to send data to other services or to use the debug node to output to the debug pane

3. Processing nodes allow you to process data; allow you to transform the data type (e.g. json, csv, xml) nodes, use the data to trigger a message (e.g. trigger, delay) nodes and to write custom code that uses the data received (e.g. function node)

Note: some nodes, like the inject and debug messages, have a button that allows you to actuate a node (in the case of the inject node) or to enable and disable a node (in the case of the debug node)

Flows consist of multiple nodes wired together, with output tabs linked to input tabs of the next node in the flow. Messages flow along the nodes carrying data from node to node.

To design, for example, a simple flow:

• click on Data Flow in the Edge Signal platform sidebar menu to open the Data Flows list

• click the Add New button (top right) and name your new flow

• this flow is now in the Data Flows list, and can be selected to open the data flow environment to start building a data flow

• start with the simplest Node - the comment Node - located in the common section of the node palette

• drag and drop a comment node onto the flow workspace

  • at the info pane on the right (switch to info tab if debug is selected), there is info on the node (node name, a unique ID, and a properties field with a node description)

  Note: more sophisticated nodes contain more info

  • Double-click the comment node to open a configuration window to add a name and detailed text if desired

• Drag and drop an inject node onto the flow workspace

  • used to generate input into a flow; the first node in the node palette under common

    • look at the info tab to see the documentation for the inject node

    Note: the name of the node on the workspace changes from inject to timestamp because the default behaviour for the node is to inject a timestamp

    • the inject node (now named timestamp) has a blue dot top right indicating that the node hasn’t been deployed since it was last changed

    • the center right grey square is the output point for the node, where you attach ‘wires’ that route output message from the inject node to the next node in the flow

To get a sense for the inject node and the flow deployment process, add a debug node to see what happens, wire them together and then deploy the flow and test it:

• Start by dragging a debug node from the common section of the node palette to the flow workspace

• Next, wire the two nodes together by clicking on the grey output point for the inject node and, holding the mouse button down, dragging towards the debug node

• An orange wire appears, which you can then attach to the grey input point on the debug node

• Click the deploy button in the window (top right)

  • a pop-up appears indicating the flow has been successfully deployed

  Note: the blue dots on the nodes disappear, indicating there are no un-deployed changes

• Before you try the flow, make sure the debug tab is selected on the right pane, then click on the left tab on the inject node and observe what appears in the debug pane:

  • the inject node, when clicked, generates a timestamp (number of milliseconds since January 1, 1970), which is converted to a message and sent along the output wire, which is delivered to the debug node as an input message

  • The debug node’s default behavior is to show any message it receives, which it does in the debug pane on the right

Augmenting a simple flow

To edit the inject node to deliver a text message rather than a timestamp:

• select the inject node in the flow and double-click it

• In the payload field, select string instead of timestamp and then type any string you like into the blank field below the payload field. Eg. “Hello – from my first Edge Signal flow”

• click ok to save the changes

  • the blue dot has appeared on the inject node (which has also returned to being called inject rather than timestamp) to indicate that you have un-deployed changes

  • Click the deploy button again to resolve that and then click the tab on the inject node

    • If you observe the debug output you’ll see that instead of a timestamp, your text has been delivered as a message to the debug node, which displays it as usual

Messages flowing between nodes

The debug node can be used to examine the messages flowing between nodes in more detail.

• All messages have three default properties:

  • the payload - the intended message

  • a message topic - a user-defined string describing what the message is about (its topic)

  • an internal identifier - an identifier for the message which can be used to trace its progress through a flow

• You can view this information if you edit the debug node configuration:

  • Select the debug node, double click and change the node output field to “Complete msg object”. If you then save this, deploy again and click the inject node, you will see in the debug pane a JSON structure that contains 3 fields: a “topic” which is currently blank, a “payload” which contains the string you set in the inject node and an internal ID field “_msgid”

  • Generally, the internal message ID field isn’t used; however, both the topic and payload are used extensively when developing flows

  • These fields are extensible, so you can define new message properties, for example, msg.location, which could be used to add the latitude and longitude values for the source of the message