Differences

This shows you the differences between two versions of the page.

--- dsa_wiki:home [2018/10/25 23:02]
wikieditor
+++ dsa_wiki:home [2021/09/20 14:35]
@@ Line 1: / Line 1: @@
-====== Distributed Services Architecture (DSA) ======
-DSA brings together entities like devices, services, and applications into one data model that updates in real time. By representing these entities as one system, DSA makes possible or simplifies various tasks, such as analytics, inter-device communication, distributed computing, and application development.
-You can read more about DSA on the [[http://iot-dsa.org/get-started/download-dsa|DSA home page]].
-----
-===== Structure of DSA =====
-DSA represents each of the entities in the system as one of the following types:
-  * **Broker**: Brokers manage security, links, subscriptions, and node permissions. Other entities in the system can operate only as permitted by a broker. A broker also saves configuration data to disk. A broker also routes data from a source to a destination.
-  * **DSLink**: DSLinks create, publish, and interact with data. DSLinks, which connect to brokers, can subscribe to data in the DSA system, to receive data whenever the data changes. You can download DSLinks from [[https://github.com/iot-dsa|IOT-DSA on Github]].
-  * **Node**: General term for the components connected by DSA. Node can be brokers, DSLinks, metrics, actions, attributes, or folders containing other nodes.
-  * **Metric**: A metric is a key/value pair. The value can be any of the data types listed in Supported Data Types, including an arbitrary value map. A metric can originate on any broker, DSLink, or other node.
-  * **Data Node**: Data nodes enable data to be stored on the broker's host server.
-  * **Action**: An action is an invocable command that can affect an entity. For example, an action might create a node or set a metric value. An action can exist on any broker, DSLink, other node, or metric.
-  * **Attribute**: An attribute is metadata for the selected entity, represented as a key/value pair. An attribute can exist on any broker, DSLink, other node, or metric.
-----
-===== Upstream and Downstream Connections =====
-A downstream entity requests permission, and an upstream entity grants or refuses that permission. A broker is always upstream from its DSLinks. A broker can be either upstream or downstream from another broker. Upstream and downstream connections are important concepts to understand when you configure permissions for multiple servers or multiple brokers on one server.
-----
-===== Connecting Two DSA Installations (Brokers) =====
-When you connect two DSA installations, one is an upstream broker and the other is a downstream broker.
-For an example, see [[http://rnd.iot-dsa.org/file/proj/rick/video/permission.html|this video]].
-To connect two DSA installations:
-  - From the downstream broker, right-click **sys** > **upstream**, and choose **Add Upstream Connection**.
-  - Specify connection settings as follows:
-  * **Name**: The node name that you want to appear in the downstream tree to represent the upstream broker.
-  * **Url**: The connection endpoint; for example ''http://localhost:8081/conn''.
-  * **Broker Name**: The name that you want to appear in the upstream tree to represent the downstream broker.
-  * **Group**: The permission group for the upstream broker.
-To remove the downstream broker from quarantine:
-  - Log into the upstream broker.
-  - Choose **sys** > **quarantine** > **Deauthorize**, and specify the downstream broker.
-----
-===== Supported Data Types =====
-DSA supports the following data types:
-  * **String**: A series of characters or an empty string.
-  * **Number**: A number or a null value.
-  * **Bool**: A true or false value.
-  * **Array**: An array object or a null value. The values in the array are of the dynamic data type. An example array of number values is ''[2,3,5,7,11]''. An example array of map values is ''[{"hello":"world","number":1},{"hello":"world"}]''.
-  * **Map**: A map object containing key/value pairs, or a null value. The key is always a string, and the value is the dynamic data type. An example value is ''{"hello":"world","primes":[2,3,5,7,11]}''.
-  * **Binary**: A byte array expressed as a string, or a null value. The string begins with ''\u001Bytes:'' and ends with a byte array encoded in base 64.
-  * **Dynamic**: A value that can be any of the above types.
-----
-===== Installing DSA =====
-To install DSA on Mac OS X, Microsoft Windows, Linux, or other platforms, follow the steps in the relevant video on [[http://iot-dsa.org/get-started/download-dsa|this page]], depending on your platform.
-If you are installing DSA on a Raspberry Pi, BeagleBone Black, or DGBox, you can also follow the steps in [[http://www.dglogik.com/company/blog/206-raspi-dsa|this blog post and video]].
-If you install DSA with DG Solution Builder, and do not have a DG Solution Builder license, follow the steps to request a license when you are prompted to do so.
-----
-===== Orientation to the DG Solution Builder Data and Metrics Panels with DSA =====
-The following image shows the [[dgsb_wiki:workspace_and_workflow:panels_and_menus:data_panel|Data panel]] and [[dgsb_wiki:workspace_and_workflow:panels_and_menus:metrics_panel|Metrics panel]] in DG Solution Builder when DSA is used.
-{{:dsa_wiki:data_metrics.png?300|}}
-|1|data node|2|downstream node|3|sys node|
-|4|sys > links node|5|metrics at the currently selected node (downstream > System)|
-----
-===== Storing Data on the DSA Server =====
-To store data on the DSA Server:
-  - In the Data panel, right-click the data node.
-  - <WRAP>Choose **Add** > **Node** or **Add** > **Value**.
-The **Value** option lets you specify a data type.</WRAP>
-  - <WRAP>Without moving the mouse pointer away from the pop-up menu, enter a node name, and optionally choose a data type.
-A top-level data node is created.</WRAP>
-  - In the Metrics panel, right-click the new data node that you created.
-  - Choose **Add** > **Node** or **Add** > **Value** to create a child of this node.
-  - Continue adding nodes until all nodes are represented.
-To set a value:
-  - In the Data panel, select the data node.
-  - In the Metrics panel, right-click the metric whose value you want to set.
-  - Choose **@set**, enter the value, and click **Invoke**.
-To invoke an action on a metric:
-  - In the Data panel, select the data node.
-  - <WRAP>In the Metrics panel, right-click the metric on which you want to invoke an action.
-A pop-up menu of available data actions is displayed.</WRAP>
-  - Choose the action, enter any required information, and click **Invoke**.
-----
-===== Installing and Updating a DSLink =====
-To install a DSLink:
-  - In the Data panel, right-click the **sys** > **links** node.
-  - Choose **Install Link**, and choose an installation method:
-    * If you have a link to a ZIP file, choose **from Url**
-    * If you have a ZIP file to upload, choose **from Zip**.
-    * If the link can be found in [[https://github.com/iot-dsa|IOT-DSA on Github]], choose **from Repository**.
-  - Specify a name for the new DSLink.
-  - <WRAP>Without moving the mouse pointer away from the pop-up window, click **Invoke**.
-When the link has been successfully installed, a "Success" message appears.</WRAP>
-  - <WRAP>To start the link, right-click the link under **sys** > **links** > **[link name]**, and choose **Start Link**.
-When the link has started successfully, it appears under **downstream** > **[link name]**.</WRAP>
-To update a DSLink:
-  - In the Data panel, right-click the link under **sys** > **links**.
-  - Choose **Update** and choose an update type, and then click **Invoke**.
-----
-===== Remote Dataflow =====
-DSA installations feature a dataflow DSLink, also referred to as //remote dataflow//. This tool is similar to DGLux5 dataflow, described [[dglux5_wiki:dataflow:home|here]], also known as //local dataflow//.
-Differences between the two are:
-  * Remote dataflow is stored on the DSA server with other DSLinks, not in the project files. Therefore, remote dataflow can run even when the project is not open in a browser.
-  * Certain blocks, such as Browser API blocks, are not available in remote dataflow.
-To create a remote dataflow model:
-  - In the Data panel, right-click **downstream** > **dataflow**.
-  - Choose **Create Dataflow**.
-  - Specify a name for the new dataflow model, and click **Invoke**.
-  - Choose the new dataflow model at **downstream** > **dataflow** > **[dataflow model name]**.
-Block property values in a remote dataflow model appear as metrics in the Metrics panel. They can be used like any other metric. The following image demonstrates an example of remote dataflow metrics.
-{{:dsa_wiki:remote_dataflow.png|}}
-----
-===== Security and Permissions =====
-See also: [[https://github.com/IOT-DSA/docs/wiki/Tokens|Tokens]].
-==== Quarantined Brokers and Links ====
-When //quarantine// is enabled on a broker, any downstream broker or link without an authorized token is held in quarantine. The system can read, subscribe to, and command nodes that are in quarantine, but a node that is in quarantine cannot access other nodes in the system.
-Whether quarantine is enabled is specified in ''dsa/dglux-server/server.json''.
-To remove a node from quarantine, you can //authorize// or //deauthorize// the node. An authorized node is granted access as the permission group that you specify. A deauthorized node is refused access and removed from the system. You can also deauthorize any downstream broker or node that has previously been authorized.
-When quarantine is disabled on a broker, any link can connect to the broker without approval.
-To authorize a broker or link:
-  - <WRAP>Right-click **sys** > **quarantine** > **Authorize**.
-A list of quarantined nodes appears.</WRAP>
-  - For **DsId**, choose the node that you want to authorize.
-  - For **Group**, choose the permission group that you want to assign.
-  - Specify a name, and click **Invoke**.
-To deauthorize a broker or link:
-  - <WRAP>Right-click **sys** > **quarantine** > **Deauthorize**.
-A list of quarantined nodes appears.</WRAP>
-  - For **Name**, choose the node that you want to authorize.
-  - Click **Invoke**.
-==== Permission Groups ====
-In DSA, you assign permission groups to users. Permission groups are defined within ''dsa/dglux-server/server.json''. Information about permission groups can be found [[https://github.com/IOT-DSA/docs/wiki/permission-list-for-the-root|here]]. Any user with "superuser" enabled has the maximum permission level and does not need a permission group defined.
-More information about DSA permissions can be found [[https://github.com/IOT-DSA/docs/wiki/DSA-Permission-Model|here]].
-==== Users ====
-To create a new user:
-  - Right-click **sys** > **users**, and choose **Create User**.
-  - Without moving the mouse pointer away from the pop-up window, enter a user name and password, and specify whether the user has superuser access.
-  - Click **Invoke**.
-To edit a username:
-  - Right-click **sys** > **users** > **[username]**, and choose **Rename User**.
-To edit a user password:
-  - Right-click **sys** > **users** > **[username]**, and choose **Change Password**.
-  - Enter the new password, and click **Invoke**.
-To edit a user's permission group, landing page, or other user properties:
-  - Right-click **sys** > **users** > **[username]**, and choose **Edit User**.
-  - Edit the properties, and click **Invoke**.
-To delete a user:
-  * Right-click **sys** > **users** > **[username]**, and choose **Remove User**.
-----
-====== The C Broker ======
-The C Broker is a small-footprint DSA broker that is intended for embedding in devices where economy of scale is important. This document tells you how to build, configure and launch the C Broker. This document assumes you are familiar with DSA and DGLux5.
-The C Broker receives data from devices by way of DSLinks, in the same way as the DGLux Server. To use the DGLux5 GUI to manage DSLinks that connect to an instance of the C Broker, install and run the LifeCycle Manager as described below. The following figure shows a standard topology on the left and an embedded topology on the right.
-{{dsa_wiki:cbrokertopology.png}}
-Differences between the DSA Server and the C Broker are as follows:
-^ Feature ^ DGLux Server/DART Broker ^ C Broker ^
-|Built-in web server |Yes |No |
-|Hosts DGLux5 GUI |Yes |No |
-|Manages users and filesystem access |Yes |No|
-**Note:** Because and the C Broker support a subset of the features of the DGLux Server, the JSON configuration files for the C Broker are not interchangeable with those of the DGLux Server.
-===== Building the C Broker =====
-To download the source code from GitHub, you must have ''git'' installed. To build the C Broker, you must have ''cmake'' 3.0.0 installed These instructions assume you are building on Ubuntu 14.04.
-To download the C Broker source code from github, issue the following command:
-<code>git clone https://github.com/IOT-DSA/sdk-dslink-c</code>
-To build the C Broker, issue the following commands in the directory where you cloned the source code:
-<code>
-mkdir build
-sh tools/build.sh
-</code>
-===== Configuring the C Broker =====
-To configure settings for the C Broker, edit the broker.json file, which is created when you build the C Broker. This file must reside in same directory as the executable.  The default broker.json file contains the following settings:
-<code>
-{
-  "http": {
-    "enabled": true,
-    "host": "0.0.0.0",
-    "port": 8080
-  },
-  "log_level": "info",
-  "allowAllLinks": true,
-  "maxQueue": 1024,
-  "defaultPermission": null,
-  "storage": {
-    "path": "."
-  }
-}
-</code>
-The following table lists valid C Broker settings.
-^ HTTP Settings ^
-|enabled | To configure the broker to accept HTTP connections, set to true. Default: true|
-|host |Specify the host name of IP address where the broker runs. Default is 0.0.0.0, which means that if the host has multiple IP addresses, the C Broker accepts connections on all of them.|
-|port |Specify the port on which the broker accepts HTTP connections. Default is 8100.|
-^ HTTPS Settings ^
-|enabled | To configure the broker to accept HTTPS connections, set to true. Default: true|
-|host |Specify the host name of IP address where the broker runs. Default is 0.0.0.0, which means that if the host has multiple IP addresses, the C Broker accepts connections on all of them.|
-|port |Specify the port on which the broker accepts HTTPS connections. Default is 8463.|
-|certName |Certificate file name (.pem file). Put certificate files in a subdirectory named “certs,” under the directory where the C Broker is running.|
-|certKeyName |Key file name (.pem file). Put key files in a subdirectory named “certs,” under the directory where the C Broker is running.|
-^ Other settings ^
-|log_level | Specifies the logging verbosity. Valid values are ''fatal'', ''error'', ''warn'', ''info'' and ''debug''. Default is ''info''.|
-|allowAllLinks |Specifies whether the C Broker accepts connections from all DSLinks or only previously-registered ones.|
-|maxQueue |The maximum number of messages from DSLinks that remain queued when the requestor has specified the qos 1 or qos 3 options. The qos (“quality of service”) settings control how the broker caches incoming data to handle disparities between the responsiveness of requestors and that of responders.) |
-|defaultPermission |Specifies the permissions to be assigned to DSLinks that do not have any permissions configured for their associated user. Valid permissions, from least to most restrictive, are ''config'', ''write'', ''read'', ''list'', and ''none''. [[https://github.com/IOT-DSA/docs/wiki/DSA-Permission-Model|(More information...)]]|
-|path |The directory that the C Broker uses for working storage for qos queue data, incoming data values, and other data that it must store locally.|
-Specify the defaultPermission setting using JSON notation; for example:
-<code>
-  "defaultPermission": [
-      [":config","config"],
-      [":write","write"],
-      [":read","read"],
-      [":user","write"],
-      ["default","read"]
-   ]
-</code>
-If you are using tokens to authenticate connections from DSLinks, token files must reside in the “tokens” subdirectory under the directory where the C Broker is running. The name of the token file must be the first 16 characters of the full token string. Specify tokens in JSON format, as follows, one token per file, as follows:
-<code>
-{"$$timeRange": null, "$$token": "47S7ybkxV6ka7Ha6zIHsFl16wiYwu4i8EGYlqdzK7NQw5Rus", "$$managed": false, "$$group": ":write"}
-</code>
-  * $$timeRange: A date and time range that specifies the start and end of the period when this token is valid. Example: 2017-03-10T00:00:00/2017-03-11T12:00:00
-  * $$token: 48-character token
-  * $$managed: To disconnect and remove all associated DSLinks when the token expires, set to true.
-  * $$group: The permission group to be assigned to the DSLink that connects using this token.
-===== Deploying the C Broker =====
-To run the C Broker on a machine other than the machine where you built it, copy the broker executable and the broker.json file to the desired location.
-To start the broker, invoke its executable on the command line. For example:
-<code>./broker &</code>
-To verify that the broker is accepting connections, issue the following command (assuming you have configured the broker to listen on port 8080):
-<code>telnet localhost 8080</code>
-===== Connecting to the C Broker =====
-To use DGLux5 to get access to the data that the C Broker receives from its DSLinks, configure a connection as follows:
-  - In the **Data** panel, expand the **sys** node and right-click upstream.
-  - Choose **Add Upstream Connection** and specify settings,
-  - Click **Invoke**.
-Settings:
-  * Name: The name to be displayed in DGLux5
-  * Url: The URL required to connect to the DSA Server. For example: http://101.0.1.197:8080/conn
-  * Broker Name: The name to be displayed by the upstream broker.
-  * Token: If required, the name of the token file containing the token needed to authenticate.
-  * Group: The group defining the permissions to be assigned to the connection.
-When the C Broker accepts the connection, it displays a success message. To view the C Broker in DGLux5, expand the **sys > upstream** node and verify that your broker instance is listed.
-{{:dsa_wiki:cbrokerconnection.png?300|}}
-The figure below shows the topology.
-{{:dsa_wiki:cbrokertopology2.png}}
-You can configure the C Broker connection to the DSA Server by manually editing configuration files that reside on the machine where the C Broker runs, as follows:
-  - In the directory where the C Broker runs, create a subdirectory named “upstream.”
-  - In the “upstream” directory, create one JSON file for each upstream connection that you want to define, using whatever file naming convention is meaningful to you. In these files, specify the connection settings in JSON, formatted as follows:
-<code>{"name": "rnd", "brokerName": "mybroker", "url": "http://rnd.iot-dsa.org", "token": "<token string>", "enabled": true, "group": ":config"}
-</code>
-Settings:
-  * name: Name of the upstream broker, must be same as the file name in the upstream folder
-  * brokerName: Name of the current broker to be displayed under the upstream broker
-  * url: The URL required to connect to the C Broker. For example: http://101.0.1.197:8080/conn
-  * token: (Optional) Token to be used when the C Broker connects.
-  * group: The permission group for the C Broker.
-===== The LifeCycle Manager =====
-The C Broker lacks the DSLink-management features of the DGLux Server. The LifeCycle Manager enables you to use DGLux5 to manage the DSLinks that connect to the C Broker.
-To build the LifeCycle Manager, your system must have the following software installed. The versions listed below are known to work. The build process and the software itself has not been tested with other versions of these packages.
-  * g++-5 14.04
-  * cmake 3.0.0
-  * libssh2 1.4.3
-  * zlib 1.2.8
-  * boost 1.54
-  * libcurl 7.35
-  * http-parser 2.1
-===== Building the  LifeCycle Manager =====
-**Note:** The github repository for the Lifecycle Manager is private. If you attempt to clone the source and are denied permission, contact DGLogik to request access to the repository.
-To download the LifeCycle source code from github, issue the following command:
-<code>git clone https://github.com/dglogik/dslink-manager</code>
-To build the LifeCycle manager, issue the following commands in the directory where you cloned the source code:
-<code>
-mkdir build
-cd build
-git submodule init
-git submodule update
-cmake -DCMAKE_CXX_COMPILER=g++-5 -DCMAKE_BUILD_TYPE=Release  ..
-make
-</code>
-To verify that the build succeeded, check that the build directory now contains an executable file named “dsmanager.”
-===== Deploying the  LifeCycle Manager =====
-To run the LifeCycle Manager on a machine other than the machine on which it was built, copy the following files from the build machine to the desired production directory:
-  * dsmanager (executable)
-  * libsdk_dslink_c.so
-  * dslink.json (To configure the display name for LifeCycle Manager, edit this file.)
-Before starting the LifeCycle Manager, verify that the C Broker is running. To start the LifeCycle manager, invoke its executable on the command line. For example, to run the LifeCycle Manager (assuming the broker is configured to listen on port 8080) in background mode, issue the following command:
-<code>./dsmanager --broker http://localhost:8080/conn</code>
-If the LifeCycle Manager started successfully, it displays “Successfully connected to the broker.” To verify that the LifeCycle Manager is accessible using DGLux5, expand the Data pane upstream node. Locate the C Broker, expand its downstream node, and verify that the LifeCycle Manager is listed and is not grayed out. To manage the DSLinks that are connected to the C Broker, right-click the LifeCycle Manager node.
-===== Troubleshooting =====
-Both the C Broker and LifeCycle Manager log all their messages to standard output and do not create any physical logs.
-----
-**Using This Wiki**
-[[dsa_wiki:dslinks:home | DSLinks]]