NEESgrid logo

Procotol Specification for the LabView NTCP plugin


Paul Hubbard, Jose Calderon, Scott Gose

hubbard@sdsc.edu
calderon@mcs.anl.gov
gose@mcs.anl.gov

Revsion history

v1.0 11/11/03 Initial posting.
v1.1 11/12/03 Typos fixed, added overview of command sequence
v1.2 5/25/04 Added transaction IDs to get/setParameter, as per code.
v1.3 6/19/04 Added cancel and multiple ControlPoint support, new connection tracking.
v1.4 11/19/04 Small edits for post to nees.org

Background and Introduction

The Labview plugin (aka lvplugin) is a code plugin for the NEESGrid Teleoperations Control Program (NTCP) (PDF) server. This plugin is a simple one that converts NTCP to and from a simple ASCII format designed for easy parsing. Communications occur over a single persistent TCP connection, initiated by NTCP. Code is included to perform all functions in LabVIEW, with several example control programs and test harnesses.

Related documentation specific to the Mini-MOST demo can be found at http://www.mcs.anl.gov/neesgrid/mmost

Getting the code

The most recent version of the code is always in the NEESGrid CVS archive (PDF) , with package name 'ntcp_plugins'. As a side note, the code NTCP server is also in the same archive, package name 'NTCP'.

Goals and Design Philosophy for the plugin

Our highest priority for the plugin was that it be reusable for other languages and platforms other than LabVIEW. This implies simplicity and consistency, which we hope that we have achieved. Your feedback is welcomed, as are design discussions.

The first choice was the communications - we picked ASCII over a single TCP connection, initiated by the NTCP server. Implications of this include:
  1. Any system wishing to use this plugin must be capable of having a process open and listen on a TCP port (i.e., have threading, or some equivalent form of multitasking and interprocess communications)
  2. All messages are tab-delimited ASCII, on a single line, variable length (Newline is the message delimiter)
  3. All messages should be human-readable.
  4. The plugin should, as much as possible, simply act as a serialization layer, with all the work being done on the client side. This means fewer assumption about client functionality, and higher implementation flexibility. As a side benefit, simpler plugin code reduces complexity and the number of bugs.
  5. Any language that can host a TCP connection and parse string can be a full-fledged NTCP controller. This opens several doors to lightweight control of small devices such as pan-tilt-zoom camera bases.
  6. We also chose to include the transaction ID in every message, so that the client side can be asynchronous and multithreaded. (This is how the LabVIEW control code was implemented)
By default, the connection is on TCP port 44000.

Connection tracking

It's worth explaining how the TCP connection is handled. Until 6/19/04, the TCP connection from the plugin to the control system was established when openSession was called, and remained in place untiil one end was stopped. This was a minor issue, and has been fixed.

Now, the TCP connection from plugin to control system is established with openSession, and closed at closeSession.

Basic Message Format

The basic command format from the NTCP server to the client is
verb [tab delimiter] TransactionID {optional parameters, tab delimited} [newline]
where the verbs are the following
For example:
propose 1203 x displacement 2.71828 y rotation 3.142
Response format is
[OK|Error] [tab] ReturnCode [tab] TransactionID {optional error message] [newline]
For example:
OK 0 Transaction1203
Error 1 TransactionFoo Parameter out of range
Note the duplication - if successful, the OK and 0 are redundant. However, I felt it useful for humans reading the protocol to have both a Boolean error indicator as well as a more-computer-useful integer denoting return code. Similarly, the optional error messages can be useful - the LabVIEW code will return different return codes and error strings for invalid axes and invalid parameters, for example. This is often invaluable for debugging.

Basic Sequence of Commands

The normal sequence of commands from NTCP to the control system looks like this:
  1. open-session
  2. set-parameter
  3. get-parameter
  4. Loop N times:
    1. propose
    2. execute
    3. get-control-point
  5. close-session
where steps 2 and 3 are semi-optional, and depend on the program running NTCP. (This is sometimes called an 'NTCP client', or 'simulation coordinator, depending on the context).

With that in mind, let's delve into some details on each command.

Detailed syntax for each verb

This section explains the parameters for each verb. For more information on the meaning and usage of each, please see the NTCP specification (PDF) and the NTCP plugin specification (PDF), as well as the NTCP helper API (PDF).

open-session

Syntax:
open-session TransactionID {parameter} {parameter}
This can optionally initialize hardware or do other system-specific work as required. This should be the very first command of a connection. The parameters are control-system-specific, meaning you can use this for whatever you need - they are optional. The LabVIEW implementation ignores the transaction ID and any parameters sent.

close-session

This mirrors open-session. Currently performs no work, but can be used as required. Should be the last command received in an NTCP session.

propose

New Feature: The latest implementation of this plugin includes the ability to handle multiple control points per propose request. The syntax of this new feature includes the keyword control-point to delimit between the multiple control points specified in the request. The structure of a propose request using this feature is as follows. 
propose TransactionID ControlPoint GeomType ParameterType Parameter [ [control-point {ControlPoint GeomType ParameterType Parameter}] ... ]
Note: The original syntax will still work so original code should not need to be modified.

Propose is really the key NTCP command. I recommend that you understand it well before writing any control software that has significant value and/or stored energy.

Syntax:
propose TransactionID ControlPoint GeomType ParameterType Parameter {GeomType ParameterType Parameter} {...}
where there can be zero to twelve parameters. The spec also allows zero parameters, for implied commands based on the control point name. I tend to find this questionable practice, and prefer to make my parameters explicit, but it's optional either way.

GeomType mirrors the NTCP spec, and is a single character:
x,y, or z
ParamType also mirrors the specification, and can be:
displacement, force, rotation or moment
Parameter is simple a scalar floating-point number. Combining these for a simple command, we have:
propose Tranaction0 ANCO-table x force 0.3 x displacement 3.1 y rotation 180.1
Note that order is not significant for the parameter triplets.

This verb is designed such that the control system should validate the proposal by its own means before returning. In the LabVIEW implementation, validation is done as follows:
    1. Control daemon tokenizes the request, recognizes it as a proposal, and puts it into a queue or pending proposals.
    2. Within the same code, a subroutine removes the proposal from the queue for validation. This was done so that proposals could be validated by external code, e.g. in a users' control program.
    3. The proposal validator reads the file 'actuators.ini' to determine which actuators are valid at present. From this file, it searches for each GeomType in the request. If an actuator with that geometry, e.g. 'x', is present, the validator compares the parameter against the static ranges defined in 'actuators.ini.'
    4. Step 3 is repeated for each geometry in the request. All must be accepted for the proposal to be succeed.
    5. Note that the total effect of this is to define a rectilinear space, flat-sided. This was sufficient for our purposes; you may well wish to enhance the validation for more interesting configurations. For example, a spherical space would require coordinate transformations and/or distance calculations to validate, rather than axis-by-axis checking.
    6. If the proposal is validated, the validator splits it up into one move per axis, and puts those commands into a queue of pending commands.
Verb notes
Once an 'OK' response has been returned, the NTCP server will usually either execute or cancel it next, depending on what the other components in the test have to say. Your code should be able to cancel transactions without ill effect; in a shameless display of hypocrisy ours does not yet do so, though the hooks are there. Neither, however, does the NTCP server itself yet manage this.

Also note that if an execute is still running, you should not receive another propose until after the move completes, otherwise you'll get 'control point busy' errors from within NTCP. Its core design assumptions include 'one and only one in-flight transaction per control point', which is rather a shame in that it disallows pipeline optimization.

execute

This simply triggers execution of a previously-accepted proposal. Obviously, the transaction ID must have been both proposed and accepted for this to be valid.

Syntax:

execute TransactionID

The control system should not return until the execution is complete.

cancel

This command cancels a pending command, usually due to a proposal being rejected by another control system somewhere else. As noted above, we cannot currently handle cancel requests. The plugin will throw an exception. The LabVIEW code has skeleton code to deal with them, but does not yet do so - all you'd have to do is walk the pending-commands queue, and remove the one with a matching transaction ID.

This should return OK if the transaction was cancelled successfully.

Syntax:
cancel TransactionID

get-control-point

This is the main feedback mechanism used by NTCP. Since NTCP does not have access to the streaming DAQ data, it needs some mechanism to get readings (e.g. force, position) back to form a closed loop; this is the mechanism.

New Feature: The latest implementation of this plugin includes the ability to query multiple control-points. Each control-point queried results in separate replies (shown below) from the plugin.

Syntax of the command is:
get-control-point TransactionID ControlPoint [ [ControlPoint] ... ]
where the transaction ID is usually ignored, but required for consistent parsing. The return format has a variable number of parameters, depending on what the given control point supports. Mirroring the propose syntax, we return up to twelve triplets of
GeomType ParamType Parameter
with the same types as 'propose' above, e.g. 'x displacement 1.414'

An example:
get-control-point DummyTransactionID003 ANCO

replies

OK 0 DummyTransactionID003 x displacement 0.0 x force -0.113 y rotation 0.0 z rotation 0.0
Verb Notes
This is an often-called method and should be asynchronous of the propose/execute logic if possible. In the LabVIEW implementation, a separate loop in the control program handles these requests and returns instant DAQ readings, enabling real-time data as a move happens. We use a LabVIEW semaphore to mark the TCP send as a critical section, so that the various threads don't collide in communicating with NTCP.

get-parameter

This is mainly used by the MATLAB code to pass simulation parameters around. The actual formatting of the parameter string is not documented as of this writing, so be cautious in your use of it. For example, MOST-SIM used this to pass both scalars (number of simulation steps) and 3x3 matrices around, both in MATLAB format. For Mini-MOST, the LabVIEW code returns a scalar defined in the control program, which is the spring stiffness of the main beam.

Syntax:
get-parameter TransactionID ParamName

Return syntax:
OK 0 ParamName Parameter
As noted above, the format of parameter needs to be negotiated.

set-parameter

This is the mirror of get-parameter, with the same notes and limitations. The plugin sends whatever it gets, and LabVIEW duly ignores it with an OK response.

Syntax:
set-parameter TransactionID ParamName Parameter

Complete conversation

To supply an example, please see this NTCP logfile. It contains a complete test run from the MCS shake table, driven by a 100-point sine wave sweep.

Other Notes

More information on the LabVIEW implementation is presently included in the Mini-MOST writeup.

None of the commands include units, which seems a significant omission. 0.5m is quite different than 0.5mm, so you must ensure that both sides agree on these before coding. The LabVIEW code includes unit support, but it'd need to be reconciled with any protocol changes to be useful.

Support

This work was supported primarily by the George E. Brown, Jr. Network for Earthquake Engineering Simulation (NEES) Program of the National Science Foundation under Award Number CMS-0117853.