Starting bootstrap

To create and start a LINC application, you have to use a python bootstrap describing all the objects, bags and rules of your application.

In this page, we describe each part of the bootstrap.

Configuration and include part

This first part allows you to import the different libraries needed by your LINC declaration. It then gets the preferences defined in your lincrc.py file (located in your home directory).

The boostrap script then uses two global variables to define the traces level: * TRACE_LEVEL: trace level of the objects * SYSTEM_TRACE_LEVEL: trace level of the internal LINC mechanisms

Other variables are defined: * DEMON_PORT: port used by the daemon * DEMON_KEY: key used by the daemon to identify the calls * LIST_OF_OBJECTS: list of script for this application * LIST_OF_SCRIPTS: list of script for this application * TOPDIR: directory in which the daemon will be executed * BINDIR: directory containing LINC binaries

# ========== Import preferences
import os, sys
SAVPATH = sys.path
sys.path.append(os.path.expanduser("~"))
import lincrc
sys.path = SAVPATH
PREFERENCES = lincrc.getPreferences()

# ========== Give access to LINC libraries
data, cmd, param = lincrc.importModules()

# ========== Set globals
TRACE_LEVEL = PREFERENCES[lincrc.TRACE_LEVEL]
SYSTEM_TRACE_LEVEL = PREFERENCES[lincrc.SYSTEM_TRACE_LEVEL]

DEMON_PORT = PREFERENCES[lincrc.DEMON_PORT]
DEMON_KEY = PREFERENCES[lincrc.DEMON_KEY]

LIST_OF_OBJECTS = []
LIST_OF_SCRIPTS = []

TOPDIR = lincrc.getCurrentDirectory()
BINDIR = PREFERENCES[lincrc.DEFAULT_INSTALL_DIRECTORY]

Application description

# ----- Application
A1 = data.ApplicationDescription()
A1.name = "My application"
A1.path = [lincrc.getCurrentDirectory()]

This structure allows to you to describe you application. You can precise name and installation path (warning: if your application is deployed in network you have to precise the path, or set of paths, used for each host).

Domains and hosts

# ----- Domains

domain1 = data.DomainDescription()
domain1.name = "mydomain.net"
domain1.bin_directory = "/home/d1/linc_install/linc"
domain1.var_directory = "/tmp/d1"
domain1.library = ["/opt/my_library"]
domain1.demon_login = DEMON_LOGIN_H1

domain2 = data.DomainDescription()
domain2.name = "otherdomain.com"
domain2.bin_directory = "c:\\Users\\d1\\linc_install\\linc"
domain2.var_directory = "c:\\temp\\d2"
domain2.library = ["d:\\my_library"]
domain2.demon_login = DEMON_LOGIN_H2


# ----- Hosts
H1 = data.HostDescription()
H1.name = "h1"
H1.domain = domain1
H1.ssh_port = 22
H1.is_local = False

H2 = data.HostDescription()
H2.name = "h2"
H2.domain = domain2

In this part, you declare the set of hosts where your application will be deployed.

HostDescription structure contains host name and an associated domain.

This domain is configured, via DomainDescription structure, in function of the host Operating System and the host configuration.

Host conection

When a LINC application uses remote hosts, LINC starts a demon on each host. This demon is started remotly via ssh.

In the host description, you can specify the ssh port to use (the default port is 22).

If you want to use the monitor interface from another machine, you must not use localhost in the host name. Instead use it's IP address or hostname.

By doing so, LINC will start the demon via ssh.

If you want prevent this (for instance to use a public address without opening ssh port), you can tell that a host (e.g. H1) is local.

Local means the host is the machine used to start the application with the bootstrap file.

If is_local is set to True, LINC will not use ssh but call a subprocess to start the demon as if the hostname was localhost.

H1.is_local = True

note: the system scans all directories contained in library list of DomainDescription (and path field of ApplicationDescription) to find participant, bag and storage classes using by LINC.
If you want to protect a directory from this scanning, add into it an empty file named "no_scan_directory".

Bags

bag_cnt = data.BagDescription()
bag_cnt.name = "Connection"
bag_cnt.fieldnames = ['departure', 'arrival']
bag_cnt.type = ("Multiset", "TupleSpace")
bag_cnt.params = {
    param.ASPECT_BLOCKING : False,
    param.ASPECT_READONLY : True}

BagDescription is a structure to describe your resource storage.

It is identified by a name (use in coordination rule) and a list of field names.

The type field contains two parts:

  • the first part defines the type of bag:
    • Multiset is the default type of bag, with tuple space,
    • to use your own bag, you need to write a bag_BagName.py file and use BagName for the bag type,
  • the second part defines the type of storage.

See chapter LINC bag and LINC storage to known how create your own kind of bag or storage.
In the params field, you can precise different parameters used by this bag (aspect, configuration, ...)

Participants

MetroObject = data.ObjectDescription()
MetroObject.name = "Metro"
MetroObject.type = "Coordinator"
MetroObject.application = A1
MetroObject.port = None
MetroObject.group = 'green'
MetroObject.host = H1
MetroObject.nameserver = NameServer
MetroObject.params = {
     param.OBJECT_TRACE_LEVEL:TRACE_LEVEL,
     param.OBJECT_HTTP_ROOT:lincrc.getCurrentDirectory()
}
MetroObject.list_of_bags = [bag_cnct]
LIST_OF_OBJECTS.append(MetroObject)

With ObjectDescription structure, you can describe a LINC participant.

It is identified by a name (used in coordinate rule) and a type.

Its type defines the kind of participant you want to use (see chapter LINC participant to create your own participant).<

The fields host, port and group define where this object will be deployed. If port is None, LINC choose a free port for this participant.

All objects declared with the same triplet (host, port, group) run in same LINC process.

Each object must be declared to a Name Server object. You must define an object of this kind in your boot strap and associate it to all your application's participants.

Like for bag, in the params field, you can precise different parameters used by this participant (such as trace level, configuration).

Coordination script

script_init = data.ScriptDescription()
script_init.application = A1
script_init.name = "init"
script_init.policy = "Automatic"
script_init.list_of_objects = [MetroObject]
LIST_OF_SCRIPTS.append(script1)

The structure ScriptDescription allows to you to declare a new rule script (containing one or more rules) used by a coordinator participant.<

You must precise its file in the name field. Extension of this file must be '.rules' or '.ls'. You can put other explicit extension or full absolute path of you rule file.

If you precise a policy, you can run many scripts identified with the same value of this field.

You must indicate coordinator objects in list_of_objects where this rule script will be running.

Final Linc boot strap

# ========== Deploy application
if __name__ == '__main__':
    cmd.full_control(LIST_OF_OBJECTS, LIST_OF_SCRIPTS, DEMON_PORT, DEMON_KEY, SYSTEM_TRACE_LEVEL)

In the end of this LINC launcher script, the full_control function of the LINC framework is called with all your configuration:

  • LIST_OF_OBJECTS Python list of participants description (ObjectDescription structure) that you use in your application.
  • LIST_OF_SCRIPTS Python list of rule scripts description (ScriptDescription structure) that you use in your application.
  • DEMON_PORT Port TCP/IP using by each Linc daemon.

A LINC daemon is a small process running in background in each host of your application. It starts all the participants running on a host.

  • DEMON_KEY This key is a security word, used by your application, to prevent an other LINC application using the same hosts to interfere with your application.
  • SYSTEM_TRACE_LEVEL Level of traces for each containers of participant.

You can precise this trace level in two ways::

  • with an integer Integer between 0 (only error mode traces) and 3 (high debug mode traces), it is used in development for detect problems in main components of a container.

In production mode, set this field to zero.

  • with a tuple of eight integers In this case, you set a trace level for each component of container.
  • Main container
  • Communication manager
  • Request queue
  • Request Manager
  • Object Manager
  • Method Manager
  • Configuration Manager
  • Monitoring

For example:

SYSTEM_TRACE_LEVEL = (1, 0 ,1, 2, 3, 1, 0, 1)

Set components Main container, Request queue, Method Manager and Monitoring to level 1 (information level) and Object Manager in level 2 (low debug) and Method Manager in level 3 (high debug).

Remarks:

SYSTEM_TRACE_LEVEL = 1

is equal to

SYSTEM_TRACE_LEVEL = (1, 0, 0, 0, 1, 1, 0, 0)

And

SYSTEM_TRACE_LEVEL = -1

is equal to SYSTEM_TRACE_LEVEL = (-1, -1, -1, -1, -1, -1, -1, -1)C

Launch syntax

To launch Linc application:

python <python Linc boot strap> [options]

The options can be:

--version            show program's version number and exit
-h, --help           show this help message and exit
--domains            concerned domains
--list_objects       print list of objects known in the python boot strap file.
--stop_all           stop all objects and container known in the python boot strap file.
--kill_all           kill all process associated with this application (objects, containers, daemons)
--check_scripts      [ All | {policy} | {script_name} ]
                     check rule script(s) in associate coordinator following the pattern:
                     All: all the script known in the python script file.
                     {policy}: all script known associated with this policy value.
                     {script_name}: one script
--clear_archives     clear all archives (logs, traces, ...) for each Daemon
--reexecute          [Num]
                     Re-execute from log archive information.
--start_objects      [ All | Local | {object_name}+ ]
                     start the list of Objects following the pattern:
                     All: all the objects known in the python boot strap file.
                     Local: only the objects belonging to the local domain
                     {object_name}+: one or more objects which names are separated by whitespace
--list_scripts       print list of scripts known in the python boot strap file.
--run_scripts        [ All | {policy} | {script_name} ]
                     launch rule script(s) in associate coordinator following the pattern:
                     All: all the script known in the python boot strap file.
                     {policy}: all script known associated with this policy value.
                     {script_name}: one script
--go                 shortcut for command '--kill_all --check_script All --start_objects All --run_scripts All' .
--list_domains       print list of domains known in the python boot strap file.
--stop_objects       [ All | Local | {object_name}+ ]
                     stop the list of Objects following the pattern:
                     All: all the objects known in the python boot strap file.
                     Local: only the objects belonging to the local domain
                     {object_name}+: one or more objects which names are separated by whitespace
--stop_daemons       stop all daemons for each host know in the python boot strap file.
--start_daemons      start all daemons for each host know in the python boot strap file.
--list_daemons       print list of all daemons and their state for each host know in the python boot strap file.
--download_archives  [Num] [import path]
                     Download log and trace archive for all daemon.
--dashboard          Open dashboard panel in web browser for each host know in the python boot strap file.

LINC application options

To create a LINC application, different options allow you to configure, optimize or precise it.

Libraries and frameworks referencing

When you install LINC on your computer, you can also install some libraries and frameworks.

For that, modify the lincframework.py file contained in your home directory.

In this python file, add your libraries paths in FRAMEWORK_PATH list variable.

All paths added in this variable will be accessed directly by your application.

You can add specific variables used in your LINC object for your needs.

In your application, you can import lincframework module to read them.

example of lincframework.py file:

  # -*- coding: utf-8 -*-
  ##########################################
  #
  # This file is part of LINC
  #
  ########################################

  FRAMEWORK_PATH = []

  FRAMEWORK_PATH.append("/home/user/pututu")

  MyConfigPort = 111000

Sources scanning

In starting of each container, LINC scans the paths of ApplicationDescription.path, DomainDescription.library and FRAMEWORK_PATH (lincframework file).

In those paths, LINC searches all python modules containing a participant, a bag or a storage class.

Those modules are registered by the container to instantiate it with objects creation.

If you want to protect a specific directory to this scanning (no python file, special code, deeper directory), you can block it by adding a empty file no_scan_directory in this folder.

This directory and its sub-folders will not be scanned by LINC.

Troubleshooting

If you application does not start or you want to have all the traces your can run the command:

python bootstrap.py --dashboard

This will open dashboard panel in web browser for each host know in the python bootstrap file. If that fails or if you want to have the traces of only one host you can ask directly to the demon running on this host at the following url:

http://DEMON_IP:DEMON_PORT/template_dashboard.html?key=KEY

http://127.0.0.1:18000/template_dashboard.html?key=key_linc