Introduction

In a previous lesson we managed resources (insert or remove) by editing the bags' contents with the help of the monitor. This was done manually, accessing one resource at a time.

In this lesson, we will show another way to insert resources via URL in a more "applicative/programmatic/practical" way. In fact, some methods of the middleware's protocol are directly accessible via URLs. We are going to build a simple program that inserts resources by pointing towards the convenient URLs.

New design of the metro network

As for now, the trom network was excessively simplistic : it was composed of 4 stations and a few links were defined between some of them.

In the new proposed design :

  • The metro network is composed of lines.
  • A line is a set of stations linked together.
  • A line is identified within the network by an associated color (line color).
  • Each standard station on a given line is linked with (and only with) all the other stations of the same line.
  • Some particular stations belong to more than one line : they are called Interconnections between lines.
  • On a given line, all the "standard" stations (i.e. not interconnections) have a name starting with the same letter. Then, an incremental numeric value completes the name of each station and defines the order of the stations along the line path (A1, A2, A3,...).

In our example, the enhanced metro network will be composed of 4 lines, respectively associated with the Red, Blue, Green, and Black colors. Standard station names respectively begin with letters A, B, C and D.

To make the interconnections quickly noticeable, their name systematically begins with an I. A numeric value is also associated with interconnections in the purpose of providing unique names within the overall network.

A map of the new trom network is presented hereafter :

 Red:     A1-A2-'''I1'''-A3-A4-A5-A6-A7-A8-'''I2'''-A9
                ''':'''                    ''':'''
 Blue:    B1-'''I3'''-'''I1'''-B2-'''I4'''-B3-B4-B5-B6 ''':'''  B7-B8-B9
             ''':'''        ''':'''              ''':'''
 Green:      ''':'''     C1-'''I4'''-C2-C3-C4-'''I5'''-'''I2'''-C5-C6-C7-C8-C9
             ''':'''                    ''':'''
 Black:   D1-'''I3'''-D2-D3-D4-D5-D6-D7-'''I5'''-D8-D9

The Connection bag, in its current form (reminded below), already enables the storage of the resources required to build such a metro network. The color of the line is stored in the line field of the resource and each link between 2 stations is stored in the departure-arrival field couple. A sample resource may look like ("Red", "A1", "A2" ).

 bag2 = CHASKI.config.data.BagDescription()
 bag2.name = "Connection"
 bag2.type = ("Multiset", "TupleSpace")
 bag2.fieldnames = ['line', 'departure', 'arrival']
 bag2.params = {}
 list_of_bags.append(bag2)

So, we have to insert all the distinct resources (combinations of stations) required to "build" our network and make it work in both ways, for each and every line. As a consequence, the complexity of inserting all of them manually greatly increased. Of course, this complexity remains true if we consider building the network by hand with initialization rules. Even if both these "manual" solutions are achievable, it is (at least) safer to build the network map with the help of a little "piece of code".

Resources insertion

Principle

As explained above, the lines definitions resources are "stored" in the Connection bag, which is contained in the Metro object.

The middleware natively integrates mechanisms to perform the basic operations of its protocol (i.e. read(), put() and get() operations) against a given object, through HTTP. We are going to use this feature to put() the required resources in the Connection bag, sending HTTP requests to the Metro object.

As we want to be sure that we can reach the Metro object, we specify the port number on which we are going to interact with it (previously, the port value was set to None, which let the middleware select the port number associated with the object). The Metro object is then declared as follows, with a port number set to 10000 :

 # - MetroObject
 MetroObject = data.ObjectDescription()
 MetroObject.name = "Metro"
 MetroObject.type = "Coordinator"
 MetroObject.port = 10000
 MetroObject.host = H1
 MetroObject.application = A1
 MetroObject.nameserver = NameServer
 MetroObject.params = {param.OBJECT_TRACE_LEVEL:TRACE_LEVEL}
 MetroObject.list_of_bags = [bag2]
 LIST_OF_OBJECTS.append(MetroObject)

Interacting with resources this way can be considered as a kind of "low-level" access. As a consequence, the operations performed (the transactions within which they are executed) have to be explicitly confirmed/committed.

  • During a first phase, the operation command is sent to the object (here, a put() operation of a given resource).
  • If the operation is evaluated as valid by the object, it returns a leaseID. This leaseID identifies the transaction within which the operation is performed.
  • Then, in a second phase, a confirmation request is sent to the object, containing the returned leaseID. This confirmation triggers a transaction commit.

Note : in the monitor, both phases are integrated in a unique user request, but the internal behavior is quite similar to that described above.

URL structure

The Connection bag (and its resources) is accessed via a URL sent to the Metro object. The base of this URL is the same as the one used to access the monitor and points toward the object through its associated port.

http://localhost:10000/Metro/MONITOR

Then, we add complementary tokens and parameters to this base URL to specify that we want to use the middleware's protocol to insert resources in the Connection bag. Of course, these tokens and parameters are "normalized" to be understood by the middleware.

The URL structure (template) used to perform operations against a given bag contained within the Metro object is detailed below :

http://localhost:10000/Metro/ACTION?operation=operation_name&bag_name=bag_name&parameters=parameters_associated_to_the_operation

This template is decomposed as :

  • localhost.localdomain:1000/Metro/ is the path to access the Metro object on the localhost.localdomain host through the 10000 port.
  • ACTION specifies that the middleware's protocol is used within the URL (the same way as MONITOR enrolled the monitor in a previous lesson).
    The end of the template structure is directly conditioned by the use of the ACTION token.
    • operation is the name of basic operation to perform. In our example, we are going to use put then confirm operations.
    • bag_name is the name of bag where the resources are managed.
    • parameters is a complementary value which contains the awaited parameters required by the operation.
      During the first put phase of the insertion, parameters is used to send a representation of the resource to be inserted
      During the second confirm phase of the insertion, parameters is used to send back the returned leaseID value.

As we now know a way to insert resources in a given bag, we are able to implement our "piece of code" to build the map of the network.

The buildMap_ACTION.py file #=

We define a separate dedicated python file, named buildMap_ACTION.py (as it is responsible for building the map of the trom network, using the ACTION path of the middleware's protocol), located in the same directory as the quinoa.py file. This file will be manually executed, simply by calling the python interpreter (see below).

We are going to manage http requests and urls, so we import the urllib module. The pickle module is also imported. As defined http://docs.python.org/library/pickle.html, the pickle module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure.. Here, the pickle module is used to

 import urllib
 import pickle

A dictionary is defined to contain the lines definition : the keys of the items are the lines colors and the associated values are the lists of the corresponding stations, in their predefined order along the line path :

 d = {
 'red':['A1', 'A2', 'I1', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'I2', 'A9'],
 'blue':['B1', 'I3', 'I1', 'B2', 'I4', 'B3', 'B4', 'B5', 'B6', 'B7', 'B8', 'B9'],
 'green':['C1', 'I4', 'C2', 'C3', 'C4', 'I5', 'I2', 'C5', 'C6', 'C7', 'C8', 'C9'],
 'black':['D1', 'I3', 'D2', 'D3', 'D4', 'D5', 'D6', 'D7', 'I5', 'D8', 'D9']
 }

This storage structure provides quite a "user-friendly" picture of our network map. But it also allows easy construction of a list of all the resources required to build the map of the network.

For each defined line (retrieved with its related dictionary key), 2 simple nested loops obtain each of the attended station couples. The line color and each given station couple are associated in a single tuple, then stored in a dedicated list.

list_of_resources_to_insert = []

for line in d.keys():
    pathline = d[line]
    for i in range(len(pathline)):
        first_station = pathline[i]
        for next_station in pathline[i + 1:]:
	    # one way
            connection = (line, first_station, next_station)
            list_of_resources_to_insert.append(connection)
	    # return
            connection = (line, next_station, first_station)
            list_of_resources_to_insert.append(connection)

Once the list of resources (stored as tuples) is completed, the insertion is performed in the 2 steps detailed below.

First step

The code sample below is quite self-explanatory : * for each resource (tuple) stored in the resource list, * a dictionary is defined and initialized with the parameters names awaited in the URL template and their associated values * the operation parameter is set to put, * the bag_name parameter is set to Connection, * the parameters parameter is set to a list representation of the given resource * the dictionary of parameters is then passed to urllib which returns its contained parameters encoded as URL parameters. * URL lib then "open" the full URL (base URL + encoded parameters), which means that the URL is sent to its expected destination. It returns a file descriptor as a reply. * the reply is read and stored as a string. * the returned string is a serialized python object. It is de-serialized with the pickle module. Only the second value of the returned parameters matters here, as it contains the leaseID mentioned above.

 for resource in list_of_resources_to_insert:
    # 1st phase on the put operation
    print 'put operation with resource=', resource
    url_parameters = {'operation':'put',
                      'bag_name':'Connection',
                      'parameters':[resource]}
    s = urllib.urlencode(url_parameters)
    fp = urllib.urlopen(url + s)
    txt = fp.read()
    dummy1, leaseid, dummy2, dummy3 = pickle.loads(txt)

Second step

The second step is required to confirm the insertion operation.

Following a similar process, a URL is built to perform a confirm operation against the Connection bag for the transaction associated with the returned leaseid.

  # 2nd phase on the put operation
  print 'confirm the put operation with lease id=', leaseid
  url_parameters = {'operation':'confirm',
                    'bag_name':'Connection',
                    'leaseid':leaseid}
  s = urllib.urlencode(url_parameters)
  fp = urllib.urlopen(url + s)
  txt = fp.read()
  a,b,c,d = pickle.loads(txt)

After this confirmation step, the resource is safely inserted into the Connection bag.

Run buildMap_ACTION.py

First, start the application with following command:

python quinoa.py --start_objects All

Next, run the buildMap script as follow:

python buildMap_ACTION.py

Then, open the monitor from your web browser by navigating to http://localhost:9999/NameServer/MONITOR.

Point to the Metro object to check that the resources have been inserted

Rules

The buildMap_ACTION.py * file is used to initialize the *Connection bag.

It would be possible to follow the same principles to insert the initial resources in the Customer and Destination bags of the People object (insert the resources with other python scripts). But, in these latter cases, using the rules is more efficient.

init.ls

Compared with the previous init.ls files, only the initial locations and destinations of the customers are updated to match the new network design.

# Bag Customer
 ::
 {
 ["People","Customer"].put("alice","A1") ;
 ["People","Customer"].put("bob","B1") ;
 ["People","Customer"].put("charles","C1") ;
 ["People","Customer"].put("denis","D1")
 }.

 # Bag Destination
 ::
 {
 ["People","Destination"].put("alice","D9") ;
 ["People","Destination"].put("bob","D9") ;
 ["People","Destination"].put("charles","D9") ;
 ["People","Destination"].put("denis","D9")
 }.
 ```

## trom.rules ##

The application rules defined in the previous *trom.rules* file did not contain
explicitly named locations. The involved locations are retrieved from the
resources read from the bags. It is not required to update this file as it also
enables journeys from direct connections to 4 changes between the initial
location and the targeted destination.


## Run the scripts ##

You should run the initialization and application scripts with the following command:

``` shell
python quinoa.py --run_scripts All

Note that, because of the combinatorial expansion of travel possibilities, the final result may not appear immediately. Please wait a few seconds or refresh several times the monitor page of the travel bag, till each customer reaches his targeted destination.