mmjobs
Thanks to this module it is possible to load several models in memory and execute them concurrently. A general synchronization mechanism based on event queues as well as two specialized IO drivers are also provided in order to ease the implementation of parallel algorithms in Mosel.
To use this module, the following line must be included in the header of the Mosel model file:uses 'mmjobs'Example
The following example shows how to compile, load, and then run a model from another model. After having started the execution, it waits for 60 seconds before stopping the secondary model if the latter has not yet finished.
model "mmjobs example" uses "mmjobs","mmsystem" declarations mymod: Model event: Event end-declarations ! Compile 'mymod.mos' to memory if compile("","mymod.mos","shmem:bim")<>0 then exit(1) end-if load(mymod,"shmem:bim") ! Load bim file from memory... fdelete("shmem:bim") ! ... and release the memory block ! Disable model output setdefstream(mymod,"","null:","null:") run(mymod) ! Start execution and wait(60) ! wait 1 min for an event if isqueueempty then ! No event has been sent... writeln("Model too long: stopping it!") stop(mymod) ! ... stop the model then wait wait end-if ! An event is available: model finished event:=getnextevent writeln("Exit status: ", getvalue(event)) writeln("Exit code : ", getexitcode(mymod)) unload(mymod) end-modelProcedures and functions
Model management
The type Model is used to reference a Mosel model. This section describes the procedures and functions available for model management: compilation of source model files, loading of bim files, execution and retrieval of model information. Note that before being used, a model has to be initialized by loading a bim file (load).
compile getexitcode getid getstatus load reset run setdefstream stop unload Synchronization
Synchronization between running models can be implemented using events. Events are characterized by a class and a value and may be exchanged between a model and its parent model. An event queue is attached to each model to collect all events sent to this model and is managed with a FIFO policy (First In – First Out). Depending on the needs, a model may check whether its queue is empty or simply suspend its execution until it has been sent an event.
The type Event represents an event in the Mosel language. Objects of type Event may be compared with = or <> and assigned with :=. The function nullevent returns an event without class and value: this is the initial value of a newly created event and no model can send an event of this kind (i.e. the class is necessarily not null).
dropnextevent getclass getfromid getnextevent getvalue isqueueempty nullevent send wait waitfor I/O drivers
The mmjobs module provides a modified version of the mem IO driver designed to be used in a multithreaded environment: memory blocks allocated by the shmem IO driver are persistent (i.e. they are not released after the model terminates) and can be used by several models. Thanks to this facility, models running concurrently may exchange data through memory by means of initialization blocks for instance.
The driver mempipe offers another communication mechanism between models: a memory pipe may be open by two models simultaneously. One of them for writing and the other one for reading. This driver also supports initialization blocks through which data is transfered in binary form.
Driver shmem
shmem:label[/minsize]The file name for this driver is a label: this is the identifier of the memory block. A label is not local to a particular model and remains valid after the end of the execution of the model having created it. All memory blocks are released when the module mmjobs is unloaded but a given memory block may also be deleted explicitly by calling the fdelete procedure of module mmsystem or by using the fremove C-function of the Native Interface.
Several models may open a given label at the same time and several read operations may be performed concurrently. However, writing to a memory block can be done by only one model at a time: if several models try to read and write from/to the same label, only one (it becomes the owner of the memory block) performs its IO operations for writing and the others are suspended until the owner closes its file descriptor to the specified label. Then, one of the waiting models is restarted and becomes the new owner: this process continues until all file descriptors to the label are closed.
Memory blocks are allocated dynamically and their size is increased automatically if necessary by pages of fixed size. When the file is closed, the system releases the memory that has not been used. In order to reduce memory fragmentation and increase efficiency (if the memory requirement is significantly larger than 4Kb), it is possible to specify an initial block size minsize (in bytes) when opening the file, e.g. for a block of 100Kb: "shmem:mymem/102400"
Driver mempipe
mempipe:[all,][noindex,]nameA memory pipe is characterized by its name. Only one model may open a pipe for reading but several models may open the same pipe for writing. However, if several models try to write to the same pipe, only one (it becomes the owner of the memory pipe) performs its IO operations and the others are suspended until the owner closes its file descriptor to the specified pipe. Then, one of the waiting models is restarted and becomes the new owner: this process continues until all file descriptors to the pipe are closed.
Pipe operations are possible only if the two ends of the pipe are open: one model for reading and at least one model for writing. There is no notion of 'end of file' in a pipe: if a model tries to read from an empty pipe (i.e. no model is writing to the other end) no error is raised and the model is suspended until something is available. Similarly trying to write to a pipe for which no model is reading from the other end is a blocking operation. In order to avoid lock ups, it is usually good practice to synchronize the models using events. For instance a model waits for a specific event before trying to read from a pipe; before starting to write to the same pipe, the other model sends the expected event.
Memory pipes may be used with initialization blocks. In this case, the pipe name can be completed by options all (all entries of dynamic arrays are transfered) and/or noindex (only values of array entries are transfered - indices do not precede).
This driver does not use labels for each record of the initialization block: it is assumed (but not checked) that both ends of the pipe are using the same sequence of records. For instance, if the writer sends an integer, a string and then an array of reals, the reader must expect an integer, a string and an array of reals: it is not allowed to skip records or change order as it is usually possible with these blocks.
If you have any comments or suggestions about these pages, please send mail to docs@dashoptimization.com.