Transparent Thoughts 🪟

Search

SearchSearch

2U.5

Jun 23, 202510 min read

From:

To:

Implement loading and unloading rolls to CCT Batching Machine.

The central abstraction the library provides is that of a “place to load an object to and from “. The existing object location mechanism is the containment component, but this is slated for deprecation/re-implementation, so this aspect of the library will be implemented as an interface. The record of loads and unloads will be append-only, forming an audit trail of the actions.

The two sources of research are the load/unload in implementation in cct-joining and the gate library.

Having read the implementation in cct-joining, the names of the unwind containers are encoded using a hard-coded prefix name i.e. CJ1_. There are custom exception paths handling invalid states in the load unload actions, there is custom logic in handling a set of unload conditions.

The first development to tackle is to generalise the notion of “a place to load things to and from”, removing the need to hard-code the container name prefixes. Something akin to this was done for the gate abstraction, so I’ll read that next.

The gate abstraction has a three-tier implementation: “CCO/DB”, “Django App”, “Usage”.

The first provides an abstract gate and WhitelistGate types, which can dynamically construct Django models, with contexts. The second provides a convenient Django app for creating whitelists and interacting with their content. Finally, the app is used in cct-batching to implement an inbox.

I am generalising the concept of recording loading and unloading objects froma production machine. The purpose of recording these actions is to maintain a record of the state of a process in terms of ‘what is being operated on’ by the process.

The very basic notion of a process is that it frames operations carried out on objects. This implementation provides a general mechanisms for tracking which objects are being operated on by a given process instance.

We define a process as an object having inputs, and inputs as objects which can be loaded and unloaded, with a present state.

We define a NamedInput to be one identified by a non-unique human-readable name. A NamedInputProcess is a Process with only NamedInputs.

A SimpleNamedInput is a NamedInput capable of being loaded with a single object of one type defined by a CCODjangoModel. This has been made redundant as the “simple” and “normal” input process have the same API so can be implemented as the same class.

A SimpleNamedInputProcess is a NamedInputProcess with a set of SimpleNamedInputs addressable by their name see above note.

API

cct_joining = NamedInputProcessType.get_type("cct_joining")

Process definition named input types

process instance process instance

???

I want to define a kind of process to be “CCT Joining” having 3 inputs: “A”, “B”, “C” all taking CCT Bulk Rolls.

I then want to define an instance of “CCT Joining” called “CJ1”.

I then want to load a bulk roll to CJ1 Input A, then inspect the state of CJ1 Input B, finally unload the Bulk Roll from Input C.

cj1 = NamedInputProcessType.get_type("cct_joining", {
	"A": (CCTBulkRoll, CCTJoiningProduct, ),
	"B": (CCTBulkRoll, CCXProducedMaterial, ),
	"C": CCTBulkRoll,
}).get_instance("cj1")
 
...
 
cj1.get_input("A").load(cct_joining_product, load_reason)
cj1.get_input("B").state
cj1.get_input("C").unload(cct_bulk_roll, unload_reason)

The first action a user performs is to describe a new process type. An aspect of this description is to name and classify inputs.

During the instantiation of an input, an object load log of that given type must exist.

Example of an object load log record: {FK("cj1"), "A", FK("CC01T1250101")}

erDiagram
	"named_input_processes.types" ||--o{ "named_input_processes.instances": ""
	"named_input_processes.instances" ||--o{ "named_input_processes.inputs": ""
	"named_input_processes.inputs" ||--o{ "named_input_processes.load_{object_type_table}": ""
	"named_input_processes.load_{object_type_table}" ||--o| "named_input_processes.unloads": ""
	"public.{object_type_table}" ||--o{ "named_input_processes.load_{object_type_table}":""

I am moving the process abstraction into a new library: process-model. I need to endow process-model with the bare-minimum Django infrastructure to unit test the implementation using the Django test framework, namely dynamically constructing NamedInputLoad classes.

So what is the bare-minimum Django infrastructure to facilitate Django unit tests?

After some trial-and-error, I got this working with a simple settings config just configuring the test DB bypass and database registry.

With the test frame in place, I need to establish the DB table creation and test that first.

Now with dynamic model generation working for NamedInputLoad, I need to define its API.

The given namedInputLoad is a record of a load event, it does not provide an indication of the present state of the NamedInput. I actually think the limits placed upon the Load and Unload types should be minimal. The limitations should be placed upon the Input type to ensure the ‘load’ and ‘unload’ actions are valid. The only constraint placed on a given load event is that there isn’t a current load event with identical input and load object and unload state.

erDiagram
	"named_input_processes.types" ||--o{ "named_input_processes.instances": ""
	"named_input_processes.instances" ||--o{ "named_input_processes.inputs": ""
	"named_input_processes.inputs" ||--o{ "named_input_processes.load_{object_type_table}": ""
	"named_input_processes.load_{object_type_table}" ||--o| "named_input_processes.unloads": ""
	"public.{object_type_table}" ||--o{ "named_input_processes.load_{object_type_table}":""

	"named_input_processes.types" {
		INT id PK "GENERATED ALWAYS AS IDENTITY"
		STR name "UNIQUE NOT NULL"
	}

	"named_input_processes.instances" {
		INT id PK "GENERATED ALWAYS AS IDENTITY"
		STR name "NOT NULL"
		INT type_id FK "NOT NULL"
		UNIQUE _ "(name, instance_id)"
	}
	
	"named_input_processes.inputs" {
		INT id PK "GENERATED ALWAYS AS IDENTITY"
		STR name "NOT NULL"
		INT instance_id FK "NOT NULL"
		UNIQUE _ "(name, instance_id)"
	}
	
	"named_input_processes.load_{object_type_table}" {
		INT id PK "GENERATED ALWAYS AS IDENTITY"
		INT input_id FK "NOT NULL"
		INT load_object_id FK "NOT NULL"
		TIMESTAMP load_time "WITH TIME ZONE NOT NULL"
		TEXT load_reason "NOT NULL"
		INT unload_id FK "UNIQUE"
		UNIQUE _ "(input_id, load_object_id, unload_id) NULLS NOT DISTINCT"
	}
	
	"named_input_processes.unloads" {
		INT id PK "GENERATED ALWAYS AS IDENTITY"
		TIMESTAMP unload_time "WITH TIME ZONE NOT NULL"
		TEXT unload_reason "NOT NULL"
	}

The process-model library provides re-usable models for common elements of every Standard Operating Procedure implemented into the IMS.

Currently, two models are provided by this library: the gate and the process.

The gate provides a mechanism for controlling the flow of objects between processes. The essential aspect of a gate is the object_passes check which either allows or denies a given object to pass the gate.

The process model provides a standard way of representing the actual state of a given process being carried out in the factory. Currently, an implementation is provided for modelling the state of objects loaded into the inputs of a given process. the process-model provides a framework for recording the operation of processes, enabling the storage and recall of both their real-time and historical state.

The processes module provides a standard way of representing the actual state of a given process being carried out in the factory.

The process state modelled by this current implementation is limited to the state of the objects loaded in the inputs of the process.

This process state is modelled by three abstract classes: ProcessType, ProcessInstance, and Input.

ProcessTypes define the blueprint for a process, it is the “on paper” description of the process.

A ProcessInstance is a real “place” where the process is carried out in the factory (or back-office etc.). This is commonly manifested as a physical production machine, but can also be an abstract, even transient instantiation. It is where the actual operation of the given ProcessType happens.

An Input (currently the only concretely implemented aspect of a process) is the part of a process that takes things in to be operated on/with as a part of the process. These inputs are most commonly occupied by physical things: Bulk Rolls, sand, cement, … which may be transformed or even consumed by the process. But inputs can also be more abstract things like a Sales Order, which may form instructions on how the process is carried out. In the case of physical inputs to a process, the objects can only be in one process instance at a time, whereas an informational input can be in many process instances at once. This is implemented here as an instruction flag on the input, indicating that it cannot modify the objects taken into it, but also relaxing the condition of that object being in more than one process at a time.

Now having implemented namedInput (see 2U.5.1), I will implement NamedInputInstance.

A NamedInputInstance is an actual version of a process operating in practice. If the processes type is “CCT Joining” then the “CJ1” machine forms the heart of a real operating instance of “CCT Joining”.

For the time being, this is not much more than a named middle-man between the ProcessType blueprint and the NamedInputs loaded with objects.

NamedInput needs a loads dict on instantiation mapping object type to a loader logger for that type.

Again, I am not going to restrict the “shape” of an instance by the DB, so whatever is provided by the mapping is what will be constructed.

I also want to leave mutating the DB until the last moment, so no calling .save() until the records are actually required.

The final step is to implement the NamedInputProcessType.

Due to the concrete type constructs making superfluous DB calls (to ensure the underlying tables exist), we run it once for each type in the input map.

named_input_process Provides an implementation of the abstract process in which inputs are indexed by a human-readable name, unique according to (type, instance, input), so we can load objects into input “A” on process instance “CJ1” of type “CCT Joining”.

The high-level API to use this module is as follows:

cj1 = NamedInputProcessType.get_type("CCT Joining", {
	"A": (CCTBulkRoll, CCTJoiningProduct, ),
	"B": (CCTBulkRoll, CCXProducedMaterial, ),
	"C": CCTBulkRoll,
}).get_instance("CJ1")
 
...
 
cj1.get_input("A").load(cct_joining_product, load_reason)
cj1.get_input("C").unload(cct_bulk_roll, unload_reason)
cj1.get_input("B").state

This first constructs a NamedInputProcessType, “CCT Joining”, defined to have three inputs: “A”, “B”, “C” which can take CCTBulkRolls, CCTJoiningProducts and CCXProducedMaterial in various combinations.

Then we get an instance of this process type: “CJ1”, which is what we actually perform our loads/unloads onto. We then call to get_input() and load() or unload() some objects. Finally, we can check the state of an input to see what is currently loaded into it.

Loads and unloads are implemented as an append-only log recording:

  • What object was loaded/unloaded
  • When it was loaded/unloaded
  • Why it was loaded/unloaded

The log is very permissive in what it allows, the only constraint is that duplicate events cannot be recorded: i.e. an object cannot be loaded twice to the same input simultaneously, it must be unloaded before it can be loaded again. This also applies to unload records. This loose restriction means that objects can be loaded to multiple different inputs simultaneously, so restrictions over this must be implemented by the application developer.

The ‘shape’ of a process is defined at runtime when namedInputProcessType.get_type() is called. This is not governed by a stored ‘shape’ in the database. The upside of this is that it makes changing the shape of a given process type very easy, and could be dynamically driven at any point in the application flow. The drawback is that no checking is performed to see if a given shape is “correct”, it simply is the shape provided in the call to get_type().

Graph View

Backlinks