Model Management

The rorodata platform has built-in support for version control of machine learning models. The model version control allows you to store trained models, and track the model metadata which can used for audits and improvements later.

Concepts

Model Repository

Each Project can have zero or more model repositories. The model repositories manage multiple versions of one model.

Model Image

Each version is called ModelImage (similar to machine image), which contains the serialized object, associated metadata, links to sources, destinations and attached files.

            +-----------------------------------------------+
            | ModelRepository A                             |
            |                                               |
            |  ModelImage - v1         ModelImage - v2      |
            | +---------------+       +---------------+     |
            | | Model v1      |       | Model v1      |     |
            | +---------------+       +---------------+     |
            | | Metadata v1   |       | Metadata v1   |     |
            | +---------------+       +---------------+     |
            +-----------------------------------------------+

            +-----------------------------------------------+
            | ModelRepository B                             |
            |                                               |
            |  ModelImage - v1         ModelImage - v2      |
            | +---------------+       +---------------+     |
            | | Model v1      |       | Model v1      |     |
            | +---------------+       +---------------+     |
            | | Metadata v1   |       | Metadata v1   |     |
            | +---------------+       +---------------+     |
            +-----------------------------------------------+

The platform uses AWS S3 to store all the model images, all the metadata is stored in relational database.

Installation

The model management API is a default with roro.

Before you start with model management: Make sure you've signed up and installed roro CLI on your system - it should only take a minute!

Model Management API

Project Class

The Project Class contains all the utilites required to get access, create model repositories.

get_current_project()

The project is a unit "workspace" in rorodata. Model artefacts are segreggated by projects. This functions allows getting the current project object.

Usage: get_current_project()

Parameters: None

Response: Project object <Project your-project-name>

Example

>> import roro
>> project = roro.get_current_project()
>> project
<Project face-detection>

get_model_repository()

This function returns reference to a model repository for the model name passed as a parameter.

Usage: Project.get_current_project(name)

Parameter: TEXT string with model name  (required)

Response: ModelRepository object <ModelRepository <your-project-name>/TEXT>

Example

>> project = roro.get_current_project()
>> model_repo = project.get_model_repository("face-detection")
<ModelRepository face-detection/VGG16-tl-model>

list_model_repositories()

This function returns a list of all the model repositories associated with the project.

Usage: Project.list_model_repositories()

Parameter: None

Response: [<ModelRepository <your-project-name>/<model-name-1>,<ModelRepository <your-project-name>/<model-name-2>, ...]

Example

>> project = roro.get_current_project()
>> model_repo = project.list_model_repositories()
[<ModelRepository face-detection/VGG16-tl-model>, <ModelRepository face-detection/facenet-tl-model>]

create_model_repository()

Creates a new model repository with a given name in the current project.

Usage: Project.create_model_repository()

Parameter: TEXT string 

Response: ModelRepository object <ModelRepository your-project-name>

Example

>> project = roro.get_current_project()
>> model_repo = project.create_model_repository("dl-models")
>> model_repo
<ModelRepository dl-models>

ModelRepository Class

The ModelRepository Class contains all the utilites required to get access, create model images. The class also contains utilities to manage model repository

get_activity()

This function returns list of all models in the repository

Usage: ModelRepository.get_activity()

Parameters: None

Response: [List of ModelImage objects]

Example

>> import roro
>> project = roro.get_current_project()

# Take the repo for required model
>> model_repo = project.get_model_repository("face-detection")
>> model_repo.get_activity()
[<ModelImage face-detection/VGG16-tl-model@4>, <ModelImage face-detection/VGG16-tl-model@3>, <ModelImage face-detection/VGG16-tl-model@2>, <ModelImage face-detection/VGG16-tl-model@1>]

new_model_image()

This function creates a new model image and returns the reference to the image. Please note, the ModelImage.save() must be called on the new model image object. The input paramater to the function is the model object of interest.

Usage: ModelRepository.new_model_image(self, model)

Parameters: Model object

Response: ModelImage object <ModelImage <your-project-name>/<model-name>>

Example

# run the training algorithm to build the model
>> model = train_machine_learning_model()

>> model_image = model_repo.new_model_image(model)

# Save the image
>> model_image['Input-Data-Source'] = 's3://image-dataset-201706'
>> model_image['Accuracy'] = 0.89
>> model_image.save(comment="Built new model using the data till June 2017")

get_model_image()

This function returns a model image for the model, given the version number and tag name. The input parameters to the function are verion string or tag name string.

Usage: ModelRepository.get_model_image(self, version=None, tag=None)

Parameters: version TEXT, tag TEXT      (either one required)

Response: ModelImage object <ModelImage <your-project-name>/<model-name>>

Example

>> import roro
>> project = roro.get_current_project()

# Take the repo for required model
>> model_repo = project.get_model_repository("face-detection")

# get the image of the latest version or any tag
>> model_image = model_repo.get_model_image(tag="latest")
<ModelImage face-detection/VGG16-tl-model@4>

get_tags()

This function returns all the tags available in the current repository.

This utility is not deprecated in roro 0.1.10 onwards


add_tag()

This function tags a specified version of model image with the given tag. The function takes string tag name as input.

Usage: ModelRepository.add_tag(self, tag=None)

Parameters: tag TEXT        (required)

Response: ModelImage object <ModelImage <your-project-name>/<model-name>@<tag>>

Example

>> import roro
>> project = roro.get_current_project()

# Take the repo for required model
>> model_repo = project.get_model_repository("face-detection")

# get the image of the latest version or any tag
>> model_image = model_repo.get_model_image(tag="latest")
>> model_image.add_tag(tag="latest")
>> model_image = model_repo.get_model_image(tag="latest")
<ModelImage face-detection/VGG16-tl-model@4>

ModelImage Class

The ModelImage Class contains all the utilites to manage model images. A ModelImage represents one version of an machine learning model and its metadata.

Metadata can be added to a model image like a dictionary, like so:

model_image['Accuracy'] = 0.83
model_image['Input-Source'] = 's3://face-detection-201706'
save()

Saves model image to current model repository. The function takes an optional comment argument - this is similar to a git commit message.

Usage: ModelImage.save(self, comment)

Parameters: Text string for the comment     (optional)

Response: ModelImage object

Example

# run the training algorithm to build the model
>> model = train_machine_learning_model()

# create a new model image with metadata
>> model_image = model_repo.new_model_image(model)
>> model_image['Input-Data-Source'] = 's3://image-dataset-201706'
>> model_image['Accuracy'] = 0.89

# save the model
>> model_image.save(comment="Built new model using the data till June 2017")

get_model()

This function returns the current model object.

Usage: ModelImage.get_model(self)

Parameters: None

Response: ModelImage object

Example

>> import roro
>> project = roro.get_current_project()

# Take the repo for required model
>> model_repo = project.get_model_repository("face-detection")

# get the image of the latest version or any tag
>> model_image = model_repo.get_model_image(tag="latest")
>> model_image.get_model()

get_details()

This function returns metadata associated with the model image.

Usage: ModelImage.get_details(self)

Parameters: None

Response: Dict with metadata for the ModelImage

Example

>> import roro
>> project = roro.get_current_project()

# Take the repo for required model
>> model_repo = project.get_model_repository("face-detection")

# get the image of the latest version or any tag
>> model_image = model_repo.get_model_image(tag="latest")
>> model_image.get_details()
'Model-ID: 4fbe8871\nModel-Name: VGG16-tl-model\nModel-Version: 4\nDate: 2017-10-02 08:32:05.944665\nTest-Score: 0.811846431485\n\n        built new face-detection model\n'

Related: get_summary() returns a summary view of the above detailed view.


set_metadata()

This function that sets new metadata to an existing model image.

Usage: ModelImage.set_metadata(self, metadata)

Parameters: Dict with metadata      (required)

Response: Updated ModelImage object

Example

>> import roro
>> project = roro.get_current_project()

# Take the repo for required model
>> model_repo = project.get_model_repository("face-detection")

# get the image of the latest version or any tag
>> model_image = model_repo.get_model_image(tag="latest")
>> model_image.get_details()
'Model-ID: 4fbe8871\nModel-Name: VGG16-tl-model\nModel-Version: 4\nDate: 2017-10-02 08:32:05.944665\nTest-Score: 0.811846431485\n\n        built new face-detection model\n'

>> metadata = {'Author': 'rorodata team'}
>> model_image.set_metadata(metadata=metadata)
>> model_image.get_details()
'Model-ID: 4fbe8871\nModel-Name: VGG16-tl-model\nModel-Version: 4\nDate: 2017-10-02 08:32:05.944665\nAuthor: rorodata team\nTest-Score: 0.811846431485\n\n        built new face-detection model\n'

version

Returns the version number of the model.

Usage: ModelImage.version

Parameters: None

Response: version number string

Example

>> model_image.version
4

name

Returns the name of the model.

Usage: ModelImage.name

Parameters: None

Response: model name string

Example

>> model_image.name
VGG16-tl-model

comment

Returns the last comment/commit message of the model.

Usage: ModelImage.comment

Parameters: None

Response: comment string

Example

>> model_image.comment
built new face-detection model

Feedback

Help us improve the documentation. Flag errors, issues or request how-tos, guides and tutorials on our #documentation channel on our Slack