Projects Package¶

`Projects` Package¶

This package contains all HPCStats Project importers.

`ProjectImporter` Module¶

This module contains the base class for all Projects importers.

class HPCStats.Importer.Projects.ProjectImporter.ProjectImporter(app, db, config)¶

Bases: HPCStats.Importer.Importer.Importer

This is the base class common to all HPCStats Projects importers. It defines a common set of attributes and generic methods.

find_domain(search)¶: Search for a Domain over the list of Domains loaded by importer in self.domains attribute. Returns None if not found.

find_project(search)¶: Search for a Project over the list of Projects loaded by importer in self.projects attribute. Returns None if not found.

`ProjectImporterCSV` Module¶

This module import projects from CSV file. The CSV file must be formatted like this:

<project_code>;<project_description>; [<domain_id>] <domain_description>; [<sector_id>] <sector_description>

Where:

The project_code is a mandatory string
The project_description is an optional string
The domain_id is an optional string
The domain_description is an optional string
The sector_id is an optional string
The sector_description is an optional string

A domain has both an ID and a description. However, a sector could have an ID but not description. The reverse is not true: there could not be a sector w/ a description w/o ID.

A project could have a domain and no sector. But the reverse is not true: there could not be project w/ a sector and w/o domain.

class HPCStats.Importer.Projects.ProjectImporterCSV.ProjectImporterCSV(app, db, config)¶

Bases: HPCStats.Importer.Projects.ProjectImporter.ProjectImporter

Main class of this module.

check()¶: Check if CSV file exists and is a proper flat file.

load()¶: Open CSV file and load project out of it. Raises Exceptions if error is found in the file. Returns the list of Projects with their Domains.

update()¶: Update loaded project (with associated domains) in database.

`ProjectImporterDummy` Module¶

Dummy Project importer

class HPCStats.Importer.Projects.ProjectImporterDummy.ProjectImporterDummy(app, db, config)¶

Bases: HPCStats.Importer.Projects.ProjectImporter.ProjectImporter

Main class of this module.

check()¶: Dummy check

load()¶: Dummy Projects load.

update()¶: Dummy Projects update.

`ProjectImporterFactory` Module¶

This module contains the factory design pattern class that builds the appropriate ProjectImporter depending on what is specified in configuration.

class HPCStats.Importer.Projects.ProjectImporterFactory.ProjectImporterFactory¶

Bases: object

This class simply delivers the factory() static method, there is not point to instanciate it with an object.

static factory(app, db, config)¶: This method returns the appropriate ProjectImporter object depending on what is specified in configuration. In case of configuration error, HPCStatsConfigurationException is raised.

`ProjectImporterSlurm` Module¶

This module import projects from wckeys in Slurm accouting database.

class HPCStats.Importer.Projects.ProjectImporterSlurm.ProjectImporterSlurm(app, db, config)¶

Bases: HPCStats.Importer.Projects.ProjectImporter.ProjectImporter

Main class of this module.

check()¶: Check if all Slurm databases are available and if we connect to them.

connect_db(cluster)¶: Connect to a cluster Slurm database and set conn/cur attributes accordingly.

disconnect_db()¶: Disconnect to currently connected Slurm DB.

load()¶: Connects to all known Slurm databases to extract project codes in jobs wckeys. If a cluster raises a HPCStatsSourceError, it prints an error and continues with the next one.

load_cluster(cluster)¶: Connect to cluster Slurm database to extract project codes from jobs wckeys. Raises HPCStatsSourceError in case of error.

update()¶

Create loaded project (with associated domains) in database if not existing. It does not update domain nor project since:

There is only one default domain which should have created with its key and name in the first place, there is no point to update it.

The project are imported without description from Slurm and are linked to default domain. Operators have probably added a description to the project and linked it to another proper domain and we must not alter these information.

Projects Package¶