Model Package¶

`Account` Module¶

Schema of the Account table in HPCStats database:

Account(
  account_uid      integer NOT NULL,
  account_gid      integer NOT NULL,
  account_creation date NOT NULL,
  account_deletion date NOT NULL,
  userhpc_id       integer NOT NULL,
  cluster_id       integer NOT NULL,
  CONSTRAINT Account_pkey PRIMARY KEY (userhpc_id, cluster_id)
)

class HPCStats.Model.Account.Account(user, cluster, uid, gid, creation_date, deletion_date, exists=None)¶

Bases: object

Model class for the Account table.

existing(db)¶: Returns True if the account already exists in database (same cluster and user), or False if not.

load(db)¶: Load the Account based on the content of the DB and set objects attributes accordingly. It calls existing() method if needed and raises HPCStatsRuntimeError if the Account is not found in DB.

save(db)¶: Insert Account in database. It first makes sure that the Account does not already exist in database yet by calling Account.existing() method if needed. If the account already exists in database, it raises HPCStatsRuntimeError.

update(db)¶: Update Account uid, gid, creation date and deletion date in database. Raises HPCStatsRuntimeError is exists is False.

HPCStats.Model.Account.load_unclosed_users_accounts(db, cluster)¶: Load (User,Account) tuples w/o account deletion date on cluster from DB.

HPCStats.Model.Account.nb_existing_accounts(db, cluster)¶: Return the number of existing account for the cluster in DB.

`Business` Module¶

Schema of the Business table in HPCStats database:

Business(
  business_code        character varying(30) NOT NULL,
  business_description text NOT NULL,
  CONSTRAINT Business_pkey PRIMARY KEY (business_code)
)

class HPCStats.Model.Business.Business(code, description)¶

Bases: object

Model class for the Business table.

existing(db)¶: Returns True if the business already exists in database (with same code), or False if not.

save(db)¶: Insert Business in database. It first makes sure that the Business does not already exist in database yet by calling Business.existing() method if needed. If the business already exists in database, it raises HPCStatsRuntimeError.

update(db)¶: Update Business description in database. Raises HPCStatsRuntimeError if exists is False.

`Cluster` Module¶

Schema of the Cluster table in HPCStats database:

Cluster(
  cluster_id   SERIAL,
  cluster_name character varying(30) NOT NULL,
  CONSTRAINT Cluster_pkey PRIMARY KEY (cluster_id),
  CONSTRAINT Cluster_unique UNIQUE (cluster_name)
)

class HPCStats.Model.Cluster.Cluster(name, cluster_id=None)¶

Bases: object

Model class for Cluster table

find(db)¶: Search the Cluster in the database based on its name. If exactly one cluster matches in database, set cluster_id attribute properly and returns its value. If more than one cluster matches, raises HPCStatsDBIntegrityError. If no cluster is found, returns None.

get_min_datetime(db)¶: Returns the start datetime of the oldest started and unfinished job on the cluster.

get_nb_accounts(db, creation_date)¶: Returns the total of users on the cluster whose account have been created defore date given in parameter.

get_nb_active_users(db, start, end)¶: Returns the total number of users who have run job(s) on the cluster between start and end datetimes in parameters.

get_nb_cpus(db)¶: Returns the total number of CPUs available on the cluster

save(db)¶: Insert Cluster in database. You must make sure that the Cluster does not already exist in database yet (typically using Cluster.find() method else there is a risk of future integrity errors because of duplicated clusters. If cluster_id attribute is set, it raises HPCStatsRuntimeError.

`Domain` Module¶

Schema of the Domain table in HPCStats database:

Domain(
  domain_id   character varying(4) NOT NULL,
  domain_name character varying(30),
  CONSTRAINT Domain_pkey PRIMARY KEY (domain_id),
  CONSTRAINT Domain_unique UNIQUE (domain_name)
)

class HPCStats.Model.Domain.Domain(key, name)¶

Bases: object

Model class for the Domain table.

existing(db)¶: Returns True if the domain already exists in database (same key), or False if not.

save(db)¶: Insert Domain in database. It first makes sure that the Domain does not already exist in database yet by calling Domain.existing() method if needed. If the domain already exists in database, it raises HPCStatsRuntimeError.

update(db)¶: Update Domain name in database. Raises HPCStatsRuntimeError if exists is False.

`Event` Module¶

Schema of the Event table in HPCStats database:

Event(
  event_id     SERIAL,
  event_type   character varying(30) NOT NULL,
  event_reason character varying(30),
  event_nbCpu  integer NOT NULL,
  event_start  timestamp NOT NULL,
  event_end    timestamp NOT NULL,
  node_id      integer NOT NULL,
  cluster_id   integer NOT NULL,
  CONSTRAINT Event_pkey PRIMARY KEY (event_id, node_id, cluster_id),
  CONSTRAINT Event_unique UNIQUE (node_id, cluster_id, event_start)
)

class HPCStats.Model.Event.Event(cluster, node, nb_cpu, start_datetime, end_datetime, event_type, reason, event_id=None)¶

Bases: object

Model class for the Event table.

find(db)¶: Search the Event in the database based on the node, the cluster, the start and the end datetime. If exactly one event matches in database, set event_id attribute properly and returns its value. If more than one event matches, raises HPCStatsDBIntegrityError. If no event is found, returns None.

merge_event(event)¶: Set Event end datetime equals to event in parameter end datetime.

save(db)¶: Insert Event in database. You must make sure that the Event does not already exist in database yet (typically using Event.find() method else there is a risk of future integrity errors because of duplicated events. If event_id attribute is set, it raises HPCStatsRuntimeError.

update(db)¶: Update the Event in DB. The event_id attribute must be set for the Event, either by passing this id to __init__() or by calling Event.find() method previously. If event_id attribute is not set, it raises HPCStatsRuntimeError.

HPCStats.Model.Event.get_datetime_end_last_event(db, cluster)¶: Returns the end datetime of the last event on the cluster in DB. Returns None if no event found.

HPCStats.Model.Event.get_datetime_start_oldest_unfinished_event(db, cluster)¶: Returns the start datetime of the oldest event on the cluster in DB. Returns None if no unfinished event found.

HPCStats.Model.Event.get_unfinished_events(db, cluster)¶: Get the list of unfinished Events on the cluster out of DB. Returns None if no unfinished event found.

`FSUsage` Module¶

Schema of the fsusage table in HPCStats database:

fsusage(
  fsusage_time   timestamp NOT NULL,
  fsusage_usage  real NOT NULL,
  fsusage_inode  real,
  filesystem_id  integer NOT NULL,
  cluster_id     integer NOT NULL,
  CONSTRAINT fsusage_pkey PRIMARY KEY (fsusage_time, filesystem_id, cluster_id)
)

class HPCStats.Model.FSUsage.FSUsage(filesystem, timestamp, usage, inode)¶

Bases: object

Model class for the fsusage table.

existing(db)¶: Returns True if the FSUsage already exists in database (same cluster, filesystem and timestamp), or False if not.

save(db)¶: Insert FSUsage in database. It first makes sure that the FSUsage does not already exist in database yet by calling FSUsage.existing() method if needed. If the fsusage already exists in database, it raises HPCStatsRuntimeError.

HPCStats.Model.FSUsage.get_last_fsusage_datetime(db, cluster, filesystem)¶: Get the datetime of the last fsusage for the fs in DB.

`Filesystem` Module¶

Schema of the filesystem table in HPCStats database:

filesystem(
  filesystem_id   SERIAL,
  filesystem_name character varying(30) NOT NULL,
  cluster_id      integer NOT NULL,
  CONSTRAINT filesystem_pkey PRIMARY KEY (filesystem_id, cluster_id),
  CONSTRAINT filesystem_unique UNIQUE (filesystem_name, cluster_id)
)

class HPCStats.Model.Filesystem.Filesystem(mountpoint, cluster, fs_id=None)¶

Bases: object

Model class for the filesystem table.

find(db)¶: Search the Filesystem in the database based on its mountpoint and cluster. If exactly one filesystem matches in database, set fs_id attribute properly and returns its value. If more than one filesystem matches, raises HPCStatsDBIntegrityError. If no filesystem is found, returns None.

save(db)¶: Insert Filesystem in database. You must make sure that the Filesystem does not already exist in database yet (typically using Filesystem.find() method else there is a risk of future integrity errors because of duplicated filesystems. If fs_id attribute is set, it raises HPCStatsRuntimeError.

`Job` Module¶

Schema of the Job table in HPCStats database:

Job(
  job_id         SERIAL,
  job_sched_id   integer NOT NULL,
  job_batch_id   character varying(30) NOT NULL,
  job_nbCpu      integer NOT NULL,
  job_name       character varying(200),
  job_state      character varying(50) NOT NULL,
  job_queue      character varying(50),
  job_account    character varying(50),
  job_department character varying(30),
  job_submission timestamp NOT NULL,
  job_start      timestamp,
  job_end        timestamp,
  userhpc_id     integer NOT NULL,
  cluster_id     integer NOT NULL,
  project_id     integer NOT NULL,
  business_code  character varying(30) NOT NULL,
  CONSTRAINT Job_pkey PRIMARY KEY (job_id),
  CONSTRAINT Job_unique UNIQUE (job_batch_id, cluster_id)
)

class HPCStats.Model.Job.Job(account, project, business, sched_id, batch_id, name, nbcpu, state, queue, job_acct, job_department, submission, start, end, job_id=None)¶

Bases: object

Model class for the Cluster table.

find(db)¶: Search the Job in the database based on its sched_id and cluster. If exactly one job matches in database, set job_id attribute properly and returns its value. If more than one job matches, raises HPCStatsDBIntegrityError. If no job is found, returns None.

save(db)¶: Insert Job in database. You must make sure that the Job does not already exist in database yet (typically using Job.find() method else there is a risk of future integrity errors because of duplicated jobs. If job_id attribute is set, it raises HPCStatsRuntimeError.

update(db)¶: Update Job sched_id, nbcpu, name, state, queue, submission, start and end in database. Raises HPCStatsRuntimeError if self.job_id is None.

HPCStats.Model.Job.get_batchid_last_job(db, cluster)¶: Return the batch_id of the last job recorded in HPCStats DB as an integer. Returns None if not found.

HPCStats.Model.Job.get_batchid_oldest_unfinished_job(db, cluster)¶: Return the batch_id of the oldest unfinished job recorded in HPCStats DB as an integer. Returns None if not found.

`Node` Module¶

Schema of the Node table in HPCStats database:

Node(
  node_id        SERIAL,
  node_name      character varying(30) NOT NULL,
  node_model     character varying(50) NOT NULL,
  node_nbCpu     integer,
  node_partition character varying(30) NOT NULL,
  node_flops     integer NOT NULL,
  node_memory    integer NOT NULL,
  cluster_id     integer NOT NULL,
  CONSTRAINT Node_pkey PRIMARY KEY (node_id, cluster_id),
  CONSTRAINT Node_unique UNIQUE (node_name, cluster_id)
)

class HPCStats.Model.Node.Node(name, cluster, model, partition, cpu, memory, flops, node_id=None)¶

Bases: object

Model class for the Node table.

find(db)¶: Search the Node in the database based on its name and cluster. If exactly one node matches in database, set node_id attribute properly and returns its value. If more than one node matches, raises HPCStatsDBIntegrityError. If no node is found, returns None.

save(db)¶: Insert Node in database. You must make sure that the Node does not already exist in database yet (typically using Node.find() method else there is a risk of future integrity errors because of duplicated nodes. If node_id attribute is set, it raises HPCStatsRuntimeError.

update(db)¶: Update Node partition, cpu, memory and flops fields in database. Raises HPCStatsRuntimeError if self.node_id is None.

`Project` Module¶

Schema of the Project table in HPCStats database:

Project(
  project_id          SERIAL,
  project_code        character varying(30) NOT NULL,
  project_description text,
  domain_id           character varying(4),
  CONSTRAINT Project_pkey PRIMARY KEY (project_id),
  CONSTRAINT Project_unique UNIQUE (project_code)
)

class HPCStats.Model.Project.Project(domain, code, description, project_id=None)¶

Bases: object

Model class for the Project table

find(db)¶: Search the Project in the database based on its code. If exactly one project matches in database, set project_id attribute properly and returns its value. If more than one project matches, raises HPCStatsDBIntegrityError. If no project is found, returns None.

load(db)¶: Load the Project based on the content of the DB and set objects attributes accordingly. The project_id attribute must have been set previously, typically by calling find() method. It raises raises HPCStatsRuntimeError if the Project is not found in DB.

save(db)¶: Insert Project in database. You must make sure that the Project does not already exist in database yet (typically using Project.find() method else there is a risk of future integrity errors because of duplicated clusters. If project_id attribute is set, it raises HPCStatsRuntimeError.

update(db)¶: Update Project description field in database. Raises HPCStatsRuntimeError if self.project_id is None.

`Run` Module¶

Schema of the Run table in HPCStats database:

Run(
  job_id         integer NOT NULL,
  node_id        integer NOT NULL,
  cluster_id     integer NOT NULL,
  CONSTRAINT Run_pkey PRIMARY KEY (job_id, node_id, cluster_id)
)

class HPCStats.Model.Run.Run(cluster, node, job)¶

Bases: object

Model class for the Run table.

existing(db)¶: Returns True if theRunt already exists in database (same cluster, node and job), or False if not.

save(db)¶: Insert Run in database. It first makes sure that the Run does not already exist in database yet by calling Run.existing() method if needed. If the Run already exists in database, it raises HPCStatsRuntimeError.

`User` Module¶

Schema of the Userhpc table in HPCStats database:

Userhpc(
  userhpc_id         SERIAL,
  userhpc_login      character varying(30) NOT NULL,
  userhpc_name       character varying(30) NOT NULL,
  userhpc_firstname  character varying(30) NOT NULL,
  userhpc_department character varying(30),
  CONSTRAINT Userhpc_pkey PRIMARY KEY (userhpc_id),
  CONSTRAINT Userhpc_unique UNIQUE (userhpc_login)
)

class HPCStats.Model.User.User(login, firstname, lastname, department, user_id=None)¶

Bases: object

Model class for the Userhpc table.

find(db)¶: Search the User in the database based on its login. If exactly one user matches in database, set user_id attribute properly and returns its value. If more than one user matches, raises HPCStatsDBIntegrityError. If no user is found, returns None.

save(db)¶: Insert User in database. You must make sure that the User does not already exist in database yet (typically using User.find() method else there is a risk of future integrity errors because of duplicated users. If user_id attribute is set, it raises HPCStatsRuntimeError.

update(db)¶: Update User firstname, lastname and department fields in database. Raises HPCStatsRuntimeError is user_id is None.

Model Package¶

Account Module¶

Business Module¶

Cluster Module¶

Domain Module¶

Event Module¶

FSUsage Module¶

Filesystem Module¶

Job Module¶

Node Module¶

Project Module¶

Run Module¶

User Module¶

`Account` Module¶

`Business` Module¶

`Cluster` Module¶

`Domain` Module¶

`Event` Module¶

`FSUsage` Module¶

`Filesystem` Module¶

`Job` Module¶

`Node` Module¶

`Project` Module¶

`Run` Module¶

`User` Module¶