Package index

Building an MDP model to a file

binaryMDPWriter()

Function for writing an HMDP model to binary files. The function defines sub-functions which can be used to define an HMDP model saved in a set of binary files.

binaryActionWriter()

Function for writing actions of a HMDP model to binary files. The function defines sub-functions which can be used to define actions saved in a set of binary files. It is assumed that the states have been defined using binaryMDPWriter and that the id of the states is known (can be retrieved using e.g. stateIdxDf).

hmpMDPWriter()

Function for writing an HMDP model to a hmp file (XML). The function define sub-functions which can be used to define an HMDP model stored in a hmp file.

getBinInfoActions()

Info about the actions in the HMDP model under consideration.

getBinInfoStates()

Info about the states in the binary files of the HMDP model under consideration.

randomHMDP()

Generate a "random" HMDP stored in a set of binary files.

Loading the MDP

loadMDP()

Load the HMDP model defined in the binary files. The model are created in memory using the external C++ library.

saveMDP()

Save the MDP to binary files

Get info about the MDP

getInfo()

Information about the MDP

plot(<HMDP>)

Plot the state-expanded hypergraph of the MDP.

plotHypergraph()

Plot parts of the state expanded hypergraph (experimental).

getHypergraph()

Return the (parts of) state-expanded hypergraph

getWIdx()

Return the index of a weight in the model. Note that index always start from zero (C++ style), i.e. the first weight, the first state at a stage etc has index 0.

Finding the optimal policy

runPolicyIteAve()

Perform policy iteration (average reward criterion) on the MDP.

runPolicyIteDiscount()

Perform policy iteration (discounted reward criterion) on the MDP.

runValueIte()

Perform value iteration on the MDP.

Get info about the current policy

setPolicy()

Modify the current policy by setting policy action of states.

getPolicy()

Get parts of the optimal policy.

runCalcWeights()

Calculate weights based on current policy. Normally run after an optimal policy has been found.

getSteadyStatePr()

Calculate the steady state transition probabilities for the founder process (level 0).

getRPO()

Calculate the retention pay-off (RPO) or opportunity cost for some states.

Convert to different file formats

convertBinary2HMP()

Convert a HMDP model stored in binary format to a hmp (XML) file. The function simply parse the binary files and create hmp files using the hmpMDPWriter().

convertHMP2Binary()

Convert a HMDP model stored in a hmp (xml) file to binary file format.