Package index • MDP2

Building an MDP model to a file

binary_mdp_writer(): Function for writing an HMDP model to binary files. The function defines sub-functions which can be used to define an HMDP model saved in a set of binary files.
binary_action_writer(): Function for writing actions of a HMDP model to binary files. The function defines sub-functions which can be used to define actions saved in a set of binary files. It is assumed that the states have been defined using binary_mdp_writer and that the id of the states is known (can be retrieved using e.g. state_idx_df).
hmp_mdp_writer(): Function for writing an HMDP model to a hmp file (XML). The function define sub-functions which can be used to define an HMDP model stored in a hmp file.
memory_mdp_writer(): Function for building an HMDP model directly in memory.
get_bin_info_actions(): Info about the actions in the HMDP model under consideration.
get_bin_info_states(): Info about the states in the binary files of the HMDP model under consideration.
random_hmdp(): Generate a "random" HMDP stored in a set of binary files.

load_mdp(): Load the HMDP model defined in the binary files. The model are created in memory using the external C++ library.
save_mdp(): Save the MDP to binary files

get_info(): Information about the MDP
plot(<HMDP>): Plot the state-expanded hypergraph of the MDP.
plot_hypergraph(): Plot parts of the state expanded hypergraph.
get_hypergraph(): Return the (parts of) state-expanded hypergraph
get_w_idx(): Return the index of a weight in the model. Note that index always start from zero (C++ style), i.e. the first weight, the first state at a stage etc has index 0.

run_policy_ite_ave(): Perform policy iteration using the average expected-weight Bellman operator on the MDP.
run_policy_ite_discount(): Perform policy iteration using the discounted expected-weight Bellman operator on the MDP.
run_value_ite(): Perform value iteration on the MDP.

set_policy(): Modify the current policy by setting policy action of states.
get_policy(): Get parts of the optimal policy.
run_calc_weights(): Calculate weights based on current policy. Normally run after an optimal policy has been found.
get_steady_state_pr(): Calculate the steady state transition probabilities for the founder process (level 0).
get_rpo(): Calculate the retention pay-off (RPO) or opportunity cost for some states.

convert_binary_to_hmp(): Convert a HMDP model stored in binary format to a hmp (XML) file. The function simply parse the binary files and create hmp files using the hmp_mdp_writer().
convert_hmp_to_binary(): Convert a HMDP model stored in a hmp (xml) file to binary file format.