Package¶
-
class
pydtmc.
MarkovChain
(p, states=None)[source]¶ Bases:
object
Defines a Markov chain with given transition matrix and state names.
- Parameters
- Raises
ValidationError – if any input argument is not compliant.
-
property
absorbing_states
[source]¶ A property representing the absorbing states of the Markov chain.
-
property
absorption_probabilities
[source]¶ A property representing the absorption probabilities of the Markov chain. If the Markov chain is not absorbing and has no transient states, then None is returned.
-
property
absorption_times
[source]¶ A property representing the absorption times of the Markov chain. If the Markov chain is not absorbing, then None is returned.
-
property
accessibility_matrix
[source]¶ A property representing the accessibility matrix of the Markov chain.
- Return type
-
property
adjacency_matrix
[source]¶ A property representing the adjacency matrix of the Markov chain.
- Return type
-
are_communicating
(state1, state2)[source]¶ The method verifies whether the given states of the Markov chain are communicating.
-
backward_committor
(states1, states2)¶ Alias of backward_committor_probabilities.
-
backward_committor_probabilities
(states1, states2)[source]¶ The method computes the backward committor probabilities between the given subsets of the state space defined by the Markov chain.
Aliases: backward_committor- Parameters
- Return type
- Returns
the backward committor probabilities if the Markov chain is ergodic, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if the two sets are not disjoint.
-
static
birth_death
(p, q, states=None)[source]¶ The method generates a birth-death Markov chain of given size and from given probabilities.
- Parameters
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if q and p have different a size or if the vector resulting from the sum of q and p contains any value greater than 1.
-
closest_reversible
(distribution, weighted=False)[source]¶ The method computes the closest reversible of the Markov chain.
Notes: the algorithm is described in Computing the nearest reversible Markov Chain (Nielsen & Weber, 2015).- Parameters
- Return type
Optional
[~MarkovChain]- Returns
a Markov chain if the algorithm finds a solution, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if a weighted Frobenius norm is used and the distribution contains zero-valued probabilities.
-
property
communicating_classes
[source]¶ A property representing the communicating classes of the Markov chain.
-
conditional_probabilities
(state)[source]¶ The method computes the probabilities, for all the states of the Markov chain, conditioned on the process being at a given state.
Aliases: conditional_distribution- Parameters
- Return type
- Returns
the conditional probabilities of the Markov chain states.
- Raises
ValidationError – if any input argument is not compliant.
-
property
determinant
[source]¶ A property representing the determinant the transition matrix of the Markov chain.
- Return type
-
property
entropy_rate
[source]¶ A property representing the entropy rate of the Markov chain. If the Markov chain is not ergodic, then None is returned.
-
property
entropy_rate_normalized
[source]¶ A property representing the entropy rate, normalized between 0 and 1, of the Markov chain. If the Markov chain is not ergodic, then None is returned.
-
expected_rewards
(steps, rewards)[source]¶ The method computes the expected rewards of the Markov chain after N steps, given the reward value of each state.
- Parameters
- Return type
- Returns
the expected rewards of each state of the Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
expected_transitions
(steps, initial_distribution=None)[source]¶ The method computes the expected number of transitions performed by the Markov chain after N steps, given the initial distribution of the states.
- Parameters
- Return type
- Returns
the expected number of transitions on each state of the Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
static
fit_map
(possible_states, walk, hyperparameter=None)[source]¶ The method fits a Markov chain using the maximum a posteriori approach.
- Parameters
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
static
fit_mle
(possible_states, walk, laplace_smoothing=False)[source]¶ The method fits a Markov chain using the maximum likelihood approach.
- Parameters
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
forward_committor
(states1, states2)¶ Alias of forward_committor_probabilities.
-
forward_committor_probabilities
(states1, states2)[source]¶ The method computes the forward committor probabilities between the given subsets of the state space defined by the Markov chain.
Aliases: forward_committor- Parameters
- Return type
- Returns
the forward committor probabilities if the Markov chain is ergodic, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if the two sets are not disjoint.
-
static
from_dictionary
(d)[source]¶ The method generates a Markov chain from the given dictionary, whose keys represent state pairs and whose values represent transition probabilities.
- Parameters
d (
Dict
[Tuple
[str
,str
],Union
[float
,int
]]) – the dictionary to transform into the transition matrix.- Return type
- Returns
a Markov chain.
- Raises
ValueError – if the transition matrix defined by the dictionary is not valid.
ValidationError – if any input argument is not compliant.
-
static
from_file
(file_path)[source]¶ The method reads a Markov chain from the given file.
Notes: every line of the file must have the following format: <state_from> <state_to> <probability>- Parameters
file_path (
str
) – the location of the file that defines the Markov chain.- Return type
- Returns
a Markov chain.
- Raises
FileNotFoundError – if the file does not exist.
OSError – if the file cannot be read.
ValueError – if the file path is invalid or if the file contains invalid data.
-
static
from_function
(f, possible_states, quadrature_interval=None, quadrature_type='newton-cotes')[source]¶ The method generates a Markov chain from the given transition function.
- Parameters
f (
Callable
[[float
,float
],float
]) – the transition function of the process.possible_states (
List
[str
]) – the possible states of the process.quadrature_type (
str
) – the quadrature type to use for the computation of nodes and weights (one of gauss-chebyshev, gauss-legendre, neiderreiter, newton-cotes, simpson or trapezoid-rule; newton-cotes by default).quadrature_interval (
Optional
[Tuple
[Union
[float
,int
],Union
[float
,int
]]]) – the quadrature interval to use for the computation of nodes and weights (if omitted, the interval [0, 1] is used).
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if the Gauss-Legendre quadrature fails to converge or if the Simpson quadrature is attempted on an even number of possible states.
-
static
from_matrix
(m, states=None)[source]¶ The method generates a Markov chain with the given state names, whose transition matrix is obtained through the normalization of the given matrix.
- Parameters
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
property
fundamental_matrix
[source]¶ A property representing the fundamental matrix of the Markov chain. If the Markov chain has no transient states, then None is returned.
-
hitting_probabilities
(states=None)[source]¶ The method computes the hitting probability, for all the states of the Markov chain, to the given set of states.
-
static
identity
(size, states=None)[source]¶ The method generates a Markov chain of given size based on an identity transition matrix.
- Parameters
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
property
is_absorbing
[source]¶ A property indicating whether the Markov chain is absorbing.
- Return type
-
is_absorbing_state
(state)[source]¶ The method verifies whether the given state of the Markov chain is absorbing.
- Parameters
- Return type
- Returns
True if the state is absorbing, False otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
is_accessible
(state_target, state_origin)[source]¶ The method verifies whether the given target state is reachable from the given origin state.
-
property
is_aperiodic
[source]¶ A property indicating whether the Markov chain is aperiodic.
- Return type
-
property
is_canonical
[source]¶ A property indicating whether the Markov chain has a canonical form.
- Return type
-
is_cyclic_state
(state)[source]¶ The method verifies whether the given state is cyclic.
- Parameters
- Return type
- Returns
True if the state is cyclic, False otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
property
is_ergodic
[source]¶ A property indicating whether the Markov chain is ergodic or not.
- Return type
-
property
is_irreducible
[source]¶ A property indicating whether the Markov chain is irreducible.
- Return type
-
is_recurrent_state
(state)[source]¶ The method verifies whether the given state is recurrent.
- Parameters
- Return type
- Returns
True if the state is recurrent, False otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
property
is_reversible
[source]¶ A property indicating whether the Markov chain is reversible.
- Return type
-
property
is_symmetric
[source]¶ A property indicating whether the Markov chain is symmetric.
- Return type
-
is_transient_state
(state)[source]¶ The method verifies whether the given state is transient.
- Parameters
- Return type
- Returns
True if the state is transient, False otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
property
kemeny_constant
[source]¶ A property representing the Kemeny’s constant of the fundamental matrix of the Markov chain. If the Markov chain is not absorbing, then None is returned.
-
property
mean_first_passage_times
[source]¶ A property representing the mean first passage times of the Markov chain. If the Markov chain is not ergodic, then None is returned.
Aliases: mfpt
-
mean_first_passage_times_between
(states_target, states_origin)[source]¶ The method computes the mean first passage times between the given subsets of the state space.
Aliases: mfpt_between- Parameters
- Return type
- Returns
the mean first passage times between the given subsets if the Markov chain is irreducible, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
mean_first_passage_times_to
(states=None)[source]¶ The method computes the mean first passage times, for all the states, to the given set of states.
Aliases: mfpt_to
-
mfpt_between
(states_target, states_origin)¶ Alias of mean_first_passage_times_between.
-
property
mixing_rate
[source]¶ A property representing the mixing rate of the Markov chain. If the SLEM (second largest eigenvalue modulus) cannot be computed, then None is returned.
-
mixing_time
(initial_distribution=None, jump=1, cutoff_type='natural')[source]¶ The method computes the mixing time of the Markov chain, given the initial distribution of the states.
- Parameters
initial_distribution (
Optional
[Iterable
[+T_co]]) – the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).jump (
int
) – the number of steps in each iteration (by default, 1).cutoff_type (
str
) – the type of cutoff to use (either natural or traditional; natural by default).
- Return type
- Returns
the mixing time if the Markov chain is ergodic, None otherwise.
- Raises
ValidationError – if any input argument is not compliant.
-
property
periods
[source]¶ A property representing the period of each communicating class defined by the Markov chain.
-
property
pi
[source]¶ A property representing the stationary distributions of the Markov chain.
Aliases: stationary_distributions, steady_states
-
predict
(steps, initial_state=None, include_initial=False, output_indices=False, seed=None)[source]¶ The method simulates the most probable outcome of a random walk of N steps.
Notes: in case of probability tie, the subsequent state is chosen uniformly at random among all the equiprobable states.- Parameters
steps (
int
) – the number of steps.initial_state (
Union
[int
,str
,None
]) – the initial state of the prediction (if omitted, it is chosen uniformly at random).include_initial (
bool
) – a boolean indicating whether to include the initial state in the output sequence (by default, False).output_indices (
bool
) – a boolean indicating whether to the output the state indices (by default, False).seed (
Optional
[int
]) – a seed to be used as RNG initializer for reproducibility purposes.
- Return type
- Returns
the sequence of states produced by the simulation.
- Raises
ValidationError – if any input argument is not compliant.
-
prior_probabilities
(hyperparameter=None)[source]¶ The method computes the prior probabilities, in logarithmic form, of the Markov chain.
- Parameters
hyperparameter (
Optional
[Iterable
[+T_co]]) – the matrix for the a priori distribution (if omitted, a default value of 1 is assigned to each parameter).- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
static
random
(size, states=None, zeros=0, mask=None, seed=None)[source]¶ The method generates a Markov chain of given size with random transition probabilities.
- Parameters
size (
int
) – the size of the chain.states (
Optional
[List
[str
]]) – the name of each state (if omitted, an increasing sequence of integers starting at 1).zeros (
int
) – the number of zero-valued transition probabilities (by default, 0).mask (
Optional
[Iterable
[+T_co]]) – a matrix representing the locations and values of fixed transition probabilities.seed (
Optional
[int
]) – a seed to be used as RNG initializer for reproducibility purposes.
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
ValueError – if the number of zero-valued transition probabilities exceeds the maximum threshold.
-
property
rank
[source]¶ A property representing the rank of the transition matrix of the Markov chain.
- Return type
-
property
recurrence_times
[source]¶ A property representing the recurrence times of the Markov chain. If the Markov chain has no recurrent states, then None is returned.
-
property
recurrent_classes
[source]¶ A property representing the recurrent classes defined by the Markov chain.
-
property
recurrent_states
[source]¶ A property representing the recurrent states of the Markov chain.
-
redistribute
(steps, initial_status=None, include_initial=False, output_last=True)[source]¶ The method simulates a redistribution of states of N steps.
- Parameters
steps (
int
) – the number of steps.initial_status (
Union
[int
,Iterable
[+T_co],None
]) – the initial state or the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).include_initial (
bool
) – a boolean indicating whether to include the initial distribution in the output sequence (by default, False).output_last (
bool
) – a boolean indicating whether to the output only the last distributions (by default, True).
- Return type
- Returns
the sequence of redistributions produced by the simulation.
- Raises
ValidationError – if any input argument is not compliant.
-
property
relaxation_rate
[source]¶ A property representing the relaxation rate of the Markov chain. If the SLEM (second largest eigenvalue modulus) cannot be computed, then None is returned.
-
static
rouwenhorst_approximation
(size, y_bar, sigma_e, rho)[source]¶ The method, using the Rouwenhorst approximation, computes the Markov chain associated with the discretized version of the following AR(1) process:
\(y_t = \bar{y} + \rho y_{t-1} + \varepsilon_t\)with \(\varepsilon_t \overset{i.i.d}{\sim} \mathcal{N}(0, \sigma)\)- Parameters
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
sensitivity
(state)[source]¶ The method computes the sensitivity matrix of the stationary distribution with respect to a given state.
-
property
spectral_gap
[source]¶ A property representing the spectral gap of the Markov chain. If the Markov chain is not ergodic, then None is returned.
-
static
tauchen_approximation
(size, b, sigma_u, rho, omega=None)[source]¶ The method, using the Tauchen approximation, computes the Markov chain associated with the discretized version of the following AR(1) process:
\(y_{t+1} = b + \rho y_t + u_{t+1}\)with \(u_t \overset{i.i.d}{\sim} \mathcal{N}(0, \sigma)\)- Parameters
size (
int
) – the size of the chain.b (
float
) – the value of the constant term \(b\).sigma_u (
float
) – the standard deviation of the random process \(u\).rho (
float
) – the value of the autocorrelation coefficient \(\rho\).omega (
Optional
[float
]) – the number of standard deviations to approximate out to (by default, 3).
- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
to_canonical
()¶ Alias of to_canonical_form.
- Return type
-
to_canonical_form
()[source]¶ The method returns the canonical form of the Markov chain.
Aliases: to_canonical- Return type
- Returns
a Markov chain.
-
to_directed_graph
(multi=True)[source]¶ The method returns a directed graph representing the Markov chain.
Aliases: to_digraph- Parameters
multi (
bool
) – a boolean indicating whether the graph is allowed to define multiple edges between two nodes (by default, True).- Return type
DiGraph
- Returns
a directed graph.
- Raises
ValidationError – if any input argument is not compliant.
-
to_file
(file_path)[source]¶ The method writes a Markov chain to the given file.
- Parameters
file_path (
str
) – the location of the file in which the Markov chain must be written.- Raises
OSError – if the file cannot be written.
ValueError – if the file path is invalid.
-
to_graph
(multi=True)¶ Alias of to_directed_graph.
- Return type
DiGraph
-
to_lazy
(inertial_weights=0.5)¶ Alias of to_lazy_chain.
- Return type
-
to_lazy_chain
(inertial_weights=0.5)[source]¶ The method returns a lazy chain by adjusting the state inertia of the original process.
- Parameters
inertial_weights (
Union
[float
,int
,Iterable
[+T_co]]) – the inertial weights to apply for the transformation (by default, 0.5).- Return type
- Returns
a Markov chain.
- Raises
ValidationError – if any input argument is not compliant.
-
to_subchain
(states)[source]¶ The method returns a subchain containing all the given states plus all the states reachable from them.
-
property
topological_entropy
[source]¶ A property representing the topological entropy of the Markov chain.
- Return type
-
property
transient_classes
[source]¶ A property representing the transient classes defined by the Markov chain.
-
property
transient_states
[source]¶ A property representing the transient states of the Markov chain.
-
transition_probability
(state_target, state_origin)[source]¶ The method computes the probability of a given state, conditioned on the process being at a given specific state.
-
walk
(steps, initial_state=None, final_state=None, include_initial=False, output_indices=False, seed=None)[source]¶ The method simulates a random walk of N steps.
- Parameters
steps (
int
) – the number of steps.initial_state (
Union
[int
,str
,None
]) – the initial state of the walk (if omitted, it is chosen uniformly at random).final_state (
Union
[int
,str
,None
]) – the final state of the walk (if specified, the simulation stops as soon as it is reached even if not all the steps have been performed).include_initial (
bool
) – a boolean indicating whether to include the initial state in the output sequence (by default, False).output_indices (
bool
) – a boolean indicating whether to the output the state indices (by default, False).seed (
Optional
[int
]) – a seed to be used as RNG initializer for reproducibility purposes.
- Return type
- Returns
the sequence of states produced by the simulation.
- Raises
ValidationError – if any input argument is not compliant.
-
pydtmc.
plot_eigenvalues
(mc, dpi=100)[source]¶ The function plots the eigenvalues of the Markov chain on the complex plane.
- Parameters
mc (~MarkovChain) – the target Markov chain.
dpi (
int
) – the resolution of the plot expressed in dots per inch (by default, 100).
- Return type
- Returns
None if Matplotlib is in interactive mode as the plot is immediately displayed, otherwise the handles of the plot.
- Raises
ValidationError – if any input argument is not compliant.
-
pydtmc.
plot_graph
(mc, nodes_color=True, nodes_type=True, edges_color=True, edges_value=True, dpi=100)[source]¶ The function plots the directed graph of the Markov chain.
Notes: Graphviz and Pydot are not required, but they provide access to extended graphs with additional features.- Parameters
mc (~MarkovChain) – the target Markov chain.
nodes_color (
bool
) – a boolean indicating whether to display colored nodes based on communicating classes (by default, True).nodes_type (
bool
) – a boolean indicating whether to use a different shape for every node type (by default, True).edges_color (
bool
) – a boolean indicating whether to display edges using a gradient based on transition probabilities, valid only for extended graphs (by default, True).edges_value (
bool
) – a boolean indicating whether to display the transition probability of every edge (by default, True).dpi (
int
) – the resolution of the plot expressed in dots per inch (by default, 100).
- Return type
- Returns
None if Matplotlib is in interactive mode as the plot is immediately displayed, otherwise the handles of the plot.
- Raises
ValidationError – if any input argument is not compliant.
-
pydtmc.
plot_redistributions
(mc, distributions, initial_status=None, plot_type='projection', dpi=100)[source]¶ The function plots a redistribution of states on the given Markov chain.
- Parameters
mc (~MarkovChain) – the target Markov chain.
distributions (
Union
[int
,List
[ndarray
]]) – a sequence of redistributions or the number of redistributions to perform.initial_status (
Union
[int
,Iterable
[+T_co],None
]) – the initial state or the initial distribution of the states (if omitted, the states are assumed to be uniformly distributed).plot_type (
str
) – the type of plot to display (either heatmap or projection; projection by default).dpi (
int
) – the resolution of the plot expressed in dots per inch (by default, 100).
- Return type
- Returns
None if Matplotlib is in interactive mode as the plot is immediately displayed, otherwise the handles of the plot.
- Raises
ValueError – if the “distributions” parameter represents a sequence of redistributions and the “initial_status” parameter does not match its first element.
ValidationError – if any input argument is not compliant.
-
pydtmc.
plot_walk
(mc, walk, initial_state=None, plot_type='histogram', dpi=100)[source]¶ The function plots a random walk on the given Markov chain.
- Parameters
mc (~MarkovChain) – the target Markov chain.
walk (
Union
[int
,List
[int
],List
[str
]]) – a sequence of states or the number of simulations to perform.initial_state (
Union
[int
,str
,None
]) – the initial state of the walk (if omitted, it is chosen uniformly at random).plot_type (
str
) – the type of plot to display (one of histogram, sequence and transitions; histogram by default).dpi (
int
) – the resolution of the plot expressed in dots per inch (by default, 100).
- Return type
- Returns
None if Matplotlib is in interactive mode as the plot is immediately displayed, otherwise the handles of the plot.
- Raises
ValueError – if the “walk” parameter represents a sequence of states and the “initial_state” parameter does not match its first element.
ValidationError – if any input argument is not compliant.