What is Probability theory?
theory in two major ways. First, the laws of probability tell us how AI systems should reason, so we design our algorithms to compute or approximate expressions derived using . Second, statistics to theoretically analyze the behavior of proposed AI systems. fundamental tool disciplines of science and engineering.
Many branches of deal mostly with entities that are entirely deterministic and certain. A programmer can usually safely assume that a CPU will execute each machine instruction flawlessly. Errors in hardware do occur but are rare enough software applications be designed to account
for them. many computer scientists and software engineers a relatively clean and certain environment, it surprising that machine learning makes heavy use of . Tbecause machine learning uncertain quantities and sometimes stochastic (non-deterministic) quantities. Uncertainty and stochasticity can arise from many sources. Researchers have made compelling arguments for quantifying uncertainty using probability since the 1980s. Many of the arguments presented here are summarized from or inspired by Pearl (1988). Nearly all activities require some ability to reason presence of uncertainty. In fact, beyond mathematical statements that are true by definition, difficult to any proposition absolutely true or any event absolutely
guaranteed to occur. There are three possible sources of uncertainty:
1. Inherent stochasticity system being modeled. , most interpretations of describe the dynamics of subatomic particles as being probabilistic. also create theoretical scenarios that we postulate random dynamics, where we assume that the cards are truly shuffled into a random order.
we want for applications. was originally developed the frequencies of events. easy how probability theory study events like drawing hand of cards game of poker. These events are often repeatable. say that an outcome probability p of occurring, it if we repeated the experiment (e.g., draw a hand of cards) infinitely , then proportion p of the repetitions would that outcome. of reasoning seem immediately applicable to propositions that repeatable. If a doctor
analyzes a patient and says that the patient 40% chance the flu, something very different—we make infinitely many replicas of the patient, there any reason to believe that different replicas of the patient would present with symptoms yet have varying underlying conditions. In the case of the doctor diagnosing the patient, we use probability to represent a degree of belief, with 1 indicating absolute certainty that the patient has the flu and 0 indicating absolute certainty that the patient have the flu. The former probability, related the rates at which events occur, is
known as frequentist probability, while the latter, qualitative levels of certainty, as Bayesian probability. If we list several properties that we expect reasoning about uncertainty , then satisfying those properties are to treat Bayesian probabilities as behaving same as frequentist probabilities. , if to compute the probability that a player will win a poker game she certain set of cards, we use same formulas
as compute the probability that a patient disease she has certain symptoms. For more details about why a littleset of common-sense assumptions imply that equivalent axioms must control both probability, see Ramsey (1926). Probability seen extension of logic to uncertainty. Logic provides of formal rules for determining what propositions are implied to
be true or false given that set of propositions is true or false. provides of formal rules for determining the likelihood of a proposition being true given the likelihood of other propositions.