Theory

Measuring Uncertainty

How do you measure “uncertainty”?

That may seem like an odd question. But let’s just dive right into it, because starting down this path of inquiry will lead us step by step to the definition of the fascinating concept of Shannon entropy.

“I’m 99% Certain”

We can start with one common way people express certainty. You might say “I’m 99% certain it will rain today”. This, of course, implies that you’re 1% uncertain. So one obvious definition of uncertainty is the inverse of certainty, or $1 - p$, where $p$ is certainty expressed as a percentage.

In this article we introduce an argument model: a set of terms for analyzing arguments by naming their parts. There are various argument models in the academic literature on argumentation theory and related fields but none provide us with precise definitions for all the concepts behind our algorithms for improving online conversations. So we will define those concepts here.

Anatomy of an Argument

Our model incorporates the basic ideas from the influential Toulmin model of argumentation first introduced in 1948, but uses a simpler model with more modern terminology.

Definition of Relevance

In the previous essay in this series, we introduced the basic ideas and terminology of Bayesian argumentation, including the concept of relevance. In this essay I explore a precise mathematical definition of relevance and the related concept of acceptance.

Argument and Information

In the previous essay in this series, we introduced the idea of relevance, and said that a premise is relevant to the conclusion iff $P(A \vert B) > P(A \vert \bar{B})$.

Consider the argument (𝐴) this is a good candidate for the job because (𝐵) he has a pulse. Having a pulse may not be a very persuasive reason to hire somebody, but it is probably quite relevant, because if the candidate did not have a pulse, the subject would probably be much less likely to want to hire him. That is $P(A \vert B) > P(A \vert \bar{B})$.

Why Accept the Premise?

In the previous essay in this series, we defined the ideas of necessity and sufficiency from the perspective of a Bayesian rational agent. If an argument is necessary, then if the subject were to reject the premise, they would decrease their acceptance of the conclusion. And if an argument is sufficient, then if the subject were to accept the premise, they would increase their acceptance of the conclusion.

This is the final article in my series on Bayesian Argumentation. To understand this essay, read the introductory article and the article on [Relevance and Acceptance] (/relevance-and-acceptance).

Relevance is Not Absolute

Relevance exists in the context of the subject’s other prior beliefs. For example, if the subject believes that ($\bar{𝐶}$) the car is out of gas, and also ($\bar{B}$) the battery is dead, then both of these are good reasons to believe ($\bar{A}$) the car won’t start.

Bayesian Argumentation: Summary of Definitions

Below is a summary of all the terms and equations defined in the essays in this series, followed by a detailed example.

• Claim: A statement that one can agree or disagree with

• Premise: A claim intended to support or oppose some conclusion

• Conclusion: The claim supported or opposed by the premise

• Argument: A premise asserted in support of or opposition to some conclusion

For an argument with premise 𝐵 and conclusion 𝐴, and a subject whose beliefs are represented by probability measure P…

“When you have eliminated the impossible, all that remains, no matter how improbable, must be the truth.”

– Sherlock Holmes (Arthur Conan Doyle)

For a long time Bayesian inference was something I understood without really understanding it. I only really got it it after reading Chapter 2 of John K. Kruschke’s textbook Doing Bayesian Data Analysis, where he describes Bayesian Inference as Reallocation of Credibility Across Possibilities

I now understand Bayesian Inference to be essentially a mathematical generalization of Sherlock Holmes’ pithy statement about eliminating the impossible. This article is my own attempt to elucidate this idea. If this essay doesn’t do the trick, you might try Bayesian Reasoning for Intelligent People by Simon DeDeo or Kruschke’s Bayesian data analysis for newcomers.

In this article we develop the basic mathematical formula for calculating the opinion of the meta-reasoner in arguments involving a single main argument thread.

In the Introduction to Distributed Bayesian Reasoning, we argue that the rules of Bayesian inference can enable a form of distributed reasoning.

In this article we introduce the idea of meta-reasoner, which is the hypothetical fully-informed average juror.

The meta-reasoner resembles the average juror in that it holds prior beliefs equal to the average beliefs of the participants, but it is fully-informed because it holds beliefs for every relevant sub-jury.

This article is part of the series on distributed Bayesian reasoning. It assumes you have read the previous article on the Basic Math.