鲁晨光中文主页：http://survivor99.com/lcg/

语义信息研究：http://www.survivor99.com/lcg/books/GIT/

用语义信息方法求检验、估计和混合模型的最大互信息和最大似然度

(信道匹配算法中文版)

摘要：不固定Shannon信道或产生假设的规则，求解最大互信息和最大似然度(等价于最大平均对数似然度)，是非常困难的，我们不得不使用迭代方法。根据鲁晨光提出的语义信息测度和R(G)函数(信息率失真函数R(D)的推广; G是语义互信息的下限)可以得到一种新的迭代方法，用于检验、估计、混合模型。该语义信息测度被解释为平均对数标准(normalized)似然度, 而似然度由真值函数通过语义贝叶斯推理产生。一组真值函数构成一个语义信道，语义信道和Shannon信道相互匹配和迭代就能得到产生最大互信息和最大平均对数似然度的Shannon信道。该算法可谓信道匹配算法，简称CM算法。迭代的收敛可以通过R(G)函数得到直观解释和证明。CM算法用于检验、估计和混合模型的几个例子显示运算简单(可以用Excel文件演示)，收敛快速(随机选择的例子，收敛需要的迭代次数大多接近5次)。对于混合模型，CM算法和EM算法类似； 但是和标准EM算法比，CM算法有更好的收敛性和更多潜在应用。

关键词：Shannon信道；语义信道；语义信息；似然度；混合模型；EM算法; 机器学习。

演示迭代的Excel文件下载：

http://survivor99.com/lcg/CM-iteration.zip

ABSTRACT：It is very difficult to solve Maximum Mutual Information (MMI) or Maximum Likelihood (ML) for all possible Shannon Channels or uncertain rules of choosing hypotheses, so that we have to use iterative methods. According to the Semantic Mutual Information (SMI) and R(G) function proposed by Chenguang Lu (1993) (where R(G) is an extension of information rate distortion function R(D), and G is the lower limit of the SMI), we can obtain a new iterative algorithm of solving the MMI and ML for tests, estimations, and mixture models. The SMI is defined by the average log normalized likelihood. The likelihood function is produced from the truth function and the prior by the semantic Bayesian inference. A group of truth functions constitute a semantic channel. Letting the semantic channel and Shannon channel mutually match and iterate, we can obtain the Shannon channel that maximizes the Shannon mutual information and the average log likelihood. This iterative algorithm is called Channels’ Matching algorithm or the CM algorithm. The convergence can be intuitively explained and proved by the R(G) function. Several iterative examples for tests, estimations, and mixture models show that the computation of the CM algorithm is simple, and can be demonstrated in excel files. For most random examples, the numbers of iterations for convergence are close to 5. For mixture models, the CM algorithm is similar to the EM algorithm; however, the CM algorithm has better convergence and more potential applications in comparison with the standard EM algorithm.

Keywords: Shannon channel; semantic channel; semantic information; likelihood; tests; estimations；mixture; EM algorithm

Excel files demostrating the CM algorithm:

http://survivor99.com/lcg/CM-iteration.zip