{"product_id":"information-theory-for-machine-learning-yehuda-setnik-9798273722620","title":"Information Theory for Machine Learning: Theorems, Proofs, and Python Implementations","description":"\u003cp\u003eThe complete graduate-level reference for entropy, divergence, and mutual information in modern machine learning, rigorously developed from measure theory to contemporary estimators and algorithms.\u003c\/p\u003e\u003cp\u003e\u003c\/p\u003e\u003cul\u003e\n\u003cli\u003eMeasure-theoretic foundations: sigma-algebras, Radon-Nikodym, conditional expectation, change of measure.\u003c\/li\u003e\n\u003cli\u003eCore measures: entropy, cross-entropy, KL, mutual information; f-divergences and Renyi divergences with variational dualities (Fenchel, Donsker-Varadhan).\u003c\/li\u003e\n\u003cli\u003eData processing and fundamental inequalities: log-sum, Pinsker, Csiszar-Kullback-Pinsker, Fano, Le Cam, Assouad; equality conditions and sufficiency.\u003c\/li\u003e\n\u003cli\u003eGaussian tools: entropy power inequality, de Bruijn identity, Fisher information, I-MMSE, Gaussian extremality.\u003c\/li\u003e\n\u003cli\u003eMaximum entropy and exponential families; log-partition convexity, Bregman geometry, Pythagorean theorems.\u003c\/li\u003e\n\u003cli\u003eFisher information and asymptotics: score, Cramer-Rao bounds, LAN, Bernstein-von Mises, asymptotic efficiency.\u003c\/li\u003e\n\u003cli\u003eInformation geometry and natural gradients: Fisher-Rao metric, dual connections, mirror descent.\u003c\/li\u003e\n\u003cli\u003eSource coding and MDL: Kraft-McMillan, NML, universal coding, compression-generalization links.\u003c\/li\u003e\n\u003cli\u003eGeneralization: PAC-Bayes bounds, mutual information bounds I(W;S), stability of SGD.\u003c\/li\u003e\n\u003cli\u003eConcentration via information: DV method, log-Sobolev and Poincare inequalities, transportation T1\/T2, hypercontractivity.\u003c\/li\u003e\n\u003cli\u003eVariational inference and divergence minimization: ELBO, alpha-divergences, EP, black-box VI with reparameterization.\u003c\/li\u003e\n\u003cli\u003eEstimating entropy and MI: plug-in, kNN, KDE, Kraskov, MINE, InfoNCE; minimax rates and consistency.\u003c\/li\u003e\n\u003cli\u003eRate-distortion and information bottleneck: Blahut-Arimoto, optimal encoders, sufficiency-compression trade-offs.\u003c\/li\u003e\n\u003cli\u003eContrastive representation learning under augmentations: alignment vs uniformity, identifiability, sample complexity.\u003c\/li\u003e\n\u003cli\u003eGenerative modeling: VAEs, bits-back coding, beta-VAE, TCVAE; likelihood calibration and posterior collapse.\u003c\/li\u003e\n\u003cli\u003eScore matching and Stein: Fisher divergence, kernel Stein discrepancies; diffusion models as score-based SDEs with likelihood estimation.\u003c\/li\u003e\n\u003cli\u003eOptimal transport with entropic regularization: Kantorovich duality, Sinkhorn, Schrodinger bridges; OT vs f-divergence objectives.\u003c\/li\u003e\n\u003cli\u003eDistributed and federated learning under communication limits: quantization, gradient coding, lower bounds via information.\u003c\/li\u003e\n\u003cli\u003ePrivacy and leakage: differential privacy, Renyi DP, moments accountant; accuracy-privacy trade-offs and inference risks.\u003c\/li\u003e\n\u003cli\u003eActive learning and Bayesian experimental design: expected information gain, submodularity, scalable estimators.\u003c\/li\u003e\n\u003c\/ul\u003e\u003cbr\u003e\u003cbr\u003e\u003cb\u003eAuthor:\u003c\/b\u003e Yehuda Setnik\u003cbr\u003e\u003cb\u003eISBN-13:\u003c\/b\u003e 9798273722620\u003cbr\u003e\u003cb\u003ePublisher:\u003c\/b\u003e Independently Published\u003cbr\u003e\u003cb\u003eLanguage:\u003c\/b\u003e English\u003cbr\u003e\u003cb\u003ePublished:\u003c\/b\u003e 11\/09\/2025\u003cbr\u003e\u003cb\u003ePages:\u003c\/b\u003e 374\u003cbr\u003e\u003cb\u003eFormat:\u003c\/b\u003e Paperback\u003cbr\u003e\u003cb\u003eWeight:\u003c\/b\u003e 1.91lbs\u003cbr\u003e\u003cb\u003eSize:\u003c\/b\u003e 11.00h x 8.50w x 0.77d","brand":"Yehuda Setnik","offers":[{"title":"Paperback","offer_id":48066747203839,"sku":"9798273722620","price":79.99,"currency_code":"USD","in_stock":true}],"thumbnail_url":"\/\/cdn.shopify.com\/s\/files\/1\/0662\/2982\/9887\/files\/img_330b5a36-091c-4f95-92e6-7fc2a82d6836.jpg?v=1768680285","url":"https:\/\/www.whiterainbookhouse.com\/products\/information-theory-for-machine-learning-yehuda-setnik-9798273722620","provider":"WR Book House","version":"1.0","type":"link"}