拉普拉斯的汪-优快云博客

原创 Optimization Algorithms

Reference:https://d2l.ai/chapter_optimization/index.htmlContentReview: Stochastic Gradient Descent (SGD)Momentum: Leaky AveragesExample: An Ill-conditioned ProblemThe Momentum MethodAdagradMotivation: Sparse FeaturesAdditional Benefits: PreconditioningTh

2021-09-11 05:09:36 480

原创 Attention Mechanisms

Reference:https://d2l.ai/chapter_attention-mechanisms/index.html 10.2-10.6ContentAttention Pooling-An ExampleAttention Scoring FunctionsAdditive AttentionScaled Dot-Product AttentionBahdanau AttentionSequence to Sequence LearningIncorporate Attention Mod

2021-09-06 17:15:28 361

原创 Modern Recurrent Neural Networks

Reference:https://d2l.ai/chapter_recurrent-modern/index.html 9.1-9.4ContentMotivationGated Recurrent Units (GRU)Reset Gate and Update GateHidden StateLong Short-Term Memory (LSTM)Input Gate, Forget Gate, and Output GateMemory CellHidden StateDeep Recurre

2021-09-05 17:59:00 260

原创 Recurrent Neural Networks

Reference:https://d2l.ai/chapter_recurrent-neural-networks/index.html (8.1 & 8.4)Pattern Recognition and Machine Learning 13.1-13.2ContentSequence ModelsAutoregressive ModelsMarkov ModelsLatent Autoregressive ModelsHidden Markov ModelsNeural Network

2021-09-03 04:11:59 294

原创 Vanishing and Exploding Gradients

Reference:https://d2l.ai/chapter_multilayer-perceptrons/numerical-stability-and-init.htmlGlorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the thirteenth international conference on artificia

2021-08-30 03:57:20 279

原创 Cross-Entropy Loss

Reference:https://en.wikipedia.org/wiki/Cross_entropyhttps://d2l.ai/chapter_linear-networks/softmax-regression.html#loss-functionDefinition: Cross-EntropyThe cross-entropy of the distribution qqq relative to a distribution ppp over a given set is defin

2021-08-30 03:55:23 295

原创 Minibatch Stochastic Gradient Descent

Reference:https://d2l.ai/chapter_linear-networks/linear-regression.htmlhttps://d2l.ai/chapter_linear-networks/linear-regression-scratch.htmlGradient descentx←x−η∂xL(x)\mathbf x\leftarrow \mathbf x-\eta \partial_{\mathbf x}\mathcal L(\mathbf x)x←x−η∂x

2021-08-30 03:54:43 305

原创 Bayesian Learning_ELBO, Variational Inference, and EM Algorithm

Reference:https://mbernste.github.io/posts/elbo/https://mbernste.github.io/posts/variational_inference/https://mbernste.github.io/posts/em/https://mbernste.github.io/posts/gmm_em/Theodoridis S. Machine learning: a Bayesian and optimization perspective

2021-08-27 04:24:13 513

原创 MA&ALA4.1&4.2_空间和子空间 (Space and Subspace)

注：本文是对Matrix Analysis and Applied Linear Algebra一书4.1节Space and Subspace和4.2节Four Fundamental Subspaces的学习笔记文章目录向量空间(Vector Space)子空间(Subspace)四个基本子空间列空间(Column Space)和行空间(Row Space)零空间(Nullspace)和左零空间(Left-Hand Nullspace)总结向量空间(Vector Space)定义一个向量空间涉及4

2021-05-10 05:02:39 552

原创 Distributed Source Coding

Reference:Elements of Information Theory, 2nd EditionSlides of EE4560, TUD文章目录IntroductionSlepian-Wolf CodingRandom BinningEncoding and Decoding SchemeOutline of Proof: AchievabilityInterpretation of Slepian-Wolf CodingIntroductionWe know how to enc

2021-04-13 06:09:10 322

原创 Network Information Theory

Reference:Elements of Information Theory, 2nd EditionSlides of EE4560, TUDContentIntroduction-Network IT FeaturesDiscrete Memoryless Multiple-Access ChannelBinary Multiplier ChannelBinary Erasure MA ChannelGaussian MA ChannelGaussian FDMA CapacityGaussi

2021-04-12 06:42:06 933

原创 Rate Distortion Theory

Reference:Elements of Information Theory, 2nd EditionSlides of EE4560, TUDContentQuantizationRate-Distortion TheoryBinary SourceGaussian SourceIndependent Gaussian VariablesGaussian Sources with MemoryAchievability of R(D)R(D)R(D) for i.i.d. SourcesFor

2021-04-01 03:32:36 654

原创 Data Compression

Reference:Elements of Information Theory, 2nd EditionSlides of EE4560, TUDContentConsequences of The AEP: Data CompressionSource CodesKraft InequalityOptimal CodesBounds on the optimal code lengthShannon CodeHuffman CodeArithmetic CodingProblem descrip

2021-03-19 05:53:22 626

原创 Channel Capacity 3: Gaussian Channel

Reference:Elements of Information Theory, 2nd EditionSlides of EE4560, TUD文章目录Differential EntropyGaussian ChannelsGaussian Channel CapacityBand-Limited ChannelParallel Gaussian ChannelsGaussian Channels with FeedbackDifferential EntropyWe now introdu

2021-03-07 03:24:15 432

原创 Channel Capacity 2: Channel Coding Theorem

Reference:Elements of Information Theory, 2nd EditionSlides of EE4560, TUDContentPreliminariesJointly Typical SequencesChannel Coding TheoremChannel with FeedbackSource-Channel SeparationPreliminariesWe analyze a communication system as shown in Figur

2021-03-02 06:37:32 458

原创 Channel Capacity 1: Discrete Channels

Reference:Elements of Information Theory, 2nd EditionSlides of EE4560, TUDContentChannel Capacity DefinitionExample ChannelsBinary Symmetrical Channel (BSC)Binary Erasure Channel (BEC)Binary Asymmetric Channel (BAC/Z-Channel)Symmetric ChannelsProperties

2021-02-25 05:05:02 700 2

原创 Asymptotic Equipartition Property

Reference:Elements of Information Theory, 2nd EditionSlides of EE4560, TUDContentAEPTypical SetHigh-probability SetsAEPIn information theory, the analog of the law of large numbers is the asymptotic equipartition property (AEP). It is a direct consequ

2021-02-22 01:57:26 535

原创 Information Measures

Information MeasuresReference:Elements of Information Theory, 2nd EditionSlides of EE4560, TUDContentInformation MeasuresEntropyMutual InformationEntropy RatesHow to measure information? →\to→ Hartley’s approach [slides 5] →\to→ Problems?no allowance

2021-02-19 00:40:11 166

原创 Feature Selection and Extraction

Reference:Pattern recognition, by Sergios Theodoridis, Konstantinos Koutroumbas (2009)Slides of CS4220, TUDContentThe Peaking Phenomenon (5.3)Class Separability Measures (5.6)DivergenceChernoff Bound and Bhattacharyya DistanceScatter MatricesFeature Sel

2020-12-30 06:37:20 387

原创 Composite Hypothesis Testing

ContentExample: DC Level in WGN with Unknown Amplitude (A>0)Example: DC Level in WGN with Unknown AmplitudeComposite Hypothesis Testing ApproachesBayesian ApproachGeneralized Likelihood Ratio Test (GLRT)Example: DC Level in WGN with Unknown Amplitude -

2020-12-18 06:58:19 339

原创 Probabilistic Models

Reference:Bishop C M. Pattern recognition and machine learning[M]. springer, 2006.Slides of CS4220, TUDContentBayesian InferenceBayes’ Theorem for Gaussian Variables (2.3.1-2.3.3)Conditional Gaussian distributionsMarginal Gaussian distributionsPartition

2020-12-17 04:29:45 282

原创 Random Signals

Reference:Kay S M. Fundamentals of statistical signal processing[M]. Prentice Hall PTR, 1993. (Vol.2 Ch. 5 - 5.6)Slides of ET4386, TUDContentExample: Energy DetectorGeneralization 1: Signals With Arbitrary Covariance MatricesEstimator-CorrelatorCanonica

2020-12-15 05:01:44 220

原创 Deterministic Signals

Reference:Kay S M. Fundamentals of statistical signal processing[M]. Prentice Hall PTR, 1993. (Vol.2 Ch. 4 - 4.4)Slides of ET4386, TUDContentMatched FiltersDevelopment of DetectorInterpretationPropertyPerformance of Matched FilterGeneralized Matched Fil

2020-12-12 05:45:26 320

原创 Support Vector Machines

Reference:Section 3.7 of Pattern recognition, by Sergios Theodoridis, Konstantinos Koutroumbas (2009)Slides of CS4220, TUDContentMargins: IntuitionSeparable ClassesNonseparable ClassesKernelsMargins: IntuitionLet xi,i=1,2,…,N,\mathbf x_{i}, i=1,2, \ld

2020-12-11 02:29:52 288

原创 Neyman Pearson Theorem

ContentIntroductionA simple detection problemGeneral problem formulationNeyman Pearson TheoremExample: DC Level in WGNReceiver Operating Characteristic (ROC)Example: Change in VarianceMinimum Probability of ErrorMinimum Bayes Risk DetectorExample: DC Level

2020-12-10 04:45:27 867

原创 General Bayesian Estimators (MMSE, MAP, LMMSE)

Reference:Kay S M. Fundamentals of statistical signal processing[M]. Prentice Hall PTR, 1993. (Ch. 11 - 11.5, Ch. 12 - 12.5)Slides of ET4386, TUDContentRisk FunctionsMinimum Mean Square Error Estimators (MMSE)PropertiesExample: Bayesian Fourier Analysis

2020-12-06 05:54:49 822

原创 The Bayesian Philosophy

ContentPrior Knowledge and EstimationChoosing a Prior PDFProperties of the Gaussian PDFBayesian Linear ModelWe now depart from the classical approach to statistical estimation in which the parameter θ\thetaθ of interest is assumed to be a deterministic bu

2020-12-02 03:13:16 156

原创 Linear Classification

Reference:Bishop C M. Pattern recognition and machine learning[M]. springer, 2006.- Chapter 4 up to and including 4.3.2ContentDiscriminant Functions (Nonprobabilistic Methods)Two classesMultiple classesLeast Squares for ClassificationFisher’s linear dis

2020-11-29 03:43:56 345

原创 Practical Estimators (MLE, BLUE, LS)

Reference:Kay S M. Fundamentals of statistical signal processing[M]. Prentice Hall PTR, 1993. (Ch. 6.1 - 6.5, Ch. 7.1 - 7.6, Ch. 8.1 - 8.5 and 8.8 - 8.9)Slides of ET4386, TUDContentMaximum Likelihood Estimator (MLE)Asymptotic Properties of the MLEExampl

2020-11-27 04:19:56 542 1

原创 Linear Regression

Reference:Bishop C M. Pattern recognition and machine learning[M]. springer, 2006.- Chapter 1 up to and including Subsection 1.2.5- Chapter 3 up to and including 3.1.2ContentLinear Regression: IntroExample: Polynomial Curve FittingProbabilistic Perspec

2020-11-23 01:36:52 337

原创 Cramer-Rao Lower Bound

Reference:Kay S M. Fundamentals of statistical signal processing[M]. Prentice Hall PTR, 1993. (Chapter 3-3.5)Slides of ET4386, TUDContentEstimator Accuracy ConsiderationsScore function and regularity conditionFisher informationCramer-Rao Lower Bound The

2020-11-17 05:58:58 805

原创 Classifiers Based on Bayes Decision Theory

Reference:Section 2.1-2.6 of Pattern recognition, by Sergios Theodoridis, Konstantinos Koutroumbas (2009)Slides of CS4220, TUDContentBayes Decision TheoryMinimizing the classification error probabilityMinimizing the average riskDiscriminant functions an

2020-11-16 06:45:44 242

原创 Minimum Variance Unbiased Estimation (MVU)

Reference:Kay S M. Fundamentals of statistical signal processing[M]. Prentice Hall PTR, 1993. (Chapter 2)Slides of ET4386, TUDContentAn ExampleMean Square Error CriterionMinimum Variance Unbiased EstimatorExistence of the Minimum Variance Unbiased Estim

2020-11-14 18:45:18 780

原创 Adaptive Filtering

Reference:Slides of EE4C03, TUDHayes M H. Statistical digital signal processing and modelingContentSteepest Descent AlgorithmStability / ConvergenceConverge rateMean-square errorLMS AlgorithmConvergence in the meanConvergence in the mean-squareNormalize

2020-11-01 03:13:30 445

原创 Optimum Filters

ContentThe FIR Wiener FilterFilteringPredictionDeconvolutionNoise cancellationDiscrete Kalman FilterThe discrete form of the Wiener filtering problem, is to design a filter to recover a signal d(n)d (n)d(n) from noisy observationsx(n)=d(n)+v(n)x(n)=d(n

2020-10-20 04:58:26 237

原创 Solutions of Maxwell‘s Equation

ReferenceSlides of EE4C05, TUDOrfanidis S J. Electromagnetic waves and antennas[J]. 2002.Jin J M. Theory and computation of electromagnetic fields[M]. John Wiley & Sons, 2011.ContentFundamentals of EM Wave RadiationEnergy transfer by EM fieldAn int

2020-10-15 05:05:35 1041

原创 Spectrum Estimation

Reference:Slides of EE4C03, TUDHayes M H. Statistical digital signal processing and modelingContentNonparametric MethodsThe PeriodogramA filter bank interpretation of the periodogramPerformance of the periodogramThe Modified PeriodogramPerformance of th

2020-10-12 03:27:37 352

原创 The Levinson Recursion

Reference:Slides of EE4C03, TUDHayes M H. Statistical digital signal processing and modelingContentDevelopment of the RecursionThe Lattice FilterFIR lattice filterIIR lattice filterPropertiesThe Schur AlgorithmCholesky Factorization of a Toeplitz Matrix

2020-10-06 03:10:48 641

原创 Transmission lines fundamentals

Transmission lines fundamentalsReference:Slides of EE4C05, TUDPozar D M. Microwave engineeringcontentTransmission lines fundamentalsWhat is a transmission lineSolutions (E⃗,H⃗\vec E,\vec HE,H) for TEM from Maxwell EquationsVoltage and current (V,IV,IV,I

2020-09-24 06:07:49 350

原创 Signal Modeling

ContentSignal modelsDetermine models: Least SquaresDetermine models: Pade ApproximationDetermine models: Prony’s MethodDM-Special case: All-pole modelingAll-pole modeling with finite dataAutocorrelation methodCovariance methodExamplesPole estimationChannel

2020-09-21 02:37:20 344

Elementary_probability_theory.pdf

This book provides an introduction to probability theory and its applications. The emphasis is on essential probabilistic reasoning, which is illustrated with a large number of samples. The fourth edition adds material related to mathematical finance as well as expansions on stable laws and martingales.

2019-06-27

Machine Learning for OpenCV_ Intelligent image processing with Python.pdf

Chapter 1: A Taste of Machine Learning Chapter 2: Working with Data in OpenCV and Python Chapter 3: First Steps in Supervised Learning Chapter 4: Representing Data and Engineering Features Chapter 5: Using Decision Trees to Make a Medical Diagnosis Chapter 6: Detecting Pedestrians with Support Vector Machines Chapter 7: Implementing a Spam Filter with Bayesian Learning Chapter 8: Discovering Hidden Structures with Unsupervised Learning Chapter 9: Using Deep Learning to Classify Handwritten Digits Chapter 10: Combining Different Algorithms into an Ensemble Chapter 11: Selecting the Right Model with Hyperparameter Tuning Chapter 12: Wrapping Up

2019-06-01

Hands-On Start to Wolfram Mathematica.pdf

WOLFRAM MATHEMATICA 实用编程指南 Part I The Complete Overview Chapter 1 The Very Basics Chapter 2 A Sample Project in Mathematica Chapter 3 Input and Output Chapter 4 Word Processing and Typesetting Chapter 5 Presenting with Slide Shows Chapter 6 Fundamentals of the Wolfram Language Chapter 7 Creating Interactive Models with a Single Command Chapter 8 Sharing Mathematica Notebooks Chapter 9 Finding Help Part II Extending Knowledge Chapter 10 2D and 3D Graphics Chapter 11 Visualizing Data Chapter 12 Styling and Customizing Graphics Chapter 13 Creating Figures and Diagrams with Graphics Primitives Chapter 14 Algebraic Manipulation and Equation Solving Chapter 15 Calculus Chapter 16 Differential Equations Chapter 17 Linear Algebra Chapter 18 Probability and Statistics Chapter 19 Importing and Exporting Data Chapter 20 Data Filtering and Manipulation Chapter 21 Working with Curated Data Chapter 22 Using Wolfram|Alpha Data in Mathematica Chapter 23 Statistical Functionality for Data Analysis Chapter 24 Creating Programs Chapter 25 Creating Parallel and GPU Programs Index

2019-06-01

Advanced PID Control-ISA (2005).pdf

作者Karl J. Astrom, Tore Hagglund，英文版。With a total of 13 chapters, this new book substantially expands on some of the topics covered in the previous versions and provides several new chapters that deal with controller design, feedforward design, replacement of the Ziegler-Nichols tuning rules, predictive control, loop and performance assessment and interaction. At this point in our book trilogy, we assume that the reader is highly familiar with control theory.

2019-05-15

空空如也

TA创建的收藏夹 TA关注的收藏夹

TA关注的人