What is the Performance Measure, learning algorithm?

 What is the Performance Measure, learning algorithm?


Introduction

In order to gauge the skills of a machine learning algorithm, we must design a quantitative measure of its performance. Usually, this performance measure P is restricted to the task T being administered by the system.
For tasks like classification, classification with missing inputs, and transcription, we frequently measure the accuracy of the model. Accuracy is simply the proportion of examples that the model produces the right output. 

Description

We will also obtain equivalent information by measuring the error rate, the proportion of examples for which the model produces incorrect output. we frequently ask the error rate because of the expected 0-1 loss. The 0-1 loss on a specific example is 0 if it’s correctly classified and 1 if it’s not. For tasks like density estimation, it doesn’t add up to live accuracy, error rate, or the other quite 0-1 loss. Instead, we must use a special performance metric that provides the model a continuous-valued score for every example. the foremost common approach is to report the typical log probability the model assigns to some examples. Usually, we have an interest in how well the machine learning algorithm performs on data that it’s not seen before since this determines how well it’ll work when deployed within the world

We, therefore, evaluate these performance measures employing a test set of knowledge that’s breakaway the info used for training the machine learning system. the selection of performance measures could seem straightforward and objective, but it’s often difficult to settle on a performance measure that corresponds well to the desired behavior of the system. In some cases, this is often because it’s difficult to make a decision on what should be measured. for instance, when performing a transcription task, should we measure the accuracy of the system at transcribing entire sequences, or should we use a more fine-grained performance measure that provides partial credit for getting some elements of the sequence correct? When performing a regression task, should we penalize the system more if it frequently makes medium-sized mistakes or if it rarely makes very large mistakes? These sorts of design choices depend upon the appliance. In other cases, we all know what quantity we might ideally wish to measure, but measuring it’s impractical. for instance, this arises frequently within the context of density estimation. Many of the simplest probabilistic models represent pro distributions only implicitly. Computing the particular probability value assigned to a selected point in space in many such models is intractable. 

In these cases, one must design an alternate criterion that also corresponds to the planning objectives, or design an honest approximation to the specified criterion. employing a test set of knowledge that’s breakaway the info used for training the machine learning system. the selection of performance measures could seem straightforward and objective, but it’s often difficult to settle on a performance measure that corresponds well to the specified behavior of the system. In some cases, this is often because it’s difficult to make a decision on what should be measured. for instance, when performing a transcription task, should we measure the accuracy of the system at transcribing entire sequences, or should we use a more fine-grained performance measure that provides partial credit for getting some elements of the sequence correct? When performing a regression task, should we penalize the system more if it frequently makes medium-sized mistakes or if it rarely makes very large mistakes? These sorts of design choices depend upon the appliance. In other cases, we all know what quantity we might ideally wish to measure, but measuring it’s impractical. for instance, this arises frequently within the context of density estimation. 

Many of the simplest probabilistic models represent probability distributions only implicitly. Computing the particular probability value assigned to a selected point in space in many such models is intractable. In these cases, one must design an alternate criterion that also corresponds to the planning objectives, or design an honest approximation to the specified criterion.

Leave a Comment