{"id":28,"date":"2018-08-24T06:59:59","date_gmt":"2018-08-24T06:59:59","guid":{"rendered":"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/?post_type=chapter&#038;p=28"},"modified":"2019-01-02T05:55:28","modified_gmt":"2019-01-02T05:55:28","slug":"design-of-learning-system","status":"publish","type":"chapter","link":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/chapter\/design-of-learning-system\/","title":{"rendered":"Design of Learning System"},"content":{"raw":"<div><span style=\"float: right\"><a href=\"https:\/\/youtu.be\/r0168kd5r34\" target=\"_blank\" rel=\"noopener\"><img src=\"http:\/\/epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/2018\/11\/download.png\" alt=\"epgp books\" width=\"75px\" height=\"75px;\" \/><\/a>\r\n<\/span><\/div>\r\n&nbsp;\r\n\r\n&nbsp;\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">Welcome to the e-PG Pathshala Lecture Series on Machine Learning. In this module we discuss the design of a learning system. However please note we do not go into the details but try to give an overview of the design process.<\/p>\r\n&nbsp;\r\n\r\n<strong>Learning Objectives:<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The learning objectives of this module are as follows:<\/p>\r\n\r\n<ul>\r\n \t<li style=\"text-align: justify\">To understand the various steps in the design of a learning system<\/li>\r\n \t<li style=\"text-align: justify\">To understand how to design a system that learns a model from data<\/li>\r\n \t<li style=\"text-align: justify\">To know about issues of feature selection and evaluation of a learning system<\/li>\r\n<\/ul>\r\n&nbsp;\r\n\r\n<strong>3.1 Basic Procedures in the Design of a Learning System<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The steps in the design of a learning system can be listed as follows:<\/p>\r\n\r\n<ul>\r\n \t<li style=\"text-align: justify\">Choose the training set X and how to represent it.<\/li>\r\n \t<li style=\"text-align: justify\">Choose exactly what is to be learnt, i.e. the <em>target function C<\/em>.<\/li>\r\n \t<li style=\"text-align: justify\">Choose how to represent the target function C.<\/li>\r\n \t<li style=\"text-align: justify\">Choose a learning algorithm to infer the target function from the set.<\/li>\r\n \t<li style=\"text-align: justify\">Find an evaluation procedure and a metric to test the learned function<\/li>\r\n<\/ul>\r\n<img class=\"size-full wp-image-29 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-4.png\" alt=\"\" width=\"577\" height=\"345\" \/>\r\n\r\n<strong>3.2 Design Cycle<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The design cycle is shown in Figure 3.1. The first step is the collection of data. The next step is the selection of features. This is an important step that can affect the overall learning effectiveness. In most cases, prior knowledge about the input data and what is to be learned is used in selecting appropriate features. The third step is model selection, which is essentially selection of a model that will be used to fit the training data. Here again prior knowledge about the data can be used to select the model. Once the model is selected, the learning step fine tunes the model by selecting parameters to generalize it. Finally the evaluation and testing step selects the parameters of the model, that fit the data and that also generalizes well.<\/p>\r\n&nbsp;\r\n\r\n<strong>3.3 Definition of Learning<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">Before we proceed, let us understand the meaning of learning in this context. We will explain learning using the example of the Hand-written character recognition problem. In this scenario we can define the problem as<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><strong>Task <em>T<\/em>: <\/strong>Recognizing hand-written characters (as given in Figure 3.2)<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><strong>Performance measure <em>P<\/em><\/strong>: percentage of characters correctly classified<strong> Training experience <em>E<\/em><\/strong>: a database of hand-written characters with their classifications<\/p>\r\n&nbsp;\r\n<p style=\"text-align: center\"><img class=\"size-full wp-image-30 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-5.png\" alt=\"\" width=\"217\" height=\"86\" \/><\/p>\r\n<p style=\"text-align: center\"><strong>Figure 3.2 Handwritten Characters<\/strong><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\">This example will be used throughout this module to explain the design steps.<\/p>\r\n&nbsp;\r\n\r\n<strong>3.4 Details of the Design of a learning system<\/strong>\r\n<ol>\r\n \t<li><strong>Collection of Data: <\/strong>As already explained, the first step is the collection of the data D={d1,d2,..dm,\u2026dn} where each data point represents the input data and corresponding output (Figure 3.3).<\/li>\r\n<\/ol>\r\n<p style=\"text-align: center\"><img class=\"size-full wp-image-31 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-6.png\" alt=\"\" width=\"419\" height=\"224\" \/><\/p>\r\n2.\u00a0<strong>Feature Selection: <\/strong>Feature Selection is essentially the process of selecting relevant features for use in model construction. The selection of features depends on the learning problem as given in Example 3.1.\r\n<table style=\"border-collapse: collapse;width: 99.8786%;height: 93px\" border=\"1\">\r\n<tbody>\r\n<tr>\r\n<td style=\"width: 100%\"><strong>Example 3.1 Feature Selection<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The students of a class have different attributes associated with them. Examples of such attributes include marks, native place, height etc. If the learning required is to find the association between native place and height, the marks feature should not be selected.<\/p>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<p style=\"text-align: justify\">Feature Selection could be in two ways; one is by reducing the number of attributes considered for each data point. This type of feature selection is called <strong><em>dimensionality reduction<\/em><\/strong> as shown in Figure 3.4. The second method is to reduce the number of data points considered where the original D={d1,d2,..dm,\u2026dn} is reduced to D={d1\u2026.dm} where m&lt;n (Figure 3.5)<\/p>\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-32 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-7.png\" alt=\"\" width=\"372\" height=\"136\" \/>\r\n<p style=\"text-align: center\"><strong>Figure 3.4 Reduction of Attributes<\/strong><\/p>\r\n<img class=\"size-full wp-image-33 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-8.png\" alt=\"\" width=\"320\" height=\"125\" \/>\r\n<div>\r\n<p style=\"text-align: center\"><strong>Figure 3.5 Reduction of Data Points<\/strong><\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\">3.\u00a0<strong>Model Selection: <\/strong>The next step is the model selection where we select a model that would most likely fit the data points. A linear model is one of the simplest models we should try to fit to the data. A model (its hypothesis) has a set of parameters; for example, a and b, the slope and the intercept in the simple linear model, shown in Figure 3.6.<\/p>\r\n\r\n<\/div>\r\n<ul>\r\n \t<li><em>y <\/em>=<em> ax <\/em>+<em> b <\/em>+ <em>e<\/em><\/li>\r\n \t<li>e = N(0, s)<\/li>\r\n<\/ul>\r\n<p style=\"text-align: center\"><img class=\"size-full wp-image-34 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-9.png\" alt=\"\" width=\"431\" height=\"193\" \/><\/p>\r\n<p style=\"text-align: justify\">An error function eneeds to be optimized. A simple example of an error function is the mean squared error given below:<\/p>\r\n\r\n<ul>\r\n \t<li>1 \u00e5(yi - f (xi))2<\/li>\r\n \t<li>n i=1n<\/li>\r\n<\/ul>\r\n&nbsp;\r\n<p style=\"text-align: justify\">In this function n is the number of data points, yi is the actual output obtained while f(xi) is the predicted output obtained by applying the function selected by the model ( Figure 3.7). We will go into the details later on in future modules.<\/p>\r\n&nbsp;\r\n<p style=\"text-align: center\"><img class=\"size-full wp-image-35 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-10.png\" alt=\"\" width=\"406\" height=\"150\" \/><\/p>\r\n<p style=\"text-align: center\"><strong>Figure 3.7The Error Function<\/strong><\/p>\r\n<p style=\"text-align: justify\">4 The Learning step involves the finding values of the parameters that minimize the error (Figure 3.8).<\/p>\r\n<img class=\"size-full wp-image-36 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-11.png\" alt=\"\" width=\"437\" height=\"191\" \/>\r\n<p style=\"text-align: justify\">5 The final step is the application of the learnt model to apply (evaluate) the learned model for predicting ys for new hitherto unseen inputs x using learned function f(x).<\/p>\r\n<img class=\"size-full wp-image-37 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-12.png\" alt=\"\" width=\"573\" height=\"234\" \/>\r\n\r\n<strong>3.5 Processing Data<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The data given to the learning system may require a lot of cleaning. Cleaning involves getting rid of errors and noise and removal of redundancies.<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><strong>Data Pre-processing: <\/strong>Data Pre-processing is another important process for effective learning. Pre-processing techniques include renaming, rescaling, discretization, abstraction, aggregation and introducing new attributes.<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\">Renaming or relabeling is the conversion of categorical values to numbers. However, this conversion may be inappropriate when used with some learning methods. Such an example is shown in Example 3.2 where numbers impose an order to the values that is not warranted.<\/p>\r\n&nbsp;\r\n<table class=\"aligncenter\" style=\"border-collapse: collapse;width: 99.8787%;height: 127px\" border=\"1\">\r\n<tbody>\r\n<tr style=\"height: 43px\">\r\n<td style=\"width: 99.8787%;height: 43px\" colspan=\"3\"><strong>Example 3.2 Relabelling<\/strong><\/td>\r\n<\/tr>\r\n<tr style=\"height: 28px\">\r\n<td style=\"width: 33.3333%;height: 28px\">Categorical Values<\/td>\r\n<td style=\"width: 33.2121%;height: 28px\">Conversion to\u00a0Numbers<\/td>\r\n<td style=\"width: 33.3333%;height: 28px\">Remarks<\/td>\r\n<\/tr>\r\n<tr style=\"height: 28px\">\r\n<td style=\"width: 33.3333%;height: 28px\">High,Normal, Low<\/td>\r\n<td style=\"width: 33.2121%;height: 28px\">2,1,0<\/td>\r\n<td style=\"width: 33.3333%;height: 28px\">Right<\/td>\r\n<\/tr>\r\n<tr style=\"height: 14px\">\r\n<td style=\"width: 33.3333%;height: 14px\">True, False, Unknown<\/td>\r\n<td style=\"width: 33.2121%;height: 14px\">2,1,0<\/td>\r\n<td style=\"width: 33.3333%;height: 14px\">Wrong<\/td>\r\n<\/tr>\r\n<tr style=\"height: 14px\">\r\n<td style=\"width: 33.3333%;height: 14px\">Red, Blue, Green<\/td>\r\n<td style=\"width: 33.2121%;height: 14px\">2,1,0<\/td>\r\n<td style=\"width: 33.3333%;height: 14px\">Wrong<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<div>\r\n<p style=\"text-align: justify\">Rescaling, also called normalization is the transferring of continuous values to some range, typically [-1,1] or [0,1]. Discretization or binning involves the conversion of continuous values to a finite set of discrete values. Another technique is abstraction where categorical values are merged together. In aggregation, actual values are replaced by values obtained with summary or aggregation operations, such as minimum value, maximum value, average, etc. Finally, sometimes new attributes that define a relationship with existing\u00a0<span style=\"text-align: initial;font-size: 1em\">attributes are introduced. An example is replacing weight and height attributes by a new attribute obesity-factor which is calculated as weight\/height. These pre-processing techniques are used only when the learning is not affected due to such pre-processing.<\/span><\/p>\r\n\r\n<\/div>\r\n&nbsp;\r\n\r\n<strong>3.5.1 Data biases<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">It is important to watch out for data biases. For this, we need to understand the data source. It is very easy to derive \u201cunexpected\u201d results when data used for analysis and learning are biased (pre-selected). The results or conclusions derived for pre-selected data do not hold for general cases (Example 3.3).<\/p>\r\n&nbsp;\r\n<table style=\"border-collapse: collapse;width: 100%\" border=\"1\">\r\n<tbody>\r\n<tr>\r\n<td style=\"width: 100%\"><strong>Example 3.3 Risks in pregnancy study<\/strong><\/td>\r\n<\/tr>\r\n<tr>\r\n<td style=\"width: 100%\">\r\n<p style=\"text-align: justify\"><strong>Survey: <\/strong>The sample survey on risks in pregnancy was sponsored by DARPA at various military hospitals. The study was conducted on a large sample of pregnant woman.<\/p>\r\n<p style=\"text-align: justify\"><strong>Conclusion: <\/strong>The factor with the largest impact on reducing risks during pregnancy (statistically significant) is a pregnant woman being single. That is the conclusion that single woman has the least risk. What is wrong with this conclusion?<\/p>\r\n<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n<div>\r\n\r\n&nbsp;\r\n\r\n<strong>3.6 Feature Selection<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">Sometimes the size (dimension) of a sample collection can be enormous. The selection of features requires prior knowledge about the characteristics about the input data. A typical example is document classification, where the document corpus can be represented by 10,000 different words. The data can be counts of occurrences of different words. Such a data collection entails the learning of too many parameters but not provide enough samples to justify the estimation of the parameters of the model.<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\">Feature selection reduces the feature sets. There are methods for removing input features. Such a technique is called dimensionality reduction. One method of dimensionality reduction is to replace inputs with features. Another method is to extract relevant inputs using a measure such as mutual information measure. Principal Component Analysis (PCA) is a method that mathematically reduces the dimension of the feature space. Another method of dimensionality reduction explained with an example of document classification is the grouping or clustering similar words using a suitable similarity measure and replacing the group of words with group label.<\/p>\r\n&nbsp;\r\n\r\n<strong>3.7 Model Selection<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The next important step in the design of the learning system is model selection. Again, prior knowledge about the data collection would help in an effective model selection; however, only an estimate can be done. Initial data analysis\u00a0<span style=\"text-align: initial;font-size: 1em\">and visualization can help to make a good guess about the form of the distribution or shape of the function. Independences and correlations among data points in the data collection can help in selecting a model. There may arise the over-fitting problem especially in the presence of bias and variance. Over-fitting is the problem of selecting a function that exactly fits the data, where we are not able to generalize in order to make predictions about unseen data.In other words, a model over fits if it fits particularities of the training set such as noise or bias.<\/span><\/p>\r\n\r\n<\/div>\r\n&nbsp;\r\n\r\n<strong>3.7.1 Avoiding Over-fitting<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">One method to avoid over-fitting is to ensure that there are sufficient number of examples in the training set. Another technique which will be later used for evaluation is the Hold Out method. In this method we hold some data out of the training set and train or fit on the training set (without data held out) and finally use the held out data for fine tuning the Model.<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\">Another important mathematical technique is the use of the concept of regularization, which is the process of introducing additional information in order to prevent <a href=\"http:\/\/en.wikipedia.org\/wiki\/Overfitting\">over-fitting. <\/a>This information is usually of the form of a penalty for complexity. A model should be selected based on the Occam's razor principle (proposed by William of Ockham) which states that the explanation of any phenomenon should make as few assumptions as possible, eliminating, the observable predictions of the explanatory hypothesis or theory. In other words, the simplest hypothesis (model) that fits almost all the data is the best compared to more complex ones; therefore, there is explicit preference towards simple models.<\/p>\r\n&nbsp;\r\n\r\n<strong>3.8 Evaluation<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">There are simple methods for evaluation and more complex methods using different methods for splitting the data.<\/p>\r\n&nbsp;\r\n\r\n<strong>3.8.1 Hold out Method<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">As already discussed the simplest evaluation method is the holdout method. In this method the data is divided into the training and test data sets. Typically 2\/3<\/p>\r\n<img class=\"size-full wp-image-38 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-13.png\" alt=\"\" width=\"484\" height=\"228\" \/>\r\n<p style=\"text-align: justify\">of the data is used as training data and the other 1\/3 is used as the testing set( Figure 3.10 (a) &amp; Figure 3.10 (b)).<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\">If we want to compare the predictive performance on a classification or a regression problem for two different learning methods then we will need to compare the error results on the test data set and choose the method with better (smaller) testing error for better generalization error.<\/p>\r\n<p style=\"text-align: center\"><img class=\"size-full wp-image-39 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-14.png\" alt=\"\" width=\"538\" height=\"350\" \/><\/p>\r\n<strong>3.8.2 Complex Methods<\/strong>\r\n\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-40 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-15.png\" alt=\"\" width=\"554\" height=\"367\" \/>\r\n<p style=\"text-align: justify\">The complex methodsuse multiple train\/test sets based on various random re-sampling schemes such as cross-validation, random sub-sampling, and Bootstrap (Figure 3.11).<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\">It is the generation of multiple training and test sets block of Figure 3.11 that will change depending on the sampling method. In random sub-sampling, simple holdout method with random split of data into 70% for training and 30% for testing is repeated k times. In the case of cross-validation sub-sampling (k-fold), the data is divided into k disjoint groups and tested on kth group where the rest of the data has been used for training, that is leave one out cross-validation. Typically a 10-fold cross-validation is used. In the case of bootstrap, the training set of size N=size of the data D is used with sampling with replacement. These concepts are shown in Figure 3.12.<\/p>\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-41 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-16.png\" alt=\"\" width=\"361\" height=\"492\" \/>\r\n<p style=\"text-align: center\"><strong>Figure 3.12 Sampling Methods<\/strong><\/p>\r\n&nbsp;\r\n\r\n<strong>3.9 Illustrative Example of the Process of Design<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">We use the example of handwritten character recognition (Figure 3.13) as an illustrative example to explain to illustrate the design issues and approaches.<\/p>\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-42 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-17.png\" alt=\"\" width=\"404\" height=\"196\" \/>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">We explain learning to perform a task from experience. Therefore, let us what is the meaning of task. Task can often be expressed through a mathematical function. In this case input can be x, output y and w the parameters that are \u201clearned\u201d. In case of classification output y will be discrete E.g. class membership, posterior probability, etc. For regression, y will be continuous. For the character recognition the task is as shown in Figure 3.14.<\/p>\r\n<img class=\"size-full wp-image-43 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-18.png\" alt=\"\" width=\"513\" height=\"260\" \/>\r\n<p style=\"text-align: justify\">The following are the steps in the design process for character recognition.<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><strong>Step 0: <\/strong>Let us treat the learning system as a black box (Figure 3.15), here we assume that the input a set of handwritten characters and the output is the letter q.<\/p>\r\n&nbsp;\r\n<p style=\"text-align: justify\"><strong>Step 1: <\/strong>Next we collect Training Examples (Experience). Without examples, our system will not learn as we are learning from examples (Figure 3.16).<\/p>\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-44 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-19.png\" alt=\"\" width=\"541\" height=\"450\" \/>\r\n\r\n<strong>Step 2: Representing Experience<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The next step is to choose a representation for the experience\/examples. In our example the sensor input can represented by an n-dimensional vector, called the feature vector, X = (x1, x2, x3, \u2026,xn). We can assume a 64-d vector to represent the 8X8 matrix of pixels (Figure 3.17)<\/p>\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-45 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-20.png\" alt=\"\" width=\"445\" height=\"196\" \/>\r\n<p style=\"text-align: justify\">In order to represent the experience, we need to know what <strong>X<\/strong> is. Therefore we need a corresponding vector <strong>D<\/strong>, which will record our knowledge (experience) about <strong>X.<\/strong> The experience <strong>E<\/strong> is a pair of vectors <strong>E<\/strong> = (<strong>X<\/strong>, <strong>D<\/strong>). Now the question is how to represent D. Assuming our system is to recognise 10 digits only, then D can be a 10-d binary vector; each correspond to one of the digits (Figure 3.18).<\/p>\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-46 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-21.png\" alt=\"\" width=\"406\" height=\"119\" \/>\r\n\r\n&nbsp;\r\n\r\n<strong>Step 3: Choose a Representation for the Black Box<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">The next step is the choosing of a representation for the black box. Here we need to choose a function F to approximate the black box and for a given X, the value of F would give the classification of X.<\/p>\r\n<img class=\"size-full wp-image-47 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-22.png\" alt=\"\" width=\"493\" height=\"134\" \/>\r\n\r\n<strong>Step 4: Learning\/Adjusting the Weights<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">We need a learning algorithm to adjust the weights such that the experience from the training data can be incorporated into the system, where experience E is represented in terms of input X and expected output D. The function F(X) would be modified with weights W to obtain the learned output L as given in Figure 3.20.<\/p>\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-48 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-23.png\" alt=\"\" width=\"566\" height=\"259\" \/>\r\n\r\n<strong>Step 5: Use\/Test the System<\/strong>\r\n\r\n&nbsp;\r\n<p style=\"text-align: justify\">After learning is completed, all parameters are fixed and an unknown input X can be presented to the system for which the system computes its answer according to the function F(W,X) (Figure 3.21).<\/p>\r\n&nbsp;\r\n\r\n<img class=\"size-full wp-image-49 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-24.png\" alt=\"\" width=\"473\" height=\"161\" \/>\r\n\r\n&nbsp;\r\n\r\n<strong>Summary<\/strong>\r\n\r\n&nbsp;\r\n\r\nIn this module the following were talked about\r\n<ul>\r\n \t<li>Explained the basic steps in the design of a learning system<\/li>\r\n \t<li>Outlined how a function is chosen to fit the data and the parameters are tuned to minimize the error<\/li>\r\n \t<li>Discussed some methods of evaluation<\/li>\r\n \t<li>Explained the Design Process using an example<\/li>\r\n<\/ul>\r\n<table>\r\n<tbody>\r\n<tr>\r\n<td><strong>you can view video on Design of Learning System<\/strong><\/td>\r\n<td><a href=\"https:\/\/youtu.be\/r0168kd5r34\" target=\"_blank\" rel=\"noopener\"><img class=\"alignnone wp-image-120\" src=\"http:\/\/epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/2018\/11\/download.png\" alt=\"\" width=\"36\" height=\"36\" \/><\/a><\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n\r\n<strong>Web Links<\/strong>\r\n<ul>\r\n \t<li style=\"text-align: justify\">http:\/\/people.cs.pitt.edu\/~milos\/courses\/cs2750-Spring03\/lectures\/class2.pdf<\/li>\r\n \t<li style=\"text-align: justify\">http:\/\/learningforward.org\/docs\/default-source\/commoncore\/comprehensive-professional-learning-system.pdf\" type=\"application\/pdf<\/li>\r\n \t<li style=\"text-align: justify\">http:\/\/digitalcommons.ilr.cornell.edu\/cgi\/viewcontent.cgi?article=1405&amp;amp;context=cahrswp\" type=\"application\/pdf<\/li>\r\n \t<li style=\"text-align: justify\">\"http:\/\/www.cse.hcmut.edu.vn\/~tru\/AI\/chapter11.pdf\" type=\"application\/pdf<\/li>\r\n \t<li style=\"text-align: justify\">http:\/\/www.holehouse.org\/mlclass\/11_Machine_Learning_System_Design.html<\/li>\r\n \t<li style=\"text-align: justify\">ssdi.di.fct.unl.pt\/pc\/0607\/files\/PCaulaT03-10-06.ppt<\/li>\r\n \t<li style=\"text-align: justify\">http:\/\/www.cse.hcmut.edu.vn\/~tru\/AI\/chapter11.pdf<\/li>\r\n \t<li style=\"text-align: justify\">https:\/\/www.physicsforums.com\/threads\/why-ockham-razor.122774\/<\/li>\r\n<\/ul>","rendered":"<div><span style=\"float: right\"><a href=\"https:\/\/youtu.be\/r0168kd5r34\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" src=\"http:\/\/epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/2018\/11\/download.png\" alt=\"epgp books\" width=\"75px\" height=\"75px;\" \/><\/a><br \/>\n<\/span><\/div>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">Welcome to the e-PG Pathshala Lecture Series on Machine Learning. In this module we discuss the design of a learning system. However please note we do not go into the details but try to give an overview of the design process.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>Learning Objectives:<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The learning objectives of this module are as follows:<\/p>\n<ul>\n<li style=\"text-align: justify\">To understand the various steps in the design of a learning system<\/li>\n<li style=\"text-align: justify\">To understand how to design a system that learns a model from data<\/li>\n<li style=\"text-align: justify\">To know about issues of feature selection and evaluation of a learning system<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><strong>3.1 Basic Procedures in the Design of a Learning System<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The steps in the design of a learning system can be listed as follows:<\/p>\n<ul>\n<li style=\"text-align: justify\">Choose the training set X and how to represent it.<\/li>\n<li style=\"text-align: justify\">Choose exactly what is to be learnt, i.e. the <em>target function C<\/em>.<\/li>\n<li style=\"text-align: justify\">Choose how to represent the target function C.<\/li>\n<li style=\"text-align: justify\">Choose a learning algorithm to infer the target function from the set.<\/li>\n<li style=\"text-align: justify\">Find an evaluation procedure and a metric to test the learned function<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-29 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-4.png\" alt=\"\" width=\"577\" height=\"345\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-4.png 577w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-4-300x179.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-4-65x39.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-4-225x135.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-4-350x209.png 350w\" sizes=\"auto, (max-width: 577px) 100vw, 577px\" \/><\/p>\n<p><strong>3.2 Design Cycle<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The design cycle is shown in Figure 3.1. The first step is the collection of data. The next step is the selection of features. This is an important step that can affect the overall learning effectiveness. In most cases, prior knowledge about the input data and what is to be learned is used in selecting appropriate features. The third step is model selection, which is essentially selection of a model that will be used to fit the training data. Here again prior knowledge about the data can be used to select the model. Once the model is selected, the learning step fine tunes the model by selecting parameters to generalize it. Finally the evaluation and testing step selects the parameters of the model, that fit the data and that also generalizes well.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>3.3 Definition of Learning<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">Before we proceed, let us understand the meaning of learning in this context. We will explain learning using the example of the Hand-written character recognition problem. In this scenario we can define the problem as<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><strong>Task <em>T<\/em>: <\/strong>Recognizing hand-written characters (as given in Figure 3.2)<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><strong>Performance measure <em>P<\/em><\/strong>: percentage of characters correctly classified<strong> Training experience <em>E<\/em><\/strong>: a database of hand-written characters with their classifications<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: center\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-30 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-5.png\" alt=\"\" width=\"217\" height=\"86\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-5.png 217w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-5-65x26.png 65w\" sizes=\"auto, (max-width: 217px) 100vw, 217px\" \/><\/p>\n<p style=\"text-align: center\"><strong>Figure 3.2 Handwritten Characters<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">This example will be used throughout this module to explain the design steps.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>3.4 Details of the Design of a learning system<\/strong><\/p>\n<ol>\n<li><strong>Collection of Data: <\/strong>As already explained, the first step is the collection of the data D={d1,d2,..dm,\u2026dn} where each data point represents the input data and corresponding output (Figure 3.3).<\/li>\n<\/ol>\n<p style=\"text-align: center\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-31 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-6.png\" alt=\"\" width=\"419\" height=\"224\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-6.png 419w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-6-300x160.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-6-65x35.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-6-225x120.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-6-350x187.png 350w\" sizes=\"auto, (max-width: 419px) 100vw, 419px\" \/><\/p>\n<p>2.\u00a0<strong>Feature Selection: <\/strong>Feature Selection is essentially the process of selecting relevant features for use in model construction. The selection of features depends on the learning problem as given in Example 3.1.<\/p>\n<table style=\"border-collapse: collapse;width: 99.8786%;height: 93px\">\n<tbody>\n<tr>\n<td style=\"width: 100%\"><strong>Example 3.1 Feature Selection<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The students of a class have different attributes associated with them. Examples of such attributes include marks, native place, height etc. If the learning required is to find the association between native place and height, the marks feature should not be selected.<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p style=\"text-align: justify\">Feature Selection could be in two ways; one is by reducing the number of attributes considered for each data point. This type of feature selection is called <strong><em>dimensionality reduction<\/em><\/strong> as shown in Figure 3.4. The second method is to reduce the number of data points considered where the original D={d1,d2,..dm,\u2026dn} is reduced to D={d1\u2026.dm} where m&lt;n (Figure 3.5)<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-32 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-7.png\" alt=\"\" width=\"372\" height=\"136\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-7.png 372w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-7-300x110.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-7-65x24.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-7-225x82.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-7-350x128.png 350w\" sizes=\"auto, (max-width: 372px) 100vw, 372px\" \/><\/p>\n<p style=\"text-align: center\"><strong>Figure 3.4 Reduction of Attributes<\/strong><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-33 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-8.png\" alt=\"\" width=\"320\" height=\"125\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-8.png 320w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-8-300x117.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-8-65x25.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-8-225x88.png 225w\" sizes=\"auto, (max-width: 320px) 100vw, 320px\" \/><\/p>\n<div>\n<p style=\"text-align: center\"><strong>Figure 3.5 Reduction of Data Points<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">3.\u00a0<strong>Model Selection: <\/strong>The next step is the model selection where we select a model that would most likely fit the data points. A linear model is one of the simplest models we should try to fit to the data. A model (its hypothesis) has a set of parameters; for example, a and b, the slope and the intercept in the simple linear model, shown in Figure 3.6.<\/p>\n<\/div>\n<ul>\n<li><em>y <\/em>=<em> ax <\/em>+<em> b <\/em>+ <em>e<\/em><\/li>\n<li>e = N(0, s)<\/li>\n<\/ul>\n<p style=\"text-align: center\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-34 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-9.png\" alt=\"\" width=\"431\" height=\"193\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-9.png 431w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-9-300x134.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-9-65x29.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-9-225x101.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-9-350x157.png 350w\" sizes=\"auto, (max-width: 431px) 100vw, 431px\" \/><\/p>\n<p style=\"text-align: justify\">An error function eneeds to be optimized. A simple example of an error function is the mean squared error given below:<\/p>\n<ul>\n<li>1 \u00e5(yi &#8211; f (xi))2<\/li>\n<li>n i=1n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">In this function n is the number of data points, yi is the actual output obtained while f(xi) is the predicted output obtained by applying the function selected by the model ( Figure 3.7). We will go into the details later on in future modules.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: center\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-35 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-10.png\" alt=\"\" width=\"406\" height=\"150\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-10.png 406w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-10-300x111.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-10-65x24.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-10-225x83.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-10-350x129.png 350w\" sizes=\"auto, (max-width: 406px) 100vw, 406px\" \/><\/p>\n<p style=\"text-align: center\"><strong>Figure 3.7The Error Function<\/strong><\/p>\n<p style=\"text-align: justify\">4 The Learning step involves the finding values of the parameters that minimize the error (Figure 3.8).<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-36 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-11.png\" alt=\"\" width=\"437\" height=\"191\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-11.png 437w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-11-300x131.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-11-65x28.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-11-225x98.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-11-350x153.png 350w\" sizes=\"auto, (max-width: 437px) 100vw, 437px\" \/><\/p>\n<p style=\"text-align: justify\">5 The final step is the application of the learnt model to apply (evaluate) the learned model for predicting ys for new hitherto unseen inputs x using learned function f(x).<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-37 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-12.png\" alt=\"\" width=\"573\" height=\"234\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-12.png 573w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-12-300x123.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-12-65x27.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-12-225x92.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-12-350x143.png 350w\" sizes=\"auto, (max-width: 573px) 100vw, 573px\" \/><\/p>\n<p><strong>3.5 Processing Data<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The data given to the learning system may require a lot of cleaning. Cleaning involves getting rid of errors and noise and removal of redundancies.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><strong>Data Pre-processing: <\/strong>Data Pre-processing is another important process for effective learning. Pre-processing techniques include renaming, rescaling, discretization, abstraction, aggregation and introducing new attributes.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">Renaming or relabeling is the conversion of categorical values to numbers. However, this conversion may be inappropriate when used with some learning methods. Such an example is shown in Example 3.2 where numbers impose an order to the values that is not warranted.<\/p>\n<p>&nbsp;<\/p>\n<table class=\"aligncenter\" style=\"border-collapse: collapse;width: 99.8787%;height: 127px\">\n<tbody>\n<tr style=\"height: 43px\">\n<td style=\"width: 99.8787%;height: 43px\" colspan=\"3\"><strong>Example 3.2 Relabelling<\/strong><\/td>\n<\/tr>\n<tr style=\"height: 28px\">\n<td style=\"width: 33.3333%;height: 28px\">Categorical Values<\/td>\n<td style=\"width: 33.2121%;height: 28px\">Conversion to\u00a0Numbers<\/td>\n<td style=\"width: 33.3333%;height: 28px\">Remarks<\/td>\n<\/tr>\n<tr style=\"height: 28px\">\n<td style=\"width: 33.3333%;height: 28px\">High,Normal, Low<\/td>\n<td style=\"width: 33.2121%;height: 28px\">2,1,0<\/td>\n<td style=\"width: 33.3333%;height: 28px\">Right<\/td>\n<\/tr>\n<tr style=\"height: 14px\">\n<td style=\"width: 33.3333%;height: 14px\">True, False, Unknown<\/td>\n<td style=\"width: 33.2121%;height: 14px\">2,1,0<\/td>\n<td style=\"width: 33.3333%;height: 14px\">Wrong<\/td>\n<\/tr>\n<tr style=\"height: 14px\">\n<td style=\"width: 33.3333%;height: 14px\">Red, Blue, Green<\/td>\n<td style=\"width: 33.2121%;height: 14px\">2,1,0<\/td>\n<td style=\"width: 33.3333%;height: 14px\">Wrong<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<div>\n<p style=\"text-align: justify\">Rescaling, also called normalization is the transferring of continuous values to some range, typically [-1,1] or [0,1]. Discretization or binning involves the conversion of continuous values to a finite set of discrete values. Another technique is abstraction where categorical values are merged together. In aggregation, actual values are replaced by values obtained with summary or aggregation operations, such as minimum value, maximum value, average, etc. Finally, sometimes new attributes that define a relationship with existing\u00a0<span style=\"text-align: initial;font-size: 1em\">attributes are introduced. An example is replacing weight and height attributes by a new attribute obesity-factor which is calculated as weight\/height. These pre-processing techniques are used only when the learning is not affected due to such pre-processing.<\/span><\/p>\n<\/div>\n<p>&nbsp;<\/p>\n<p><strong>3.5.1 Data biases<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">It is important to watch out for data biases. For this, we need to understand the data source. It is very easy to derive \u201cunexpected\u201d results when data used for analysis and learning are biased (pre-selected). The results or conclusions derived for pre-selected data do not hold for general cases (Example 3.3).<\/p>\n<p>&nbsp;<\/p>\n<table style=\"border-collapse: collapse;width: 100%\">\n<tbody>\n<tr>\n<td style=\"width: 100%\"><strong>Example 3.3 Risks in pregnancy study<\/strong><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 100%\">\n<p style=\"text-align: justify\"><strong>Survey: <\/strong>The sample survey on risks in pregnancy was sponsored by DARPA at various military hospitals. The study was conducted on a large sample of pregnant woman.<\/p>\n<p style=\"text-align: justify\"><strong>Conclusion: <\/strong>The factor with the largest impact on reducing risks during pregnancy (statistically significant) is a pregnant woman being single. That is the conclusion that single woman has the least risk. What is wrong with this conclusion?<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<div>\n<p>&nbsp;<\/p>\n<p><strong>3.6 Feature Selection<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">Sometimes the size (dimension) of a sample collection can be enormous. The selection of features requires prior knowledge about the characteristics about the input data. A typical example is document classification, where the document corpus can be represented by 10,000 different words. The data can be counts of occurrences of different words. Such a data collection entails the learning of too many parameters but not provide enough samples to justify the estimation of the parameters of the model.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">Feature selection reduces the feature sets. There are methods for removing input features. Such a technique is called dimensionality reduction. One method of dimensionality reduction is to replace inputs with features. Another method is to extract relevant inputs using a measure such as mutual information measure. Principal Component Analysis (PCA) is a method that mathematically reduces the dimension of the feature space. Another method of dimensionality reduction explained with an example of document classification is the grouping or clustering similar words using a suitable similarity measure and replacing the group of words with group label.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>3.7 Model Selection<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The next important step in the design of the learning system is model selection. Again, prior knowledge about the data collection would help in an effective model selection; however, only an estimate can be done. Initial data analysis\u00a0<span style=\"text-align: initial;font-size: 1em\">and visualization can help to make a good guess about the form of the distribution or shape of the function. Independences and correlations among data points in the data collection can help in selecting a model. There may arise the over-fitting problem especially in the presence of bias and variance. Over-fitting is the problem of selecting a function that exactly fits the data, where we are not able to generalize in order to make predictions about unseen data.In other words, a model over fits if it fits particularities of the training set such as noise or bias.<\/span><\/p>\n<\/div>\n<p>&nbsp;<\/p>\n<p><strong>3.7.1 Avoiding Over-fitting<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">One method to avoid over-fitting is to ensure that there are sufficient number of examples in the training set. Another technique which will be later used for evaluation is the Hold Out method. In this method we hold some data out of the training set and train or fit on the training set (without data held out) and finally use the held out data for fine tuning the Model.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">Another important mathematical technique is the use of the concept of regularization, which is the process of introducing additional information in order to prevent <a href=\"http:\/\/en.wikipedia.org\/wiki\/Overfitting\">over-fitting. <\/a>This information is usually of the form of a penalty for complexity. A model should be selected based on the Occam&#8217;s razor principle (proposed by William of Ockham) which states that the explanation of any phenomenon should make as few assumptions as possible, eliminating, the observable predictions of the explanatory hypothesis or theory. In other words, the simplest hypothesis (model) that fits almost all the data is the best compared to more complex ones; therefore, there is explicit preference towards simple models.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>3.8 Evaluation<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">There are simple methods for evaluation and more complex methods using different methods for splitting the data.<\/p>\n<p>&nbsp;<\/p>\n<p><strong>3.8.1 Hold out Method<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">As already discussed the simplest evaluation method is the holdout method. In this method the data is divided into the training and test data sets. Typically 2\/3<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-38 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-13.png\" alt=\"\" width=\"484\" height=\"228\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-13.png 484w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-13-300x141.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-13-65x31.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-13-225x106.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-13-350x165.png 350w\" sizes=\"auto, (max-width: 484px) 100vw, 484px\" \/><\/p>\n<p style=\"text-align: justify\">of the data is used as training data and the other 1\/3 is used as the testing set( Figure 3.10 (a) &amp; Figure 3.10 (b)).<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">If we want to compare the predictive performance on a classification or a regression problem for two different learning methods then we will need to compare the error results on the test data set and choose the method with better (smaller) testing error for better generalization error.<\/p>\n<p style=\"text-align: center\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-39 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-14.png\" alt=\"\" width=\"538\" height=\"350\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-14.png 538w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-14-300x195.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-14-65x42.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-14-225x146.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-14-350x228.png 350w\" sizes=\"auto, (max-width: 538px) 100vw, 538px\" \/><\/p>\n<p><strong>3.8.2 Complex Methods<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-40 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-15.png\" alt=\"\" width=\"554\" height=\"367\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-15.png 554w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-15-300x199.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-15-65x43.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-15-225x149.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-15-350x232.png 350w\" sizes=\"auto, (max-width: 554px) 100vw, 554px\" \/><\/p>\n<p style=\"text-align: justify\">The complex methodsuse multiple train\/test sets based on various random re-sampling schemes such as cross-validation, random sub-sampling, and Bootstrap (Figure 3.11).<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">It is the generation of multiple training and test sets block of Figure 3.11 that will change depending on the sampling method. In random sub-sampling, simple holdout method with random split of data into 70% for training and 30% for testing is repeated k times. In the case of cross-validation sub-sampling (k-fold), the data is divided into k disjoint groups and tested on kth group where the rest of the data has been used for training, that is leave one out cross-validation. Typically a 10-fold cross-validation is used. In the case of bootstrap, the training set of size N=size of the data D is used with sampling with replacement. These concepts are shown in Figure 3.12.<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-41 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-16.png\" alt=\"\" width=\"361\" height=\"492\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-16.png 361w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-16-220x300.png 220w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-16-65x89.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-16-225x307.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-16-350x477.png 350w\" sizes=\"auto, (max-width: 361px) 100vw, 361px\" \/><\/p>\n<p style=\"text-align: center\"><strong>Figure 3.12 Sampling Methods<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p><strong>3.9 Illustrative Example of the Process of Design<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">We use the example of handwritten character recognition (Figure 3.13) as an illustrative example to explain to illustrate the design issues and approaches.<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-42 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-17.png\" alt=\"\" width=\"404\" height=\"196\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-17.png 404w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-17-300x146.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-17-65x32.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-17-225x109.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-17-350x170.png 350w\" sizes=\"auto, (max-width: 404px) 100vw, 404px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">We explain learning to perform a task from experience. Therefore, let us what is the meaning of task. Task can often be expressed through a mathematical function. In this case input can be x, output y and w the parameters that are \u201clearned\u201d. In case of classification output y will be discrete E.g. class membership, posterior probability, etc. For regression, y will be continuous. For the character recognition the task is as shown in Figure 3.14.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-43 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-18.png\" alt=\"\" width=\"513\" height=\"260\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-18.png 513w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-18-300x152.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-18-65x33.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-18-225x114.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-18-350x177.png 350w\" sizes=\"auto, (max-width: 513px) 100vw, 513px\" \/><\/p>\n<p style=\"text-align: justify\">The following are the steps in the design process for character recognition.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><strong>Step 0: <\/strong>Let us treat the learning system as a black box (Figure 3.15), here we assume that the input a set of handwritten characters and the output is the letter q.<\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\"><strong>Step 1: <\/strong>Next we collect Training Examples (Experience). Without examples, our system will not learn as we are learning from examples (Figure 3.16).<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-44 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-19.png\" alt=\"\" width=\"541\" height=\"450\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-19.png 541w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-19-300x250.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-19-65x54.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-19-225x187.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-19-350x291.png 350w\" sizes=\"auto, (max-width: 541px) 100vw, 541px\" \/><\/p>\n<p><strong>Step 2: Representing Experience<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The next step is to choose a representation for the experience\/examples. In our example the sensor input can represented by an n-dimensional vector, called the feature vector, X = (x1, x2, x3, \u2026,xn). We can assume a 64-d vector to represent the 8X8 matrix of pixels (Figure 3.17)<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-45 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-20.png\" alt=\"\" width=\"445\" height=\"196\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-20.png 445w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-20-300x132.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-20-65x29.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-20-225x99.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-20-350x154.png 350w\" sizes=\"auto, (max-width: 445px) 100vw, 445px\" \/><\/p>\n<p style=\"text-align: justify\">In order to represent the experience, we need to know what <strong>X<\/strong> is. Therefore we need a corresponding vector <strong>D<\/strong>, which will record our knowledge (experience) about <strong>X.<\/strong> The experience <strong>E<\/strong> is a pair of vectors <strong>E<\/strong> = (<strong>X<\/strong>, <strong>D<\/strong>). Now the question is how to represent D. Assuming our system is to recognise 10 digits only, then D can be a 10-d binary vector; each correspond to one of the digits (Figure 3.18).<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-46 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-21.png\" alt=\"\" width=\"406\" height=\"119\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-21.png 406w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-21-300x88.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-21-65x19.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-21-225x66.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-21-350x103.png 350w\" sizes=\"auto, (max-width: 406px) 100vw, 406px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><strong>Step 3: Choose a Representation for the Black Box<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">The next step is the choosing of a representation for the black box. Here we need to choose a function F to approximate the black box and for a given X, the value of F would give the classification of X.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-47 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-22.png\" alt=\"\" width=\"493\" height=\"134\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-22.png 493w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-22-300x82.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-22-65x18.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-22-225x61.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-22-350x95.png 350w\" sizes=\"auto, (max-width: 493px) 100vw, 493px\" \/><\/p>\n<p><strong>Step 4: Learning\/Adjusting the Weights<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">We need a learning algorithm to adjust the weights such that the experience from the training data can be incorporated into the system, where experience E is represented in terms of input X and expected output D. The function F(X) would be modified with weights W to obtain the learned output L as given in Figure 3.20.<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-48 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-23.png\" alt=\"\" width=\"566\" height=\"259\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-23.png 566w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-23-300x137.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-23-65x30.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-23-225x103.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-23-350x160.png 350w\" sizes=\"auto, (max-width: 566px) 100vw, 566px\" \/><\/p>\n<p><strong>Step 5: Use\/Test the System<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify\">After learning is completed, all parameters are fixed and an unknown input X can be presented to the system for which the system computes its answer according to the function F(W,X) (Figure 3.21).<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-49 aligncenter\" src=\"http:\/\/csp15.epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-24.png\" alt=\"\" width=\"473\" height=\"161\" srcset=\"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-24.png 473w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-24-300x102.png 300w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-24-65x22.png 65w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-24-225x77.png 225w, https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-content\/uploads\/sites\/65\/2018\/08\/Untitled-24-350x119.png 350w\" sizes=\"auto, (max-width: 473px) 100vw, 473px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p><strong>Summary<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>In this module the following were talked about<\/p>\n<ul>\n<li>Explained the basic steps in the design of a learning system<\/li>\n<li>Outlined how a function is chosen to fit the data and the parameters are tuned to minimize the error<\/li>\n<li>Discussed some methods of evaluation<\/li>\n<li>Explained the Design Process using an example<\/li>\n<\/ul>\n<table>\n<tbody>\n<tr>\n<td><strong>you can view video on Design of Learning System<\/strong><\/td>\n<td><a href=\"https:\/\/youtu.be\/r0168kd5r34\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-120\" src=\"http:\/\/epgpbooks.inflibnet.ac.in\/wp-content\/uploads\/2018\/11\/download.png\" alt=\"\" width=\"36\" height=\"36\" \/><\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><strong>Web Links<\/strong><\/p>\n<ul>\n<li style=\"text-align: justify\">http:\/\/people.cs.pitt.edu\/~milos\/courses\/cs2750-Spring03\/lectures\/class2.pdf<\/li>\n<li style=\"text-align: justify\">http:\/\/learningforward.org\/docs\/default-source\/commoncore\/comprehensive-professional-learning-system.pdf&#8221; type=&#8221;application\/pdf<\/li>\n<li style=\"text-align: justify\">http:\/\/digitalcommons.ilr.cornell.edu\/cgi\/viewcontent.cgi?article=1405&amp;amp;context=cahrswp&#8221; type=&#8221;application\/pdf<\/li>\n<li style=\"text-align: justify\">&#8220;http:\/\/www.cse.hcmut.edu.vn\/~tru\/AI\/chapter11.pdf&#8221; type=&#8221;application\/pdf<\/li>\n<li style=\"text-align: justify\">http:\/\/www.holehouse.org\/mlclass\/11_Machine_Learning_System_Design.html<\/li>\n<li style=\"text-align: justify\">ssdi.di.fct.unl.pt\/pc\/0607\/files\/PCaulaT03-10-06.ppt<\/li>\n<li style=\"text-align: justify\">http:\/\/www.cse.hcmut.edu.vn\/~tru\/AI\/chapter11.pdf<\/li>\n<li style=\"text-align: justify\">https:\/\/www.physicsforums.com\/threads\/why-ockham-razor.122774\/<\/li>\n<\/ul>\n","protected":false},"author":3,"menu_order":3,"template":"","meta":{"pb_show_title":"on","pb_short_title":"","pb_subtitle":"","pb_authors":[],"pb_section_license":""},"chapter-type":[],"contributor":[],"license":[],"class_list":["post-28","chapter","type-chapter","status-publish","hentry"],"part":3,"_links":{"self":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/pressbooks\/v2\/chapters\/28","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/pressbooks\/v2\/chapters"}],"about":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/wp\/v2\/types\/chapter"}],"author":[{"embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/wp\/v2\/users\/3"}],"version-history":[{"count":6,"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/pressbooks\/v2\/chapters\/28\/revisions"}],"predecessor-version":[{"id":453,"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/pressbooks\/v2\/chapters\/28\/revisions\/453"}],"part":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/pressbooks\/v2\/parts\/3"}],"metadata":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/pressbooks\/v2\/chapters\/28\/metadata\/"}],"wp:attachment":[{"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/wp\/v2\/media?parent=28"}],"wp:term":[{"taxonomy":"chapter-type","embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/pressbooks\/v2\/chapter-type?post=28"},{"taxonomy":"contributor","embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/wp\/v2\/contributor?post=28"},{"taxonomy":"license","embeddable":true,"href":"https:\/\/ebooks.inflibnet.ac.in\/csp15\/wp-json\/wp\/v2\/license?post=28"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}