{"id":357,"date":"2018-07-13T10:39:18","date_gmt":"2018-07-13T10:39:18","guid":{"rendered":"http:\/\/itp7.epgpbooks.inflibnet.ac.in\/?post_type=chapter&p=357"},"modified":"2019-05-16T08:58:21","modified_gmt":"2019-05-16T08:58:21","slug":"awk-scripting","status":"publish","type":"chapter","link":"https:\/\/ebooks.inflibnet.ac.in\/itp7\/chapter\/awk-scripting\/","title":{"rendered":"AWK Scripting"},"content":{"raw":"
\"epgp<\/a>\r\n<\/span><\/div>\r\n \r\n\r\nOverview of AWK<\/strong>\r\n\r\n \r\n

AWK is a scripting language supported by most of the Linux distributions. AWK supports text processing and is useful in report generation. It is generally used for data extraction from flat files. AWK usually works on streams of textual data to perform set of actions for data extraction. AWK is widely used to generate reports from flat files or from log files. Let us understand the applicability of AWK in networking domain. Suppose, we are using NS2 simulator that simulates large network. This simulator will generate trace files which contain the records of each packet transfer happening in the network. The amount of records generated by NS2 can be in thousands, when we want to generate a summary of the packet transfers, AWK can be used to extract certain records according to user\u201fs criteria. AWK is a widely used tool for reporting purpose.<\/p>\r\n \r\n

AWK was created in 1977 at Bell Labs. The name, AWK is derived from the beginning letters of the surnames of its developers\/authors who were Alfred Aho, Peter Weinberger and Brian Kernighan. Later, during 1985-88, AWK was revised by Paul Rubin, Jay Fenalson and Richard Stallman. The revised AWK version was released in 1988 as GNU AWK. As of now, AWK is available in various Linux distributions, AWK has many variants like gawk, mawk, nawk etc.<\/p>\r\n \r\n\r\nFile Structure and AWK<\/strong>\r\n\r\n \r\n

AWK reads from a file or from the standard input. The output of AWK is to its standard output. AWK adheres to the concepts of \"file\", \"record\" and \"field\" as used in flat file model. A structured text file consists of records, usually, each line of a file represent a single record. So, each line is a record for AWK. AWK operates on one record at a time. A record consists of field(s) which are structured as columns, these fields are separated by space(s) or tab(s) by default. Each filed can be accessed by prefixing \u201e$\u201f sign, so the first field can be accessed as $1, field 2 with $2, and so on, however, the field $0 refers to the whole record. Hence a structure file containing rows are identified as records and columns are identified as fields. The following figure is an example of flat file structure where rows are identified as different records and columns are identified as fields:<\/p>\r\n

\"\"<\/p>\r\n

Figure 1: Flat File Model(Source: https:\/\/commons.wikimedia.org\/wiki\/File:Flat_File_Model.svg)<\/p>\r\n \r\n\r\nLet us take an example to understand the structure of file. Suppose a system with \/etc\/passwd file contains the following entries in its tail section:\r\n\r\n \r\n\r\nhardik:x:504:504::\/home\/hardik:\/bin\/bash hiren:x:505:505::\/home\/hiren:\/bin\/bash rakesh:x:506:506::\/home\/rakesh:\/bin\/bash<\/strong>\r\n\r\n \r\n\r\nAWK will see this file as follows:\r\n

1 line indicates 1 record, so in total there are 3 records in the above mentioned file. 1 record indicates 7 fields in the above case, the records are separated by \u201c:\u201d. It must be noted that by default, the field separator is space.<\/p>\r\n \r\n

AWK is a programming language. AWK uses specific structure of program to perform pattern matching. An awk program is a sequence of statements that take the following form:<\/p>\r\n

\"\"<\/p>\r\n \r\n\r\nIn general, the format is as follows:\r\n\r\n \r\n\r\nawk options \u201eselection criteria {actions}\u201f file(s)<\/strong>\r\n

Where, pattern acts as a selector that determines whether the action is to be executed. Patterns can be: arithmetic expressions, relational expressions, regular expressions string-valued or boolean expressions. These actions can be a\u00a0sequence of statements terminated by newlines or semicolons. AWK programs can either be executed on command prompt or written in a file. The following figure illustrates use of pattern matching in AWK:<\/p>\r\n

\"\"<\/p>\r\n

Figure 2: AWK patterns and field<\/strong><\/p>\r\n \r\n\r\nNow, let us see how AWK can be used to fetch various patterns from the file.\r\n\r\nAWK Command to fetch the userid of user \u201chardik\u201d:\r\n\r\n$awk -F\":\" \u201e\/hardik\/ {print $1 \" \" $3}\u201f \/etc\/passwd<\/strong>\r\n\r\nOutput:\r\n\r\nhardik 504\r\n\r\nAnother way to write the command is:\r\n\r\n$awk \u201eBEGIN { FS=\u201c:\u201d } \/hardik\/ {print $1 \" \" $3}\u201f \/etc\/passwd<\/strong>\r\n\r\nOutput:\r\n\r\nhardik 504\r\n\r\n \r\n

In the above example, print command is used to output text on standard output. The output text is always terminated with a newline. Here, the \u201eF\u201f option is used to specify field separator which is \u201e:\u201f. The print command can be used as follows to display the first field:<\/p>\r\n \r\n\r\nprint $1<\/strong>\r\n\r\n \r\n\r\nThe following command displays the first field and third field of the current record:\r\n\r\nprint $1, $3<\/strong>\r\n

Let us see other few examples to understand the use of AWK. In the following example, we will consider a student record file. The content of file \u201estmarks.dat\u201f are student no, name, marks in subject-1, marks in subject-2 and marks in subject-3. Each record is a line in the file, a space separates each field. The content of file as shown below:<\/p>\r\n

\"\"<\/p>\r\n

Figure 3: Content of stmarks.dat file<\/p>\r\n \r\n

To display the initial two fields of file \u201estmarks.dat\u201f, we use the print command as shown earlier. However, here the field separator is space so there is no need to explicitly specify the field separator using the \u201eF\u201f option. The following figure demonstrates use of print command with awk:<\/p>\r\n

\"\"<\/p>\r\n

Figure 4: Using print command with awk<\/p>\r\n \r\n\r\nUsing operators with AWK<\/strong>\r\n

AWK supports a wide range of operators. The following table lists various comparison operators supported\u00a0 by AWX:<\/p>\r\n

\"\".<\/p>\r\nThe following screen-shot demonstrates an example where comparision operator is applied. In the following screen, the command lists the student number and student name who have scored marks above 60 in subject-1:\r\n

\"\"<\/p>\r\nThe following screen-shot demonstrates an example where comparision operator is applied. In the following screen, the command lists the student number and student name who have scored marks above 60 in subject-1:\r\n

\"\"<\/p>\r\n

Figure 5: Using comparison operator in awk Using Arithmetic Operators with AWK<\/strong><\/p>\r\n \r\n

AWK supports arithmetic operators like addition (+), subtraction (-), multiplication (*), division (\/), remainder (%), exponentiation (^), increment (++) and decrement (--) with pattern matching.The following illustration demonstrates the use of addition operator to display those records where the total of marks of three subjects is above 180:<\/p>\r\n\"\"\r\n

Figure 6: Using arithmetic operators with AWK<\/p>\r\n \r\n

In the following figure, we have applied multiple conditions to selectively filter records. The following example demonstrates the display of only those records where the marks in 3 different subjects are above 50:<\/p>\r\n

\"\"<\/p>\r\n

Figure 7: Comparison and && operator with AWK<\/p>\r\n \r\n\r\nUsing regular expression with AWK<\/strong>\r\n\r\n \r\n

Let us see an example how regular expression can be used with AWK. In the following illustration, we are listing those records where the student name begin with \u201eR\u201f:<\/p>\r\n

\"\"<\/p>\r\n

Figure 8: Using regular expression with AWK<\/p>\r\n \r\n

Using Built-in variables with AWK<\/strong><\/p>\r\n

AWK supports the use of both user-defined variables and built-in variables. In this section, let us explore the built-in variables supported by AWK. The built-in variables are used in AWK script<\/p>\r\n \r\n\r\nThe following variables are supported by AWK:\r\n