awk “Aho Weiberger and Kernighan” 三个作者的姓的第一个字母awk是Linux下的一个命令同时也是一种语言解析引擎awk具备完整的编程特性。比如执行命令网络请求等精通awk,是一个Linux工作者的必备技能语法awk ‘pattern{action}’awk pattern语法awk理论上可以代替grepawk ‘pattern{action}’ ,默认以空格分隔awk ‘BBEGIN{}END{}’ 开始和结束 awk ‘/Running/’ 正则匹配 awk ‘/aa/,/bb/’ 区间选择 awk ‘$2~/xxx/’ 字段匹配这里指从第2个字段开始匹配包含xxx内容的行 awk ’NR2’ 取第二行 awk ’NR1’ 去掉第一行awk的字段数据处理-F参数指定字段分隔符BEGIN{FS‘_’}也可以表示分隔符$0 代表原来的行 $1 代表第一个字段 $N 代表第N个字段 $NF 代表最后一个字段下面以一个在nginx.log中查找返回状态码非200的请求响应数目的需求为例演示awk的基础用法有一份nginx.log文件打开后内容格式如下220.181.108.111 - -[05/Dec/2018:00:11:42 0000]GET /topics/15225/show_wechat HTTP/1.1200 1684-Mozilla/5.0 (compatible; Baiduspider/2.0; http://www.baidu.com/search/spider.html)0.029 0.029.216.244.66.241 - -[05/Dec/2018:00:11:42 0000]GET /topics/10052/replies/85845/reply_suggest HTTP/1.1301 5-Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, helpmoz.com)0.016 0.016.216.244.66.241 - -[05/Dec/2018:00:11:42 0000]GET /topics/10040?order_bycreated_at HTTP/1.1301 5-Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, helpmoz.com)0.002 0.002.216.244.66.241 - -[05/Dec/2018:00:11:42 0000]GET /topics/10043/replies/85544/reply_suggest HTTP/1.1301 5-Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, helpmoz.com)0.001 0.001.216.244.66.241 - -[05/Dec/2018:00:11:44 0000]GET /topics/10075/replies/89029/edit HTTP/1.1301 5-Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, helpmoz.com)0.001 0.001.216.244.66.241 - -[05/Dec/2018:00:11:44 0000]GET /topics/10075/replies/89631/edit HTTP/1.1301 5-Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, helpmoz.com)0.001 0.001.216.244.66.241 - -[05/Dec/2018:00:11:45 0000]GET /topics/10075?order_bycreated_at HTTP/1.1301 5-Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, helpmoz.com)0.000 0.000.216.244.66.241 - -[05/Dec/2018:00:11:45 0000]GET /topics/10075?order_bylike HTTP/1.1301 5-Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, helpmoz.com)0.001 0.001.223.71.41.98 - -[05/Dec/2018:00:11:46 0000]GET /cable HTTP/1.1101 60749-Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:63.0) Gecko/20100101 Firefox/63.02608.898 2608.898.113.87.161.17 - -[05/Dec/2018:00:11:39 0000]GET /cable HTTP/1.1101 3038-Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36112.418 112.418.216.244.66.241 - -[05/Dec/2018:00:11:46 0000]GET /topics/10079/replies/119591/edit HTTP/1.1301 5-Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, helpmoz.com)0.001 0.001.216.244.66.241 - -[05/Dec/2018:00:11:46 0000]GET /topics/10089?localezh-TW HTTP/1.1301 5-Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, helpmoz.com)0.002 0.002.观察log内容可以发现以空格为分隔符状态码在第九个字段位置这里我们用awk命令从第九个字段位置开始匹配非200的状态码并打印出来命令awk $9!~/200/{print $9} nginx.log[avbxb9efockpz ~]$ awk $9!~/200/{print $9} nginx.log 301 301 301 301 301 301 301 301 301 ......#剩余部分省略再对取出的数据进行排序-去重-按数字的倒叙进行排列命令awk $9!~/200/{print $9} nginx.log | sort | uniq -c | sort -nr命令含义sort: 按从小到大进行排序uniq-c :去重相邻 -nr: 按数字进行倒叙排序 -n:按数字进行排序结果展示[sqavbxb9efockpz ~]$awk$9!~/200/{print$9}nginx.log|sort|uniq-c|sort-nr 433 101 304 301 266 404 152 302 7 401 5 304 2 499 2 422 1 500再结合awk ‘BBEGIN{}END{}’ 命令以统计当前用户数目的例子来展示命令用法使用cat /etc/passwd命令来查看本机用户我们需要提取出用户名称并加上数字序号显示出来达到这种效果1 nobody 2 root 3 daemon 4 _uucp 5 _taskgated 6 _networkd 7 _installassistant 8 _lp 9 _postfix......#后面的省略用户信息localhost:~ qinzhen$cat/etc/passwd### User Database## Note that this file is consulted directly only when the system is running# in single-user mode. At other times this information is provided by# Open Directory.## See the opendirectoryd(8) man page for additional information about# Open Directory.##nobody:*:-2:-2:Unprivileged User:/var/empty:/usr/bin/false root:*:0:0:System Administrator:/var/root:/bin/sh daemon:*:1:1:System Services:/var/root:/usr/bin/false _uucp:*:4:4:Unix to Unix Copy Protocol:/var/spool/uucp:/usr/sbin/uucico _taskgated:*:13:13:Task Gate Daemon:/var/empty:/usr/bin/false _networkd:*:24:24:Network Services:/var/networkd:/usr/bin/false _installassistant:*:25:25:Install Assistant:/var/empty:/usr/bin/false _lp:*:26:26:Printing Services:/var/spool/cups:/usr/bin/false _postfix:*:27:27:Postfix Mail Server:/var/spool/postfix:/usr/bin/false _scsd:*:31:31:Service Configuration Service:/var/empty:/usr/bin/false _ces:*:32:32:Certificate Enrollment Service:/var/empty:/usr/bin/false _appstore:*:33:33:Mac App Store Service:/var/empty:/usr/bin/false _mcxalr:*:54:54:MCX AppLaunch:/var/empty:/usr/bin/false......#后面的省略思路-awk运行前先定义序号索引0用来递增保存用户-利用awk将用户提取出来按索引分别保存-切片结束后再按行数进行循环将数字序号与第一步保存的信息拼接打印注意cat /etc/passwd打印出的结果中最上方的注释需要处理跳过cat/etc/passwd|awk-F:BEGINE{userindex0}{user[userindex]$1;userindex}END{for(i0;iNR;i)print i1, user[i10]}|less