Useful commands to see general information on the firewall resources been used, interface and traffic statistics, and traffic counters.
System and resources
show system resources is the same as top in linux, adding follow will keep it auto refreshing until stopped with CTRL + C.
ste@HME-PAL-OEW1> show system resources follow top - 17:41:48 up 25 days, 9:26, 1 user, load average: 0.82, 0.43, 0.33 Tasks: 222 total, 2 running, 167 sleeping, 0 stopped, 1 zombie %Cpu(s): 8.9 us, 1.7 sy, 0.0 ni, 87.0 id, 0.0 wa, 2.2 hi, 0.2 si, 0.0 st KiB Mem : 6612900 total, 280932 free, 2637248 used, 3694720 buff/cache KiB Swap: 4095996 total, 3526652 free, 569344 used. 886092 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 6592 root 20 0 66.7g 2.6g 2.6g S 15.9 41.0 5349:19 pan_task 5341 root 20 0 0 0 0 S 3.0 0.0 968:27.46 kni_single 5415 root 20 0 472768 25248 0 S 0.7 0.4 147:20.85 distributord 2924 root 0 -20 3047528 2.6g 2.6g S 0.3 40.8 61:20.15 masterd_apps 2967 root 15 -5 168604 7140 1988 S 0.3 0.1 64:58.25 sysd 5293 nobody 20 0 52840 2200 1632 S 0.3 0.0 19:50.62 redis-server 5298 nobody 20 0 50280 228 72 S 0.3 0.0 19:43.07 redis-server 5411 root 20 0 1764808 374444 122964 S 0.3 5.7 62:49.87 devsrvr 5416 root 20 0 520088 47852 208 S 0.3 0.7 45:03.03 iotd 5418 root 20 0 741304 150176 112504 S 0.3 2.3 218:54.33 useridd 5486 nobody 20 0 50280 2032 1640 S 0.3 0.0 30:57.99 redis-server 5493 nobody 20 0 58984 1708 1172 S 0.3 0.0 92:21.15 redis-server 5538 root 20 0 1745340 244996 12768 S 0.3 3.7 14:15.94 logrcvr 5547 root 20 0 721500 74572 9632 S 0.3 1.1 181:41.14 dnsproxyd 6889 root 20 0 66.7g 2.6g 2.5g S 0.3 40.5 8:04.76 sdwand 7061 root 20 0 66.7g 2.6g 2.5g S 0.3 40.5 63:38.55 pan_dha 8730 root 20 0 160764 15512 4 S 0.3 0.2 28:41.15 envoy 17075 ste 20 0 121636 2960 2400 R 0.3 0.0 0:00.04 top 28650 root 20 0 0 0 0 I 0.3 0.0 3:38.61 kworker/0:1-eve 1 root 20 0 4388 4 0 S 0.0 0.0 0:10.67 init
The running firewall processes and PID can be viewed using the command
show system software status.
If running any tasks that are CPU intensive such as logging of the packet flows, this command is useful for keeping an eye on what resources are using up what percent of the CPUs. By default it will be for the last 60 seconds, can view the average and maximum CPU usage by filtering on the last 1-60 seconds, 1-60 minutes, 1-24 hours, 1-7 days or 1-13 weeks.
ste@HME-PAL-OEW1> show running resource-monitor hour last 3 ste@HME-PAL-OEW1> show running resource-monitor Resource monitoring sampling data (per second): CPU load sampling by group: flow_lookup : 1% flow_fastpath : 1% flow_slowpath : 1% flow_forwarding : 1% flow_mgmt : 1% flow_ctrl : 1% nac_result : 0% flow_np : 1% dfa_result : 0% module_internal : 1% aho_result : 0% zip_result : 0% pktlog_forwarding : 1% send_out : 1% flow_host : 1% send_host : 1% fpga_result : 0% CPU load (%) during last 60 seconds: core 0 1 * 1
Both these commands provide really nice realtime statistics on session traffic (packet rate, throughput, number of sessions) and applications (sessions, packets, bytes) that keep refreshing until quit.
ste@HME-PAL-OEW1> show system statistics session System Statistics: ('q' to quit, 'h' for help) Device is up : 18 days 3 hours 12 mins 55 sec Packet rate : 120/s Throughput : 165 Kbps Total active sessions : 61 Active TCP sessions : 37 Active UDP sessions : 21 Active ICMP sessions : 3 ste@HME-PAL-OEW1> show system statistics application Top 20 Application Statistics: ('q' to quit, 'h' for help) Virtual System: vsys1 application sessions packets bytes -------------------------------- ---------- ------------ ------------ stun 1184 2581768 2093735764 itunes-base 3181 1823844 1698765716 twitter-base 936 1719312 1628091621 icloud-base 7005 1240232 910446082 rtp-base 3 735148 807039702 http-video 23 9402047 9396393333 avira-antivir-update 277 527288 537032712
A few useful commands for looking at the dataplane interface utilisation.
all Interface name, speed, duplex, state, MAC, IP address and zoneshow interface ethernet1/1 Settings and traffic counters including transmitted, received dropped and errorsshow counter interface all Traffic counters including transmitted, received dropped and errorsshow counter rate ethernet1/1 Number of tx/rx packets and Mbps for last second
Netstat shows information related to the management plane.
show netstat interfaces yes
Shows management, loopback and tap interfaces as well as counters (*all yes* also shows tcpdumpshow netstat route yes Shows the management routing tableshow netstat all yes Show the ports open and the state of the connections on themshow netstat programs yes Shows ports open anD connections but now includes program and PID
I dont really understand the purpose of the ‘system state’ commands yet, seem to be hardware capabilities as well as counters.
show system state filter-pretty sys.s1.p*
Shows counters on all interfacesshow system state browser Interface stats in realtime, do Shift + L, port_stats, Y, U
System logs can viewed by severity and further filtered down based on the object, subtype and event. Juts like traffic logs can filter the time-frame and add
csv-output equal yes to display the output in csv format.
show log system receive_time in [last-60-seconds | last-15-minutes | last-hour | last-6-hrs | last-12-hrs | last-24-hrs] show log system receive_time in [last-7-days | last-30-days | last-calendar-day | last-calendar-month] show log system start-time [equal | not-equal] <YYYY/MM/DD@hh:mm:ss> show log system end-time [equal | not-equal] <YYYY/MM/DD@hh:mm:ss> ste@HME-PAL-OEW1> show log system severity > equal equal > greater-than-or-equal greater-than-or-equal > less-than-or-equal less-than-or-equal > not-equal not-equal ste@HME-PAL-OEW1> show log system severity greater-than-or-equal critical critical high high informational informational low low medium medium ste@HME-PAL-OEW1> show log system object equal "ethernet1/2" subtype equal "port" Time Severity Subtype Object EventID ID Description =============================================================================== 2022/09/28 08:18:06 info port ethern link-ch 0 Port ethernet1/2: Down Unknown duplex 2022/09/28 08:18:06 info port ethern link-ch 0 Port ethernet1/2: Up 10Gb/s-full duplex 2022/09/28 08:18:06 info port ethern link-ch 0 Port ethernet1/2: MAC Up
The CLI doesn’t provide auto-completion on object, subtype or event, for this reason the GUI maybe a bit easier for looking at system logs.
The majority of the other non-system logs are grouped under plugins-log and mp-log (dp-log on some platforms). To view and filter use tail, less, follow and grep.
? View all log files such asls plugins-log ? View all log files for pluginstail mplog pan_boot.logless mp-log configd.logless mp-log dhcpd.logtail mp-log routed.logtail follow yesmp-log bfd.logtail follow yesmp-log pan_ifmgr.log
A disk quota is assigned to the different log types, logs will be purged when the quota is exceeded. The allocated quota and current usage levels can be checked with the command
show system logdb-quota.
All sessions traversing the firewall are tracked by the processes that touch them with global counters incremented for each step that a packet takes and for each packet in a session.
show counter global provides information about the processes/actions taken on the packets passing through the device; whether they are dropped, NAT-ed, decrypted and so on. This is a list of all the possible global counters.
ste@HME-PAL-OEW1> show counter global filter + aspect Counter aspect + category Counter category + delta Difference from last read + packet-filter Counters for packet that matches debug filter + severity Counter severity + value value
Global counters can be filtered based on severity, category and aspect.
- Severity: Are 4 levels of severity, info (default for all counters), drop (indicate something that was intentionally discarded like due to a security policy), error (packets that are malformed and are discarded), or warn (system level error or abnormality in received packets)
- Category: Indicates which process this counter is related to (dfa (APP-ID algorithm engine), appid, dlp, flowpacket, uid, zip, nat, etc)
- Aspect: More detail regarding which stage a packet was in when the counter was incremented (parse, session, and forward are three stages of flow)
Drop counters are normally the ones of real interests as it keeps a count of all drops, what is causing them (such as flow_policy_deny for packets that were dropped by a security rule) and how many packets were dropped. By running the command multiple times with delta yes can see the drop counters since the last time the command was run making it easier to see if counters are increasing.
show counter global
View all global countersshow counter global filter delta yes View only global counters that have changed since ran last commandshow counter global name ? Lists all the counters that are availablesshow counter global filter category Lists available processes such as cluster, device, dlp, flow, nat, proxy, ssl, tcp, url, etcshow counter global filter aspect Lists available stages such as arp, bfd, dos, forward, offload, pktproc, etcshow counter global filter value non-zero Show only counters without non-zero valuesshow counter global name flow_policy_deny Show this one counter with a brief descriptionshow counter global filter severity drop Show only drop countersshow counter global filter severity dropdelta yes Show only drop counters that have happened since last command was run
Global counters can be narrowed down to specific traffic flows by using the same filters that packet captures use. Before starting is best to clear all counters and unmark any sessions that were marked by the previous filter.
debug dataplane packet-diag clear all
Clear all capture settings, counters and filtersdebug dataplane packet-diag clear filter-marked-session all Unmark any sessions that were marked by previous filtersdebug dataplane packet-diag set filter match destination 220.127.116.11 Configure upto 4 filtersdebug dataplane packet-diag set filter match source 18.104.22.168show counter global filter delta yespacket-filter yesseverity drop Show only drop counters for flows that match this filter
Palo KB article on global counters and filtering them
Some really good info on counters and lots of other CLI show commands
Some handy cheatsheets from Mastering Palo Alto Networks by Tom Piens