Improve scan file

[ghost]

Improve scan file column delimiter specification
Place the important cha channel bssid at the front

hcxdumptool -i $INTERFACE --rds=1 -F >ScanAP.list
Or --output-format csv

CHA   MAC-AP    ESSID    BEACON  RESPONSE RSSI                  SCAN:  2412/1
------------------------------------------------------------------------------
1    | 4456e3******  | -28  | **********    | 10:11:52  | 10:11:52 |
5    | 3447d4******  | -32  | *******        |               |  10:11:52 |
6    | bc9df5******  | -52  | *                 | 10:11:52  |               |
11  | b8f888******   | -62  |                   |10:11:52   | 10:11:52 |
10  | 3647d4******  | -42  |                   |                |              |
8    | 70192f******   | -20  | **********    |                |              |

[ZerBea]

Done by this commit (replaced " " by "|"):
https://github.com/ZerBea/hcxdumptool/commits/master/

It makes no sense to print more characters (" | " or "\t") than a single one as delimiter, because the status line becomes too long.

[ghost]

Very good !

[ghost]

@ZerBea

 local scan_file="/tmp/scan-$(date +%s).csv"
    sudo hcxdumptool -i $MON_INTERFACE --rcascan=p -F >"$scan_file" &
local scan_pid=$!

When it is run background, output Scan file is empty,
but not every time, but it always encounters abnormal (but is more stable when running in the foreground window)

seems that a parameter needs to be add to make it suitable for running in background
--output-scan file

When running it in background, it is easy to cause driver to crash
ERROR(s) during runtime (mostly caused by a broken driver)

[ZerBea]

If desired to run hcxdumptool in background or to redirect the status to a file, it is mandatory to disable the entire internal screen handling of hcxdumptool. If not, hcxdumptool throws ERRORs because it can't handle the screen (because there is no screen to handle).
The option "--rdt" disables the entire screen handling on all real time displays:
--rdt : disable TIOCGWINSZ for real time displays

All ERRORs are gone:

# hcxdumptool --rcascan=active --rdt > test2.log &
# killall hcxdumptool
#

More information is here:
#504 (reply in thread)

[ghost]

Oh, ok thanks !

[ZerBea]

BTW:
There is no need to close a discussion. Just mark it as answered.

[ghost]

There is a problem with the scanning part, seems that compatibility with the network card is not good enough
Using the new Git version, hcxdump Linux kernel 6.12 test
card list
rt5572
rt3070
rtl8812bu
Unable to scan many known APs

It shouldn't be a driver issue
Because I compared it with other tools

[ZerBea]

I can't confirm this running an official Linux Kernel
Test target responds to all tested devices:

$ uname -r
6.12.23-1-lts

$ sudo hcxdumptool --rcascan=active --rds=2 -c 11a

ID 148f:5572 Ralink Technology, Corp. RT5572 Wireless Adapter
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|13:58:00|13:57:51| -52|ccce1edc3bee|AP_7272

148f:3070 Ralink Technology, Corp. RT2870/RT3070 Wireless Adapter
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:03:32|14:03:30| -61|ccce1edc3bee|AP_7272

ID 0cf3:9271 Qualcomm Atheros Communications AR9271 802.11n
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:05:57|14:05:53| -51|ccce1edc3bee|AP_7272

ID 148f:761a Ralink Technology, Corp. MT7610U ("Archer T2U" 2.4G+5G WLAN Adapter
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:07:04|14:07:01| -51|ccce1edc3bee|AP_7272

ID 0e8d:7612 MediaTek Inc. MT7612U 802.11a/b/g/n/ac Wireless Adapter
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:09:56|14:09:52| -54|ccce1edc3bee|AP_7272

ID 2357:0108 TP-Link TL-WN822N Version 4 RTL8192EU
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:15:33|14:15:30| -54|ccce1edc3bee|AP_7272

ID 2357:0109 TP-Link TL-WN823N v2/v3 [Realtek RTL8192EU]
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:16:21|14:16:17| -57|ccce1edc3bee|AP_7272

ID 2357:010c TP-Link TL-WN722N v2/v3 [Realtek RTL8188EUS]
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:17:19|14:17:19| -49|ccce1edc3bee|AP_7272

ID 148f:5370 Ralink Technology, Corp. RT5370 Wireless Adapter
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:18:44|14:18:40| -50|ccce1edc3bee|AP_7272

ID 7392:7710 Edimax Technology Co., Ltd Edimax Wi-Fi
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:19:59|14:19:56| -57|ccce1edc3bee|AP_7272

ID 0b05:17d1 ASUSTek Computer, Inc. AC51 802.11a/b/g/n/ac Wireless Adapter [Mediatek MT7610U]
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:36:13|14:36:09| -62|ccce1edc3bee|AP_7272

ID 0bda:8187 Realtek Semiconductor Corp. RTL8187 Wireless Adapter
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:38:19|14:38:17| -24|ccce1edc3bee|AP_7272

ID 0e8d:7961 MediaTek Inc. Wireless_Device
CHA| BEACON |RESPONSE|RSSI|   MAC-AP   |ESSID                (SCAN:  2462/11)
---+--------+--------+----+------------+--------------------------------------
 11|14:40:29|14:40:27| -47|ccce1edc3bee|AP_7272

rtl8812bu is not part of the official Linux kernel:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/net/wireless/realtek/rtw88?h=v6.12.23
I guess you're running a special distribution kernel with patched drivers.

[ghost]

My
$ uname -r
6.12.13-amd64

[ghost]

If each channel AP list is displayed separately, scanning becomes meaningless
Because it has already jumped to the next channel, the previous AP list cannot be seen anymore
So, all AP columns need to be displayed in the same window

[ZerBea]

Ok, I understand, Please try latest commit.
Running in rcascan mode, "--rds=3" shows all APs on all channels sorted by RSSI.

-rds=<digit>    : enable real time display
                    attack mode:
                     0 = off(default)
                     1 = show APs on current channel, show CLIENTs (M1M2ROGUE)
                     2 = show all APs (M1M2, M1M2M3 or PMKID), show CLIENTs (M1M2ROGUE)
                     3 = show all APs, show CLIENTs (M1M2ROGUE)
                     columns:
                      E = encryption (e)ncrypted / (o)pen
                      A = AKM (p)re-shared key
                      1 = received M1
                      2 = received M1M2
                      3 = received M1M2M3
                      P = received PMKID
                    rcascan mode
                     0 = show APs on current channel sorted by BEACON timestamp
                     1 = show APs on current channel sorted by PROBERESPONSE timestamp
                     2 = show APs on current channel sorted by RSSI
                     3 = show APs of all channels sorted by RSSI

[ghost]

Yeah, rds=3 is necessary
Now everything is back to normal !

[ZerBea]

Looks like sorting by RSSI makes the display less "noisy" than displaying all APs on all channels sorted by BEACON or PROBERESPONSE. The RSSI does not change as often as the BEACON time (every 100 ms).

[ghost]

hcxdumptool -F --rcascan=a --rds=3 -t 1
hold time must be >= 5 seconds

Scan is forced to stay on the channel for 5s
If scan 2/5g channels, need 300s !

[ZerBea]

Looks like some interfaces / drivers are really slow.
By this commit:
c858e31
when running in rcascan mode, stay time is not longer hard coded to 5 seconds and be controlled by "-t" option.

[ZerBea]

This is a discussion, so I add some important information (must read and must understand) for other readers here:

All (new) features and especially the real time status display have a price tag!

Process an incoming BEACON (usually an AP transmit a BEACON every 102.4 ms):

process80211beacon_rcascan() took 0.000004 seconds to execute 
process80211beacon_rcascan() took 0.000003 seconds to execute 
process80211beacon_rcascan() took 0.000003 seconds to execute 
process80211beacon_rcascan() took 0.000002 seconds to execute 
process80211beacon_rcascan() took 0.000004 seconds to execute 
process80211beacon_rcascan() took 0.000002 seconds to execute 
process80211beacon_rcascan() took 0.000003 seconds to execute 
process80211beacon_rcascan() took 0.000003 seconds to execute 

Process an incoming PROBERESPONSE:

process80211proberesponse_rcascan() took 0.000001 seconds to execute 
process80211proberesponse_rcascan() took 0.000001 seconds to execute 
process80211proberesponse_rcascan() took 0.000001 seconds to execute 
process80211proberesponse_rcascan() took 0.000001 seconds to execute 
process80211proberesponse_rcascan() took 0.000002 seconds to execute 
process80211proberesponse_rcascan() took 0.000001 seconds to execute 
process80211proberesponse_rcascan() took 0.000001 seconds to execute 
process80211proberesponse_rcascan() took 0.000001 seconds to execute 

Show real time display:
show_realtime_rca() took 0.000174 seconds to execute

The more traffic on the channel && the more options == hcxdumptool becomes sluggish!
The kernel starts to drop packets:

^C
336 Packet(s) captured by kernel
125 Packet(s) dropped by kernel
exit on sigterm

[ghost]

Ok thanks

[ghost]

Looks like some interfaces / drivers are really slow.

@ZerBea
Additionally, it may be important to note that some code may cause driver crashes,
Because I found that using hcxdump can easily cause network card issues
Can add? If there is a problem with the driver, it can automatically reset the driver and disconnect the interface to reconnect

Other tools
They seem to reduce and avoid such problems, with a low failure rate and high compatibility

[ZerBea]

Good point!
hcxdumptool adheres the specification of NETLINK.
Unfortunately some drivers don't do this. As a cause, they might crash.

I do not add code (that makes hcxdumptool more sluggish) to workaround firmware or driver bugs!
The list of this bugs is long:
https://patchwork.kernel.org/project/linux-wireless/list/
https://bugzilla.kernel.org/

[ghost]

when running in rcascan mode, stay time is not longer hard coded to 5 seconds and be controlled by "-t" option.

Test parameters are not working
hcxdumptool -F --rcascan=a --rds=3 -t 1
hold time must be >= 5 seconds

[ZerBea]

By latest commit:
c2105a7
minimum stay time is now 2 seconds,

$ sudo hcxdumptool --rcascan=active -t 1
hold time must be > 1 second

$ sudo hcxdumptool --rcascan=active -t 2
This is a highly experimental penetration testing tool!
...

Even if you now can set values of 2,3,4, that does not make sense.
5 seconds stay time is a good value to give all APs on the channel the chance to respond. That is not the case on shorter stay time.
A stay time e.g. of 1 only allows to get a response of < 5 APs.

[ZerBea]

Please take a look at this table:
#512 (reply in thread)
If the stay time on the channel is e.g. one second, you only get this one (BEACON and response within the same second):
11|14:17:19|14:17:19| -49|ccce1edc3bee|AP_7272

while this one get lost (2 seconds between BEACON and response):
11|14:03:32|14:03:30| -61|ccce1edc3bee|AP_7272

and this one get lost (4 seconds between BEACON and response)::
11|14:16:21|14:16:17| -57|ccce1edc3bee|AP_7272

I strongly recommend not to use stay times below 5 seconds.

[ZerBea]

The difference:
Official hcxdumptool: (mostly useless) status display && a lot of (mostly useless) options && a lot of (mostly) useless features included:
#512 (reply in thread)

^C
336 Packet(s) captured by kernel
125 Packet(s) dropped by kernel
exit on sigterm

My test version (no status display && no useless options && no useless features && 100% tailored to the mission by BPF) running against three test target APs (channel 1, 6 and 11). Each of them has one connected CLIENT.

^C
packets captured..: 34357
packets dropped...: 0
eventflag:........: 0001

Absolutely no dropped packets.

[ghost]

The parameters still don't work
sudo hcxdumptool --rcascan=active -t 2
Observation still stays for >5s (SCAN: 2412/11)

Discovered issues with other parameters as well : --exitoneapol=4
Captured m1m2 or 1m2m3 still not exiting
sudo hcxdumptool -i wlan0 -c 6a --bpf=attack.bpf --exitoneapol=4

[ZerBea]

Option -t is the stay time on a channel before hcxdumptool moves to the next channel. If you set -t 2 hcxdumptool stays exact for 2 second on a channel.
Option -t is not the refresh time of the display. The refresh time of the display is hard coded to refresh it every 5 seconds.
Due to performance reasons I do not change this. Doing this every second, the value of dropped packets increase incredible.
That's still halfway justifiable: show_realtime_rca() took 0.000174 seconds to execute
Doing this every second drops performance incredibly.

To fix exitoneapol, I need some time, because I'm not at home at the moment.

[ZerBea]

I think I found a solution. Now the refresh time of the real time display is equal to the stay time on the channel. Unfortunately the performance impact is huge on "t < 5".

[ZerBea]

Now exitoneapol should be fixed too:

sudo hcxdumptool -c 1a --rds=3 --m2max=0 --exitoneapol=2 -i wlp5s0f3u3
...
1352 Packet(s) captured by kernel
38 Packet(s) dropped by kernel
exit on EAPOL M1M2M3

sudo hcxdumptool -c 1a --rds=3 --m2max=0 --exitoneapol=4 -i wlp5s0f3u3
...
695 Packet(s) captured by kernel
70 Packet(s) dropped by kernel
exit on EAPOL M1M2

[ZerBea]

Some more tests confirm that the impact of the real time display is huge:

# time hcxdumptool -i wlp5s0f3u3 --m2max=0 --exitoneapol=2 -c 1a

306 Packet(s) captured by kernel
0 Packet(s) dropped by kernel
exit on EAPOL M1M2M3

real	0m7,277s
user	0m0,000s
sys	0m0,027s


# time hcxdumptool -i wlp5s0f3u3 --m2max=0 --exitoneapol=2 -c 1a --rds=3
...
2023 Packet(s) captured by kernel
42 Packet(s) dropped by kernel
exit on EAPOL M1M2M3

real	0m43,481s
user	0m0,043s
sys	0m0,096s

# time hcxdumptool -i wlp5s0f3u3 --m2max=0 --exitoneapol=2 -c 1a --rds=3 --rdt > log.file
...
2331 Packet(s) captured by kernel
20 Packet(s) dropped by kernel
exit on EAPOL M1M2M3

real	0m45,692s
user	0m0,011s
sys	0m0,110s

The same applies to "t < 5" and -F option.
All this options can cause a significant loss of performance!