Adds network capture decryption support to http scanners

[cgranleese-r7]

This pull request adds enhanced support for network capture decryption for http scanner modules. By writing to the sslkeylogfile it enables network capture decryption which is useful to decrypt TLS traffic in Wireshark.

This is a follow on to #20024 and rapid7/rex-socket#74.

Verification

• Start msfconsole
• Test the changes against some scanner/http/* modules.
• The modules should complete
• Run ls -la and you should now see a file called sslkeylogfile.txt
• Code changes are sane

[OmegaGateway333]

well i hope it works for the best.

[sjanusz-r7]

Works for me 🎉 I used a script to automate the testing of scanner/http modules.

I did have to update rex-socket manually to pull in the updated gem:

bundle update rex-socket

Then, I set up an example SSL Docker container as a target with this:

docker run -p 18080:8080 -p 18443:443 vulhub/openssl:1.0.1c-with-nginx

In this example, I'm using ./sslkeylogfile.txt as the output file.

Then I did:

bundle exec msfconsole -q

When the Console loaded up, I went into:

irb

I set up Wireshark to listen with the following filter:

tcp.port == 18443 && http

In this IRB prompt, I have pasted in the following script:

rejected = ['crawl', 'brute', 'dir', 'ad_login', 'enum_wayback', 'exchange_proxylogon', 'etherpad_duo_login', 'es_file_explorer_open_port', 'synology_forget_passwd_user_enum', 'xpath'].freeze
http_scanner_modules = ::Msf::Modules::Metadata::Cache.instance.find('ref_name' => [['scanner/http'], []]).reject { |x| rejected.any? { |rej| x.ref_name.include?(rej) } }

results = []

http_scanner_modules.each do |mod|
  module_instance = framework.modules.create(mod.ref_name)
  module_instance.import_defaults
  # module_instance.required_cred_options.each { |opt| opt = 'sjanusz-r7' }
  module_instance.datastore['USERNAME'] = 'sjanusz-r7'
  module_instance.datastore['PASSWORD'] = 'sjanusz-r7'
  module_instance.datastore['RHOSTS'] = '127.0.0.1'
  module_instance.datastore['SSL'] = true
  module_instance.datastore['RPORT'] = 18443
  module_instance.datastore['TARGETURI'] = '/'
  module_instance.datastore['DOMAIN'] = '/'
  module_instance.datastore['SSLKeyLogFile'] = './sslkeylogfile.txt'
  puts "Validating & Running module: #{module_instance.fullname}"
  begin
    module_instance.options.validate(module_instance.datastore)
    module_instance.run_simple
  rescue ::StandardError => e
    puts e.message
    results << { status: :fail, mod: module_instance }
    next
  end
  results << { status: :success, mod: module_instance }
end

puts "Successful module runs: #{results.select{|x| x[:status] == :success}.count }"
puts "Failed module runs: #{results.select{|x| x[:status] == :fail}.count }"

nil

You should see some entries in WireShark, all of which are 400 Bad Request. For example:

Set up the WireShark TLS Key File using this link: https://wiki.wireshark.org/TLS#using-the-pre-master-secret
Now, you should see the list of packets has increased, and you will be able to see the actual HTTP requests. For example: