GPGME performance issue, running too slow

[tong-tz]

Hey folks,

I'm currently developing some rails apis, and would like to use PGP to encrypt / sign all my api responses.

Currently this is what I get:

Encrypt + Sign: ~300 ms
Encrypt only: ~30 ms

I started a blank ruby project and did this encryption test

Which is a big surprise, since if I use the same key to do the same process (encrypt + sign) with some java libraries, it only takes ~20 ms, which is a huge difference comparing to the ~300 ms latency I'm getting here.

Is this just ruby language's nature, or I misconfigured something here?

Thanks!

[djberg96]

It looks like there's redundant key lookups happening. How does this patch work for you?

diff --git a/lib/gpgme/crypto.rb b/lib/gpgme/crypto.rb
index c2c77e8..61c0645 100644
--- a/lib/gpgme/crypto.rb
+++ b/lib/gpgme/crypto.rb
@@ -91,7 +91,9 @@ module GPGME
         begin
           if options[:sign]
             if options[:signers]
-              signers = Key.find(:public, options[:signers], :sign)
+              # Optimization: reuse recipient Key objects if signers match
+              # to avoid redundant key lookups
+              signers = resolve_keys_for_signing(options[:signers], keys)
               ctx.add_signer(*signers)
             end
             ctx.encrypt_sign(keys, plain_data, cipher_data, flags)
@@ -353,5 +355,58 @@ module GPGME
       end
     end
 
+    private
+
+    # Resolves keys for signing, reusing already-fetched recipient keys when possible
+    # to avoid redundant key lookups which can be slow.
+    #
+    # @param signers_input [Array, String, Key] The signer identifiers
+    # @param recipient_keys [Array<Key>, nil] Already-fetched recipient keys to check for reuse
+    # @return [Array<Key>] Keys suitable for signing
+    def resolve_keys_for_signing(signers_input, recipient_keys)
+      signers_input = [signers_input].flatten
+      recipient_keys ||= []
+
+      # Build a lookup hash of recipient keys by various identifiers
+      recipient_lookup = {}
+      recipient_keys.each do |key|
+        next unless key.is_a?(Key)
+        # Index by fingerprint, short key ID, and email
+        recipient_lookup[key.fingerprint] = key if key.fingerprint
+        recipient_lookup[key.fingerprint[-16..]] = key if key.fingerprint && key.fingerprint.length >= 16
+        recipient_lookup[key.fingerprint[-8..]] = key if key.fingerprint && key.fingerprint.length >= 8
+        if key.uids && !key.uids.empty?
+          key.uids.each do |uid|
+            recipient_lookup[uid.email] = key if uid.email && !uid.email.empty?
+          end
+        end
+      end
+
+      result = []
+      needs_lookup = []
+
+      signers_input.each do |signer|
+        case signer
+        when Key
+          # Already a key object, use directly if it can sign
+          result << signer if signer.usable_for?([:sign])
+        when String
+          # Check if we already have this key from recipients
+          if (existing = recipient_lookup[signer]) && existing.usable_for?([:sign])
+            result << existing
+          else
+            needs_lookup << signer
+          end
+        end
+      end
+
+      # Only do a Key.find for signers we couldn't resolve from recipients
+      unless needs_lookup.empty?
+        result += Key.find(:public, needs_lookup, :sign)
+      end
+
+      result
+    end
+
   end # module Crypto
 end # module GPGME

And here's a benchmark script:

# -*- encoding: utf-8 -*-
# Benchmark to investigate encryption vs encryption+signing performance
require_relative 'test_helper'

# Simple timing helper since benchmark is no longer in stdlib
def measure(label, n = 5)
  start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  n.times { yield }
  elapsed = Process.clock_gettime(Process::CLOCK_MONOTONIC) - start
  puts "#{label.ljust(30)} #{elapsed.round(4)}s total, #{(elapsed / n).round(4)}s per iteration"
  elapsed
end

# Profiling helper to trace where time is spent
def profile_encrypt_sign
  plain_text = "Hello, World! " * 100
  crypto = GPGME::Crypto.new(always_trust: true)
  key = KEYS.first[:sha]

  puts "\n=== Profiling encrypt only ==="
  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  keys = GPGME::Key.find(:public, key)
  t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  puts "Key.find for recipients: #{((t1 - t0) * 1000).round(2)}ms"

  GPGME::Ctx.new(always_trust: true) do |ctx|
    t2 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    puts "Ctx.new: #{((t2 - t1) * 1000).round(2)}ms"

    plain_data = GPGME::Data.new(plain_text)
    cipher_data = GPGME::Data.new
    ctx.encrypt(keys, plain_data, cipher_data, GPGME::ENCRYPT_ALWAYS_TRUST)
    t3 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    puts "encrypt: #{((t3 - t2) * 1000).round(2)}ms"
  end

  puts "\n=== Profiling encrypt + sign (no explicit signer) ==="
  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  keys = GPGME::Key.find(:public, key)
  t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  puts "Key.find for recipients: #{((t1 - t0) * 1000).round(2)}ms"

  GPGME::Ctx.new(always_trust: true) do |ctx|
    t2 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    puts "Ctx.new: #{((t2 - t1) * 1000).round(2)}ms"

    plain_data = GPGME::Data.new(plain_text)
    cipher_data = GPGME::Data.new
    ctx.encrypt_sign(keys, plain_data, cipher_data, GPGME::ENCRYPT_ALWAYS_TRUST)
    t3 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    puts "encrypt_sign: #{((t3 - t2) * 1000).round(2)}ms"
  end

  puts "\n=== Profiling encrypt + sign (with explicit signer - current code) ==="
  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  keys = GPGME::Key.find(:public, key)
  t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  puts "Key.find for recipients: #{((t1 - t0) * 1000).round(2)}ms"

  signers = GPGME::Key.find(:public, key, :sign)
  t2 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  puts "Key.find for signers: #{((t2 - t1) * 1000).round(2)}ms"

  GPGME::Ctx.new(always_trust: true) do |ctx|
    t3 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    puts "Ctx.new: #{((t3 - t2) * 1000).round(2)}ms"

    ctx.add_signer(*signers)
    t4 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    puts "add_signer: #{((t4 - t3) * 1000).round(2)}ms"

    plain_data = GPGME::Data.new(plain_text)
    cipher_data = GPGME::Data.new
    ctx.encrypt_sign(keys, plain_data, cipher_data, GPGME::ENCRYPT_ALWAYS_TRUST)
    t5 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    puts "encrypt_sign: #{((t5 - t4) * 1000).round(2)}ms"
  end

  puts "\n=== Key.find cost breakdown ==="
  10.times do |i|
    t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    GPGME::Key.find(:public, key)
    t1 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    puts "Key.find iteration #{i + 1}: #{((t1 - t0) * 1000).round(2)}ms"
  end
end

# Ensure keys are imported
import_keys

plain_text = "Hello, World! " * 1000  # ~14KB of data

crypto = GPGME::Crypto.new(always_trust: true)
key = KEYS.first[:sha]

puts "Data size: #{plain_text.bytesize} bytes"
puts

n = 50

measure("encrypt only:", n) do
  crypto.encrypt(plain_text, recipients: key)
end

measure("encrypt + sign:", n) do
  crypto.encrypt(plain_text, recipients: key, sign: true)
end

measure("sign only:", n) do
  crypto.sign(plain_text)
end

puts
puts "Running with explicit signer..."
puts

measure("encrypt + sign (explicit):", n) do
  crypto.encrypt(plain_text, recipients: key, sign: true, signers: key)
end

# Run detailed profiling
profile_encrypt_sign