fingerprint injection makes browsers detectable as headless

[ishfx]

Describe the bug
Simply injecting fingerprint-suite (using newInjectedContext or newInjectedPage), with or without options, for headless or headfull browser, makes the browser detected as headless in : https://arh.antoinevastel.com/bots/areyouheadless.

• headless = detected
• headless + stealth = undetected
• headless + fingerprint-suite = detected !!
• headless + stealth + fingerprint-suite = detected !!
• headfull = undetected
• headfull + stealth = undetected
• headfull + fingerprint-suite = detected !!
• headfull + stealth + fingerprint-suite = detected !!

Every time fingerprint suite is used, even without any option, it makes the browser detectable.

To Reproduce

const { chromium: playwright } = require('playwright-extra')
const { newInjectedContext } = require('fingerprint-injector');

playwright.launch({ headless: false }).then(test)

async function test(browser) {
  const context = await newInjectedContext(browser, {}); // DETECTED
  // const context = await brower.newContext(); // UNDETECTED

  const page = await context.newPage();
  await page.goto('https://arh.antoinevastel.com/bots/areyouheadless');
  await page.screenshot({ path: 'detected.png', fullPage: true });
  await browser.close()
}

Expected behavior
Injecting the fingerprint-suite shouldn't make the browser be detected as headless.

System information:

• OS: Arch Linux x86_64 - 6.3.2-arch1-1
• Node.js version: v16.20.0

It seems like using the "chrome" browser ( browsers: ["chrome"] ) is the only one triggering the detection.

// headless mode
Device: desktop | Os: windows | Browser: chrome | Status: You are Chrome headless
Device: desktop | Os: windows | Browser: firefox | Status: You are not Chrome headless
Device: desktop | Os: windows | Browser: edge | Status: You are not Chrome headless

Device: desktop | Os: macos | Browser: chrome | Status: You are Chrome headless
Device: desktop | Os: macos | Browser: firefox | Status: You are not Chrome headless
Device: desktop | Os: macos | Browser: edge | Status: You are not Chrome headless
Device: desktop | Os: macos | Browser: safari | Status: You are not Chrome headless

Device: desktop | Os: linux | Browser: chrome | Status: You are Chrome headless
Device: desktop | Os: linux | Browser: firefox | Status: You are not Chrome headless
Device: desktop | Os: linux | Browser: edge | Status: You are not Chrome headless

Device: mobile | Os: android | Browser: chrome | Status: You are not Chrome headless
Device: mobile | Os: android | Browser: firefox | Status: You are not Chrome headless
Device: mobile | Os: android | Browser: edge | Status: You are not Chrome headless

Device: mobile | Os: ios | Browser: edge | Status: You are not Chrome headless
Device: mobile | Os: ios | Browser: safari | Status: You are not Chrome headless

// headfull mode
Device: desktop | Os: windows | Browser: chrome | Status: You are Chrome headless
Device: desktop | Os: windows | Browser: firefox | Status: You are not Chrome headless
Device: desktop | Os: windows | Browser: edge | Status: You are not Chrome headless

Device: desktop | Os: macos | Browser: chrome | Status: You are Chrome headless
Device: desktop | Os: macos | Browser: firefox | Status: You are not Chrome headless
Device: desktop | Os: macos | Browser: edge | Status: You are not Chrome headless
Device: desktop | Os: macos | Browser: safari | Status: You are not Chrome headless

Device: desktop | Os: linux | Browser: chrome | Status: You are Chrome headless
Device: desktop | Os: linux | Browser: firefox | Status: You are not Chrome headless
Device: desktop | Os: linux | Browser: edge | Status: You are not Chrome headless

Device: mobile | Os: android | Browser: chrome | Status: You are Chrome headless
Device: mobile | Os: android | Browser: firefox | Status: You are not Chrome headless
Device: mobile | Os: android | Browser: edge | Status: You are not Chrome headless

Device: mobile | Os: ios | Browser: edge | Status: You are not Chrome headless
Device: mobile | Os: ios | Browser: safari | Status: You are not Chrome headless

const { chromium } = require("playwright");
const { FingerprintInjector } = require("fingerprint-injector");
const { FingerprintGenerator } = require("fingerprint-generator");


async function runBrowser(device, os, browserType) {
	const browser = await chromium.launch({
		headless: false
	});

	// chrome, firefox, edge, safari

	const { fingerprint, headers } = await new FingerprintGenerator().getFingerprint({
        devices: [device],
        operatingSystems: [os],
        browsers: [browserType]
    });

    const context = await browser.newContext({
        userAgent: fingerprint.navigator.userAgent,
        colorScheme: 'dark',
        viewport: {
            width: fingerprint.screen.width,
            height: fingerprint.screen.height,
        },
        extraHTTPHeaders: {
            'accept-language': headers['accept-language'],
        },
    });

    await new FingerprintInjector().attachFingerprintToPlaywright(context, { fingerprint, headers });

    const page = await context.newPage();

    await page.goto("https://arh.antoinevastel.com/bots/areyouheadless", { waitUntil: "load"});

    const value = await page.evaluate(() => document.querySelector('#res p').textContent);

    console.log(`Device: ${device} | Os: ${os} | Browser: ${browserType} | Status: ${value}`);

    await browser.close();

}

[ishfx]

@steinpigs thanks for this test!

[ishfx]

any takers ?

[barjin]

Sorry for the delay, I actually started looking into this last week - but unfortunately didn't come to any conclusion. The 'Are you headless' website seems to utilize some kind of ML-like regression, where it collects the browser fingerprint and then sends it to the server, which decides whether the fingerprint is valid or not.

Since this is not an actual bot-protection service, the priority on this is a bit lower, but I'll definitely continue looking into this. Thanks for your patience! :)

[ishfx]

No problem, I understand. Thank you for the update. However, I wanted to mention that the injected evasions in the fingerprint-injector compromise the anonymity of the fingerprint, rendering it unusable. This is an important factor to consider.

[mrfussion]

I'm not sure if this works here, but you should find a way to use 'new' in headless
playwright.launch({ headless: new }).then(test)

[ishfx]

I'm not sure if this works here, but you should find a way to use 'new' in headless playwright.launch({ headless: new }).then(test)

Thanks for the inputs, but that's not the issue here.
Plus, the headless new option is only for puppeteer and not playwright : https://github.com/microsoft/playwright/blob/9bca9f1b4ff3478c20a526c8f8b41ab8ab9be6e6/packages/playwright-core/src/client/types.ts#L105

[abhisheksurve45]

Debugged this, It is getting flagged due to webDriver property in the fingerprint being set to true.

The https://arh.antoinevastel.com/bots/areyouheadless plugin has a check

https://github.com/antoinevastel/fpscanner/blob/master/src/fpScanner.js#L119

@barjin

[qkxie]

@abhisheksurve45 So why did not fingerprint-suite set webDirver to false?

[tenkuken]

fingerprint-injector/fingerprint-injector.js
await page.setExtraHTTPHeaders(this.onlyInjectableHeaders(headers, browserVersion));

without extraheader will pass the detection.

[tenkuken]

fingerprint-injector/fingerprint-injector.js await page.setExtraHTTPHeaders(this.onlyInjectableHeaders(headers, browserVersion));

without extraheader will pass the detection.

Removing accept-language will fix this issue:
fingerprint-injector/fingerprint-injector.js
await page.setExtraHTTPHeaders(this.onlyInjectableHeaders(headers, browserVersion));
++ delete extraHeaders['accept-language'];

[FdelMazo]

Can confirm @tenkuken patch works! I added accept-language to the requestHeaders array (which is an array of headers to be filtered out) and my tests with areyouheadless works both with headless: old and headless: new on puppeteer

fingerprint-suite/packages/fingerprint-injector/src/fingerprint-injector.ts

Lines 36 to 53 in 098d592

	private onlyInjectableHeaders(headers: Record<string, string>, browserName?: string): Record<string, string> {
	const requestHeaders = [
	'accept-encoding',
	'accept',
	'cache-control',
	'pragma',
	'sec-fetch-dest',
	'sec-fetch-mode',
	'sec-fetch-site',
	'sec-fetch-user',
	'upgrade-insecure-requests',
	];
	const filteredHeaders = { ...headers };
	requestHeaders.forEach((header) => {
	delete filteredHeaders[header];
	});