malformed attribute selector

[brandonscholet]

Hi! I looked through a lot of the internet at your discussions on the error "malformed attribute selector". In one of my projects I'm depending on selectors written by a large group of people.

I was wondering if you had a regex that could detect invalid selectors, such as those missing a quote or something.

It would be really wonderful, and save time, as I could just ignore the selector instead of the current try catch solution

[brandonscholet]

Also, I just wanted to say thank you for what you do.

[facelessuser]

I don't have just a simple regex to tell if an entire complex selector is good. In general, I'm basically tokenizing the selectors.

To better explain, I start at the beginning and I look for allowed things identifiers (tags), classes (identifiers preceded by .), pseudo-class names (identifiers preceded by :). If a pseudo class is followed by (, and it is recognized as a pseudo class that takes functional options, then we switch context and parse according to the rules of that pseudo-class. If we see a [, we switch context to handle an attribute.

I'm mainly giving a general overview, the exact details may be a little different, but you get the idea. There is no simple magic regex that can tell if the entire selector is valid, we just take it apart, piece by piece, making sure each piece makes sense in context of what came before, and if it doesn't, we halt by throwing an error.

[brandonscholet]

That's fair. Thank you for your time!

You should check out my tool wappybird. It's built on wapppalyzer. I use it to scan very large ranges for penetration tests to search for technologies.

[facelessuser]

@brandonscholet One thing you could do is compile the selector to test it in your own validation function.

>>> import soupsieve as sv
>>> def is_valid_selector(sel):
...     try:
...         sv.compile(sel)
...     except (sv.SelectorSyntaxError, NotImplementedError):
...         return False
...     return True
...
>>> is_valid_selector('> this')
False
>>> is_valid_selector('tag.class[attr=value]')
True

[brandonscholet]

You are a gentlemen and a scholar. It's quick and easy to add to the guy's script. and adds no time to the larger amount of things to parse through without adding to execution time.

[facelessuser]

Yeah, we cache recently compiled patterns, so subsequent compiles should take much less time.

[facelessuser]

Just a note for anyone else checking this issue out. Soupsieve also offers its own API. It was designed this way as we were not sure when we originally implemented this if BeautifulSoup would accept us as the official CSS selector implementation.

We can build a query before using it, and use the query directly by passing in the BeautifulSoup tag.

>>> from bs4 import BeautifulSoup
>>> import soupsieve as sv
>>> soup = BeautifulSoup('<div><span>text</span></div>', 'html.parser')
>>> query = sv.compile('div > span')
>>> query.select(soup)
[<span>text</span>]

This allows you control of the selecting process. You can catch bad selectors and then choose to continue or not continue with the query without having to issue a recompile. The API also allows performing matches with match and finding the closest parent with closest, etc.

One of these days, I may see if BeautifulSop can add closest and match into their API as well, but they are always usable through our API.

[gvozdila]

Hi! I want to join discussion with a question. If user-formed selector could harm while sv.compile() ?
Cause if we pass some none-typical string to soupsieve.compile(), than we can hang it.

>>> from bs4 import BeautifulSoup
>>> import soupsieve as sv
>>> selector = sv.compile("[\]!=D4XXXXXXXXXXXXXXXXXXXXXXX")

Hangs with python3.11.2 and soupsieve 2.3.2 or 2.8.2

I'm sure this string expands into something like 42^^GIGANUMBER, but looks like one should check it anyway before passing to compile()

[facelessuser]

@gvozdila, it would be appreciated if, in the future, a new issue is opened for new cases, as opposed to commenting on old closed cases.

As for your specific case, the problem is actually that we just had a bad refactor. We originaly had a pattern that checked for some valid characters for identifiers r'[somechars]+', and when we created a pattern variable to replace this, we mistakenly repeated the identifier when we shouldn't fr'{IDENTIFER}+'. It should have just been fr'{IDENTIFER}'.

Anyway, the case has been identified and fixed here: #289