Add domain blacklist or toggle to disable/enable

[KyleMit]

One possible issue with the ubiquity of applying to every page, is it'll also show up on work emails and important stuff. Users faced with this friction are likely to disable all-together, rather than keep installed for more whimsical uses on other sites.

The icon should add a drop down menu similar to stylish or something that offers the ability to turn off styles for a particular domain

That probably forces you to add a management page that allows you to view and manage all locally stored info.

Some lower hanging fruit would just be to allow toggling on and off ad hoc for a particular viewing of a page. Since you're applying inline styles, you can add a class to each wrapped element like .colorify and then apply the rule

.colorify-override .colorify {
   color: inherit !important
}

The toggle can be applied from the icon menu or from a registered keyboard shortcut