Skip to content

A group described by its relation to X is extracted as X #578

Description

@WaylandYang

Two sentences from the AI-companies corpus and what the extractor made of them:

Former OpenAI personnel have founded competing AI companies Anthropic, SpaceXAI, Safe Superintelligence Inc., and Thinking Machines Lab. (openai.txt, section 2)

became OpenAI —founded→ Anthropic at 90%. It ranked first among the ten paths paths_between found between the two companies, and the answer to "how are OpenAI and Anthropic related" opened with "OpenAI founded Anthropic".

Over one hundred companies using OpenAI contacted Anthropic, according to The Information. (removal-of-sam-altman-from-openai.txt, section 18)

became OpenAI —contacted→ Anthropic, again the first path, in the re-extraction that followed #568.

The pattern is one thing: the sentence's subject is a group described by its relation to X, and the extractor writes X. "Former OpenAI personnel" is not OpenAI; "companies using OpenAI" is not OpenAI. The named entity inside the description is the only name in sight, and the model reaches for it. #568 does not catch this: the fact arrives with subject: "OpenAI", a declared, typed, well-known entity, and every guard passes it. The founded relation was minted by auto-extension with no domain or range, so the direction check had nothing to test either.

What should come out

The information in both sentences is real and worth keeping; it is about the object, and the subject has no name.

  • Anthropic —founded_by→ "former OpenAI personnel": a value fact on the named participant, the description kept verbatim as the literal, the evidence quote attached. When the corpus later names the people (it does: Dario and Daniela Amodei), their worksFor / founder edges carry the real relation, and paths_between finds it through them.
  • Anthropic —contacted_by→ "over one hundred companies using OpenAI", likewise.
  • No edge between OpenAI and Anthropic from either sentence. A relation between two organisations that the text never states must not be minted from a modifier.

If the sentence has no named participant on either side, it yields nothing; a dropped fact is cheaper than a false edge that sorts first.

Where the fix goes

Two layers, the second so the first is measured rather than trusted:

  1. The extractor's contract. The prompt already says "write Y as the subject and X as the object" for direction; it says nothing about subjects that are descriptions. Add the rule and the example: a subject is the entity the sentence predicates about; a group described by its relation to an entity ("former X employees", "companies using X", "X's investors") is never written as X; state the fact from the named side, the description as the value, or leave it out.
  2. A guard at ingest, with a signal. A fact whose subject name occurs in its own quote only as part of a longer noun phrase (former OpenAI …, OpenAI personnel|employees|staff|researchers|alumni|investors|customers, using|of|from|at OpenAI right before the verb) is not asserted with that subject. It is recorded under a new reason subject_shortened with the phrase as the example, so the rate can be read per base and per model. The guard does not try to rewrite the fact: the rewrite is the model's job under rule 1, and the guard measures how often it still fails.

Not this: dropping every fact whose subject also appears elsewhere in the quote (that is most facts), or a second model call per fact.

Acceptance

Re-extract openai.txt and removal-of-sam-altman-from-openai.txt:

  • neither sentence yields an OpenAI–Anthropic edge; each yields the value fact on Anthropic, or nothing;
  • subject_shortened signals are counted and their examples read as group descriptions;
  • paths_between("OpenAI", "Anthropic") on the result puts the Amodei path first among the two-hop ones and has no one-hop edge whose evidence is either sentence;
  • the guard's tests pin the phrase shapes above and leave "OpenAI announced GPT-4" alone.

Related: #559 / #568 (an undeclared object no longer becomes a node; this is the subject-side twin), #558 / #565 (where the false edge surfaced), #560 / #569 (the minted founded relation with no signature).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions