Still something wrong with Notes/Disclosures - perhaps time for a new approach

[mpreiss9]

Feature Category

• [X ] New API functionality
• Performance improvement
• Developer experience improvement
• Documentation enhancement
• Tool/utility addition

Problem Statement

The new Notes and Disclosures capability is not really working out.

notes = xbrl.statements.notes()
disclosures = xbrl.statements.disclosures()

do not produce all the notes and disclosures. And many of the ones provided aren't helpful as they are malformed or textual.

Take for example AMAT 2022 10-K. (Any other year would work too). In the physical filing we see Note 8, Balance Sheet Detail. This Note has several tables: Inventories, Other Current Assets, PPE, Deferred Income Taxes and Other Assets, Accounts Payable and Accrued Expenses, Other Liabilities. Each table takes a line item from the Balance Sheet and expands on it, which is extremely useful as the Balance Sheet is very brief (only 5 items in Liabilities!).

Neither notes nor disclosures using xbrl.statements as above will provide these tables. They are completely missing. HOWEVER, if the user knows to do:

specific_notes = xbrl.statements.search('Balance Sheet Detail')

then these items are found.

Who would benefit from this feature?

• Beginner Python users working with SEC filings
• [ X] Financial analysts and researchers
• [ X] Advanced developers building financial applications
• Data scientists working with financial datasets

Proposed Solution

Perhaps we should move to the approach Arelle uses. It's also what the SEC uses to produce the csv datasets. Each table in the filing is provided it's own number and can be accessed individually.

I'd consider this rather low priority. After examining about 100 physical 10-K reports, it does seem like there is rough uniformity in how Notes/Disclosures are labelled. I think by building a manageable list of search terms it should be possible to access most of the information desired, using xbrl.statements.search

---

Feature requests are evaluated based on EdgarTools' core principles: Simple yet powerful, accurate financials, beginner-friendly, and joyful UX.

[mpreiss9]

Looking at old issues, I think this is another way of looking at #336

[Rud854]

After reading the whole text, I think this could be a start. But it's about more than Notes/ Disclosures.
. But sure I suppose for the sake of this, we can try to look at from this way

[mpreiss9]

@Rud854 Agreed your issue is about more than notes/disclosures. what I meant was that the Arelle or SEC flat file approach for creating a numeric table id for every table would necessarily also solve the notes/disclosures issue. it's a superset solution.

[dgunning]

I am working on using the SEC Financial Statement and Notes as a different way to get at the data - I assume this is the SEC flat file approach you mention. The catch is it requires some infrastructure to serve quickly - there is data preprocessing and storage and some engineering to get really per company data fast - and I would have to offer this as a cost. This would be much lower than existing commercial offerings but it's still a cost. I also intend to backport some of this to the open source library edgartools if I get it performant enough

[mpreiss9]

I'm kind of surprised you want to incorporate SEC Financial Statement and Notes into edgartools. As we've discussed, the flat files will always have a lag of from 2 to 6 weeks compared to the data now coming in from edgartools. (During government shutdowns the lag will be even longer). As a result, what comes out of edgartools will become unpredictable as some data will be current as of yesterday's filing and some will be older. I think this will hurt the user experience. I know I would want to know from one day to the next what data I'm getting.
As you and I have discussed, I am very familiar with SEC Financial Statement and Notes having built multiple iterations of software to use it. I decided to move to edgartools precisely because I was tired of the time lag and after the last government shutdown and recent SEC headcount reductions I wasn't comfortable assuming these files would continue to be updated.