Polars Functionality

[Rud854]

So it's another long term request for me. I understand that Edgartools runs with Pandas and that is great honestly. However, for me I began to see with larger financial datasets that I possess in my overall workflow that pandas is not fast enough on its own. If it's a minor suggestion, to allow a way for you to automatically allow a Polars Dataframe. A lot of my workflow is moving on to Polars. So for example what can work is

Income_df = income.statement_todataframe(add polars= Boolean)
Using a backend function to convert from pandas to Polars.
polars_df =pl. from_pandas (pandas dataframe)

I'm not sure how to implement this.

As a PS how is the textblock to dataframe feature. That issue was #336

[kalavrcom]

I use Polars exclusively after making the switch away from Pandas. Finance is still dominated by Pandas though, so it will likely be a slow transition. In the meantime, you can easily convert from Pandas to Polars dataframes by using pl.from_pandas(pandas_df). https://docs.pola.rs/api/python/stable/reference/api/polars.from_pandas.html

Sometimes you have to fiddle with the Pandas index/indices but still very workable.

[dgunning]

Is it a matter of code convenience since the rest of your workflow is in Polars or do you actually anticipate performance gains when working with edgartools? I hadn't made the change to polars as yet because pandas is still so widely used.

It should not be that difficult to add polars support, I just didn't know it was worth it

[Rud854]

I think it's both convenience and for performance gains to be able to integrate Polars to my large workflows especially if I'm pulling a lot of filings and aggregating them. To be honest, from my job and stuff polars is being slowly transitioned from pandas. The syntax is a lot better and smaller too.

[Rud854]

I use Polars exclusively after making the switch away from Pandas. Finance is still dominated by Pandas though, so it will likely be a slow transition. In the meantime, you can easily convert from Pandas to Polars dataframes by using pl.from_pandas(pandas_df). https://docs.pola.rs/api/python/stable/reference/api/polars.from_pandas.html

Sometimes you have to fiddle with the Pandas index/indices but still very workable.

Yeah I agree. Im aware you can but I think would be a bit easier to allow a Polars option with Edgar tools like Edgartools would be one step ahead of the curve.

and yes it's tricky with Pandas indexes sometimes.