A Comprehensive DeFi Risk Frame Work for the Interdependent systems?

[CarboClanC]

DeFi microcosm is much greater than the sum of its parts. What are the relevant frameworks we can leverage to evaluate smart contract risk AND financial risks associated with highly interdependent DeFi stack?
We should not made the same mistake we have made in the past in traditional financial markets in assuming independence of underlying shocks. Even if we understand every single line of code in every single system, how can we know how would they interact? How would the edge case in one system's smart contract design translate into financial tail risk in another? Just some food for thought...

[anthonyleezhang]

Writing down a few thoughts from a discussion I had with @CarboClanC :

• DeFi is a lot of independent systems, but they're (to my understanding) all linked to prices of a few crypto assets, notably ETH. These kinds of systems are historically vulnerable to big, correlated crashes. The designers of e.g. DAI seem to have had this in mind, and have over-collateralized to try to fight these issues, but I think it's just hard to rule out events way in the tail of the distributions. Simulations just historically aren't good at predicting these kinds of "black swan" events, it's hard to predict what will happen. So I think it's a useful exercise to ask "what will happen to the DeFi ecosystem if ETH crashes, leading to DAI CDP fire sales, further fueling the ETH crash, etc."

• One interesting point from traditional finance: compare the crash and the subsequent great depression of 1929 to the more recent 2008 crash. Arguably, having a "lender of last resort", the Fed, able to step in and take over all the bad debt in a "black swan event" is what saved the financial sector in 2008 from larger scale collapse. The 1929 Fed had less knowledge and less tools to combat the recession, and thus let some very large fraction (I think like 1/4) of all banks fail. The network of contracts that is the financial system then took a long time to recover.

• DeFi doesn't at present have a lender of last resort, as far as I understand. There's no one to step in to take on bad debt or negotiate changes to contract terms in these kinds of "black swan" events. That's kind of the selling point of the whole system: rules are written into the code, no individual can unilaterally change them. This is good if you think there are big rent-seeking financial institutions tweaking the rules for their own profit. This is bad if you think that occasionally the Fed's ability to tweak the rules adds stability to the system.

• If we are interested in stability of the DeFi system and resilience against black swan/tail risk kinds of events, maybe one thing to think about is to add ``panic buttons'' to products like DAI, UMA, MARKET, etc. For example, if the price of ETH drops suddenly, perhaps code could be triggered to allow slower sales of collateral, slow down margin calls, forgive some debt, etc., in the interests of systemic stability.

• More tentatively, imagine a kind of "decentralized central bank" -- a (basically) democratic institution whose power is linked to these kinds of price-linked "panic buttons", who can then institute emergency policies designed to save as much of the DeFi ecosystem as possible in these kinds of black swan/tail risk scenarios.

[jasonge27]

The interaction between agents in the Defi stack can be simulated since the dynamics are strictly codified. If the agents are bots with a certain strategy (policy function), we can train the policy function so that the bots try to "unbalance" the system. Given enough computational power, the bots should be able to learn how to find the financial black swan just like how OpenAI trained its bots to play 5v5 Dota. This simulation was not possible with traditional finance because credit risk is hard to evaluate -- if my counter-party goes bankrupt, they may default on their contracts with me. Defi hopefully eliminates credit risk and makes tail events more predictable.

[anthonyleezhang]

I don't really buy this argument.

• We may be talking about different things, but DeFi isn't just a program to simulate because it has humans in the loop. ETH prices decline a bit, CDPs are closed and auctioned off. This can be simulated, but how does the market react to this? Do ETH holders in the broader world panic and take all the liquidity out of the market? That part isn't a program.

• DOTA is an "easy" problem in a sense, for two reasons. One, it's a game, so you can play the computer against itself; two, it's zero-sum. Winning a zero-sum game against a human is actually easier than saving a non-zero-sum economy with unpredictable human participants.

• ML kinds of approaches are even harder to apply in these kinds of "black swan" events. We just don't know how the psychology of the market behaves in situations we've never seen the market in before. So long as some element of the DeFi system involves nontrivial human decision making I think this is going to be an issue.

[anthonyleezhang]

And let me emphasize, I am not saying the system is fundamentally broken, or that doing simulations is useless, I am just saying it would maybe be good to be over-careful and not to trust our models too much, because historically (in traditional finance) models and simulations haven't worked well in extreme situations. Best to have an overabundance of caution. The design of, e.g. DAI I think is going in the right direction by over-collateralizing by large amounts. I am basically throwing out the idea of doing more kinds of "what if" scenarios, and perhaps looking into creating "panic buttons" triggered by large collapses that allow breaking/relaxing/changing the rules in extreme situations.

[jasonge27]

I agree that market dynamics are not easy to be simulated. I think my point is that the black swan events for DeFi may be more easily found via simulation than the traditional financial market because there's no counter-party risk here. Counterparty risk event such as Lehman Brothers bankruptcy was hard to predict but turned out to be a key event. But I agree that counter-party risk may not be the real bottleneck for simulating black swan events.

The unpredictable human behavior is indeed another factor but it's a bit off-topic. Bots that are initialized with a random policy (maybe with some panic mode or risk aversion / other behavioral finance models) could be a starting point for modeling such market behavioral. It's limited by the model space of the simulation mechanism and should not be trusted as a "safety guarantee" as @anthonyleezhang argued.

Finding black swan may be too hard a problem to start with. What're the lower hanging fruits now in this area? Is there a better framework for the stress test and risk management for DeFi?

[mindaoyang]

Actually, there are 'last resort lenders’ in DeFi setups today, in the case the DAI, the last resort fall back to MKR holders, ie to recap the system in the event of Global Settlement, similarly, dForce token holder also will step in the recap the system, in the event USDx encountering black-swan events, the issues could arise for those protocols without any platform token embedded in the system to safeguard the platform.

A bigger and more unpredictable risks are from those interdependent components, i.e over-reliance on Maker's ETH oracle could trigger cascading risks among interconnected protocols and smart contract risk is certainly next to watch.

In traditional finance, risks are quite easy to ring-fence, thus handled, but not in DeFi, where most contracts are not expected to have exit hatches or brakes, it could amplify the network effect of black-swan risks. Maybe there will be protocol or framework developed for ring-fencing interdependence risks, like DeFi's own BASEL protocols.

[anthonyleezhang]

Thanks @mindaoyang , I don't have super great knowledge of DAI's structure or others. I basically agree that the network of interdependencies creates risks that are opaque and hard to understand. (I think this is a problem with any system with multiple components that evolves organically, I don't think we understand the network of cascading risks in traditional finance perfectly either)

[jclancy93]

Super interesting discussion, thanks for pinging me @CarboClanC.

Finding black swan may be too hard a problem to start with. What're the lower hanging fruits now in this area? Is there a better framework for the stress test and risk management for DeFi?

I agree with this. Finding black swan risks is something the traditional financial system still struggles with. No model is perfect and there is always going to be some level of model risk. However, I don't think that means we shouldn't attempt to better model and understand these systems.

As DeFi continues to grow, there will be many users of these systems that are unable to understand all of these interdependent risks. There will also be financial products offered that make different risk-reward tradeoffs. One of the responsibilities of the people developing and pushing these systems forward is to better communicate and mitigate risks to users.