The Data Science Behind Sommelier’s First Aave Cellar

With the upcoming launch of our first Cellar, we wanted to give you an technical overview of the Aave stablecoin strategy that Sommelier will be deploying and highlight our value prop for Aave lenders: If you intend to lend stablecoins on Aave, you’re better off putting that liquidity into this Cellar. Let's dive in!

What are Cellars?

Sommelier Cellars provide the functionality to execute strategies more sophisticated and complex than was previously possible in DeFi. At a high level, a strategy is executed through the Sommelier protocol using two “entities”: a strategy provider and a cellar contract.

The strategy provider sends action-recommendations through Sommelier’s validator set to be executed on the cellar contract. The strategy provider has the freedom to leverage any type of off-chain computation to arrive at these strategy recommendations. The only constraint is that the recommended actions are within the parameters of what the cellar contract allows.

Cellar design is itself an art - too much functionality opens the door to security vulnerabilities (e.g. strategy providers misbehaving), and too little functionality limits the flexibility of the type of strategies that can be run on the cellar. We’ll cover the details of cellar design in a different blog post.

The goal of this blog post is two-fold:

1. Provide a technical overview of the Aave stablecoin strategy that Sommelier will be launching as the first Cellar.
2. Set a baseline for the standards that the Sommelier community should expect regarding new Cellar proposals.

What does a Cellar Proposal consist of?

One focus of this blog post is to describe baseline standards that the community should expect regarding new strategy / cellar proposals. We will discuss the various components that a prospective strategy provider should consider before making a governance proposal proposing a new strategy, and we demonstrate these considerations concretely using the Aave Stablecoin Cellar that is launching soon.

The first question a cellar proposal should answer is: why is this strategy worth running? While this is a holistic question, there are several ways to answer quantitatively. In particular, we can leverage historical data to assess the performance of the proposed cellar. We refer to this process - of using historical data to assess a strategy - as “backtesting.”

Some questions that we considered in devising the Aave cellar + strategy are the following:

• What is the alpha of the strategy? I.e. is this strategy capturing an opportunity in the market? Does it optimize something that’s worth optimizing?

• Does this strategy need to be run on a cellar, or would a simple smart contract alone suffice? I.e. how much opportunity is there to leverage off-chain computation to improve yield?

• How does our cellar perform historically?

• Does the strategy make sense for validators to run? I.e. over what timespan do profits outweigh gas fees?

• Does it scale with TVL? What are the TVL limits of the strategy?

We will outline our process in answering each of these questions with respect to the Aave cellar.

Finally, we will show through our backtesting results that our Aave Stablecoin Cellar succeeds in selecting the optimal stablecoin lending position on Aave.

What is the alpha of the strategy?

This is the first step of the backtesting process. In our case, answering this question involved data exploration of Aave stablecoin deposit APYs (via data provider by Flipside Crypto).

This plot highlights several interesting features about Aave deposit supply rates:

• Supply rates can be highly volatile, even for stablecoins

• There are periods of higher volatility and periods of lower volatility

• It’s not obvious which stablecoin is best to lend. Instead of looking at current highest APY we should be computing something like an integral (or weighted-average) of the supply rate curves.

This suggests that there is opportunity for a strategy that outperforms the baseline of simply picking the stablecoin with the current highest APY.

Why does this strategy need to be run on a Cellar?

In other words: can this strategy be captured by just a smart contract? There are several ways to determine which stablecoin will be best to lend:

• Looking at a historical weighted average

• Chasing volatility to capture the spikes in APY

• More sophisticated time-series forecasting (e.g. machine learning based approaches)

The latter two approaches - particularly time-series forecasting - aren’t feasible to do on-chain.

This means our strategy does benefit from the flexibility that the Sommelier architecture provides.

The final strategy that we decided on uses time-series forecasting to make a decision on whether to enter a particular stablecoin lending position. Note that this approach is infeasible to run on a smart contract.

In general, any type of sophisticated machine learning techniques will need to be run off-chain. The Sommelier architecture provides the solution to leverage this complex, off-chain computation in a secure, decentralized way.

How does the Cellar perform historically?

This is the crux of “backtesting”. To answer this question we implemented the following in Python:

• An automated strategy which generates action recommendations (e.g. “Deposit DAI into a lending pool”)

• A data pipeline to feed (both historical and live) Aave data (curated by Flipside) to the automated strategy

• A Cellar simulator to take actions from the automated strategy and simulate Aave lending positions

The data pipeline and cellar simulator are fixed implementation costs. They tell us how the automated strategy would have performed based on historical data. The automated strategy, on the other hand, is an iterative process. We can use our simulators to refine the strategy to yield the best results on historical data.

The result of one backtest is given in the following table:

Let’s breakdown the table.

• Each row represents an update generated from the automated strategy. We can observe that the `timestamp` increases in 12 hour increments - this is because we set up our strategy to be updated every 12 hours in backtesting. In live deployment the strategy will update every hour.

• The `lending_token` column tells us which stablecoin the strategy is currently lending (at the timestamp).

• The `latest_action` column tells us the action that was performed at that time, for example executing the *deposit* action in row 3.

• The `lending_amount` tells us the value of the current lending position. Notice that the number increases after an `interest_accrued` event. This corresponds to interest being compounded on the current lending position.

• On the other hand, `inactive_liquidity` is the amount of assets the Cellar is **holding** without lending. It goes to 0 after *deposit* is called. If the strategy executed *withdraw* then the entire lending amount would turn into inactive liquidity.

• Since we’re manually simulating a compounding lending position, we need data on interest rates. This is the `supply_rate` column. It tells us the interest rate at the given `timestamp`. We’ll discuss `supply_rate_calc` in the next section.

• The `interest_made` column gives us the cumulative interest made up until that timestamp.

• The `_yield` column gives us the yield up until that point in the backtest. You may notice that `_yield` is negative when `interest_made` is positive. This is because we factor the cost of gas into our `_yield` calculation.

In the above table we are only showing results for the first two days of the backtest. The full table actually spans several months.

Does the strategy make sense for validators to run?

We mentioned above that our calculated yield incorporates gas fees. In the Sommelier architecture it’s important to keep in mind who pays gas fees for what transaction. Any function call that the strategy provider makes to the Cellar is passed through the validator set, therefore the validators pay the gas on these calls.

In the Aave Cellar, these are the `rebalance` calls: withdrawing from one stablecoin lending position and depositing into another.

This means that we need to choose our `rebalance` calls carefully. Too frequently and validators aren’t incentivized to run the strategy.

For our strategy, we use our time-series modeling to forecast interest rates, and only call `rebalance` if the difference in interest between the current lending position and prospective position is enough to repay the gas fees within a reasonable time period (the specific time period is a parameter of the strategy). This, of course, depends on Cellar TVL.

Does the Cellar scale with TVL?

For any strategy, there is a TVL (Total Value Locked) limit where the strategy is no longer effective. In Aave, a large lending position actually affects the interest rates. In our `CellarSimulator` we account for the size of our position in our interest rate forecasting (and backtesting). This means that the strategy can automatically optimize for the best lending position given the Cellar TVL.

We can compare the historical performance of our strategy to a holding strategy for various stablecoins at various TVLs:

TVL remains fixed over the duration of the backtest, i.e. users don’t add or remove liquidity. The performance of our strategy tracks the best performing stablecoin at each TVL using only historical data

Some features of the strategy from the above plot:

• The strategy doesn’t do anything (APY = 0%) when TVL is below a certain point. This is because the interest at these low TVLs doesn’t justify the gas

• At low TVLs, historically FEI is the best performing stablecoin

• At high TVLs only certain stablecoins are viable

Note that we need to account for the size of our position in backtesting performance and interest rate forecasting. This is the `supply_rate_calc` column in the above table. We take our position and recalculate Aave’s supply rate based on the effect of our simulated lending position.

This is also why the FEI curve decreases as TVL increases: the FEI Aave pool is relatively small and therefore large TVL’s neutralize the interest.

Strategists may choose to put a cap on TVL to maximize performance of the strategy.

In the above plot, we assumed Cellar TVL remained fixed throughout the duration of the backtest. In reality, users will be continuously adding/removing liquidity from the Cellar. We decided to simulate this flow of liquidity by assuming a linearly growing TVL and adding random noise.

TVL starts at $100k and grows linearly over the course of the backtest to the Final TVL on the x-axis. We also add Gaussian noise at each time-step to simulate randomness in add/remove liquidity events. Our strategy better adapts to growing TVL than a fixed strategy that only lends a single stablecoin.

Each point on each curve in the above plot corresponds to a single backtest. If we zoom in to a specific “Final TVL” value we produce the following plot:

Performance of our strategy vs. holding individual stablecoins in a Cellar that increases in TVL from $100k to $25MM over the course of 4 months. Our strategy obtains the highest yield by adapting to the changes in TVL.

We’re actively working on simulating different kinds of (non-linear) TVL growth curves to make our strategy even more robust.

The Verdict: Cellar Performance

Our backtesting results demonstrate that this Aave Stablecoin Cellar achieves nearly optimal performance in selecting the best stablecoin to lend on Aave. In particular, this Cellar accounts for volatility, position size, and gas costs while dynamically rebalancing to the best available position.

The value prop is straightforward: if you intend to lend stablecoins on Aave, you’re better off putting that liquidity into this Cellar.

Conclusion

In this post we’ve discussed some of the considerations a prospective strategist / cellar creator should think about when proposing a new strategy or Cellar. We described our generic backtesting framework and dove into specifics of how we approached the upcoming Aave Stablecoin Cellar.

One important point that we haven’t addressed is the programming overhead for performing strategy backtesting. In an ideal world, a strategist could run a strategy on Sommelier with minimal (if any) programming experience. Even with programming experience, building backtesting infrastructure is non-trivial. To lower this barrier, we are working on developing generic and flexible tools to support backtesting and deploying new Cellar strategies on Sommelier.