Oracles are not a one size fit all solution, in fact redundancy should be used depending on your use cases and risk tolerance
Developers building real world applications for web3 rely on oracles to relay and publish data on-chain. Oracles are not a one size fit all solution, in fact redundancy should be used depending on your use cases and risk tolerance. Switchboard’s goal is to generalize data feeds and leave it up to the developer to configure and tune their data flow. Today’s article looks at how Switchboard feeds differ from other providers, and how you can configure your data feed to increase economic security.
We also have an infographic which summarizes this article: https://docs.switchboard.xyz/img/Data_Feed_Best_Practices.png
Oracles Overview
A Switchboard feed belongs to a single oracle queue, which is a realm of oracles that get assigned to update requests in a round-robin fashion. The feed’s config dictates how update requests are invoked and routed through the network, along with a set of job accounts that define how data is sourced off-chain. Switchboard data feeds can fetch data from http endpoints, websockets, or on-chain sources. Switchboard feeds are meant to be generic enough to meet any use case and leave it up to the on-chain consumers to configure.
Oracle Consensus
Switchboard uses rounds to open and close a batch of oracle responses. A Switchboard feed has a specified minUpdateDelay which determines the minimum time a round is open for in order to give the oracles enough time to respond. When a new round is opened, the data feed gets assigned to a new batch of oracles, which is configurable with the parameter oracleRequestBatchSize. After minOracleResults responses are received, the on-chain program calculates the result using the median of the oracle responses. The median ensures large outliers do not drastically skew the accepted result, unlike averages. Oracles who responded within the queue’s configured parameters are rewarded, while the oracles who respond outside this threshold are slashed (if the queue has slashingEnabled). As you can see, the oracle consensus can be configured by the downstream user to provide the security necessary for their use case. Check out the data feed docs for more information on the lifecycle of Switchboard data feed updates.
This differs from other oracle solutions like Pyth where data feeds are sourced from trusted market participants directly. This results in a faster update rate but makes no guarantees on the number of sources that responded. With Switchboard feeds you can guarantee a set number of sources and oracles have responded before accepting the result on-chain.
Democratizing Data Feeds
Switchboard is a decentralized protocol that allows users to publish and request updates from a pool of oracles. Switchboard oracles allocate their resources to updates requested from on-chain consumers. This leaves it up to the consumer to decide what data is brought on-chain.
Switchboard feeds are public meaning anyone is free to read the value on-chain so sometimes it is ideal to piggy back on an existing feed. This works well for some protocols but leaves you vulnerable to sudden changes in your on-chain data flow. This is a risk vector that can be mitigated by owning your own data feed and being your feed’s authority.
If you’re using a public data feed, consider contributing to the feeds lease to extend its lifetime.
💡 NOTE: The feed authority differs from the lease authority. Only the lease authority is permitted to withdraw funds. When contributing to public leases check the lease authority is set to an empty public key or else the feed authority could withdraw the funds for themselves.
Own Your Data
The feed authority has the power to modify the feed, including:
change the job definitions and adapt to market conditions such as if a new exchange siphoned volume from the current sources
speed up or slow down the feed depending on changing use cases
change the reporting parameters to save on cost
move the feed to a new queue for increased security or reliability
complete ownership over your on-chain data flow
DAO
A feed authority could even be set to a DAO where the quorum votes on changes to a feeds configuration. This is true decentralization at play where the settings of a feed are optimized for a variety of use cases where voting power could be controlled by lease contributions or tokens staked.
Tuning a Switchboard Feed
Now that we have some background on how switchboard oracles publish data on-chain, let’s look at how to configure a Switchboard feed.
Data Source Diversity
The first step when building a feed is determining where you are sourcing your data from. Each data source should correspond to a job account, which is just a collection of Switchboard tasks, used to instruct the oracles on how to fetch and transform data. Checkout the job directory for some example definitions. You can also view a catalog of curated job definitions in the Switchboard publisher.
Job Diversity — Ensure you’re sourcing data from multiple, reliable sources.
Job Weights — Add job weights for sources with a higher degree of truth, such as exchanges where price discovery is likely to happen. Oracles submit the weighted median as their final result so jobs with a higher weight have more influence on the result.
minJobResults — Ensure the oracles have a reliable set of data before responding on-chain. If only 1 out of 12 sources respond, you will not have a reliable result. But if you set this to 12 out of 12 and one of your sources is failing, you will encounter a stale feed, so ensure you have some margin for single source failures.
Queue Selection
You should select a queue that will provide your feed with a sufficient level of security.
Oracle Reputation — you should select a queue with a proven track record of honest reporting. Oracles store various metrics on-chain that can be used to determine their reliability. Oracles are assigned to update request in a round robin fashion where assignment is pseudo-random but this gives a fairly accurate picture of a queues overall reliability.
Oracle minStake — you should select a queue that requires oracles to stake a respectable sum of capital before joining the queue. This will be used to slash oracles for misreporting feeds.
Queue Reward — your selected queue should reward oracles sufficiently enough such that they are incentivized to report honestly, yet not too high where update request are prohibitively expensive. Switchboard DAO queues currently use 12500 lamports as the oracle reward, per update request.
Queue Permissions — high availability feeds should be on a queue where unpermissionedFeedsEnabled is set to false. This prevents data feeds from being able to join and spam the queue with update requests.
Slashing — feeds securing financial applications should be on a queue with slashingEnabled to dissuade oracles from malicious behavior.
You may notice Switchboard DAO queues do not have all of these settings enabled yet. Switchboard DAO queues currently rely on vetting trusted oracle operators to bootstrap the network. As the network grows, the queues will be further decentralized and rely on these parameters to enforce security. Stay tuned for more information on the Switchboard DAO and how new oracles will be permitted by the network.
Data Availability
Configure when and how often you need new data on-chain.
Crank / Automatic Updates — any data feed that has sufficient queue permissions is able to join a crank. A crank is a scheduling mechanism for data feeds that will invoke a new round after each minUpdateDelay with some added jitter to prevent predictable oracle assignment. Data feeds not on a crank should set the option disableCrank and will need to manually requests new updates or have their own scheduling mechanism.
minUpdateDelay — determines how often new data can be requested on-chain. This should be set depending on your use case. Decentralized exchanges will need faster updates than a lending protocol.
varianceThreshold — is the change percentage required between a previous round and the current oracle result before a value is published on-chain. This is used to conserve lease fees for the feed operator. Highly available feeds should set this to 0 so values are always reported on-chain.
forceReportPeriod — is the max staleness for a feed. If the varianceThreshold was not exceeded but a set number of seconds have passed, the oracle will respond on-chain. This should almost always be used with varianceThreshold to ensure your feed is updating as expected and not stale for another reason.
💡 NOTE: See the check_staleness function in the rust crate when integrating.
Oracle Consensus
Configure how many oracles are assigned to a request and how many must respond to accept a result.
oracleRequestBatchSize — sets the number of oracles assigned to each update request. The quickest way to increase feed security is to request more oracles each update round because it requires a higher degree of oracle collusion in order to affect the on-chain result. In reality, this increases the overall cost of a feed so it’s a careful consideration for feed operators when configuring a feed.
minOracleResults — set the minimum number of oracles that must respond before accepting a result on-chain. This ensures you have a sufficient data set before the on-chain program takes the median of the oracle responses.
The number of oracles assigned to an update request should always be less than the number of oracles required to respond. There are a variety of reasons that may cause an oracle response to fail, such as Solana network degradation, individual oracle network issues, or transaction spamming. Your feed’s lease is only deducted when a round was successfully closed based on the feed’s minOracleResults.
Cost
The two main methods to reduce a feeds cost is to:
increase the minUpdateDelay so updates are requested less often
add a varianceTolerance so changes are only reported when a given percentage change threshold is reached. This will greatly reduce the number of updates on-chain during periods of low volatility so if you are using a staleness check in your on-chain program, make sure to pair it with forceReportPeriod so a value is always reported after a given staleness interval.
Another way to reduce cost is to set the feed authority to a DAO and share the feed cost with other on-chain consumers. A DAO can be used to vote on feed changes and can help democratize data on-chain. If you’re using a public data feed, consider contributing to the feeds lease to extend its lifetime.
Redundancy
While Switchboard feeds have a proven track record, no system is fault tolerant without some level of redundancy. If possible, you should pair a Switchboard feed with another oracle provider or a TWAP feed to ensure you are viewing reliable data. Stay tuned for a future article on advanced data feed usages.
Recommended Feed Configuration
The following table highlights some recommended settings. This will not encompass all use cases — use at your own discretion. Find us on Discord to answer any configuration questions.
Feed Maintenance
The following highlights some basic maintenance you should employ when working with a Switchboard feed.
Monitor Lease Funds — you should monitor your feeds lease account for low balances. When a feed’s lease is emptied, it will no longer process new updates until it has enough to reward the oracles processing the update. We are working with a few web3 messaging services to enable wallet notifications when leases are low on funds.
Monitor Crank Eviction — if a lease is emptied, it will also be evicted from its crank. Switchboard feeds act like public utilities where anyone is free to re-push it to a crank, as long as it doesn’t have disableCrank set to true.
Monitor Data Sources — Sometimes APIs change or move. High availability feeds should have some basic routine health checks to ensure their on-chain data is updating as expected.
Integration Checklist
The switchboard-v2 crate is used to integrate Switchboard into your on-chain programs. You can also view some example programs in the switchboard-v2 repo.
Check aggregator is owned by the Switchboard program
Check the latest round has more oracle responses than the minOracleResults (get_result)
Optional, check the feed has not exceeded a given confidence interval (check_confidence_interval)
Optional, check the feed has been updated recently (check_staleness)
TWAPs and Historical Data
Switchboard feed owners can elect to add a history buffer to their feed. This will automatically store the last N accepted results on-chain, where N is configurable by the user from 1 to 350,000 samples, with the current Solana account size limits. On-chain consumers can also read from the history buffer and validate a subset of the history to ensure the current oracle price has not skewed too far from the recent samples.
Once a feed has a history buffer, anyone can create a separate TWAP feed that reads the history buffer and calculates the time weighted average. This is useful for feeds that may have low liquidity and suffer from frequent price swings.
More advanced usages of TWAP include the ability to source the TWAP price at settlement each week. Let us say we’re an options protocol with contracts settling at 5PM UTC every Friday and we need a way to source the 1hr TWAP leading up to settlement. We also need to ensure that value is always calculated for that 1hr window so our liquidators are using the most accurate price observed on-chain. With Switchboard, you can build a feed that will always resolve to that 1hr window each week. You can see it in action here on devnet. https://switchboard.xyz/explorer/2/9wChHmbtuLbjGYt9tdH7guLY9zaqT56veSxTba18k5N3
Switchboard data feeds aim to be as general as possible to meet all use cases. If you require a more complicated data feed, do not hesitate to reach out because we may be able to build a custom solution that will benefit not only you but other data feed users.
Feed Creation Walkthrough
I’m a brand new lending protocol and I need a new data feed for XYZ. Let’s walk through some considerations I might have for how to publish this feed.
Queue Selection
This protocol will obviously be successful and have a significant total value locked (TVL) so I should choose a secure queue. I look at a queue that already has an XYZ feed but I notice the queue isn’t as secure as I need; maybe it has a low minimum stake requirement for oracles or doesn’t slash oracles for misreporting. So I check the Switchboard DAO and find a queue that requires oracles to stake 10 SOL, has slashing enabled, requires feeds to be permitted to join, and has a proven track record of honest reporting … perfect, we found the queue we’ll create the feed for.
💡 NOTE: Data feed owners can always move their feed to a new queue after creation.
Data Source Diversity
Next, we’ll need to find where to source the data. I see 4 centralized exchanges and 2 decentralized exchanges. Upon further inspection, I notice one of the CEXs has questionable volume and one of the DEXs has no volume — so we should ignore these sources. So we’ll create the feed with 3 CEXs and 1 DEX. This is good but let’s see if we can do better. We see theres a Saber pool for this asset so we can use the lpExchangeRateTask to fetch the price. We can also use the jupiterSwapTask to aggregate through various on-chain sources and get us the best price. Awesome, we have 3 CEXs, 1 DEX, a Saber pool source, and a Jupiter Swap source.
We want the feed result to be calculated with enough sources so we have an accurate depiction of the price but if we keep it too strict, a single source failing could halt the feed — maybe one of the CEX sources had a data center outage. Your minJobResults should be at least one less than the total number of sources to account for this. To give us enough margin, we’ll set ours to 4, so 4 out of 6 sources have to respond in order for the oracle to publish a price update.
We notice the majority of the volume for this source is on a single CEX where the majority of price discovery occurs — we should definitely weigh this source higher than the rest. Here’s what that may look like.
Data Availability
A lending protocol may not need the most up-to-date prices as compared to a trading platform. So we will set the minUpdateDelaySeconds to 15. This should be fast enough for our lending platform — we can always adjust if needed.
We’re a lending protocol on-chain so we need the data to be readily available — so we will choose to push the feed onto a crank to enable automatic updates. The crank is an incentivisation mechanism to regularly schedule feed updates. A sport betting platform may opt-out of a crank since they will only need the data when an event is settled.
Oracle Consensus
We’re planning on securing financial capital with our data feeds so we need to ensure a malicious oracle cannot skew the price feed in its favor. To do this we will set the oracleRequestBatchSize to at least 8. This means 5+ oracles would have to collude in order to skew a feeds accepted value on-chain; remember we’re using the median and not the average as the final result. We’ll then set minOracleResults to 7 to account for any oracle networking issues that may prevent them from responding. If only 7 oracles respond, at least 4 will need to be acting in good faith to produce an honest result — which is a very reasonable assumption to make since we selected our queue carefully.
Cost
Woohoo, we’re almost done. We’ve built the job definitions, configured and tuned the settings, so we head over to the Switchboard publisher to build the feed. We get to checkout and notice the cost may be a bit high for a lending protocol that is just starting out. No worries, let’s look at some ways to conserve cost.
Let’s look at minUpdateDelaySeconds. Updating this feed every 15 seconds may be a bit overkill. We can increase this to 30 to double the length of the feed.
The XYZ feed is a relatively low volatility feed, meaning the prices updated on-chain may be the same or about the same each update round. To account for this we can set the varianceThreshold to 0.1%, which instructs the oracles to skip reporting a value if the value hasn’t differed by 0.1% since the last reported value on-chain. This will greatly extend the life of your feed but makes it hard to estimate the length of time left on the feed since it will depend on the feeds volatility and market conditions. Our feed will now only incur a cost when a feeds value has changed significantly.
Great, we’ve found some ways to reduce cost without sacrificing security. But what if the XYZ asset suffers some loss of interest and reduce volatility? Your feed may fail to update for hours on end if the price hasn’t moved by the set varianceThreshold. This can be mitigated by setting a forceReportPeriod, which instructs the oracles to always report a result on-chain even if the variance threshold wasn’t exceeded, or max staleness. We will set forceReportPeriod to 900 so the feed will always report a value on-chain every 15minutes. When integrating on-chain, we should always be checking if the feed was updated within the forceReportPeriod interval to ensure the feed is healthy.
Conclusion
This was just a basic overview of how to think about configuring a feed to meet your use case. Each on-chain application will have their own set of requirements for how often a feed needs to be updated and what level of security is needed. Switchboard has provided the tools to generalize feed creation to meet most developers use cases. Please stop by the Discord with any questions on configuring or integrating Switchboard feeds.