Technical Architecture

PowerQuery connects to our main datastore, a Postgres database, and executes user-defined queries in an efficient manner. When a user enters the site (whether they are authenticated or not), they can use a SQL IDE to perform analysis on any of our public tables. Queries are ordered in a queue and executed on the core database directly.

For authentication, PowerQuery relies purely on Web3 principles. User credentials are stored in a Postgres backend, but no passwords are required. The authentication flow is as follows:

1. The user requests a random nonce from the server

2. The server generates a random nonce and assigns it to the user’s address in the database, returning it to the user

3. The user signs the nonce with their Ethereum wallet, and sends the signature to the server

4. The server verifies that the signature matches the nonce and the address stored, logging the user in if the verification passes

Only the user’s address is stored, no other personal details are required. The user’s address is then used as the default username.

PowerQuery also has a caching layer that minimizes the required resources for creating data products. When a user executes a saved query, e.g. on a dashboard or for an individual data product, the query result is cached in a Redis instance, and the next time somebody accesses that query, the result is instantly returned from the cache, saving on computation. Users can specify the timeout of the cache when they create a data product, if they would like more up-to -date results.

Results can be displayed as raw tables and visualizations, but the full datasets can also be downloaded and exported in a variety of formats, including CSV and JSON. Charts can be shared and embedded, and they will be tied to your wallet. We limit the amount of rows that can be exported, so users are encouraged to perform aggregate queries that can still perform complex analysis on hundreds of millions of rows.

In the future we will boast even more expansive data coverage, including multiple blockchains, and full trade histories for symbol pairs. User queries and query results will also be encrypted by the user’s public keys, meaning that the data you collect and analytics you perform will be end-to-end encrypted. The L3 Atom community will also be involved in creating new visualization types and further customization options.